2023-07-21T22:05:57.256 INFO:root:teuthology version: 0.0.1.dev153+g79491cb 2023-07-21T22:05:57.257 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --name rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}} --verbose --archive /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737 -- /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737/orig.config.yaml 2023-07-21T22:05:57.313 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T22:06:02.421 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737 branch: wip-rishabh-2023Jul13-base-2 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}} email: null first_in_suite: false job_id: '7347737' kernel: client: sha1: distro flavor: default kdb: true sha1: c685da6f56cc77f2cfb9c4221be3960522616caf ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul13-base-2 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13-base-2 sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13-base-2 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi107.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi190.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 79491cb69dd37a1eafb93b3471998174bdfa2fd9 timestamp: 2023-07-21_21:30:53 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-21T22:06:02.421 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa; will attempt to use it 2023-07-21T22:06:02.422 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks 2023-07-21T22:06:02.422 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-21T22:06:02.424 INFO:teuthology.task.internal:Checking packages... 2023-07-21T22:06:02.456 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea' 2023-07-21T22:06:02.457 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-21T22:06:02.457 INFO:teuthology.packaging:ref: None 2023-07-21T22:06:02.458 INFO:teuthology.packaging:tag: None 2023-07-21T22:06:02.458 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13-base-2 2023-07-21T22:06:02.458 INFO:teuthology.packaging:sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:06:02.459 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul13-base-2 2023-07-21T22:06:07.974 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4735.gbffe049d 2023-07-21T22:06:07.974 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-21T22:06:07.982 INFO:teuthology.task.internal:no buildpackages task found 2023-07-21T22:06:07.982 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-21T22:06:07.991 INFO:teuthology.task.internal:Saving configuration 2023-07-21T22:06:08.012 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-21T22:06:08.034 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T22:06:08.064 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:59:02.935156', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T22:06:08.089 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:59:02.933574', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T22:06:08.117 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:59:02.931783', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T22:06:08.117 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-21T22:06:08.124 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-21T22:06:08.124 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-21T22:06:08.124 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-21T22:06:08.125 INFO:teuthology.run_tasks:Running task console_log... 2023-07-21T22:06:08.236 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd16d843550>, signals=[15]) 2023-07-21T22:06:08.238 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-21T22:06:08.244 INFO:teuthology.task.internal:Opening connections... 2023-07-21T22:06:08.245 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2023-07-21T22:06:08.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:06:08.316 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2023-07-21T22:06:08.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:06:08.385 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2023-07-21T22:06:08.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:06:08.455 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-21T22:06:08.462 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2023-07-21T22:06:08.481 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2023-07-21T22:06:08.482 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2023-07-21T22:06:08.539 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T22:06:08.539 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T22:06:08.539 INFO:teuthology.orchestra.run.smithi107.stdout:ID="rhel" 2023-07-21T22:06:08.540 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="fedora" 2023-07-21T22:06:08.540 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="8.6" 2023-07-21T22:06:08.540 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el8" 2023-07-21T22:06:08.540 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T22:06:08.540 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2023-07-21T22:06:08.540 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T22:06:08.540 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T22:06:08.541 INFO:teuthology.orchestra.run.smithi107.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T22:06:08.541 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T22:06:08.541 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:06:08.541 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T22:06:08.541 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T22:06:08.541 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T22:06:08.542 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T22:06:08.542 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2023-07-21T22:06:08.570 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2023-07-21T22:06:08.589 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2023-07-21T22:06:08.589 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-07-21T22:06:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T22:06:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T22:06:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:ID="rhel" 2023-07-21T22:06:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="fedora" 2023-07-21T22:06:08.646 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="8.6" 2023-07-21T22:06:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el8" 2023-07-21T22:06:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T22:06:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2023-07-21T22:06:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T22:06:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T22:06:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T22:06:08.647 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T22:06:08.648 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-21T22:06:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T22:06:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T22:06:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T22:06:08.648 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T22:06:08.648 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-07-21T22:06:08.670 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2023-07-21T22:06:08.689 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2023-07-21T22:06:08.690 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2023-07-21T22:06:08.748 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T22:06:08.748 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T22:06:08.748 INFO:teuthology.orchestra.run.smithi190.stdout:ID="rhel" 2023-07-21T22:06:08.749 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="fedora" 2023-07-21T22:06:08.749 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="8.6" 2023-07-21T22:06:08.749 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el8" 2023-07-21T22:06:08.749 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T22:06:08.749 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2023-07-21T22:06:08.749 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T22:06:08.749 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T22:06:08.750 INFO:teuthology.orchestra.run.smithi190.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T22:06:08.750 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T22:06:08.750 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:06:08.750 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T22:06:08.750 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T22:06:08.750 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T22:06:08.750 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T22:06:08.751 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2023-07-21T22:06:08.770 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-21T22:06:08.781 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-21T22:06:08.922 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-21T22:06:08.922 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2023-07-21T22:06:08.925 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2023-07-21T22:06:08.928 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2023-07-21T22:06:08.946 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-21T22:06:08.952 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-21T22:06:08.953 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2023-07-21T22:06:08.984 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2023-07-21T22:06:08.986 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2023-07-21T22:06:09.011 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-21T22:06:09.095 INFO:teuthology.run_tasks:Running task kernel... 2023-07-21T22:06:09.142 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'c685da6f56cc77f2cfb9c4221be3960522616caf'} 2023-07-21T22:06:09.144 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-07-21T22:06:09.145 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-07-21T22:06:09.145 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T22:06:09.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:06:09.165 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2023-07-21T22:06:09.224 INFO:teuthology.orchestra.run.smithi107.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-21T22:06:09.224 INFO:teuthology.task.kernel:Running kernel on smithi107: 4.18.0-372.9.1.el8.x86_64 2023-07-21T22:06:09.225 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum install -y kernel 2023-07-21T22:06:10.362 INFO:teuthology.orchestra.run.smithi107.stdout:Updating Subscription Management repositories. 2023-07-21T22:06:10.362 INFO:teuthology.orchestra.run.smithi107.stdout:Unable to read consumer identity 2023-07-21T22:06:12.380 INFO:teuthology.orchestra.run.smithi107.stdout:Copr repo for python3-asyncssh owned by ceph 5.6 kB/s | 3.5 kB 00:00 2023-07-21T22:06:23.404 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 1.4 MB/s | 16 MB 00:11 2023-07-21T22:06:26.904 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 393 kB/s | 24 kB 00:00 2023-07-21T22:06:27.598 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-07-21T22:06:27.656 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2023-07-21T22:06:27.657 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2023-07-21T22:06:27.657 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2023-07-21T22:06:27.767 DEBUG:teuthology.orchestra.run.smithi107:> echo no | sudo yum reinstall kernel || true 2023-07-21T22:06:28.133 INFO:teuthology.orchestra.run.smithi107.stdout:Updating Subscription Management repositories. 2023-07-21T22:06:28.134 INFO:teuthology.orchestra.run.smithi107.stdout:Unable to read consumer identity 2023-07-21T22:06:28.396 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:02 ago on Fri 21 Jul 2023 10:06:26 PM UTC. 2023-07-21T22:06:28.439 INFO:teuthology.orchestra.run.smithi107.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-21T22:06:28.496 INFO:teuthology.orchestra.run.smithi107.stderr:Error: No packages marked for reinstall. 2023-07-21T22:06:28.542 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum reinstall -y kernel || true 2023-07-21T22:06:28.868 INFO:teuthology.orchestra.run.smithi107.stdout:Updating Subscription Management repositories. 2023-07-21T22:06:28.869 INFO:teuthology.orchestra.run.smithi107.stdout:Unable to read consumer identity 2023-07-21T22:06:29.125 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:03 ago on Fri 21 Jul 2023 10:06:26 PM UTC. 2023-07-21T22:06:29.164 INFO:teuthology.orchestra.run.smithi107.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-21T22:06:29.186 INFO:teuthology.orchestra.run.smithi107.stderr:Error: No packages marked for reinstall. 2023-07-21T22:06:29.231 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2023-07-21T22:06:29.310 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-07-21T22:06:29.311 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-07-21T22:06:29.311 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-07-21T22:06:29.311 DEBUG:teuthology.parallel:result is None 2023-07-21T22:06:29.312 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-21T22:06:29.319 INFO:teuthology.task.internal:Creating test directory... 2023-07-21T22:06:29.320 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T22:06:29.322 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T22:06:29.324 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T22:06:29.345 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-21T22:06:29.355 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-21T22:06:29.362 INFO:teuthology.task.internal:Creating archive directory... 2023-07-21T22:06:29.363 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T22:06:29.378 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T22:06:29.385 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T22:06:29.427 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-21T22:06:29.436 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-21T22:06:29.436 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T22:06:29.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:06:29.470 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T22:06:29.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:06:29.489 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T22:06:29.492 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T22:06:29.514 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T22:06:29.522 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T22:06:29.538 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T22:06:29.557 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T22:06:29.572 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T22:06:29.579 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T22:06:29.601 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T22:06:29.602 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-21T22:06:29.610 INFO:teuthology.task.internal:Configuring sudo... 2023-07-21T22:06:29.610 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T22:06:29.612 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T22:06:29.624 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T22:06:29.671 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-21T22:06:29.680 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-21T22:06:29.680 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T22:06:29.682 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T22:06:29.693 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T22:06:29.728 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T22:06:39.788 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T22:06:39.830 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T22:06:39.917 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T22:06:39.960 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:06:39.961 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T22:06:40.030 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T22:06:40.078 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T22:06:40.134 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T22:06:40.217 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T22:06:40.263 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-07-21T22:06:40.263 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T22:06:40.332 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T22:06:40.380 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T22:06:40.481 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T22:06:40.527 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T22:06:40.570 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-07-21T22:06:40.571 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T22:06:40.638 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2023-07-21T22:06:40.640 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2023-07-21T22:06:40.643 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2023-07-21T22:06:40.700 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T22:06:40.704 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T22:06:40.735 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T22:06:41.145 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-21T22:06:41.157 INFO:teuthology.task.internal:Starting timer... 2023-07-21T22:06:41.157 INFO:teuthology.run_tasks:Running task pcp... 2023-07-21T22:06:41.173 INFO:teuthology.run_tasks:Running task selinux... 2023-07-21T22:06:41.183 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-07-21T22:06:41.183 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2023-07-21T22:06:41.261 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2023-07-21T22:06:41.263 DEBUG:teuthology.orchestra.run.smithi132:> sudo service auditd rotate 2023-07-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi132.stdout:Rotating logs: 2023-07-21T22:06:41.322 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2023-07-21T22:06:41.382 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2023-07-21T22:06:41.384 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-21T22:06:41.384 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2023-07-21T22:06:41.411 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2023-07-21T22:06:41.412 DEBUG:teuthology.orchestra.run.smithi132:> /usr/sbin/getenforce 2023-07-21T22:06:41.432 INFO:teuthology.orchestra.run.smithi132.stdout:Permissive 2023-07-21T22:06:41.433 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2023-07-21T22:06:41.453 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2023-07-21T22:06:41.454 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi132.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2023-07-21T22:06:41.454 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T22:06:41.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:06:41.482 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T22:06:41.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:06:41.510 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T22:06:41.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:06:41.538 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-21T22:06:41.539 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2023-07-21T22:06:41.566 DEBUG:teuthology.orchestra.run.smithi132:> sudo /usr/sbin/setenforce permissive 2023-07-21T22:06:41.592 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2023-07-21T22:06:41.619 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-21T22:06:41.632 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-21T22:06:41.639 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-21T22:06:41.784 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-21T22:06:41.809 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-21T22:06:41.811 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi107.front.sepia.ceph.com,smithi132.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-21T22:16:19.773 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi132.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2023-07-21T22:16:19.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-21T22:16:19.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:16:24.850 DEBUG:teuthology.orchestra.run.smithi107:> true 2023-07-21T22:16:24.919 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-21T22:16:24.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-07-21T22:16:24.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:16:29.995 DEBUG:teuthology.orchestra.run.smithi132:> true 2023-07-21T22:16:30.068 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-07-21T22:16:30.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-07-21T22:16:30.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:16:30.140 DEBUG:teuthology.orchestra.run.smithi190:> true 2023-07-21T22:16:30.211 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-07-21T22:16:30.212 INFO:teuthology.run_tasks:Running task clock... 2023-07-21T22:16:30.222 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-21T22:16:30.223 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T22:16:30.224 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:16:30.226 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T22:16:30.226 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:16:30.229 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T22:16:30.229 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:16:30.261 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T22:16:30.263 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T22:16:30.277 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T22:16:30.280 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T22:16:30.286 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T22:16:30.302 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T22:16:30.352 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: ntpd: command not found 2023-07-21T22:16:30.354 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2023-07-21T22:16:30.365 INFO:teuthology.orchestra.run.smithi132.stdout:506 Cannot talk to daemon 2023-07-21T22:16:30.366 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2023-07-21T22:16:30.380 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T22:16:30.381 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T22:16:30.383 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2023-07-21T22:16:30.394 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T22:16:30.395 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2023-07-21T22:16:30.395 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T22:16:30.410 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T22:16:30.424 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T22:16:30.460 INFO:teuthology.orchestra.run.smithi132.stderr:bash: ntpq: command not found 2023-07-21T22:16:30.463 INFO:teuthology.orchestra.run.smithi107.stderr:bash: ntpq: command not found 2023-07-21T22:16:30.464 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:16:30.464 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2023-07-21T22:16:30.464 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:16:30.464 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:16:30.464 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:16:30.465 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:16:30.468 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:16:30.468 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2023-07-21T22:16:30.468 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:16:30.468 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:16:30.468 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:16:30.469 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:16:30.491 INFO:teuthology.orchestra.run.smithi190.stderr:bash: ntpq: command not found 2023-07-21T22:16:30.494 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:16:30.494 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2023-07-21T22:16:30.495 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:16:30.495 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:16:30.495 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:16:30.495 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:16:30.496 INFO:teuthology.run_tasks:Running task pexec... 2023-07-21T22:16:30.505 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-21T22:16:30.506 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi107.front.sepia.ceph.com 2023-07-21T22:16:30.506 DEBUG:teuthology.orchestra.run.smithi107:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T22:16:30.506 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi132.front.sepia.ceph.com 2023-07-21T22:16:30.507 DEBUG:teuthology.orchestra.run.smithi132:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T22:16:30.507 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi190.front.sepia.ceph.com 2023-07-21T22:16:30.507 DEBUG:teuthology.orchestra.run.smithi190:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T22:16:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:Updating Subscription Management repositories. 2023-07-21T22:16:30.878 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2023-07-21T22:16:30.906 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-07-21T22:16:31.722 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:12 ago on Fri 21 Jul 2023 10:14:19 PM UTC. 2023-07-21T22:16:31.743 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:47 ago on Fri 21 Jul 2023 10:14:44 PM UTC. 2023-07-21T22:16:31.744 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:02:01 ago on Fri 21 Jul 2023 10:14:30 PM UTC. 2023-07-21T22:16:33.466 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2023-07-21T22:16:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2023-07-21T22:16:33.467 INFO:teuthology.orchestra.run.smithi190.stdout: Package Architecture Version Repository Size 2023-07-21T22:16:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2023-07-21T22:16:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:Resetting modules: 2023-07-21T22:16:33.468 INFO:teuthology.orchestra.run.smithi190.stdout: container-tools 2023-07-21T22:16:33.468 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:16:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2023-07-21T22:16:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2023-07-21T22:16:33.468 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:16:33.469 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2023-07-21T22:16:33.470 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2023-07-21T22:16:33.470 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2023-07-21T22:16:33.470 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2023-07-21T22:16:33.470 INFO:teuthology.orchestra.run.smithi132.stdout:Resetting modules: 2023-07-21T22:16:33.470 INFO:teuthology.orchestra.run.smithi132.stdout: container-tools 2023-07-21T22:16:33.470 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-21T22:16:33.471 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2023-07-21T22:16:33.471 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2023-07-21T22:16:33.471 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-21T22:16:33.599 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2023-07-21T22:16:33.600 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2023-07-21T22:16:33.600 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2023-07-21T22:16:33.600 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2023-07-21T22:16:33.600 INFO:teuthology.orchestra.run.smithi107.stdout:Resetting modules: 2023-07-21T22:16:33.601 INFO:teuthology.orchestra.run.smithi107.stdout: container-tools 2023-07-21T22:16:33.601 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:16:33.601 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2023-07-21T22:16:33.601 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2023-07-21T22:16:33.601 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:16:34.416 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:49 ago on Fri 21 Jul 2023 10:14:44 PM UTC. 2023-07-21T22:16:34.421 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:02:03 ago on Fri 21 Jul 2023 10:14:30 PM UTC. 2023-07-21T22:16:34.541 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:15 ago on Fri 21 Jul 2023 10:14:19 PM UTC. 2023-07-21T22:16:37.073 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2023-07-21T22:16:37.207 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2023-07-21T22:16:37.315 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2023-07-21T22:16:37.530 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-07-21T22:16:37.695 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2023-07-21T22:16:37.763 INFO:teuthology.orchestra.run.smithi107.stdout:Updating Subscription Management repositories. 2023-07-21T22:16:38.320 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:53 ago on Fri 21 Jul 2023 10:14:44 PM UTC. 2023-07-21T22:16:38.483 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:02:07 ago on Fri 21 Jul 2023 10:14:30 PM UTC. 2023-07-21T22:16:38.551 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:19 ago on Fri 21 Jul 2023 10:14:19 PM UTC. 2023-07-21T22:16:40.040 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2023-07-21T22:16:40.132 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================================ 2023-07-21T22:16:40.132 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2023-07-21T22:16:40.133 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================================ 2023-07-21T22:16:40.133 INFO:teuthology.orchestra.run.smithi190.stdout:Installing group/module packages: 2023-07-21T22:16:40.133 INFO:teuthology.orchestra.run.smithi190.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T22:16:40.133 INFO:teuthology.orchestra.run.smithi190.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T22:16:40.133 INFO:teuthology.orchestra.run.smithi190.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T22:16:40.133 INFO:teuthology.orchestra.run.smithi190.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T22:16:40.134 INFO:teuthology.orchestra.run.smithi190.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T22:16:40.134 INFO:teuthology.orchestra.run.smithi190.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T22:16:40.134 INFO:teuthology.orchestra.run.smithi190.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T22:16:40.134 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2023-07-21T22:16:40.134 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T22:16:40.134 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T22:16:40.134 INFO:teuthology.orchestra.run.smithi190.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T22:16:40.135 INFO:teuthology.orchestra.run.smithi190.stdout:Installing module profiles: 2023-07-21T22:16:40.135 INFO:teuthology.orchestra.run.smithi190.stdout: container-tools/common 2023-07-21T22:16:40.135 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:16:40.135 INFO:teuthology.orchestra.run.smithi190.stdout:Enabling module streams: 2023-07-21T22:16:40.135 INFO:teuthology.orchestra.run.smithi190.stdout: container-tools rhel8 2023-07-21T22:16:40.135 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:16:40.135 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2023-07-21T22:16:40.136 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================================ 2023-07-21T22:16:40.136 INFO:teuthology.orchestra.run.smithi190.stdout:Install 10 Packages 2023-07-21T22:16:40.136 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:16:40.137 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 18 M 2023-07-21T22:16:40.137 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 65 M 2023-07-21T22:16:40.137 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2023-07-21T22:16:40.196 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2023-07-21T22:16:40.252 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2023-07-21T22:16:40.288 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================================ 2023-07-21T22:16:40.289 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2023-07-21T22:16:40.289 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================================ 2023-07-21T22:16:40.289 INFO:teuthology.orchestra.run.smithi132.stdout:Installing group/module packages: 2023-07-21T22:16:40.289 INFO:teuthology.orchestra.run.smithi132.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T22:16:40.289 INFO:teuthology.orchestra.run.smithi132.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T22:16:40.289 INFO:teuthology.orchestra.run.smithi132.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T22:16:40.290 INFO:teuthology.orchestra.run.smithi132.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T22:16:40.290 INFO:teuthology.orchestra.run.smithi132.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T22:16:40.290 INFO:teuthology.orchestra.run.smithi132.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T22:16:40.290 INFO:teuthology.orchestra.run.smithi132.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T22:16:40.290 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2023-07-21T22:16:40.290 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T22:16:40.290 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T22:16:40.291 INFO:teuthology.orchestra.run.smithi132.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T22:16:40.291 INFO:teuthology.orchestra.run.smithi132.stdout:Installing module profiles: 2023-07-21T22:16:40.291 INFO:teuthology.orchestra.run.smithi132.stdout: container-tools/common 2023-07-21T22:16:40.291 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-21T22:16:40.291 INFO:teuthology.orchestra.run.smithi132.stdout:Enabling module streams: 2023-07-21T22:16:40.291 INFO:teuthology.orchestra.run.smithi132.stdout: container-tools rhel8 2023-07-21T22:16:40.291 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-21T22:16:40.292 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2023-07-21T22:16:40.292 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================================ 2023-07-21T22:16:40.292 INFO:teuthology.orchestra.run.smithi132.stdout:Install 10 Packages 2023-07-21T22:16:40.292 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-21T22:16:40.293 INFO:teuthology.orchestra.run.smithi190.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 159 kB/s | 25 kB 00:00 2023-07-21T22:16:40.293 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 18 M 2023-07-21T22:16:40.293 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 65 M 2023-07-21T22:16:40.293 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2023-07-21T22:16:40.315 INFO:teuthology.orchestra.run.smithi190.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 226 kB/s | 41 kB 00:00 2023-07-21T22:16:40.341 INFO:teuthology.orchestra.run.smithi190.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 455 kB/s | 94 kB 00:00 2023-07-21T22:16:40.344 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================================================ 2023-07-21T22:16:40.345 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2023-07-21T22:16:40.345 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================================================ 2023-07-21T22:16:40.345 INFO:teuthology.orchestra.run.smithi107.stdout:Installing group/module packages: 2023-07-21T22:16:40.345 INFO:teuthology.orchestra.run.smithi107.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T22:16:40.346 INFO:teuthology.orchestra.run.smithi107.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T22:16:40.346 INFO:teuthology.orchestra.run.smithi107.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T22:16:40.346 INFO:teuthology.orchestra.run.smithi107.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T22:16:40.346 INFO:teuthology.orchestra.run.smithi107.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T22:16:40.346 INFO:teuthology.orchestra.run.smithi107.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T22:16:40.346 INFO:teuthology.orchestra.run.smithi107.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T22:16:40.346 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2023-07-21T22:16:40.347 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T22:16:40.347 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T22:16:40.347 INFO:teuthology.orchestra.run.smithi107.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T22:16:40.347 INFO:teuthology.orchestra.run.smithi107.stdout:Installing module profiles: 2023-07-21T22:16:40.347 INFO:teuthology.orchestra.run.smithi107.stdout: container-tools/common 2023-07-21T22:16:40.347 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:16:40.347 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling module streams: 2023-07-21T22:16:40.348 INFO:teuthology.orchestra.run.smithi107.stdout: container-tools rhel8 2023-07-21T22:16:40.348 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:16:40.348 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2023-07-21T22:16:40.348 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================================================ 2023-07-21T22:16:40.348 INFO:teuthology.orchestra.run.smithi107.stdout:Install 10 Packages 2023-07-21T22:16:40.348 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:16:40.349 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 18 M 2023-07-21T22:16:40.349 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 65 M 2023-07-21T22:16:40.350 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2023-07-21T22:16:40.356 INFO:teuthology.orchestra.run.smithi132.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.4 MB/s | 94 kB 00:00 2023-07-21T22:16:40.366 INFO:teuthology.orchestra.run.smithi190.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 664 kB/s | 49 kB 00:00 2023-07-21T22:16:40.399 INFO:teuthology.orchestra.run.smithi190.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-21T22:16:40.406 INFO:teuthology.orchestra.run.smithi132.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 216 kB/s | 25 kB 00:00 2023-07-21T22:16:40.409 INFO:teuthology.orchestra.run.smithi107.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.5 MB/s | 94 kB 00:00 2023-07-21T22:16:40.432 INFO:teuthology.orchestra.run.smithi132.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 290 kB/s | 41 kB 00:00 2023-07-21T22:16:40.434 INFO:teuthology.orchestra.run.smithi107.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 286 kB/s | 25 kB 00:00 2023-07-21T22:16:40.441 INFO:teuthology.orchestra.run.smithi190.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.9 MB/s | 149 kB 00:00 2023-07-21T22:16:40.456 INFO:teuthology.orchestra.run.smithi132.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 491 kB/s | 49 kB 00:00 2023-07-21T22:16:40.459 INFO:teuthology.orchestra.run.smithi107.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 364 kB/s | 41 kB 00:00 2023-07-21T22:16:40.475 INFO:teuthology.orchestra.run.smithi190.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.8 MB/s | 213 kB 00:00 2023-07-21T22:16:40.484 INFO:teuthology.orchestra.run.smithi107.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 656 kB/s | 49 kB 00:00 2023-07-21T22:16:40.490 INFO:teuthology.orchestra.run.smithi132.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-21T22:16:40.515 INFO:teuthology.orchestra.run.smithi132.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-21T22:16:40.518 INFO:teuthology.orchestra.run.smithi107.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-21T22:16:40.541 INFO:teuthology.orchestra.run.smithi132.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-07-21T22:16:40.543 INFO:teuthology.orchestra.run.smithi107.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-21T22:16:40.568 INFO:teuthology.orchestra.run.smithi107.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.2 MB/s | 213 kB 00:00 2023-07-21T22:16:40.625 INFO:teuthology.orchestra.run.smithi190.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 28 MB/s | 8.0 MB 00:00 2023-07-21T22:16:40.650 INFO:teuthology.orchestra.run.smithi132.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-07-21T22:16:40.660 INFO:teuthology.orchestra.run.smithi107.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-07-21T22:16:40.684 INFO:teuthology.orchestra.run.smithi190.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 11 MB/s | 2.2 MB 00:00 2023-07-21T22:16:40.766 INFO:teuthology.orchestra.run.smithi132.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-07-21T22:16:40.767 INFO:teuthology.orchestra.run.smithi190.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 22 MB/s | 7.0 MB 00:00 2023-07-21T22:16:40.768 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:16:40.768 INFO:teuthology.orchestra.run.smithi190.stdout:Total 29 MB/s | 18 MB 00:00 2023-07-21T22:16:40.811 INFO:teuthology.orchestra.run.smithi107.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 26 MB/s | 7.0 MB 00:00 2023-07-21T22:16:40.867 INFO:teuthology.orchestra.run.smithi132.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-07-21T22:16:40.867 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:16:40.868 INFO:teuthology.orchestra.run.smithi132.stdout:Total 32 MB/s | 18 MB 00:00 2023-07-21T22:16:40.902 INFO:teuthology.orchestra.run.smithi107.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-07-21T22:16:40.903 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:16:40.903 INFO:teuthology.orchestra.run.smithi107.stdout:Total 33 MB/s | 18 MB 00:00 2023-07-21T22:16:40.967 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2023-07-21T22:16:40.986 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2023-07-21T22:16:40.986 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2023-07-21T22:16:41.067 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2023-07-21T22:16:41.086 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2023-07-21T22:16:41.086 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2023-07-21T22:16:41.103 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2023-07-21T22:16:41.121 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2023-07-21T22:16:41.122 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2023-07-21T22:16:41.166 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2023-07-21T22:16:41.170 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2023-07-21T22:16:41.267 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2023-07-21T22:16:41.271 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2023-07-21T22:16:41.303 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2023-07-21T22:16:41.307 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2023-07-21T22:16:41.669 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2023-07-21T22:16:41.797 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2023-07-21T22:16:41.833 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2023-07-21T22:16:41.853 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T22:16:41.976 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T22:16:41.999 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T22:16:42.022 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T22:16:42.146 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T22:16:42.184 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T22:16:42.215 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T22:16:42.365 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T22:16:42.379 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T22:16:42.425 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T22:16:42.571 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T22:16:42.573 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T22:16:42.775 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T22:16:42.940 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T22:16:42.963 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T22:16:43.529 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T22:16:43.707 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T22:16:43.716 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T22:16:44.419 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T22:16:44.560 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T22:16:44.600 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T22:16:44.629 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T22:16:44.795 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T22:16:44.817 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T22:16:44.818 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T22:16:44.978 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T22:16:44.981 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T22:16:44.982 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T22:16:45.130 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T22:16:45.159 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T22:16:45.245 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T22:16:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T22:16:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T22:16:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T22:16:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T22:16:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T22:16:45.246 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T22:16:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T22:16:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T22:16:45.247 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T22:16:45.395 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T22:16:45.396 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T22:16:45.396 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T22:16:45.396 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T22:16:45.396 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T22:16:45.396 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T22:16:45.396 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T22:16:45.397 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T22:16:45.397 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T22:16:45.397 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T22:16:45.455 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T22:16:45.456 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T22:16:45.456 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T22:16:45.456 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T22:16:45.456 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T22:16:45.456 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T22:16:45.457 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T22:16:45.457 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T22:16:45.457 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T22:16:45.457 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T22:16:46.413 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T22:16:46.414 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:02:01 ago on Fri 21 Jul 2023 10:14:44 PM UTC. 2023-07-21T22:16:46.591 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T22:16:46.592 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:02:16 ago on Fri 21 Jul 2023 10:14:30 PM UTC. 2023-07-21T22:16:46.602 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T22:16:46.603 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:27 ago on Fri 21 Jul 2023 10:14:19 PM UTC. 2023-07-21T22:16:49.157 INFO:teuthology.orchestra.run.smithi190.stdout:Installed products updated. 2023-07-21T22:16:49.356 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:16:49.356 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2023-07-21T22:16:49.356 INFO:teuthology.orchestra.run.smithi190.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:16:49.356 INFO:teuthology.orchestra.run.smithi190.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T22:16:49.356 INFO:teuthology.orchestra.run.smithi190.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:16:49.357 INFO:teuthology.orchestra.run.smithi190.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T22:16:49.357 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T22:16:49.357 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T22:16:49.357 INFO:teuthology.orchestra.run.smithi190.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:16:49.357 INFO:teuthology.orchestra.run.smithi190.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:16:49.358 INFO:teuthology.orchestra.run.smithi190.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T22:16:49.358 INFO:teuthology.orchestra.run.smithi190.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T22:16:49.358 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:16:49.358 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2023-07-21T22:16:49.505 DEBUG:teuthology.parallel:result is None 2023-07-21T22:16:49.638 INFO:teuthology.orchestra.run.smithi132.stdout:Installed products updated. 2023-07-21T22:16:49.652 INFO:teuthology.orchestra.run.smithi107.stdout:Installed products updated. 2023-07-21T22:16:49.837 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-21T22:16:49.837 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2023-07-21T22:16:49.837 INFO:teuthology.orchestra.run.smithi132.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:16:49.837 INFO:teuthology.orchestra.run.smithi132.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi132.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi132.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi132.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi132.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:16:49.838 INFO:teuthology.orchestra.run.smithi132.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T22:16:49.839 INFO:teuthology.orchestra.run.smithi132.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T22:16:49.839 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-21T22:16:49.839 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2023-07-21T22:16:49.844 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:16:49.845 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2023-07-21T22:16:49.845 INFO:teuthology.orchestra.run.smithi107.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:16:49.845 INFO:teuthology.orchestra.run.smithi107.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T22:16:49.845 INFO:teuthology.orchestra.run.smithi107.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:16:49.845 INFO:teuthology.orchestra.run.smithi107.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T22:16:49.845 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi107.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi107.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi107.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi107.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:16:49.846 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2023-07-21T22:16:49.979 DEBUG:teuthology.parallel:result is None 2023-07-21T22:16:49.983 DEBUG:teuthology.parallel:result is None 2023-07-21T22:16:49.984 INFO:teuthology.run_tasks:Running task install... 2023-07-21T22:16:49.992 DEBUG:teuthology.task.install:project ceph 2023-07-21T22:16:49.992 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea'}} 2023-07-21T22:16:49.993 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea'} 2023-07-21T22:16:49.993 INFO:teuthology.task.install:Using flavor: default 2023-07-21T22:16:49.999 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-21T22:16:49.999 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-21T22:16:50.000 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'tag': None, 'wait_for_package': False} 2023-07-21T22:16:50.000 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:16:50.003 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'tag': None, 'wait_for_package': False} 2023-07-21T22:16:50.003 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:16:50.004 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'tag': None, 'wait_for_package': False} 2023-07-21T22:16:50.004 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:17:05.634 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/ 2023-07-21T22:17:05.634 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4735.gbffe049d 2023-07-21T22:17:05.741 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/ 2023-07-21T22:17:05.742 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4735.gbffe049d 2023-07-21T22:17:05.745 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/ 2023-07-21T22:17:05.746 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4735.gbffe049d 2023-07-21T22:17:06.269 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T22:17:06.269 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:17:06.269 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T22:17:06.272 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T22:17:06.272 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-07-21T22:17:06.272 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T22:17:06.286 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T22:17:06.286 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-07-21T22:17:06.286 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T22:17:06.304 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T22:17:06.305 DEBUG:teuthology.orchestra.run.smithi132:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T22:17:06.306 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T22:17:06.306 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T22:17:06.319 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T22:17:06.319 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T22:17:06.376 DEBUG:teuthology.orchestra.run.smithi132:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T22:17:06.377 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T22:17:06.391 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T22:17:06.457 DEBUG:teuthology.orchestra.run.smithi132:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T22:17:06.458 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T22:17:06.472 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T22:17:06.524 INFO:teuthology.orchestra.run.smithi132.stdout:check_obsoletes = 1 2023-07-21T22:17:06.525 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2023-07-21T22:17:06.526 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum clean all 2023-07-21T22:17:06.526 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2023-07-21T22:17:06.540 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2023-07-21T22:17:06.541 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2023-07-21T22:17:06.898 INFO:teuthology.orchestra.run.smithi107.stdout:Updating Subscription Management repositories. 2023-07-21T22:17:06.901 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2023-07-21T22:17:06.914 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-07-21T22:17:07.196 INFO:teuthology.orchestra.run.smithi107.stdout:44 files removed 2023-07-21T22:17:07.205 INFO:teuthology.orchestra.run.smithi132.stdout:44 files removed 2023-07-21T22:17:07.230 INFO:teuthology.orchestra.run.smithi190.stdout:44 files removed 2023-07-21T22:17:07.243 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T22:17:07.252 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T22:17:07.276 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T22:17:07.573 INFO:teuthology.orchestra.run.smithi107.stdout:Updating Subscription Management repositories. 2023-07-21T22:17:07.584 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2023-07-21T22:17:07.608 INFO:teuthology.orchestra.run.smithi190.stdout:Updating Subscription Management repositories. 2023-07-21T22:17:08.927 INFO:teuthology.orchestra.run.smithi132.stdout:ceph packages for x86_64 89 kB/s | 81 kB 00:00 2023-07-21T22:17:08.949 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 86 kB/s | 81 kB 00:00 2023-07-21T22:17:08.950 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 88 kB/s | 81 kB 00:00 2023-07-21T22:17:09.895 INFO:teuthology.orchestra.run.smithi132.stdout:ceph noarch packages 10 kB/s | 9.4 kB 00:00 2023-07-21T22:17:09.900 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 10 kB/s | 9.4 kB 00:00 2023-07-21T22:17:09.917 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 10 kB/s | 9.4 kB 00:00 2023-07-21T22:17:10.867 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 1.6 kB/s | 1.5 kB 00:00 2023-07-21T22:17:10.884 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 1.5 kB/s | 1.5 kB 00:00 2023-07-21T22:17:11.003 INFO:teuthology.orchestra.run.smithi132.stdout:ceph source packages 1.4 kB/s | 1.5 kB 00:01 2023-07-21T22:17:11.568 INFO:teuthology.orchestra.run.smithi190.stdout:Copr repo for python3-asyncssh owned by ceph 5.1 kB/s | 3.5 kB 00:00 2023-07-21T22:17:11.609 INFO:teuthology.orchestra.run.smithi107.stdout:Copr repo for python3-asyncssh owned by ceph 4.9 kB/s | 3.5 kB 00:00 2023-07-21T22:17:11.613 INFO:teuthology.orchestra.run.smithi132.stdout:Copr repo for python3-asyncssh owned by ceph 5.9 kB/s | 3.5 kB 00:00 2023-07-21T22:17:13.876 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 16 MB 00:02 2023-07-21T22:17:15.221 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 4.4 MB/s | 16 MB 00:03 2023-07-21T22:17:17.335 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 483 kB/s | 24 kB 00:00 2023-07-21T22:17:18.320 INFO:teuthology.orchestra.run.smithi107.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 MB/s | 53 MB 00:00 2023-07-21T22:17:18.714 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 400 kB/s | 24 kB 00:00 2023-07-21T22:17:19.650 INFO:teuthology.orchestra.run.smithi132.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 MB/s | 47 MB 00:00 2023-07-21T22:17:23.867 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 16 MB 00:12 2023-07-21T22:17:26.520 INFO:teuthology.orchestra.run.smithi107.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-07-21T22:17:27.509 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 148 kB/s | 24 kB 00:00 2023-07-21T22:17:27.742 INFO:teuthology.orchestra.run.smithi132.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 56 MB/s | 53 MB 00:00 2023-07-21T22:17:28.428 INFO:teuthology.orchestra.run.smithi190.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-07-21T22:17:36.912 INFO:teuthology.orchestra.run.smithi190.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 47 MB 00:01 2023-07-21T22:17:37.492 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:17:37.493 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:17:37.508 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T22:17:37.508 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T22:17:37.510 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T22:17:37.511 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T22:17:37.513 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T22:17:37.516 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T22:17:37.518 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T22:17:37.650 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2023-07-21T22:17:37.662 INFO:teuthology.orchestra.run.smithi107.stdout:=========================================================================================================================== 2023-07-21T22:17:37.663 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2023-07-21T22:17:37.663 INFO:teuthology.orchestra.run.smithi107.stdout:=========================================================================================================================== 2023-07-21T22:17:37.664 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2023-07-21T22:17:37.664 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T22:17:37.664 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T22:17:37.664 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.4 k 2023-07-21T22:17:37.664 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 5.4 M 2023-07-21T22:17:37.664 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 848 k 2023-07-21T22:17:37.665 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 165 k 2023-07-21T22:17:37.665 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 1.5 M 2023-07-21T22:17:37.665 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 129 k 2023-07-21T22:17:37.665 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 1.7 M 2023-07-21T22:17:37.665 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 7.4 M 2023-07-21T22:17:37.665 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 47 k 2023-07-21T22:17:37.665 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 13 M 2023-07-21T22:17:37.666 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 48 M 2023-07-21T22:17:37.666 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 258 k 2023-07-21T22:17:37.666 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 164 k 2023-07-21T22:17:37.666 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 258 k 2023-07-21T22:17:37.666 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 18 k 2023-07-21T22:17:37.666 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T22:17:37.667 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T22:17:37.667 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T22:17:37.667 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 31 k 2023-07-21T22:17:37.667 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 771 k 2023-07-21T22:17:37.667 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 139 k 2023-07-21T22:17:37.668 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T22:17:37.668 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T22:17:37.668 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T22:17:37.668 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T22:17:37.668 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T22:17:37.668 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T22:17:37.668 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 197 k 2023-07-21T22:17:37.669 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 370 k 2023-07-21T22:17:37.669 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 364 k 2023-07-21T22:17:37.669 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 107 k 2023-07-21T22:17:37.669 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 94 k 2023-07-21T22:17:37.669 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.4 M 2023-07-21T22:17:37.669 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 179 k 2023-07-21T22:17:37.669 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T22:17:37.670 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2023-07-21T22:17:37.670 INFO:teuthology.orchestra.run.smithi107.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T22:17:37.670 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.7 M 2023-07-21T22:17:37.670 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.6 M 2023-07-21T22:17:37.670 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2023-07-21T22:17:37.670 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T22:17:37.671 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 22 M 2023-07-21T22:17:37.671 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 23 k 2023-07-21T22:17:37.671 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 2.3 M 2023-07-21T22:17:37.672 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 240 k 2023-07-21T22:17:37.672 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 4.6 M 2023-07-21T22:17:37.672 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 18 M 2023-07-21T22:17:37.673 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 15 k 2023-07-21T22:17:37.673 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 24 k 2023-07-21T22:17:37.673 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T22:17:37.673 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T22:17:37.673 INFO:teuthology.orchestra.run.smithi107.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T22:17:37.673 INFO:teuthology.orchestra.run.smithi107.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T22:17:37.674 INFO:teuthology.orchestra.run.smithi107.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T22:17:37.674 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 137 k 2023-07-21T22:17:37.674 INFO:teuthology.orchestra.run.smithi107.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T22:17:37.674 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T22:17:37.674 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T22:17:37.674 INFO:teuthology.orchestra.run.smithi107.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T22:17:37.674 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T22:17:37.675 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T22:17:37.675 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 473 k 2023-07-21T22:17:37.675 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T22:17:37.675 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.7 M 2023-07-21T22:17:37.675 INFO:teuthology.orchestra.run.smithi107.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T22:17:37.676 INFO:teuthology.orchestra.run.smithi107.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T22:17:37.676 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T22:17:37.676 INFO:teuthology.orchestra.run.smithi107.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T22:17:37.676 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T22:17:37.676 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T22:17:37.676 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T22:17:37.676 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T22:17:37.677 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T22:17:37.677 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T22:17:37.677 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T22:17:37.677 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T22:17:37.678 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T22:17:37.678 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T22:17:37.678 INFO:teuthology.orchestra.run.smithi107.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T22:17:37.678 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T22:17:37.678 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 45 k 2023-07-21T22:17:37.678 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 119 k 2023-07-21T22:17:37.678 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T22:17:37.679 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T22:17:37.679 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T22:17:37.679 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T22:17:37.679 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T22:17:37.679 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T22:17:37.679 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T22:17:37.679 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T22:17:37.679 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T22:17:37.680 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T22:17:37.680 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T22:17:37.680 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:17:37.680 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T22:17:37.680 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T22:17:37.680 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T22:17:37.680 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T22:17:37.681 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T22:17:37.681 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T22:17:37.681 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T22:17:37.681 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T22:17:37.681 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T22:17:37.681 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T22:17:37.681 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T22:17:37.682 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T22:17:37.682 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T22:17:37.682 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T22:17:37.682 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T22:17:37.682 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T22:17:37.682 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T22:17:37.682 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T22:17:37.683 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:17:37.683 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T22:17:37.683 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T22:17:37.683 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T22:17:37.683 INFO:teuthology.orchestra.run.smithi107.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T22:17:37.684 INFO:teuthology.orchestra.run.smithi107.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T22:17:37.684 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T22:17:37.684 INFO:teuthology.orchestra.run.smithi107.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T22:17:37.685 INFO:teuthology.orchestra.run.smithi107.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T22:17:37.685 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T22:17:37.685 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T22:17:37.685 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T22:17:37.686 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T22:17:37.686 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T22:17:37.686 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T22:17:37.687 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T22:17:37.687 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T22:17:37.688 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T22:17:37.688 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T22:17:37.688 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling module streams: 2023-07-21T22:17:37.689 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql 10 2023-07-21T22:17:37.689 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:17:37.689 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2023-07-21T22:17:37.689 INFO:teuthology.orchestra.run.smithi107.stdout:=========================================================================================================================== 2023-07-21T22:17:37.690 INFO:teuthology.orchestra.run.smithi107.stdout:Install 128 Packages 2023-07-21T22:17:37.690 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 3 Packages 2023-07-21T22:17:37.691 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:17:37.692 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 205 M 2023-07-21T22:17:37.692 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2023-07-21T22:17:38.411 INFO:teuthology.orchestra.run.smithi107.stdout:(1/131): ceph-18.0.0-4735.gbffe049d.el8.x86_64. 16 kB/s | 6.4 kB 00:00 2023-07-21T22:17:39.023 INFO:teuthology.orchestra.run.smithi132.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:17:39.024 INFO:teuthology.orchestra.run.smithi132.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:17:39.039 INFO:teuthology.orchestra.run.smithi132.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T22:17:39.040 INFO:teuthology.orchestra.run.smithi132.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T22:17:39.040 INFO:teuthology.orchestra.run.smithi132.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T22:17:39.041 INFO:teuthology.orchestra.run.smithi132.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T22:17:39.043 INFO:teuthology.orchestra.run.smithi132.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T22:17:39.047 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T22:17:39.048 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T22:17:39.103 INFO:teuthology.orchestra.run.smithi107.stdout:(2/131): ceph-fuse-18.0.0-4735.gbffe049d.el8.x8 1.2 MB/s | 848 kB 00:00 2023-07-21T22:17:39.178 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2023-07-21T22:17:39.190 INFO:teuthology.orchestra.run.smithi132.stdout:=========================================================================================================================== 2023-07-21T22:17:39.191 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2023-07-21T22:17:39.191 INFO:teuthology.orchestra.run.smithi132.stdout:=========================================================================================================================== 2023-07-21T22:17:39.191 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2023-07-21T22:17:39.191 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T22:17:39.191 INFO:teuthology.orchestra.run.smithi132.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T22:17:39.192 INFO:teuthology.orchestra.run.smithi132.stdout: ceph x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.4 k 2023-07-21T22:17:39.192 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 5.4 M 2023-07-21T22:17:39.192 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 848 k 2023-07-21T22:17:39.192 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 165 k 2023-07-21T22:17:39.192 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 1.5 M 2023-07-21T22:17:39.193 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 129 k 2023-07-21T22:17:39.193 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 1.7 M 2023-07-21T22:17:39.193 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 7.4 M 2023-07-21T22:17:39.193 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 47 k 2023-07-21T22:17:39.193 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 13 M 2023-07-21T22:17:39.193 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 48 M 2023-07-21T22:17:39.193 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 258 k 2023-07-21T22:17:39.194 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 164 k 2023-07-21T22:17:39.194 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 258 k 2023-07-21T22:17:39.194 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 18 k 2023-07-21T22:17:39.194 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T22:17:39.194 INFO:teuthology.orchestra.run.smithi132.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T22:17:39.194 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T22:17:39.194 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 31 k 2023-07-21T22:17:39.195 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 771 k 2023-07-21T22:17:39.195 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 139 k 2023-07-21T22:17:39.195 INFO:teuthology.orchestra.run.smithi132.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T22:17:39.195 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T22:17:39.195 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T22:17:39.195 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T22:17:39.195 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T22:17:39.196 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T22:17:39.196 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 197 k 2023-07-21T22:17:39.196 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 370 k 2023-07-21T22:17:39.196 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 364 k 2023-07-21T22:17:39.196 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 107 k 2023-07-21T22:17:39.196 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 94 k 2023-07-21T22:17:39.197 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.4 M 2023-07-21T22:17:39.197 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 179 k 2023-07-21T22:17:39.197 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T22:17:39.197 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrading: 2023-07-21T22:17:39.197 INFO:teuthology.orchestra.run.smithi132.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T22:17:39.197 INFO:teuthology.orchestra.run.smithi132.stdout: librados2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.7 M 2023-07-21T22:17:39.198 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.6 M 2023-07-21T22:17:39.198 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2023-07-21T22:17:39.198 INFO:teuthology.orchestra.run.smithi132.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T22:17:39.198 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 22 M 2023-07-21T22:17:39.198 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 23 k 2023-07-21T22:17:39.198 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 2.3 M 2023-07-21T22:17:39.198 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 240 k 2023-07-21T22:17:39.199 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 4.6 M 2023-07-21T22:17:39.199 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 18 M 2023-07-21T22:17:39.199 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 15 k 2023-07-21T22:17:39.199 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 24 k 2023-07-21T22:17:39.199 INFO:teuthology.orchestra.run.smithi132.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T22:17:39.199 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T22:17:39.199 INFO:teuthology.orchestra.run.smithi132.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T22:17:39.200 INFO:teuthology.orchestra.run.smithi132.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T22:17:39.200 INFO:teuthology.orchestra.run.smithi132.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T22:17:39.200 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 137 k 2023-07-21T22:17:39.200 INFO:teuthology.orchestra.run.smithi132.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T22:17:39.200 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T22:17:39.201 INFO:teuthology.orchestra.run.smithi132.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T22:17:39.201 INFO:teuthology.orchestra.run.smithi132.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T22:17:39.201 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T22:17:39.201 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T22:17:39.201 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 473 k 2023-07-21T22:17:39.201 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T22:17:39.202 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.7 M 2023-07-21T22:17:39.202 INFO:teuthology.orchestra.run.smithi132.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T22:17:39.202 INFO:teuthology.orchestra.run.smithi132.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T22:17:39.202 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T22:17:39.202 INFO:teuthology.orchestra.run.smithi132.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T22:17:39.203 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T22:17:39.203 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T22:17:39.203 INFO:teuthology.orchestra.run.smithi132.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T22:17:39.203 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T22:17:39.203 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T22:17:39.203 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T22:17:39.203 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T22:17:39.204 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T22:17:39.204 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T22:17:39.204 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T22:17:39.204 INFO:teuthology.orchestra.run.smithi132.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T22:17:39.204 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T22:17:39.204 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 45 k 2023-07-21T22:17:39.204 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 119 k 2023-07-21T22:17:39.205 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T22:17:39.205 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T22:17:39.205 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T22:17:39.205 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T22:17:39.205 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T22:17:39.205 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T22:17:39.205 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T22:17:39.206 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T22:17:39.206 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T22:17:39.206 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T22:17:39.206 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T22:17:39.206 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:17:39.206 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T22:17:39.206 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T22:17:39.207 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T22:17:39.207 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T22:17:39.207 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T22:17:39.207 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T22:17:39.207 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T22:17:39.208 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T22:17:39.208 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T22:17:39.208 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T22:17:39.208 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T22:17:39.208 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T22:17:39.209 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T22:17:39.209 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T22:17:39.209 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T22:17:39.209 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T22:17:39.209 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T22:17:39.209 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T22:17:39.210 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:17:39.210 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T22:17:39.210 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T22:17:39.210 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T22:17:39.210 INFO:teuthology.orchestra.run.smithi132.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T22:17:39.210 INFO:teuthology.orchestra.run.smithi132.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T22:17:39.210 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T22:17:39.211 INFO:teuthology.orchestra.run.smithi132.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T22:17:39.211 INFO:teuthology.orchestra.run.smithi132.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T22:17:39.211 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T22:17:39.211 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T22:17:39.211 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T22:17:39.211 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T22:17:39.211 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T22:17:39.212 INFO:teuthology.orchestra.run.smithi132.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T22:17:39.212 INFO:teuthology.orchestra.run.smithi132.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T22:17:39.212 INFO:teuthology.orchestra.run.smithi132.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T22:17:39.212 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T22:17:39.212 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T22:17:39.213 INFO:teuthology.orchestra.run.smithi132.stdout:Enabling module streams: 2023-07-21T22:17:39.213 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql 10 2023-07-21T22:17:39.213 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-21T22:17:39.214 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2023-07-21T22:17:39.214 INFO:teuthology.orchestra.run.smithi132.stdout:=========================================================================================================================== 2023-07-21T22:17:39.214 INFO:teuthology.orchestra.run.smithi132.stdout:Install 128 Packages 2023-07-21T22:17:39.214 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrade 3 Packages 2023-07-21T22:17:39.214 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-21T22:17:39.215 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 205 M 2023-07-21T22:17:39.215 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2023-07-21T22:17:39.228 INFO:teuthology.orchestra.run.smithi107.stdout:(3/131): ceph-immutable-object-cache-18.0.0-473 1.3 MB/s | 165 kB 00:00 2023-07-21T22:17:39.642 INFO:teuthology.orchestra.run.smithi132.stdout:(1/131): ceph-18.0.0-4735.gbffe049d.el8.x86_64. 16 kB/s | 6.4 kB 00:00 2023-07-21T22:17:39.945 INFO:teuthology.orchestra.run.smithi107.stdout:(4/131): ceph-mds-18.0.0-4735.gbffe049d.el8.x86 3.2 MB/s | 2.3 MB 00:00 2023-07-21T22:17:40.303 INFO:teuthology.orchestra.run.smithi107.stdout:(5/131): ceph-base-18.0.0-4735.gbffe049d.el8.x8 2.4 MB/s | 5.4 MB 00:02 2023-07-21T22:17:40.471 INFO:teuthology.orchestra.run.smithi107.stdout:(6/131): ceph-mgr-18.0.0-4735.gbffe049d.el8.x86 2.9 MB/s | 1.5 MB 00:00 2023-07-21T22:17:40.642 INFO:teuthology.orchestra.run.smithi132.stdout:(2/131): ceph-fuse-18.0.0-4735.gbffe049d.el8.x8 848 kB/s | 848 kB 00:01 2023-07-21T22:17:40.868 INFO:teuthology.orchestra.run.smithi132.stdout:(3/131): ceph-immutable-object-cache-18.0.0-473 736 kB/s | 165 kB 00:00 2023-07-21T22:17:41.221 INFO:teuthology.orchestra.run.smithi107.stdout:(7/131): ceph-mon-18.0.0-4735.gbffe049d.el8.x86 5.0 MB/s | 4.6 MB 00:00 2023-07-21T22:17:42.376 INFO:teuthology.orchestra.run.smithi132.stdout:(4/131): ceph-mds-18.0.0-4735.gbffe049d.el8.x86 1.5 MB/s | 2.3 MB 00:01 2023-07-21T22:17:42.671 INFO:teuthology.orchestra.run.smithi107.stdout:(8/131): ceph-common-18.0.0-4735.gbffe049d.el8. 4.7 MB/s | 22 MB 00:04 2023-07-21T22:17:42.793 INFO:teuthology.orchestra.run.smithi132.stdout:(5/131): ceph-base-18.0.0-4735.gbffe049d.el8.x8 1.5 MB/s | 5.4 MB 00:03 2023-07-21T22:17:42.821 INFO:teuthology.orchestra.run.smithi107.stdout:(9/131): ceph-selinux-18.0.0-4735.gbffe049d.el8 162 kB/s | 24 kB 00:00 2023-07-21T22:17:43.243 INFO:teuthology.orchestra.run.smithi132.stdout:(6/131): ceph-mgr-18.0.0-4735.gbffe049d.el8.x86 1.8 MB/s | 1.5 MB 00:00 2023-07-21T22:17:43.305 INFO:teuthology.orchestra.run.smithi107.stdout:(10/131): ceph-radosgw-18.0.0-4735.gbffe049d.el 6.5 MB/s | 13 MB 00:02 2023-07-21T22:17:43.499 INFO:teuthology.orchestra.run.smithi107.stdout:(11/131): ceph-osd-18.0.0-4735.gbffe049d.el8.x8 5.8 MB/s | 18 MB 00:03 2023-07-21T22:17:43.530 INFO:teuthology.orchestra.run.smithi107.stdout:(12/131): cephfs-mirror-18.0.0-4735.gbffe049d.e 1.1 MB/s | 258 kB 00:00 2023-07-21T22:17:43.631 INFO:teuthology.orchestra.run.smithi107.stdout:(13/131): libcephfs-devel-18.0.0-4735.gbffe049d 237 kB/s | 31 kB 00:00 2023-07-21T22:17:43.780 INFO:teuthology.orchestra.run.smithi107.stdout:(14/131): libcephsqlite-18.0.0-4735.gbffe049d.e 916 kB/s | 137 kB 00:00 2023-07-21T22:17:43.914 INFO:teuthology.orchestra.run.smithi107.stdout:(15/131): libcephfs2-18.0.0-4735.gbffe049d.el8. 2.0 MB/s | 771 kB 00:00 2023-07-21T22:17:43.939 INFO:teuthology.orchestra.run.smithi107.stdout:(16/131): librados-devel-18.0.0-4735.gbffe049d. 876 kB/s | 139 kB 00:00 2023-07-21T22:17:44.165 INFO:teuthology.orchestra.run.smithi107.stdout:(17/131): libradosstriper1-18.0.0-4735.gbffe049 1.8 MB/s | 473 kB 00:00 2023-07-21T22:17:44.277 INFO:teuthology.orchestra.run.smithi132.stdout:(7/131): ceph-common-18.0.0-4735.gbffe049d.el8. 4.4 MB/s | 22 MB 00:05 2023-07-21T22:17:44.315 INFO:teuthology.orchestra.run.smithi107.stdout:(18/131): python3-ceph-argparse-18.0.0-4735.gbf 298 kB/s | 45 kB 00:00 2023-07-21T22:17:44.344 INFO:teuthology.orchestra.run.smithi132.stdout:(8/131): ceph-mon-18.0.0-4735.gbffe049d.el8.x86 3.0 MB/s | 4.6 MB 00:01 2023-07-21T22:17:44.473 INFO:teuthology.orchestra.run.smithi107.stdout:(19/131): python3-ceph-common-18.0.0-4735.gbffe 751 kB/s | 119 kB 00:00 2023-07-21T22:17:44.485 INFO:teuthology.orchestra.run.smithi132.stdout:(9/131): ceph-selinux-18.0.0-4735.gbffe049d.el8 172 kB/s | 24 kB 00:00 2023-07-21T22:17:44.632 INFO:teuthology.orchestra.run.smithi107.stdout:(20/131): python3-cephfs-18.0.0-4735.gbffe049d. 1.2 MB/s | 197 kB 00:00 2023-07-21T22:17:44.790 INFO:teuthology.orchestra.run.smithi107.stdout:(21/131): python3-rados-18.0.0-4735.gbffe049d.e 2.3 MB/s | 370 kB 00:00 2023-07-21T22:17:44.966 INFO:teuthology.orchestra.run.smithi107.stdout:(22/131): python3-rbd-18.0.0-4735.gbffe049d.el8 2.0 MB/s | 364 kB 00:00 2023-07-21T22:17:45.175 INFO:teuthology.orchestra.run.smithi107.stdout:(23/131): librgw2-18.0.0-4735.gbffe049d.el8.x86 5.5 MB/s | 6.7 MB 00:01 2023-07-21T22:17:45.192 INFO:teuthology.orchestra.run.smithi107.stdout:(24/131): python3-rgw-18.0.0-4735.gbffe049d.el8 472 kB/s | 107 kB 00:00 2023-07-21T22:17:45.392 INFO:teuthology.orchestra.run.smithi107.stdout:(25/131): rbd-fuse-18.0.0-4735.gbffe049d.el8.x8 435 kB/s | 94 kB 00:00 2023-07-21T22:17:45.576 INFO:teuthology.orchestra.run.smithi107.stdout:(26/131): rbd-nbd-18.0.0-4735.gbffe049d.el8.x86 980 kB/s | 179 kB 00:00 2023-07-21T22:17:45.734 INFO:teuthology.orchestra.run.smithi107.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4735.g 146 kB/s | 23 kB 00:00 2023-07-21T22:17:45.909 INFO:teuthology.orchestra.run.smithi107.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4735.gbffe049 737 kB/s | 129 kB 00:00 2023-07-21T22:17:46.452 INFO:teuthology.orchestra.run.smithi132.stdout:(10/131): ceph-radosgw-18.0.0-4735.gbffe049d.el 6.2 MB/s | 13 MB 00:02 2023-07-21T22:17:46.510 INFO:teuthology.orchestra.run.smithi107.stdout:(29/131): rbd-mirror-18.0.0-4735.gbffe049d.el8. 2.6 MB/s | 3.4 MB 00:01 2023-07-21T22:17:46.551 INFO:teuthology.orchestra.run.smithi107.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4735.gbffe0 2.7 MB/s | 1.7 MB 00:00 2023-07-21T22:17:46.627 INFO:teuthology.orchestra.run.smithi132.stdout:(11/131): cephfs-mirror-18.0.0-4735.gbffe049d.e 1.4 MB/s | 258 kB 00:00 2023-07-21T22:17:46.727 INFO:teuthology.orchestra.run.smithi107.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4735.gbf 1.3 MB/s | 240 kB 00:00 2023-07-21T22:17:46.885 INFO:teuthology.orchestra.run.smithi107.stdout:(32/131): ceph-mgr-rook-18.0.0-4735.gbffe049d.e 299 kB/s | 47 kB 00:00 2023-07-21T22:17:46.903 INFO:teuthology.orchestra.run.smithi132.stdout:(12/131): ceph-osd-18.0.0-4735.gbffe049d.el8.x8 4.8 MB/s | 18 MB 00:03 2023-07-21T22:17:46.928 INFO:teuthology.orchestra.run.smithi132.stdout:(13/131): libcephfs-devel-18.0.0-4735.gbffe049d 103 kB/s | 31 kB 00:00 2023-07-21T22:17:47.027 INFO:teuthology.orchestra.run.smithi107.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4735.gb 103 kB/s | 15 kB 00:00 2023-07-21T22:17:47.103 INFO:teuthology.orchestra.run.smithi132.stdout:(14/131): libcephfs2-18.0.0-4735.gbffe049d.el8. 3.8 MB/s | 771 kB 00:00 2023-07-21T22:17:47.128 INFO:teuthology.orchestra.run.smithi132.stdout:(15/131): libcephsqlite-18.0.0-4735.gbffe049d.e 685 kB/s | 137 kB 00:00 2023-07-21T22:17:47.170 INFO:teuthology.orchestra.run.smithi107.stdout:(34/131): ceph-volume-18.0.0-4735.gbffe049d.el8 1.8 MB/s | 258 kB 00:00 2023-07-21T22:17:47.229 INFO:teuthology.orchestra.run.smithi132.stdout:(16/131): librados-devel-18.0.0-4735.gbffe049d. 1.1 MB/s | 139 kB 00:00 2023-07-21T22:17:47.263 INFO:teuthology.orchestra.run.smithi132.stdout:(17/131): libradosstriper1-18.0.0-4735.gbffe049 3.4 MB/s | 473 kB 00:00 2023-07-21T22:17:47.303 INFO:teuthology.orchestra.run.smithi107.stdout:(35/131): cephadm-18.0.0-4735.gbffe049d.el8.noa 1.2 MB/s | 164 kB 00:00 2023-07-21T22:17:47.405 INFO:teuthology.orchestra.run.smithi132.stdout:(18/131): python3-ceph-argparse-18.0.0-4735.gbf 316 kB/s | 45 kB 00:00 2023-07-21T22:17:47.436 INFO:teuthology.orchestra.run.smithi107.stdout:(36/131): cephfs-top-18.0.0-4735.gbffe049d.el8. 135 kB/s | 18 kB 00:00 2023-07-21T22:17:47.538 INFO:teuthology.orchestra.run.smithi132.stdout:(19/131): python3-ceph-common-18.0.0-4735.gbffe 891 kB/s | 119 kB 00:00 2023-07-21T22:17:47.697 INFO:teuthology.orchestra.run.smithi132.stdout:(20/131): python3-cephfs-18.0.0-4735.gbffe049d. 1.2 MB/s | 197 kB 00:00 2023-07-21T22:17:47.801 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:17:47.802 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:17:47.815 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T22:17:47.816 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T22:17:47.817 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T22:17:47.819 INFO:teuthology.orchestra.run.smithi190.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T22:17:47.821 INFO:teuthology.orchestra.run.smithi190.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T22:17:47.824 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T22:17:47.825 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T22:17:47.864 INFO:teuthology.orchestra.run.smithi132.stdout:(21/131): python3-rados-18.0.0-4735.gbffe049d.e 2.2 MB/s | 370 kB 00:00 2023-07-21T22:17:47.955 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2023-07-21T22:17:47.964 INFO:teuthology.orchestra.run.smithi132.stdout:(22/131): librgw2-18.0.0-4735.gbffe049d.el8.x86 9.2 MB/s | 6.7 MB 00:00 2023-07-21T22:17:47.967 INFO:teuthology.orchestra.run.smithi190.stdout:=========================================================================================================================== 2023-07-21T22:17:47.967 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2023-07-21T22:17:47.967 INFO:teuthology.orchestra.run.smithi190.stdout:=========================================================================================================================== 2023-07-21T22:17:47.967 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2023-07-21T22:17:47.967 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T22:17:47.968 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T22:17:47.968 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.4 k 2023-07-21T22:17:47.968 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 5.4 M 2023-07-21T22:17:47.968 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 848 k 2023-07-21T22:17:47.968 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 165 k 2023-07-21T22:17:47.968 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 1.5 M 2023-07-21T22:17:47.968 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 129 k 2023-07-21T22:17:47.969 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 1.7 M 2023-07-21T22:17:47.969 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 7.4 M 2023-07-21T22:17:47.969 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 47 k 2023-07-21T22:17:47.969 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 13 M 2023-07-21T22:17:47.969 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 48 M 2023-07-21T22:17:47.969 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 258 k 2023-07-21T22:17:47.969 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 164 k 2023-07-21T22:17:47.970 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 258 k 2023-07-21T22:17:47.970 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 18 k 2023-07-21T22:17:47.970 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T22:17:47.970 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T22:17:47.970 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T22:17:47.970 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 31 k 2023-07-21T22:17:47.970 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 771 k 2023-07-21T22:17:47.971 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 139 k 2023-07-21T22:17:47.971 INFO:teuthology.orchestra.run.smithi190.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T22:17:47.971 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T22:17:47.971 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T22:17:47.971 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T22:17:47.971 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T22:17:47.972 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T22:17:47.972 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 197 k 2023-07-21T22:17:47.972 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 370 k 2023-07-21T22:17:47.972 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 364 k 2023-07-21T22:17:47.973 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 107 k 2023-07-21T22:17:47.973 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 94 k 2023-07-21T22:17:47.973 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.4 M 2023-07-21T22:17:47.973 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 179 k 2023-07-21T22:17:47.973 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T22:17:47.973 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2023-07-21T22:17:47.973 INFO:teuthology.orchestra.run.smithi190.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T22:17:47.974 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.7 M 2023-07-21T22:17:47.974 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.6 M 2023-07-21T22:17:47.974 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2023-07-21T22:17:47.974 INFO:teuthology.orchestra.run.smithi190.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T22:17:47.974 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 22 M 2023-07-21T22:17:47.974 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 23 k 2023-07-21T22:17:47.974 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 2.3 M 2023-07-21T22:17:47.974 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 240 k 2023-07-21T22:17:47.975 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 4.6 M 2023-07-21T22:17:47.975 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 18 M 2023-07-21T22:17:47.975 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 15 k 2023-07-21T22:17:47.975 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 24 k 2023-07-21T22:17:47.975 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T22:17:47.975 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T22:17:47.975 INFO:teuthology.orchestra.run.smithi190.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T22:17:47.976 INFO:teuthology.orchestra.run.smithi190.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T22:17:47.976 INFO:teuthology.orchestra.run.smithi190.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T22:17:47.976 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 137 k 2023-07-21T22:17:47.976 INFO:teuthology.orchestra.run.smithi190.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T22:17:47.976 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T22:17:47.976 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T22:17:47.976 INFO:teuthology.orchestra.run.smithi190.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T22:17:47.976 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T22:17:47.977 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T22:17:47.977 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 473 k 2023-07-21T22:17:47.977 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T22:17:47.977 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.7 M 2023-07-21T22:17:47.977 INFO:teuthology.orchestra.run.smithi190.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T22:17:47.977 INFO:teuthology.orchestra.run.smithi190.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T22:17:47.977 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T22:17:47.977 INFO:teuthology.orchestra.run.smithi190.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T22:17:47.978 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T22:17:47.978 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T22:17:47.978 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T22:17:47.978 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T22:17:47.978 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T22:17:47.978 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T22:17:47.978 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T22:17:47.979 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T22:17:47.979 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T22:17:47.979 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T22:17:47.979 INFO:teuthology.orchestra.run.smithi190.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T22:17:47.979 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T22:17:47.979 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 45 k 2023-07-21T22:17:47.979 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 119 k 2023-07-21T22:17:47.980 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T22:17:47.980 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T22:17:47.980 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T22:17:47.980 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T22:17:47.980 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T22:17:47.980 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T22:17:47.980 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T22:17:47.980 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T22:17:47.981 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T22:17:47.981 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T22:17:47.981 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T22:17:47.981 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:17:47.981 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T22:17:47.981 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T22:17:47.981 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T22:17:47.982 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T22:17:47.982 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T22:17:47.982 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T22:17:47.982 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T22:17:47.982 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T22:17:47.982 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T22:17:47.982 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T22:17:47.983 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T22:17:47.983 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T22:17:47.983 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T22:17:47.983 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T22:17:47.983 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T22:17:47.983 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T22:17:47.984 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T22:17:47.984 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T22:17:47.984 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:17:47.984 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T22:17:47.984 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T22:17:47.984 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T22:17:47.984 INFO:teuthology.orchestra.run.smithi190.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T22:17:47.984 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T22:17:47.985 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T22:17:47.985 INFO:teuthology.orchestra.run.smithi190.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T22:17:47.985 INFO:teuthology.orchestra.run.smithi190.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T22:17:47.985 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T22:17:47.985 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T22:17:47.985 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T22:17:47.985 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T22:17:47.986 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T22:17:47.986 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T22:17:47.986 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T22:17:47.986 INFO:teuthology.orchestra.run.smithi190.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T22:17:47.986 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T22:17:47.986 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T22:17:47.986 INFO:teuthology.orchestra.run.smithi190.stdout:Enabling module streams: 2023-07-21T22:17:47.986 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql 10 2023-07-21T22:17:47.987 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:17:47.987 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2023-07-21T22:17:47.987 INFO:teuthology.orchestra.run.smithi190.stdout:=========================================================================================================================== 2023-07-21T22:17:47.987 INFO:teuthology.orchestra.run.smithi190.stdout:Install 128 Packages 2023-07-21T22:17:47.987 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 3 Packages 2023-07-21T22:17:47.987 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:17:47.988 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 205 M 2023-07-21T22:17:47.988 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2023-07-21T22:17:48.006 INFO:teuthology.orchestra.run.smithi132.stdout:(23/131): python3-rbd-18.0.0-4735.gbffe049d.el8 2.5 MB/s | 364 kB 00:00 2023-07-21T22:17:48.090 INFO:teuthology.orchestra.run.smithi132.stdout:(24/131): python3-rgw-18.0.0-4735.gbffe049d.el8 852 kB/s | 107 kB 00:00 2023-07-21T22:17:48.132 INFO:teuthology.orchestra.run.smithi132.stdout:(25/131): rbd-fuse-18.0.0-4735.gbffe049d.el8.x8 754 kB/s | 94 kB 00:00 2023-07-21T22:17:48.307 INFO:teuthology.orchestra.run.smithi132.stdout:(26/131): rbd-nbd-18.0.0-4735.gbffe049d.el8.x86 1.0 MB/s | 179 kB 00:00 2023-07-21T22:17:48.449 INFO:teuthology.orchestra.run.smithi132.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4735.g 163 kB/s | 23 kB 00:00 2023-07-21T22:17:48.508 INFO:teuthology.orchestra.run.smithi132.stdout:(28/131): rbd-mirror-18.0.0-4735.gbffe049d.el8. 8.3 MB/s | 3.4 MB 00:00 2023-07-21T22:17:48.608 INFO:teuthology.orchestra.run.smithi132.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4735.gbffe049 812 kB/s | 129 kB 00:00 2023-07-21T22:17:48.825 INFO:teuthology.orchestra.run.smithi132.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4735.gbffe0 5.4 MB/s | 1.7 MB 00:00 2023-07-21T22:17:48.853 INFO:teuthology.orchestra.run.smithi107.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 216 kB/s | 306 kB 00:01 2023-07-21T22:17:48.853 INFO:teuthology.orchestra.run.smithi190.stdout:(1/131): ceph-18.0.0-4735.gbffe049d.el8.x86_64. 12 kB/s | 6.4 kB 00:00 2023-07-21T22:17:48.945 INFO:teuthology.orchestra.run.smithi107.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 3.0 MB/s | 7.4 MB 00:02 2023-07-21T22:17:48.967 INFO:teuthology.orchestra.run.smithi132.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4735.gbf 1.7 MB/s | 240 kB 00:00 2023-07-21T22:17:49.045 INFO:teuthology.orchestra.run.smithi107.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 306 kB/s | 59 kB 00:00 2023-07-21T22:17:49.101 INFO:teuthology.orchestra.run.smithi132.stdout:(32/131): ceph-mgr-rook-18.0.0-4735.gbffe049d.e 355 kB/s | 47 kB 00:00 2023-07-21T22:17:49.251 INFO:teuthology.orchestra.run.smithi132.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4735.gb 97 kB/s | 15 kB 00:00 2023-07-21T22:17:49.476 INFO:teuthology.orchestra.run.smithi132.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 8.5 MB/s | 7.4 MB 00:00 2023-07-21T22:17:49.493 INFO:teuthology.orchestra.run.smithi132.stdout:(35/131): ceph-volume-18.0.0-4735.gbffe049d.el8 1.0 MB/s | 258 kB 00:00 2023-07-21T22:17:49.618 INFO:teuthology.orchestra.run.smithi132.stdout:(36/131): cephadm-18.0.0-4735.gbffe049d.el8.noa 1.1 MB/s | 164 kB 00:00 2023-07-21T22:17:49.643 INFO:teuthology.orchestra.run.smithi132.stdout:(37/131): cephfs-top-18.0.0-4735.gbffe049d.el8. 120 kB/s | 18 kB 00:00 2023-07-21T22:17:49.712 INFO:teuthology.orchestra.run.smithi190.stdout:(2/131): ceph-fuse-18.0.0-4735.gbffe049d.el8.x8 988 kB/s | 848 kB 00:00 2023-07-21T22:17:49.870 INFO:teuthology.orchestra.run.smithi190.stdout:(3/131): ceph-immutable-object-cache-18.0.0-473 1.0 MB/s | 165 kB 00:00 2023-07-21T22:17:49.979 INFO:teuthology.orchestra.run.smithi107.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 73 kB/s | 75 kB 00:01 2023-07-21T22:17:50.202 INFO:teuthology.orchestra.run.smithi132.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 105 kB/s | 59 kB 00:00 2023-07-21T22:17:50.204 INFO:teuthology.orchestra.run.smithi107.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 194 kB/s | 44 kB 00:00 2023-07-21T22:17:50.344 INFO:teuthology.orchestra.run.smithi132.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 531 kB/s | 75 kB 00:00 2023-07-21T22:17:50.429 INFO:teuthology.orchestra.run.smithi132.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 378 kB/s | 306 kB 00:00 2023-07-21T22:17:50.529 INFO:teuthology.orchestra.run.smithi107.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 321 kB/s | 476 kB 00:01 2023-07-21T22:17:50.553 INFO:teuthology.orchestra.run.smithi132.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 349 kB/s | 44 kB 00:00 2023-07-21T22:17:50.737 INFO:teuthology.orchestra.run.smithi132.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2023-07-21T22:17:50.762 INFO:teuthology.orchestra.run.smithi132.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 887 kB/s | 185 kB 00:00 2023-07-21T22:17:50.763 INFO:teuthology.orchestra.run.smithi107.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 141 kB/s | 33 kB 00:00 2023-07-21T22:17:50.780 INFO:teuthology.orchestra.run.smithi107.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 322 kB/s | 185 kB 00:00 2023-07-21T22:17:50.788 INFO:teuthology.orchestra.run.smithi190.stdout:(4/131): ceph-mds-18.0.0-4735.gbffe049d.el8.x86 2.5 MB/s | 2.3 MB 00:00 2023-07-21T22:17:50.838 INFO:teuthology.orchestra.run.smithi132.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 328 kB/s | 33 kB 00:00 2023-07-21T22:17:50.871 INFO:teuthology.orchestra.run.smithi132.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-07-21T22:17:50.947 INFO:teuthology.orchestra.run.smithi107.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 89 kB/s | 16 kB 00:00 2023-07-21T22:17:50.963 INFO:teuthology.orchestra.run.smithi132.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-07-21T22:17:51.097 INFO:teuthology.orchestra.run.smithi132.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 609 kB/s | 81 kB 00:00 2023-07-21T22:17:51.172 INFO:teuthology.orchestra.run.smithi132.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.2 MB/s | 384 kB 00:00 2023-07-21T22:17:51.214 INFO:teuthology.orchestra.run.smithi132.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-07-21T22:17:51.246 INFO:teuthology.orchestra.run.smithi190.stdout:(5/131): ceph-base-18.0.0-4735.gbffe049d.el8.x8 1.8 MB/s | 5.4 MB 00:02 2023-07-21T22:17:51.288 INFO:teuthology.orchestra.run.smithi190.stdout:(6/131): ceph-mgr-18.0.0-4735.gbffe049d.el8.x86 3.1 MB/s | 1.5 MB 00:00 2023-07-21T22:17:51.297 INFO:teuthology.orchestra.run.smithi107.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 334 kB/s | 173 kB 00:00 2023-07-21T22:17:51.298 INFO:teuthology.orchestra.run.smithi132.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2023-07-21T22:17:51.423 INFO:teuthology.orchestra.run.smithi132.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 389 kB/s | 49 kB 00:00 2023-07-21T22:17:51.548 INFO:teuthology.orchestra.run.smithi132.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 470 kB/s | 59 kB 00:00 2023-07-21T22:17:51.672 INFO:teuthology.orchestra.run.smithi107.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 217 kB/s | 81 kB 00:00 2023-07-21T22:17:51.682 INFO:teuthology.orchestra.run.smithi132.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 452 kB/s | 60 kB 00:00 2023-07-21T22:17:51.857 INFO:teuthology.orchestra.run.smithi132.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 1.8 MB/s | 1.2 MB 00:00 2023-07-21T22:17:51.933 INFO:teuthology.orchestra.run.smithi132.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2023-07-21T22:17:51.991 INFO:teuthology.orchestra.run.smithi132.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 116 kB/s | 16 kB 00:00 2023-07-21T22:17:52.075 INFO:teuthology.orchestra.run.smithi132.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 237 kB/s | 34 kB 00:00 2023-07-21T22:17:52.114 INFO:teuthology.orchestra.run.smithi107.stdout:(48/131): ceph-test-18.0.0-4735.gbffe049d.el8.x 5.1 MB/s | 48 MB 00:09 2023-07-21T22:17:52.133 INFO:teuthology.orchestra.run.smithi132.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-07-21T22:17:52.139 INFO:teuthology.orchestra.run.smithi107.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 24 kB/s | 11 kB 00:00 2023-07-21T22:17:52.298 INFO:teuthology.orchestra.run.smithi107.stdout:(50/131): python3-cherrypy-18.4.0-1.el8.noarch. 285 kB/s | 384 kB 00:01 2023-07-21T22:17:52.392 INFO:teuthology.orchestra.run.smithi132.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 194 kB/s | 61 kB 00:00 2023-07-21T22:17:52.418 INFO:teuthology.orchestra.run.smithi132.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 66 kB/s | 19 kB 00:00 2023-07-21T22:17:52.472 INFO:teuthology.orchestra.run.smithi190.stdout:(7/131): ceph-mon-18.0.0-4735.gbffe049d.el8.x86 3.8 MB/s | 4.6 MB 00:01 2023-07-21T22:17:52.498 INFO:teuthology.orchestra.run.smithi107.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 249 kB/s | 49 kB 00:00 2023-07-21T22:17:52.518 INFO:teuthology.orchestra.run.smithi132.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 191 kB/s | 24 kB 00:00 2023-07-21T22:17:52.543 INFO:teuthology.orchestra.run.smithi132.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 233 kB/s | 29 kB 00:00 2023-07-21T22:17:52.644 INFO:teuthology.orchestra.run.smithi132.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 220 kB/s | 27 kB 00:00 2023-07-21T22:17:52.698 INFO:teuthology.orchestra.run.smithi107.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 295 kB/s | 59 kB 00:00 2023-07-21T22:17:52.777 INFO:teuthology.orchestra.run.smithi132.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.1 MB/s | 255 kB 00:00 2023-07-21T22:17:52.861 INFO:teuthology.orchestra.run.smithi132.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.1 MB/s | 251 kB 00:00 2023-07-21T22:17:52.886 INFO:teuthology.orchestra.run.smithi132.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 564 kB/s | 61 kB 00:00 2023-07-21T22:17:52.899 INFO:teuthology.orchestra.run.smithi107.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 302 kB/s | 60 kB 00:00 2023-07-21T22:17:52.970 INFO:teuthology.orchestra.run.smithi132.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 786 kB/s | 85 kB 00:00 2023-07-21T22:17:52.995 INFO:teuthology.orchestra.run.smithi132.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 211 kB/s | 23 kB 00:00 2023-07-21T22:17:53.014 INFO:teuthology.orchestra.run.smithi132.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 3.6 MB/s | 70 kB 00:00 2023-07-21T22:17:53.066 INFO:teuthology.orchestra.run.smithi107.stdout:(54/131): python3-jaraco-functools-2.0-4.el8.no 18 kB/s | 18 kB 00:00 2023-07-21T22:17:53.266 INFO:teuthology.orchestra.run.smithi107.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 78 kB/s | 16 kB 00:00 2023-07-21T22:17:53.314 INFO:teuthology.orchestra.run.smithi132.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 46 MB/s | 14 MB 00:00 2023-07-21T22:17:53.356 INFO:teuthology.orchestra.run.smithi132.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-21T22:17:53.398 INFO:teuthology.orchestra.run.smithi132.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-21T22:17:53.441 INFO:teuthology.orchestra.run.smithi132.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.0 MB/s | 259 kB 00:00 2023-07-21T22:17:53.482 INFO:teuthology.orchestra.run.smithi132.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-07-21T22:17:53.524 INFO:teuthology.orchestra.run.smithi132.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 931 kB/s | 39 kB 00:00 2023-07-21T22:17:53.608 INFO:teuthology.orchestra.run.smithi132.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 8.1 MB/s | 688 kB 00:00 2023-07-21T22:17:53.613 INFO:teuthology.orchestra.run.smithi190.stdout:(8/131): ceph-common-18.0.0-4735.gbffe049d.el8. 4.2 MB/s | 22 MB 00:05 2023-07-21T22:17:53.641 INFO:teuthology.orchestra.run.smithi107.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 90 kB/s | 34 kB 00:00 2023-07-21T22:17:53.650 INFO:teuthology.orchestra.run.smithi132.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 13 MB/s | 538 kB 00:00 2023-07-21T22:17:53.692 INFO:teuthology.orchestra.run.smithi132.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-21T22:17:53.734 INFO:teuthology.orchestra.run.smithi132.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-07-21T22:17:53.780 INFO:teuthology.orchestra.run.smithi190.stdout:(9/131): ceph-selinux-18.0.0-4735.gbffe049d.el8 146 kB/s | 24 kB 00:00 2023-07-21T22:17:53.783 INFO:teuthology.orchestra.run.smithi107.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 320 kB/s | 283 kB 00:00 2023-07-21T22:17:53.784 INFO:teuthology.orchestra.run.smithi132.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-07-21T22:17:53.868 INFO:teuthology.orchestra.run.smithi132.stdout:(81/131): thrift-0.13.0-2.el8.x86_64.rpm 1.9 MB/s | 1.7 MB 00:00 2023-07-21T22:17:53.952 INFO:teuthology.orchestra.run.smithi132.stdout:(82/131): python3-babel-2.5.1-7.el8.noarch.rpm 29 MB/s | 4.8 MB 00:00 2023-07-21T22:17:53.977 INFO:teuthology.orchestra.run.smithi132.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 245 kB/s | 27 kB 00:00 2023-07-21T22:17:54.000 INFO:teuthology.orchestra.run.smithi107.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 284 kB/s | 61 kB 00:00 2023-07-21T22:17:54.002 INFO:teuthology.orchestra.run.smithi132.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 6.2 MB/s | 320 kB 00:00 2023-07-21T22:17:54.027 INFO:teuthology.orchestra.run.smithi132.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 870 kB/s | 44 kB 00:00 2023-07-21T22:17:54.078 INFO:teuthology.orchestra.run.smithi132.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-07-21T22:17:54.153 INFO:teuthology.orchestra.run.smithi132.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 25 MB/s | 3.7 MB 00:00 2023-07-21T22:17:54.178 INFO:teuthology.orchestra.run.smithi132.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-07-21T22:17:54.203 INFO:teuthology.orchestra.run.smithi132.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-07-21T22:17:54.220 INFO:teuthology.orchestra.run.smithi132.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.0 MB/s | 126 kB 00:00 2023-07-21T22:17:54.233 INFO:teuthology.orchestra.run.smithi107.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 332 kB/s | 196 kB 00:00 2023-07-21T22:17:54.254 INFO:teuthology.orchestra.run.smithi132.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 396 kB/s | 20 kB 00:00 2023-07-21T22:17:54.279 INFO:teuthology.orchestra.run.smithi132.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.7 MB/s | 103 kB 00:00 2023-07-21T22:17:54.297 INFO:teuthology.orchestra.run.smithi190.stdout:(10/131): ceph-osd-18.0.0-4735.gbffe049d.el8.x8 5.9 MB/s | 18 MB 00:03 2023-07-21T22:17:54.313 INFO:teuthology.orchestra.run.smithi132.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-21T22:17:54.355 INFO:teuthology.orchestra.run.smithi132.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-21T22:17:54.400 INFO:teuthology.orchestra.run.smithi107.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 143 kB/s | 24 kB 00:00 2023-07-21T22:17:54.431 INFO:teuthology.orchestra.run.smithi190.stdout:(11/131): cephfs-mirror-18.0.0-4735.gbffe049d.e 1.9 MB/s | 258 kB 00:00 2023-07-21T22:17:54.456 INFO:teuthology.orchestra.run.smithi132.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 28 MB/s | 4.9 MB 00:00 2023-07-21T22:17:54.498 INFO:teuthology.orchestra.run.smithi132.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-07-21T22:17:54.564 INFO:teuthology.orchestra.run.smithi190.stdout:(12/131): ceph-radosgw-18.0.0-4735.gbffe049d.el 6.4 MB/s | 13 MB 00:02 2023-07-21T22:17:54.590 INFO:teuthology.orchestra.run.smithi132.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-07-21T22:17:54.590 INFO:teuthology.orchestra.run.smithi190.stdout:(13/131): libcephfs-devel-18.0.0-4735.gbffe049d 196 kB/s | 31 kB 00:00 2023-07-21T22:17:54.592 INFO:teuthology.orchestra.run.smithi107.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 152 kB/s | 29 kB 00:00 2023-07-21T22:17:54.615 INFO:teuthology.orchestra.run.smithi132.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2023-07-21T22:17:54.665 INFO:teuthology.orchestra.run.smithi132.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 16 MB/s | 808 kB 00:00 2023-07-21T22:17:54.740 INFO:teuthology.orchestra.run.smithi190.stdout:(14/131): libcephfs2-18.0.0-4735.gbffe049d.el8. 4.3 MB/s | 771 kB 00:00 2023-07-21T22:17:54.749 INFO:teuthology.orchestra.run.smithi132.stdout:(100/131): postgresql-server-10.21-2.module+el8 32 MB/s | 5.1 MB 00:00 2023-07-21T22:17:54.765 INFO:teuthology.orchestra.run.smithi190.stdout:(15/131): libcephsqlite-18.0.0-4735.gbffe049d.e 784 kB/s | 137 kB 00:00 2023-07-21T22:17:54.784 INFO:teuthology.orchestra.run.smithi107.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 143 kB/s | 27 kB 00:00 2023-07-21T22:17:54.890 INFO:teuthology.orchestra.run.smithi190.stdout:(16/131): librados-devel-18.0.0-4735.gbffe049d. 924 kB/s | 139 kB 00:00 2023-07-21T22:17:54.924 INFO:teuthology.orchestra.run.smithi190.stdout:(17/131): libradosstriper1-18.0.0-4735.gbffe049 2.9 MB/s | 473 kB 00:00 2023-07-21T22:17:55.099 INFO:teuthology.orchestra.run.smithi190.stdout:(18/131): python3-ceph-argparse-18.0.0-4735.gbf 255 kB/s | 45 kB 00:00 2023-07-21T22:17:55.166 INFO:teuthology.orchestra.run.smithi132.stdout:(101/131): ceph-test-18.0.0-4735.gbffe049d.el8. 4.5 MB/s | 48 MB 00:10 2023-07-21T22:17:55.183 INFO:teuthology.orchestra.run.smithi132.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 108 kB/s | 47 kB 00:00 2023-07-21T22:17:55.224 INFO:teuthology.orchestra.run.smithi190.stdout:(19/131): python3-ceph-common-18.0.0-4735.gbffe 950 kB/s | 119 kB 00:00 2023-07-21T22:17:55.234 INFO:teuthology.orchestra.run.smithi107.stdout:(63/131): python3-simplegeneric-0.8.1-17.el8.no 15 kB/s | 19 kB 00:01 2023-07-21T22:17:55.258 INFO:teuthology.orchestra.run.smithi132.stdout:(103/131): postgresql-10.21-2.module+el8.6.0+15 2.5 MB/s | 1.5 MB 00:00 2023-07-21T22:17:55.275 INFO:teuthology.orchestra.run.smithi132.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 492 kB/s | 43 kB 00:00 2023-07-21T22:17:55.300 INFO:teuthology.orchestra.run.smithi132.stdout:(105/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.5 MB/s | 200 kB 00:00 2023-07-21T22:17:55.326 INFO:teuthology.orchestra.run.smithi132.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 958 kB/s | 48 kB 00:00 2023-07-21T22:17:55.351 INFO:teuthology.orchestra.run.smithi132.stdout:(107/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-07-21T22:17:55.374 INFO:teuthology.orchestra.run.smithi190.stdout:(20/131): python3-cephfs-18.0.0-4735.gbffe049d. 1.3 MB/s | 197 kB 00:00 2023-07-21T22:17:55.376 INFO:teuthology.orchestra.run.smithi132.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 851 kB/s | 43 kB 00:00 2023-07-21T22:17:55.402 INFO:teuthology.orchestra.run.smithi132.stdout:(109/131): libsepol-devel-2.9-3.el8.x86_64.rpm 862 kB/s | 87 kB 00:00 2023-07-21T22:17:55.427 INFO:teuthology.orchestra.run.smithi132.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 1.4 MB/s | 109 kB 00:00 2023-07-21T22:17:55.452 INFO:teuthology.orchestra.run.smithi132.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-07-21T22:17:55.477 INFO:teuthology.orchestra.run.smithi132.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 371 kB/s | 19 kB 00:00 2023-07-21T22:17:55.501 INFO:teuthology.orchestra.run.smithi107.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 356 kB/s | 255 kB 00:00 2023-07-21T22:17:55.502 INFO:teuthology.orchestra.run.smithi132.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-07-21T22:17:55.536 INFO:teuthology.orchestra.run.smithi132.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-07-21T22:17:55.550 INFO:teuthology.orchestra.run.smithi190.stdout:(21/131): python3-rados-18.0.0-4735.gbffe049d.e 2.1 MB/s | 370 kB 00:00 2023-07-21T22:17:55.562 INFO:teuthology.orchestra.run.smithi132.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 305 kB/s | 18 kB 00:00 2023-07-21T22:17:55.595 INFO:teuthology.orchestra.run.smithi132.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.7 MB/s | 559 kB 00:00 2023-07-21T22:17:55.612 INFO:teuthology.orchestra.run.smithi132.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.6 MB/s | 238 kB 00:00 2023-07-21T22:17:55.646 INFO:teuthology.orchestra.run.smithi132.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-07-21T22:17:55.671 INFO:teuthology.orchestra.run.smithi132.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-07-21T22:17:55.696 INFO:teuthology.orchestra.run.smithi132.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-07-21T22:17:55.718 INFO:teuthology.orchestra.run.smithi107.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 283 kB/s | 61 kB 00:00 2023-07-21T22:17:55.722 INFO:teuthology.orchestra.run.smithi132.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-07-21T22:17:55.747 INFO:teuthology.orchestra.run.smithi132.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-07-21T22:17:55.750 INFO:teuthology.orchestra.run.smithi190.stdout:(22/131): python3-rbd-18.0.0-4735.gbffe049d.el8 1.8 MB/s | 364 kB 00:00 2023-07-21T22:17:55.781 INFO:teuthology.orchestra.run.smithi132.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 980 kB/s | 58 kB 00:00 2023-07-21T22:17:55.806 INFO:teuthology.orchestra.run.smithi132.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 654 kB/s | 39 kB 00:00 2023-07-21T22:17:55.840 INFO:teuthology.orchestra.run.smithi132.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-07-21T22:17:55.865 INFO:teuthology.orchestra.run.smithi132.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-07-21T22:17:55.882 INFO:teuthology.orchestra.run.smithi132.stdout:(127/131): NetworkManager-initscripts-updown-1. 3.2 MB/s | 138 kB 00:00 2023-07-21T22:17:55.892 INFO:teuthology.orchestra.run.smithi190.stdout:(23/131): python3-rgw-18.0.0-4735.gbffe049d.el8 754 kB/s | 107 kB 00:00 2023-07-21T22:17:55.933 INFO:teuthology.orchestra.run.smithi132.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.9 MB/s | 2.3 MB 00:00 2023-07-21T22:17:55.960 INFO:teuthology.orchestra.run.smithi107.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 346 kB/s | 251 kB 00:00 2023-07-21T22:17:55.991 INFO:teuthology.orchestra.run.smithi132.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-21T22:17:56.009 INFO:teuthology.orchestra.run.smithi190.stdout:(24/131): librgw2-18.0.0-4735.gbffe049d.el8.x86 6.0 MB/s | 6.7 MB 00:01 2023-07-21T22:17:56.034 INFO:teuthology.orchestra.run.smithi190.stdout:(25/131): rbd-fuse-18.0.0-4735.gbffe049d.el8.x8 666 kB/s | 94 kB 00:00 2023-07-21T22:17:56.144 INFO:teuthology.orchestra.run.smithi107.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 201 kB/s | 85 kB 00:00 2023-07-21T22:17:56.176 INFO:teuthology.orchestra.run.smithi190.stdout:(26/131): rbd-nbd-18.0.0-4735.gbffe049d.el8.x86 1.2 MB/s | 179 kB 00:00 2023-07-21T22:17:56.343 INFO:teuthology.orchestra.run.smithi190.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4735.g 138 kB/s | 23 kB 00:00 2023-07-21T22:17:56.380 INFO:teuthology.orchestra.run.smithi107.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 55 kB/s | 23 kB 00:00 2023-07-21T22:17:56.411 INFO:teuthology.orchestra.run.smithi107.stdout:(69/131): python3-kubernetes-11.0.0-6.el8.noarc 279 kB/s | 1.2 MB 00:04 2023-07-21T22:17:56.436 INFO:teuthology.orchestra.run.smithi107.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2023-07-21T22:17:56.469 INFO:teuthology.orchestra.run.smithi190.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4735.gbffe049 1.0 MB/s | 129 kB 00:00 2023-07-21T22:17:56.478 INFO:teuthology.orchestra.run.smithi107.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 700 kB/s | 47 kB 00:00 2023-07-21T22:17:56.504 INFO:teuthology.orchestra.run.smithi107.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 3.2 MB/s | 200 kB 00:00 2023-07-21T22:17:56.529 INFO:teuthology.orchestra.run.smithi107.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 851 kB/s | 43 kB 00:00 2023-07-21T22:17:56.554 INFO:teuthology.orchestra.run.smithi107.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-21T22:17:56.579 INFO:teuthology.orchestra.run.smithi107.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 961 kB/s | 48 kB 00:00 2023-07-21T22:17:56.605 INFO:teuthology.orchestra.run.smithi107.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-21T22:17:56.630 INFO:teuthology.orchestra.run.smithi107.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 852 kB/s | 43 kB 00:00 2023-07-21T22:17:56.655 INFO:teuthology.orchestra.run.smithi107.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-07-21T22:17:56.680 INFO:teuthology.orchestra.run.smithi107.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-07-21T22:17:56.722 INFO:teuthology.orchestra.run.smithi107.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2023-07-21T22:17:56.750 INFO:teuthology.orchestra.run.smithi107.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 271 kB/s | 19 kB 00:00 2023-07-21T22:17:56.769 INFO:teuthology.orchestra.run.smithi190.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4735.gbffe0 5.7 MB/s | 1.7 MB 00:00 2023-07-21T22:17:56.775 INFO:teuthology.orchestra.run.smithi107.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.9 MB/s | 155 kB 00:00 2023-07-21T22:17:56.809 INFO:teuthology.orchestra.run.smithi107.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-07-21T22:17:56.834 INFO:teuthology.orchestra.run.smithi107.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-07-21T22:17:56.867 INFO:teuthology.orchestra.run.smithi107.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-07-21T22:17:56.886 INFO:teuthology.orchestra.run.smithi190.stdout:(30/131): rbd-mirror-18.0.0-4735.gbffe049d.el8. 3.9 MB/s | 3.4 MB 00:00 2023-07-21T22:17:56.893 INFO:teuthology.orchestra.run.smithi107.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-07-21T22:17:56.943 INFO:teuthology.orchestra.run.smithi107.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-07-21T22:17:56.993 INFO:teuthology.orchestra.run.smithi107.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 19 MB/s | 2.3 MB 00:00 2023-07-21T22:17:57.016 INFO:teuthology.orchestra.run.smithi132.stdout:(130/131): librados2-18.0.0-4735.gbffe049d.el8. 3.2 MB/s | 3.7 MB 00:01 2023-07-21T22:17:57.018 INFO:teuthology.orchestra.run.smithi107.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-07-21T22:17:57.036 INFO:teuthology.orchestra.run.smithi190.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4735.gbf 1.6 MB/s | 240 kB 00:00 2023-07-21T22:17:57.044 INFO:teuthology.orchestra.run.smithi107.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-07-21T22:17:57.069 INFO:teuthology.orchestra.run.smithi107.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-07-21T22:17:57.094 INFO:teuthology.orchestra.run.smithi107.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-21T22:17:57.119 INFO:teuthology.orchestra.run.smithi107.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 770 kB/s | 39 kB 00:00 2023-07-21T22:17:57.166 INFO:teuthology.orchestra.run.smithi132.stdout:(131/131): librbd1-18.0.0-4735.gbffe049d.el8.x8 2.8 MB/s | 3.6 MB 00:01 2023-07-21T22:17:57.170 INFO:teuthology.orchestra.run.smithi107.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 8.4 MB/s | 644 kB 00:00 2023-07-21T22:17:57.170 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:17:57.171 INFO:teuthology.orchestra.run.smithi132.stdout:Total 11 MB/s | 205 MB 00:17 2023-07-21T22:17:57.195 INFO:teuthology.orchestra.run.smithi107.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.2 MB/s | 171 kB 00:00 2023-07-21T22:17:57.211 INFO:teuthology.orchestra.run.smithi190.stdout:(32/131): ceph-mgr-rook-18.0.0-4735.gbffe049d.e 271 kB/s | 47 kB 00:00 2023-07-21T22:17:57.220 INFO:teuthology.orchestra.run.smithi107.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-21T22:17:57.262 INFO:teuthology.orchestra.run.smithi107.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-21T22:17:57.296 INFO:teuthology.orchestra.run.smithi107.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-07-21T22:17:57.338 INFO:teuthology.orchestra.run.smithi107.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-21T22:17:57.362 INFO:teuthology.orchestra.run.smithi190.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4735.gb 97 kB/s | 15 kB 00:00 2023-07-21T22:17:57.380 INFO:teuthology.orchestra.run.smithi107.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-07-21T22:17:57.413 INFO:teuthology.orchestra.run.smithi107.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.1 MB/s | 39 kB 00:00 2023-07-21T22:17:57.472 INFO:teuthology.orchestra.run.smithi107.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-07-21T22:17:57.514 INFO:teuthology.orchestra.run.smithi107.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 13 MB/s | 538 kB 00:00 2023-07-21T22:17:57.587 INFO:teuthology.orchestra.run.smithi190.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 9.1 MB/s | 7.4 MB 00:00 2023-07-21T22:17:57.612 INFO:teuthology.orchestra.run.smithi190.stdout:(35/131): ceph-volume-18.0.0-4735.gbffe049d.el8 1.0 MB/s | 258 kB 00:00 2023-07-21T22:17:57.664 INFO:teuthology.orchestra.run.smithi107.stdout:(104/131): python3-scipy-1.0.0-21.module+el8.5. 29 MB/s | 14 MB 00:00 2023-07-21T22:17:57.690 INFO:teuthology.orchestra.run.smithi107.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 896 kB/s | 157 kB 00:00 2023-07-21T22:17:57.715 INFO:teuthology.orchestra.run.smithi107.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-07-21T22:17:57.746 INFO:teuthology.orchestra.run.smithi190.stdout:(36/131): cephadm-18.0.0-4735.gbffe049d.el8.noa 1.0 MB/s | 164 kB 00:00 2023-07-21T22:17:57.748 INFO:teuthology.orchestra.run.smithi107.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-07-21T22:17:57.771 INFO:teuthology.orchestra.run.smithi190.stdout:(37/131): cephfs-top-18.0.0-4735.gbffe049d.el8. 113 kB/s | 18 kB 00:00 2023-07-21T22:17:57.782 INFO:teuthology.orchestra.run.smithi107.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 797 kB/s | 27 kB 00:00 2023-07-21T22:17:57.824 INFO:teuthology.orchestra.run.smithi107.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-21T22:17:57.859 INFO:teuthology.orchestra.run.smithi107.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-07-21T22:17:57.933 INFO:teuthology.orchestra.run.smithi107.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2023-07-21T22:17:57.975 INFO:teuthology.orchestra.run.smithi107.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-07-21T22:17:57.991 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2023-07-21T22:17:58.042 INFO:teuthology.orchestra.run.smithi107.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 20 MB/s | 3.7 MB 00:00 2023-07-21T22:17:58.067 INFO:teuthology.orchestra.run.smithi107.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.4 MB/s | 225 kB 00:00 2023-07-21T22:17:58.093 INFO:teuthology.orchestra.run.smithi107.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-07-21T22:17:58.110 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2023-07-21T22:17:58.111 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2023-07-21T22:17:58.118 INFO:teuthology.orchestra.run.smithi107.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.4 MB/s | 126 kB 00:00 2023-07-21T22:17:58.143 INFO:teuthology.orchestra.run.smithi107.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-07-21T22:17:58.169 INFO:teuthology.orchestra.run.smithi107.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-21T22:17:58.219 INFO:teuthology.orchestra.run.smithi107.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-07-21T22:17:58.255 INFO:teuthology.orchestra.run.smithi190.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 122 kB/s | 59 kB 00:00 2023-07-21T22:17:58.261 INFO:teuthology.orchestra.run.smithi107.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-21T22:17:58.361 INFO:teuthology.orchestra.run.smithi107.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 25 MB/s | 4.9 MB 00:00 2023-07-21T22:17:58.403 INFO:teuthology.orchestra.run.smithi107.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-21T22:17:58.478 INFO:teuthology.orchestra.run.smithi107.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-07-21T22:17:58.496 INFO:teuthology.orchestra.run.smithi190.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 408 kB/s | 306 kB 00:00 2023-07-21T22:17:58.503 INFO:teuthology.orchestra.run.smithi107.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.4 MB/s | 354 kB 00:00 2023-07-21T22:17:58.522 INFO:teuthology.orchestra.run.smithi190.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 281 kB/s | 75 kB 00:00 2023-07-21T22:17:58.554 INFO:teuthology.orchestra.run.smithi107.stdout:(125/131): postgresql-contrib-10.21-2.module+el 16 MB/s | 808 kB 00:00 2023-07-21T22:17:58.637 INFO:teuthology.orchestra.run.smithi107.stdout:(126/131): postgresql-server-10.21-2.module+el8 32 MB/s | 5.1 MB 00:00 2023-07-21T22:17:58.664 INFO:teuthology.orchestra.run.smithi190.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 308 kB/s | 44 kB 00:00 2023-07-21T22:17:58.679 INFO:teuthology.orchestra.run.smithi107.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-07-21T22:17:58.764 INFO:teuthology.orchestra.run.smithi190.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-07-21T22:17:58.914 INFO:teuthology.orchestra.run.smithi190.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 220 kB/s | 33 kB 00:00 2023-07-21T22:17:58.940 INFO:teuthology.orchestra.run.smithi190.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 673 kB/s | 185 kB 00:00 2023-07-21T22:17:59.065 INFO:teuthology.orchestra.run.smithi190.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 108 kB/s | 16 kB 00:00 2023-07-21T22:17:59.090 INFO:teuthology.orchestra.run.smithi190.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2023-07-21T22:17:59.224 INFO:teuthology.orchestra.run.smithi190.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.4 MB/s | 384 kB 00:00 2023-07-21T22:17:59.229 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2023-07-21T22:17:59.233 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2023-07-21T22:17:59.249 INFO:teuthology.orchestra.run.smithi190.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 512 kB/s | 81 kB 00:00 2023-07-21T22:17:59.374 INFO:teuthology.orchestra.run.smithi190.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 74 kB/s | 11 kB 00:00 2023-07-21T22:17:59.400 INFO:teuthology.orchestra.run.smithi190.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 117 kB/s | 18 kB 00:00 2023-07-21T22:17:59.558 INFO:teuthology.orchestra.run.smithi190.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 308 kB/s | 49 kB 00:00 2023-07-21T22:17:59.642 INFO:teuthology.orchestra.run.smithi190.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 4.4 MB/s | 1.2 MB 00:00 2023-07-21T22:17:59.675 INFO:teuthology.orchestra.run.smithi190.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 502 kB/s | 59 kB 00:00 2023-07-21T22:17:59.759 INFO:teuthology.orchestra.run.smithi190.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 515 kB/s | 60 kB 00:00 2023-07-21T22:17:59.801 INFO:teuthology.orchestra.run.smithi190.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2023-07-21T22:17:59.804 INFO:teuthology.orchestra.run.smithi107.stdout:(128/131): librados2-18.0.0-4735.gbffe049d.el8. 3.1 MB/s | 3.7 MB 00:01 2023-07-21T22:17:59.880 INFO:teuthology.orchestra.run.smithi107.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 10 MB/s | 798 kB 00:00 2023-07-21T22:17:59.893 INFO:teuthology.orchestra.run.smithi190.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 117 kB/s | 16 kB 00:00 2023-07-21T22:17:59.927 INFO:teuthology.orchestra.run.smithi190.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 268 kB/s | 34 kB 00:00 2023-07-21T22:18:00.043 INFO:teuthology.orchestra.run.smithi190.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.3 MB/s | 196 kB 00:00 2023-07-21T22:18:00.069 INFO:teuthology.orchestra.run.smithi190.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 433 kB/s | 61 kB 00:00 2023-07-21T22:18:00.185 INFO:teuthology.orchestra.run.smithi190.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 132 kB/s | 19 kB 00:00 2023-07-21T22:18:00.520 INFO:teuthology.orchestra.run.smithi190.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 53 kB/s | 24 kB 00:00 2023-07-21T22:18:00.538 INFO:teuthology.orchestra.run.smithi107.stdout:(130/131): librbd1-18.0.0-4735.gbffe049d.el8.x8 1.9 MB/s | 3.6 MB 00:01 2023-07-21T22:18:00.686 INFO:teuthology.orchestra.run.smithi190.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 165 kB/s | 27 kB 00:00 2023-07-21T22:18:00.745 INFO:teuthology.orchestra.run.smithi190.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 52 kB/s | 29 kB 00:00 2023-07-21T22:18:00.845 INFO:teuthology.orchestra.run.smithi190.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.6 MB/s | 255 kB 00:00 2023-07-21T22:18:00.970 INFO:teuthology.orchestra.run.smithi190.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 491 kB/s | 61 kB 00:00 2023-07-21T22:18:01.071 INFO:teuthology.orchestra.run.smithi190.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 772 kB/s | 251 kB 00:00 2023-07-21T22:18:01.121 INFO:teuthology.orchestra.run.smithi190.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 568 kB/s | 85 kB 00:00 2023-07-21T22:18:01.238 INFO:teuthology.orchestra.run.smithi190.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 137 kB/s | 23 kB 00:00 2023-07-21T22:18:01.263 INFO:teuthology.orchestra.run.smithi190.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-21T22:18:01.339 INFO:teuthology.orchestra.run.smithi190.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 625 kB/s | 47 kB 00:00 2023-07-21T22:18:01.397 INFO:teuthology.orchestra.run.smithi190.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 3.3 MB/s | 200 kB 00:00 2023-07-21T22:18:01.448 INFO:teuthology.orchestra.run.smithi190.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 852 kB/s | 43 kB 00:00 2023-07-21T22:18:01.506 INFO:teuthology.orchestra.run.smithi190.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-07-21T22:18:01.556 INFO:teuthology.orchestra.run.smithi190.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 965 kB/s | 48 kB 00:00 2023-07-21T22:18:01.607 INFO:teuthology.orchestra.run.smithi190.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-21T22:18:01.649 INFO:teuthology.orchestra.run.smithi190.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-07-21T22:18:01.691 INFO:teuthology.orchestra.run.smithi190.stdout:(77/131): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2023-07-21T22:18:01.749 INFO:teuthology.orchestra.run.smithi190.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-07-21T22:18:01.816 INFO:teuthology.orchestra.run.smithi190.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2023-07-21T22:18:01.866 INFO:teuthology.orchestra.run.smithi190.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2023-07-21T22:18:01.925 INFO:teuthology.orchestra.run.smithi190.stdout:(81/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.6 MB/s | 155 kB 00:00 2023-07-21T22:18:02.080 INFO:teuthology.orchestra.run.smithi107.stdout:(131/131): thrift-0.13.0-2.el8.x86_64.rpm 299 kB/s | 1.7 MB 00:05 2023-07-21T22:18:02.084 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:18:02.084 INFO:teuthology.orchestra.run.smithi107.stdout:Total 8.4 MB/s | 205 MB 00:24 2023-07-21T22:18:02.192 INFO:teuthology.orchestra.run.smithi190.stdout:(82/131): thrift-0.13.0-2.el8.x86_64.rpm 1.6 MB/s | 1.7 MB 00:01 2023-07-21T22:18:02.234 INFO:teuthology.orchestra.run.smithi190.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 1.8 MB/s | 559 kB 00:00 2023-07-21T22:18:02.293 INFO:teuthology.orchestra.run.smithi190.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 179 kB/s | 18 kB 00:00 2023-07-21T22:18:02.326 INFO:teuthology.orchestra.run.smithi190.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.8 MB/s | 605 kB 00:00 2023-07-21T22:18:02.352 INFO:teuthology.orchestra.run.smithi190.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-07-21T22:18:02.419 INFO:teuthology.orchestra.run.smithi190.stdout:(87/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 25 MB/s | 2.3 MB 00:00 2023-07-21T22:18:02.444 INFO:teuthology.orchestra.run.smithi190.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 787 kB/s | 72 kB 00:00 2023-07-21T22:18:02.469 INFO:teuthology.orchestra.run.smithi190.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-07-21T22:18:02.494 INFO:teuthology.orchestra.run.smithi190.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-07-21T22:18:02.511 INFO:teuthology.orchestra.run.smithi190.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-07-21T22:18:02.550 INFO:teuthology.orchestra.run.smithi190.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-21T22:18:02.570 INFO:teuthology.orchestra.run.smithi190.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 659 kB/s | 39 kB 00:00 2023-07-21T22:18:02.604 INFO:teuthology.orchestra.run.smithi190.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-07-21T22:18:02.629 INFO:teuthology.orchestra.run.smithi190.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-07-21T22:18:02.654 INFO:teuthology.orchestra.run.smithi190.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-21T22:18:02.696 INFO:teuthology.orchestra.run.smithi190.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-21T22:18:02.747 INFO:teuthology.orchestra.run.smithi190.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-07-21T22:18:02.797 INFO:teuthology.orchestra.run.smithi190.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-07-21T22:18:02.856 INFO:teuthology.orchestra.run.smithi190.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-07-21T22:18:02.898 INFO:teuthology.orchestra.run.smithi190.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 935 kB/s | 39 kB 00:00 2023-07-21T22:18:02.922 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2023-07-21T22:18:02.948 INFO:teuthology.orchestra.run.smithi190.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-07-21T22:18:03.043 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2023-07-21T22:18:03.043 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2023-07-21T22:18:03.082 INFO:teuthology.orchestra.run.smithi190.stdout:(103/131): python3-scipy-1.0.0-21.module+el8.5. 30 MB/s | 14 MB 00:00 2023-07-21T22:18:03.140 INFO:teuthology.orchestra.run.smithi190.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 2.7 MB/s | 538 kB 00:00 2023-07-21T22:18:03.165 INFO:teuthology.orchestra.run.smithi190.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.8 MB/s | 157 kB 00:00 2023-07-21T22:18:03.208 INFO:teuthology.orchestra.run.smithi190.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2023-07-21T22:18:03.291 INFO:teuthology.orchestra.run.smithi190.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 5.6 MB/s | 713 kB 00:00 2023-07-21T22:18:03.349 INFO:teuthology.orchestra.run.smithi190.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 455 kB/s | 27 kB 00:00 2023-07-21T22:18:03.408 INFO:teuthology.orchestra.run.smithi190.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 24 MB/s | 4.8 MB 00:00 2023-07-21T22:18:03.433 INFO:teuthology.orchestra.run.smithi190.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 3.7 MB/s | 320 kB 00:00 2023-07-21T22:18:03.450 INFO:teuthology.orchestra.run.smithi190.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-07-21T22:18:03.517 INFO:teuthology.orchestra.run.smithi190.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 6.7 MB/s | 457 kB 00:00 2023-07-21T22:18:03.584 INFO:teuthology.orchestra.run.smithi190.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 25 MB/s | 3.7 MB 00:00 2023-07-21T22:18:03.601 INFO:teuthology.orchestra.run.smithi190.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.6 MB/s | 225 kB 00:00 2023-07-21T22:18:03.627 INFO:teuthology.orchestra.run.smithi190.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-07-21T22:18:03.652 INFO:teuthology.orchestra.run.smithi190.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-07-21T22:18:03.677 INFO:teuthology.orchestra.run.smithi190.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 399 kB/s | 20 kB 00:00 2023-07-21T22:18:04.056 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2023-07-21T22:18:04.078 INFO:teuthology.orchestra.run.smithi190.stdout:(118/131): ceph-test-18.0.0-4735.gbffe049d.el8. 4.6 MB/s | 48 MB 00:10 2023-07-21T22:18:04.103 INFO:teuthology.orchestra.run.smithi190.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 230 kB/s | 103 kB 00:00 2023-07-21T22:18:04.144 INFO:teuthology.orchestra.run.smithi190.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 1.5 MB/s | 709 kB 00:00 2023-07-21T22:18:04.145 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2023-07-21T22:18:04.149 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2023-07-21T22:18:04.170 INFO:teuthology.orchestra.run.smithi190.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-07-21T22:18:04.228 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T22:18:04.229 INFO:teuthology.orchestra.run.smithi190.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-07-21T22:18:04.279 INFO:teuthology.orchestra.run.smithi190.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 7.0 MB/s | 354 kB 00:00 2023-07-21T22:18:04.345 INFO:teuthology.orchestra.run.smithi190.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 18 MB/s | 4.9 MB 00:00 2023-07-21T22:18:04.413 INFO:teuthology.orchestra.run.smithi190.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-07-21T22:18:04.413 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T22:18:04.454 INFO:teuthology.orchestra.run.smithi190.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-07-21T22:18:04.513 INFO:teuthology.orchestra.run.smithi190.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-07-21T22:18:04.605 INFO:teuthology.orchestra.run.smithi190.stdout:(128/131): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2023-07-21T22:18:04.660 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-21T22:18:04.664 INFO:teuthology.orchestra.run.smithi190.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-21T22:18:04.821 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-21T22:18:04.927 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-21T22:18:05.154 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-21T22:18:05.554 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-21T22:18:05.770 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:18:05.814 INFO:teuthology.orchestra.run.smithi190.stdout:(130/131): librbd1-18.0.0-4735.gbffe049d.el8.x8 2.7 MB/s | 3.6 MB 00:01 2023-07-21T22:18:05.989 INFO:teuthology.orchestra.run.smithi190.stdout:(131/131): librados2-18.0.0-4735.gbffe049d.el8. 2.4 MB/s | 3.7 MB 00:01 2023-07-21T22:18:05.992 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:18:05.992 INFO:teuthology.orchestra.run.smithi190.stdout:Total 11 MB/s | 205 MB 00:18 2023-07-21T22:18:06.174 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:18:06.370 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:18:06.492 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:18:06.679 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:18:06.780 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:18:06.813 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2023-07-21T22:18:06.932 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2023-07-21T22:18:06.932 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2023-07-21T22:18:06.964 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 9/134 2023-07-21T22:18:07.208 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T22:18:07.406 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T22:18:07.795 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T22:18:08.041 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2023-07-21T22:18:08.045 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2023-07-21T22:18:08.048 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T22:18:08.195 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:18:08.295 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:18:08.445 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:18:08.518 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:18:08.886 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-21T22:18:09.038 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-21T22:18:09.186 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-21T22:18:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2023-07-21T22:18:09.426 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T22:18:09.600 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T22:18:09.801 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-21T22:18:09.802 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T22:18:09.916 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T22:18:09.978 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-21T22:18:10.111 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T22:18:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T22:18:10.584 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-21T22:18:10.727 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 5/134 2023-07-21T22:18:10.786 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-21T22:18:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 5/134 2023-07-21T22:18:11.285 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:18:11.412 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:18:11.502 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-21T22:18:11.586 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:18:11.708 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:18:11.759 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-21T22:18:11.874 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:18:11.978 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-21T22:18:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-21T22:18:12.215 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-21T22:18:12.256 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T22:18:12.383 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-21T22:18:12.583 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-21T22:18:12.700 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T22:18:12.820 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T22:18:13.007 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2023-07-21T22:18:13.026 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 26/134 2023-07-21T22:18:13.119 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T22:18:13.152 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T22:18:13.171 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 27/134 2023-07-21T22:18:13.320 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T22:18:13.347 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T22:18:13.477 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:18:13.479 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T22:18:13.574 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T22:18:13.584 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:18:13.727 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:18:13.788 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T22:18:13.875 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:18:14.071 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 28/134 2023-07-21T22:18:14.072 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-21T22:18:14.178 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T22:18:14.277 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-21T22:18:14.395 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 5/134 2023-07-21T22:18:14.464 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-21T22:18:14.585 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T22:18:14.702 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-21T22:18:14.713 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T22:18:14.771 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 5/134 2023-07-21T22:18:14.893 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T22:18:14.945 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:18:14.953 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-21T22:18:15.071 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:18:15.203 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-21T22:18:15.245 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:18:15.342 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:18:15.402 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-21T22:18:15.500 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:18:15.504 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T22:18:15.656 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-21T22:18:15.668 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T22:18:15.908 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T22:18:16.131 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T22:18:16.263 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T22:18:16.451 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T22:18:16.512 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T22:18:16.748 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T22:18:16.870 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:18:16.974 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:18:17.112 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:18:17.138 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T22:18:17.155 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-21T22:18:17.257 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:18:17.383 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-21T22:18:17.473 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-21T22:18:17.478 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-21T22:18:17.575 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-21T22:18:17.640 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-21T22:18:17.679 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T22:18:17.786 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T22:18:17.801 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-21T22:18:17.862 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T22:18:17.928 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T22:18:17.932 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 26/134 2023-07-21T22:18:18.075 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T22:18:18.086 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 27/134 2023-07-21T22:18:18.099 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 36/134 2023-07-21T22:18:18.212 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T22:18:18.272 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 37/134 2023-07-21T22:18:18.306 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 28/134 2023-07-21T22:18:18.425 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-21T22:18:18.492 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T22:18:18.620 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-21T22:18:18.812 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-21T22:18:18.832 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T22:18:18.978 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T22:18:19.005 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-21T22:18:19.215 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-21T22:18:19.413 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-21T22:18:19.573 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T22:18:19.744 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T22:18:19.750 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-21T22:18:19.996 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-21T22:18:20.186 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-21T22:18:20.271 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T22:18:20.387 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-21T22:18:20.442 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T22:18:20.668 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-21T22:18:20.694 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-21T22:18:20.850 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-21T22:18:20.993 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-21T22:18:21.096 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T22:18:21.209 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-21T22:18:21.274 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-21T22:18:21.276 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 26/134 2023-07-21T22:18:21.397 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 27/134 2023-07-21T22:18:21.404 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-21T22:18:21.581 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-21T22:18:21.599 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 28/134 2023-07-21T22:18:21.639 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-21T22:18:21.794 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T22:18:21.796 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-21T22:18:21.835 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-21T22:18:21.963 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-21T22:18:22.103 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-21T22:18:22.244 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-21T22:18:22.303 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T22:18:22.493 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T22:18:22.546 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-21T22:18:22.689 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 36/134 2023-07-21T22:18:22.789 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-21T22:18:22.839 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 37/134 2023-07-21T22:18:22.973 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-21T22:18:22.991 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-21T22:18:23.100 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-21T22:18:23.158 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-21T22:18:23.283 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T22:18:23.336 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-21T22:18:23.356 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T22:18:23.476 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-21T22:18:23.529 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-21T22:18:23.564 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-21T22:18:23.565 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T22:18:23.770 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-21T22:18:23.772 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-21T22:18:23.925 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T22:18:23.971 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-21T22:18:23.993 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T22:18:24.143 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-21T22:18:24.187 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-21T22:18:24.224 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-21T22:18:24.337 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-21T22:18:24.438 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-21T22:18:24.515 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-21T22:18:24.560 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-21T22:18:24.601 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-21T22:18:24.676 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-21T22:18:24.759 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T22:18:24.835 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-21T22:18:24.900 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-21T22:18:24.945 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T22:18:25.003 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-21T22:18:25.171 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-21T22:18:25.207 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-21T22:18:25.235 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T22:18:25.383 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-21T22:18:25.406 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-21T22:18:25.457 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T22:18:25.605 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T22:18:25.605 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T22:18:25.751 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-21T22:18:25.770 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T22:18:25.787 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T22:18:25.950 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 36/134 2023-07-21T22:18:25.983 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T22:18:26.051 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-21T22:18:26.092 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 37/134 2023-07-21T22:18:26.170 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T22:18:26.242 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-21T22:18:26.252 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-21T22:18:26.335 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T22:18:26.378 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-21T22:18:26.487 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-21T22:18:26.529 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T22:18:26.577 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T22:18:26.655 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T22:18:26.673 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T22:18:26.695 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-21T22:18:26.873 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-21T22:18:26.925 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T22:18:27.018 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-21T22:18:27.075 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-21T22:18:27.147 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T22:18:27.210 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T22:18:27.215 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-21T22:18:27.275 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T22:18:27.331 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T22:18:27.387 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-21T22:18:27.456 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-21T22:18:27.557 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T22:18:27.587 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-21T22:18:27.622 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-21T22:18:27.778 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-21T22:18:27.791 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T22:18:27.792 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-21T22:18:27.943 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-21T22:18:27.962 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-21T22:18:27.967 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T22:18:28.112 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-21T22:18:28.193 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T22:18:28.239 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-21T22:18:28.305 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-21T22:18:28.491 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-21T22:18:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-21T22:18:28.689 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-21T22:18:28.737 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-21T22:18:28.983 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-21T22:18:29.043 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-21T22:18:29.123 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T22:18:29.276 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T22:18:29.285 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T22:18:29.328 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-21T22:18:29.508 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T22:18:29.571 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-21T22:18:29.595 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T22:18:29.772 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T22:18:29.798 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-21T22:18:29.866 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T22:18:29.902 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T22:18:30.040 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-21T22:18:30.059 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T22:18:30.209 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T22:18:30.291 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T22:18:30.350 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-21T22:18:30.485 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T22:18:30.533 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T22:18:30.551 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-21T22:18:30.668 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T22:18:30.719 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T22:18:30.727 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-21T22:18:30.860 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T22:18:30.939 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-21T22:18:30.996 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T22:18:31.114 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-21T22:18:31.228 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T22:18:31.279 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-21T22:18:31.435 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T22:18:31.500 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T22:18:31.558 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-21T22:18:31.646 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T22:18:31.650 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:18:31.724 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:18:31.823 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-21T22:18:31.883 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 93/134 2023-07-21T22:18:31.889 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T22:18:32.064 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-21T22:18:32.090 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T22:18:32.294 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-21T22:18:32.298 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T22:18:32.442 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T22:18:32.500 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T22:18:32.629 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T22:18:32.860 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T22:18:33.074 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T22:18:33.213 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T22:18:33.353 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T22:18:33.542 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T22:18:33.593 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T22:18:33.868 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T22:18:34.062 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T22:18:34.147 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T22:18:34.227 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T22:18:34.379 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T22:18:34.459 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T22:18:34.499 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T22:18:34.725 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T22:18:34.745 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T22:18:34.935 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T22:18:34.947 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T22:18:35.107 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T22:18:35.331 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T22:18:35.517 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:18:35.583 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T22:18:35.697 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T22:18:35.699 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:18:35.749 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T22:18:35.855 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:18:35.932 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:18:35.950 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T22:18:36.105 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 93/134 2023-07-21T22:18:36.290 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:18:36.580 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:18:36.663 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:18:36.799 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:18:37.028 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T22:18:37.313 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T22:18:37.558 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T22:18:37.885 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T22:18:38.120 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T22:18:38.307 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T22:18:39.110 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T22:18:39.258 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:18:39.314 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:18:39.500 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 93/134 2023-07-21T22:18:39.776 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:18:39.988 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:18:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:18:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:18:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:18:41.146 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:18:43.166 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:18:43.350 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:18:43.944 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:18:44.196 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:18:44.282 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:18:44.624 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:18:55.574 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:18:55.574 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /sys 2023-07-21T22:18:55.574 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /proc 2023-07-21T22:18:55.575 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /dev 2023-07-21T22:18:55.575 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /run 2023-07-21T22:18:55.575 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /mnt 2023-07-21T22:18:55.575 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /var/tmp 2023-07-21T22:18:55.575 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /home 2023-07-21T22:18:55.575 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /tmp 2023-07-21T22:18:55.575 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /dev 2023-07-21T22:18:55.575 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-21T22:18:55.904 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:18:57.473 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:18:57.644 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:18:57.862 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:18:58.079 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:18:58.544 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:18:58.738 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:18:58.755 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:18:59.824 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:18:59.824 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2023-07-21T22:18:59.824 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2023-07-21T22:18:59.824 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /dev 2023-07-21T22:18:59.825 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /run 2023-07-21T22:18:59.825 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2023-07-21T22:18:59.825 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2023-07-21T22:18:59.825 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2023-07-21T22:18:59.825 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2023-07-21T22:18:59.825 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /dev 2023-07-21T22:18:59.825 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:19:00.233 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:19:00.427 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:19:00.587 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:19:00.710 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 102/134 2023-07-21T22:19:00.935 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 103/134 2023-07-21T22:19:01.112 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:19:01.327 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:19:01.635 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:19:01.804 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:19:02.010 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:19:02.230 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:19:02.400 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:19:02.594 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:19:02.745 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:19:02.836 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:19:02.863 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:19:03.017 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:19:03.034 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:19:03.237 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 107/134 2023-07-21T22:19:03.457 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:19:03.492 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:19:03.633 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:19:03.633 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2023-07-21T22:19:03.634 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2023-07-21T22:19:03.634 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /dev 2023-07-21T22:19:03.634 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /run 2023-07-21T22:19:03.634 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2023-07-21T22:19:03.634 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2023-07-21T22:19:03.634 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2023-07-21T22:19:03.634 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2023-07-21T22:19:03.635 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /dev 2023-07-21T22:19:03.635 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:19:03.636 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:19:03.647 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:19:03.980 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:19:04.661 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:19:04.804 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:19:04.928 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 110/134 2023-07-21T22:19:04.938 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 102/134 2023-07-21T22:19:05.116 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:19:05.121 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 103/134 2023-07-21T22:19:05.149 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:19:05.303 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:19:05.332 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:19:05.345 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:19:05.536 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:19:05.550 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:19:05.558 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:19:05.718 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:19:05.859 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:19:05.881 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:19:05.937 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:19:06.036 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:19:06.138 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:19:06.599 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:19:06.738 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:19:06.755 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:19:06.819 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:19:06.974 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:19:07.066 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:19:07.419 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 107/134 2023-07-21T22:19:07.611 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:19:07.647 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:19:07.815 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:19:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:19:08.440 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:19:08.595 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:19:08.719 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 102/134 2023-07-21T22:19:08.893 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 103/134 2023-07-21T22:19:09.079 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:19:09.104 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 110/134 2023-07-21T22:19:09.295 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:19:09.302 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:19:09.336 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:19:09.465 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:19:09.507 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:19:09.542 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:19:09.657 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:19:10.002 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:19:10.132 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:19:10.503 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:19:10.554 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:19:10.670 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:19:10.761 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:19:10.839 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 115/134 2023-07-21T22:19:10.985 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-21T22:19:11.112 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 107/134 2023-07-21T22:19:11.174 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-21T22:19:11.303 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:19:11.337 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:19:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:19:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:19:12.047 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-21T22:19:12.384 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-21T22:19:12.571 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 119/134 2023-07-21T22:19:12.773 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-21T22:19:12.798 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 110/134 2023-07-21T22:19:12.939 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 121/134 2023-07-21T22:19:12.965 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:19:12.999 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:19:13.120 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:19:13.136 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T22:19:13.161 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:19:13.329 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:19:13.674 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:19:13.820 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:19:14.644 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:19:15.308 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:19:15.546 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 115/134 2023-07-21T22:19:15.593 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:19:15.610 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:19:15.785 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 124/134 2023-07-21T22:19:15.845 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-21T22:19:15.991 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 117/134 2023-07-21T22:19:16.051 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 125/134 2023-07-21T22:19:16.248 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-21T22:19:16.356 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 126/134 2023-07-21T22:19:16.388 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T22:19:16.581 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T22:19:16.653 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T22:19:16.785 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T22:19:16.793 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T22:19:16.826 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-21T22:19:16.967 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 121/134 2023-07-21T22:19:16.974 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-21T22:19:17.106 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T22:19:17.168 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-21T22:19:17.246 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-21T22:19:17.459 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-21T22:19:17.472 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-21T22:19:17.472 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:19:17.654 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:19:17.654 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:19:17.881 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:19:18.066 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:19:18.340 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:19:18.595 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:19:18.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 1/134 2023-07-21T22:19:18.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2/134 2023-07-21T22:19:18.595 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 3/134 2023-07-21T22:19:18.596 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 4/134 2023-07-21T22:19:18.596 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 5/134 2023-07-21T22:19:18.596 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:19:18.596 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:19:18.596 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:19:18.596 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 9/134 2023-07-21T22:19:18.596 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 10/134 2023-07-21T22:19:18.597 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 11/134 2023-07-21T22:19:18.597 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 12/134 2023-07-21T22:19:18.597 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 13/134 2023-07-21T22:19:18.597 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 14/134 2023-07-21T22:19:18.597 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 15/134 2023-07-21T22:19:18.597 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 16/134 2023-07-21T22:19:18.598 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 17/134 2023-07-21T22:19:18.598 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 18/134 2023-07-21T22:19:18.598 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 19/134 2023-07-21T22:19:18.598 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 20/134 2023-07-21T22:19:18.598 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 21/134 2023-07-21T22:19:18.598 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 22/134 2023-07-21T22:19:18.598 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 23/134 2023-07-21T22:19:18.599 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 24/134 2023-07-21T22:19:18.599 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 25/134 2023-07-21T22:19:18.599 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 26/134 2023-07-21T22:19:18.599 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 27/134 2023-07-21T22:19:18.606 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 28/134 2023-07-21T22:19:18.606 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 29/134 2023-07-21T22:19:18.606 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 30/134 2023-07-21T22:19:18.606 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 31/134 2023-07-21T22:19:18.606 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 32/134 2023-07-21T22:19:18.607 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 33/134 2023-07-21T22:19:18.607 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 34/134 2023-07-21T22:19:18.607 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 35/134 2023-07-21T22:19:18.607 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 36/134 2023-07-21T22:19:18.607 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 37/134 2023-07-21T22:19:18.607 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 38/134 2023-07-21T22:19:18.607 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T22:19:18.608 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T22:19:18.608 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T22:19:18.608 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T22:19:18.608 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T22:19:18.608 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T22:19:18.609 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T22:19:18.609 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T22:19:18.609 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T22:19:18.609 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T22:19:18.609 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T22:19:18.609 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T22:19:18.610 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T22:19:18.610 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T22:19:18.610 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T22:19:18.610 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T22:19:18.610 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T22:19:18.610 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T22:19:18.610 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T22:19:18.611 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T22:19:18.611 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T22:19:18.611 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T22:19:18.611 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T22:19:18.611 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T22:19:18.612 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T22:19:18.612 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T22:19:18.612 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T22:19:18.613 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T22:19:18.613 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T22:19:18.613 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T22:19:18.613 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T22:19:18.614 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T22:19:18.614 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T22:19:18.614 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-21T22:19:18.615 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-21T22:19:18.615 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-21T22:19:18.615 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-21T22:19:18.615 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-21T22:19:18.616 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-21T22:19:18.616 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-21T22:19:18.616 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-21T22:19:18.616 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-21T22:19:18.617 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-21T22:19:18.617 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-21T22:19:18.617 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-21T22:19:18.617 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-21T22:19:18.618 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-21T22:19:18.618 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-21T22:19:18.618 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-21T22:19:18.619 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-21T22:19:18.619 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-21T22:19:18.619 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-21T22:19:18.619 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-21T22:19:18.619 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-21T22:19:18.619 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-21T22:19:18.619 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-21T22:19:18.619 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-21T22:19:18.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-21T22:19:18.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-21T22:19:18.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-21T22:19:18.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-21T22:19:18.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-21T22:19:18.620 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-21T22:19:18.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-21T22:19:18.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-21T22:19:18.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-21T22:19:18.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-21T22:19:18.621 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-21T22:19:18.622 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-21T22:19:18.622 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-21T22:19:18.622 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-21T22:19:18.622 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-21T22:19:18.622 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-21T22:19:18.622 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-21T22:19:18.623 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-21T22:19:18.623 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-21T22:19:18.623 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-21T22:19:18.623 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-21T22:19:18.624 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-21T22:19:18.624 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-21T22:19:18.624 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-21T22:19:18.625 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-21T22:19:18.625 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-21T22:19:18.625 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T22:19:18.625 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-21T22:19:18.626 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-21T22:19:18.626 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-21T22:19:18.626 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-21T22:19:18.627 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-21T22:19:18.627 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T22:19:18.627 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 129/134 2023-07-21T22:19:18.628 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T22:19:18.628 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 131/134 2023-07-21T22:19:18.628 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:19:18.628 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T22:19:19.192 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 115/134 2023-07-21T22:19:19.247 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:19:19.493 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-21T22:19:19.496 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:19:19.516 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:19:19.654 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 124/134 2023-07-21T22:19:19.664 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 117/134 2023-07-21T22:19:19.905 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-21T22:19:19.921 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 125/134 2023-07-21T22:19:20.052 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T22:19:20.195 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 126/134 2023-07-21T22:19:20.242 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T22:19:20.498 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-21T22:19:20.581 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T22:19:20.682 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 121/134 2023-07-21T22:19:20.779 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T22:19:20.812 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T22:19:20.865 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T22:19:21.071 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T22:19:21.084 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T22:19:21.091 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:19:21.091 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:54 ago on Fri 21 Jul 2023 10:17:27 PM UTC. 2023-07-21T22:19:21.271 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T22:19:21.279 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T22:19:21.414 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-21T22:19:21.415 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:19:21.607 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:19:21.607 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:19:21.868 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:19:22.078 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:19:22.609 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:19:22.610 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 1/134 2023-07-21T22:19:22.610 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2/134 2023-07-21T22:19:22.610 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 3/134 2023-07-21T22:19:22.610 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 4/134 2023-07-21T22:19:22.610 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 5/134 2023-07-21T22:19:22.610 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:19:22.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:19:22.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:19:22.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 9/134 2023-07-21T22:19:22.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 10/134 2023-07-21T22:19:22.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 11/134 2023-07-21T22:19:22.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 12/134 2023-07-21T22:19:22.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 13/134 2023-07-21T22:19:22.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 14/134 2023-07-21T22:19:22.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 15/134 2023-07-21T22:19:22.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 16/134 2023-07-21T22:19:22.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 17/134 2023-07-21T22:19:22.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 18/134 2023-07-21T22:19:22.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 19/134 2023-07-21T22:19:22.613 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 20/134 2023-07-21T22:19:22.613 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 21/134 2023-07-21T22:19:22.613 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 22/134 2023-07-21T22:19:22.613 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 23/134 2023-07-21T22:19:22.614 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 24/134 2023-07-21T22:19:22.614 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 25/134 2023-07-21T22:19:22.614 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 26/134 2023-07-21T22:19:22.614 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 27/134 2023-07-21T22:19:22.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 28/134 2023-07-21T22:19:22.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 29/134 2023-07-21T22:19:22.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 30/134 2023-07-21T22:19:22.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 31/134 2023-07-21T22:19:22.616 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 32/134 2023-07-21T22:19:22.616 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 33/134 2023-07-21T22:19:22.616 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 34/134 2023-07-21T22:19:22.616 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 35/134 2023-07-21T22:19:22.617 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 36/134 2023-07-21T22:19:22.617 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 37/134 2023-07-21T22:19:22.618 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 38/134 2023-07-21T22:19:22.618 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T22:19:22.618 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T22:19:22.618 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T22:19:22.619 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T22:19:22.619 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T22:19:22.619 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T22:19:22.619 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T22:19:22.620 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T22:19:22.620 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T22:19:22.620 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T22:19:22.620 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T22:19:22.620 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T22:19:22.620 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T22:19:22.620 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T22:19:22.621 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T22:19:22.621 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T22:19:22.621 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T22:19:22.621 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T22:19:22.621 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T22:19:22.621 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T22:19:22.622 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T22:19:22.622 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T22:19:22.622 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T22:19:22.622 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T22:19:22.622 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T22:19:22.622 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T22:19:22.622 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T22:19:22.623 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T22:19:22.623 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T22:19:22.623 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T22:19:22.623 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T22:19:22.623 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T22:19:22.623 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T22:19:22.623 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-21T22:19:22.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-21T22:19:22.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-21T22:19:22.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-21T22:19:22.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-21T22:19:22.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-21T22:19:22.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-21T22:19:22.625 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-21T22:19:22.625 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-21T22:19:22.625 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-21T22:19:22.625 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-21T22:19:22.625 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-21T22:19:22.625 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-21T22:19:22.626 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-21T22:19:22.626 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-21T22:19:22.626 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-21T22:19:22.626 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-21T22:19:22.626 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-21T22:19:22.626 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-21T22:19:22.626 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-21T22:19:22.627 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-21T22:19:22.627 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-21T22:19:22.627 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-21T22:19:22.627 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-21T22:19:22.627 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-21T22:19:22.627 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-21T22:19:22.627 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-21T22:19:22.627 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-21T22:19:22.628 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-21T22:19:22.628 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-21T22:19:22.628 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-21T22:19:22.628 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-21T22:19:22.628 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-21T22:19:22.628 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-21T22:19:22.629 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-21T22:19:22.629 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-21T22:19:22.629 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-21T22:19:22.629 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-21T22:19:22.629 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-21T22:19:22.629 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-21T22:19:22.630 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-21T22:19:22.630 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-21T22:19:22.630 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-21T22:19:22.630 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-21T22:19:22.630 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-21T22:19:22.630 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-21T22:19:22.631 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-21T22:19:22.631 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-21T22:19:22.631 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-21T22:19:22.631 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-21T22:19:22.631 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-21T22:19:22.631 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-21T22:19:22.632 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-21T22:19:22.632 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-21T22:19:22.632 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-21T22:19:22.632 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-21T22:19:22.632 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-21T22:19:22.632 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 129/134 2023-07-21T22:19:22.632 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T22:19:22.633 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 131/134 2023-07-21T22:19:22.633 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:19:22.633 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T22:19:22.992 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:19:23.260 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:19:23.278 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:19:23.402 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 124/134 2023-07-21T22:19:23.685 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 125/134 2023-07-21T22:19:23.994 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 126/134 2023-07-21T22:19:24.108 INFO:teuthology.orchestra.run.smithi132.stdout:Installed products updated. 2023-07-21T22:19:24.321 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-21T22:19:24.321 INFO:teuthology.orchestra.run.smithi132.stdout:Upgraded: 2023-07-21T22:19:24.321 INFO:teuthology.orchestra.run.smithi132.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T22:19:24.321 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.321 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.322 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2023-07-21T22:19:24.322 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T22:19:24.322 INFO:teuthology.orchestra.run.smithi132.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T22:19:24.322 INFO:teuthology.orchestra.run.smithi132.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T22:19:24.323 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.323 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.323 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.323 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.323 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:24.323 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.323 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.323 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.324 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:24.324 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:24.324 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:24.324 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:24.324 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:24.324 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.324 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.325 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:24.325 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.325 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.325 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.325 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:24.326 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:24.326 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.326 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:24.326 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T22:19:24.326 INFO:teuthology.orchestra.run.smithi132.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T22:19:24.326 INFO:teuthology.orchestra.run.smithi132.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T22:19:24.327 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T22:19:24.327 INFO:teuthology.orchestra.run.smithi132.stdout: jq-1.6-3.el8.x86_64 2023-07-21T22:19:24.327 INFO:teuthology.orchestra.run.smithi132.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T22:19:24.327 INFO:teuthology.orchestra.run.smithi132.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T22:19:24.327 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T22:19:24.327 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.327 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.327 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.328 INFO:teuthology.orchestra.run.smithi132.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T22:19:24.328 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:19:24.328 INFO:teuthology.orchestra.run.smithi132.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T22:19:24.328 INFO:teuthology.orchestra.run.smithi132.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T22:19:24.328 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:19:24.328 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T22:19:24.328 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.329 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.329 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T22:19:24.329 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.329 INFO:teuthology.orchestra.run.smithi132.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T22:19:24.329 INFO:teuthology.orchestra.run.smithi132.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T22:19:24.329 INFO:teuthology.orchestra.run.smithi132.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T22:19:24.329 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T22:19:24.329 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T22:19:24.330 INFO:teuthology.orchestra.run.smithi132.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T22:19:24.330 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T22:19:24.330 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T22:19:24.330 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T22:19:24.330 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T22:19:24.330 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T22:19:24.330 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T22:19:24.331 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T22:19:24.331 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T22:19:24.331 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:19:24.331 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:19:24.331 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:19:24.331 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T22:19:24.331 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T22:19:24.331 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T22:19:24.332 INFO:teuthology.orchestra.run.smithi132.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T22:19:24.332 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T22:19:24.332 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.333 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.333 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.333 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T22:19:24.333 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T22:19:24.333 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T22:19:24.333 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T22:19:24.333 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T22:19:24.334 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T22:19:24.334 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T22:19:24.334 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T22:19:24.334 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T22:19:24.334 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T22:19:24.334 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T22:19:24.334 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T22:19:24.334 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T22:19:24.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T22:19:24.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T22:19:24.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T22:19:24.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T22:19:24.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T22:19:24.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T22:19:24.335 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T22:19:24.336 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T22:19:24.336 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T22:19:24.336 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T22:19:24.336 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T22:19:24.336 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T22:19:24.337 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T22:19:24.337 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T22:19:24.337 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T22:19:24.337 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T22:19:24.337 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.337 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.337 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T22:19:24.338 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T22:19:24.338 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.338 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T22:19:24.338 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T22:19:24.338 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T22:19:24.338 INFO:teuthology.orchestra.run.smithi132.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T22:19:24.339 INFO:teuthology.orchestra.run.smithi132.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T22:19:24.339 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T22:19:24.339 INFO:teuthology.orchestra.run.smithi132.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T22:19:24.339 INFO:teuthology.orchestra.run.smithi132.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T22:19:24.339 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T22:19:24.339 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T22:19:24.339 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T22:19:24.340 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T22:19:24.340 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T22:19:24.340 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.340 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.340 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:24.340 INFO:teuthology.orchestra.run.smithi132.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T22:19:24.340 INFO:teuthology.orchestra.run.smithi132.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T22:19:24.340 INFO:teuthology.orchestra.run.smithi132.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T22:19:24.341 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T22:19:24.341 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T22:19:24.341 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T22:19:24.341 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-21T22:19:24.342 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2023-07-21T22:19:24.399 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T22:19:24.500 DEBUG:teuthology.parallel:result is None 2023-07-21T22:19:24.576 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T22:19:24.661 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T22:19:24.851 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T22:19:24.864 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T22:19:25.010 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T22:19:25.017 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T22:19:25.038 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:19:25.038 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:59 ago on Fri 21 Jul 2023 10:17:25 PM UTC. 2023-07-21T22:19:25.153 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-21T22:19:25.154 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:19:25.329 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:19:25.330 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:19:25.567 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:19:25.767 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:19:26.322 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:19:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 1/134 2023-07-21T22:19:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2/134 2023-07-21T22:19:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 3/134 2023-07-21T22:19:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 4/134 2023-07-21T22:19:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 5/134 2023-07-21T22:19:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:19:26.323 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:19:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:19:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 9/134 2023-07-21T22:19:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 10/134 2023-07-21T22:19:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 11/134 2023-07-21T22:19:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 12/134 2023-07-21T22:19:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 13/134 2023-07-21T22:19:26.324 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 14/134 2023-07-21T22:19:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 15/134 2023-07-21T22:19:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 16/134 2023-07-21T22:19:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 17/134 2023-07-21T22:19:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 18/134 2023-07-21T22:19:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 19/134 2023-07-21T22:19:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 20/134 2023-07-21T22:19:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 21/134 2023-07-21T22:19:26.325 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 22/134 2023-07-21T22:19:26.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 23/134 2023-07-21T22:19:26.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 24/134 2023-07-21T22:19:26.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 25/134 2023-07-21T22:19:26.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 26/134 2023-07-21T22:19:26.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 27/134 2023-07-21T22:19:26.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 28/134 2023-07-21T22:19:26.326 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 29/134 2023-07-21T22:19:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 30/134 2023-07-21T22:19:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 31/134 2023-07-21T22:19:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 32/134 2023-07-21T22:19:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 33/134 2023-07-21T22:19:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 34/134 2023-07-21T22:19:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 35/134 2023-07-21T22:19:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 36/134 2023-07-21T22:19:26.327 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 37/134 2023-07-21T22:19:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 38/134 2023-07-21T22:19:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T22:19:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T22:19:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T22:19:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T22:19:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T22:19:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T22:19:26.328 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T22:19:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T22:19:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T22:19:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T22:19:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T22:19:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T22:19:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T22:19:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T22:19:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T22:19:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T22:19:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T22:19:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T22:19:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T22:19:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T22:19:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T22:19:26.330 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T22:19:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T22:19:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T22:19:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T22:19:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T22:19:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T22:19:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T22:19:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T22:19:26.331 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T22:19:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T22:19:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T22:19:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T22:19:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-21T22:19:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-21T22:19:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-21T22:19:26.332 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-21T22:19:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-21T22:19:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-21T22:19:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-21T22:19:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-21T22:19:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-21T22:19:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-21T22:19:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-21T22:19:26.333 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-21T22:19:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-21T22:19:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-21T22:19:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-21T22:19:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-21T22:19:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-21T22:19:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-21T22:19:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-21T22:19:26.334 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-21T22:19:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-21T22:19:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-21T22:19:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-21T22:19:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-21T22:19:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-21T22:19:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-21T22:19:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-21T22:19:26.335 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-21T22:19:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-21T22:19:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-21T22:19:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-21T22:19:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-21T22:19:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-21T22:19:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-21T22:19:26.336 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-21T22:19:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-21T22:19:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-21T22:19:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-21T22:19:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-21T22:19:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-21T22:19:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-21T22:19:26.337 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-21T22:19:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-21T22:19:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-21T22:19:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-21T22:19:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-21T22:19:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-21T22:19:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-21T22:19:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-21T22:19:26.338 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-21T22:19:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-21T22:19:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-21T22:19:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-21T22:19:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-21T22:19:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-21T22:19:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-21T22:19:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-21T22:19:26.339 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 129/134 2023-07-21T22:19:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T22:19:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 131/134 2023-07-21T22:19:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:19:26.340 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T22:19:28.059 INFO:teuthology.orchestra.run.smithi107.stdout:Installed products updated. 2023-07-21T22:19:28.272 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:19:28.273 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2023-07-21T22:19:28.273 INFO:teuthology.orchestra.run.smithi107.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T22:19:28.273 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.273 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.273 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2023-07-21T22:19:28.273 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T22:19:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T22:19:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T22:19:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.274 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:28.275 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.276 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T22:19:28.277 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T22:19:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T22:19:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T22:19:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: jq-1.6-3.el8.x86_64 2023-07-21T22:19:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T22:19:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T22:19:28.278 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T22:19:28.279 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.279 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.279 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.279 INFO:teuthology.orchestra.run.smithi107.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T22:19:28.280 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:19:28.280 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T22:19:28.280 INFO:teuthology.orchestra.run.smithi107.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T22:19:28.280 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:19:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T22:19:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T22:19:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T22:19:28.281 INFO:teuthology.orchestra.run.smithi107.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T22:19:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T22:19:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T22:19:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T22:19:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T22:19:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T22:19:28.282 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T22:19:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T22:19:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T22:19:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T22:19:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T22:19:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T22:19:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T22:19:28.283 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:19:28.284 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:19:28.284 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:19:28.284 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T22:19:28.285 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T22:19:28.285 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T22:19:28.285 INFO:teuthology.orchestra.run.smithi107.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T22:19:28.285 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T22:19:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T22:19:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T22:19:28.286 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T22:19:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T22:19:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T22:19:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T22:19:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T22:19:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T22:19:28.287 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T22:19:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T22:19:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T22:19:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T22:19:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T22:19:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T22:19:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T22:19:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T22:19:28.288 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T22:19:28.289 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T22:19:28.289 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T22:19:28.289 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T22:19:28.289 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T22:19:28.290 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T22:19:28.290 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T22:19:28.290 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T22:19:28.290 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T22:19:28.290 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T22:19:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T22:19:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T22:19:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T22:19:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T22:19:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T22:19:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T22:19:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T22:19:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T22:19:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T22:19:28.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T22:19:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T22:19:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T22:19:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T22:19:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T22:19:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T22:19:28.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T22:19:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T22:19:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T22:19:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T22:19:28.294 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T22:19:28.295 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T22:19:28.295 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T22:19:28.295 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T22:19:28.295 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T22:19:28.296 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:19:28.296 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2023-07-21T22:19:28.469 DEBUG:teuthology.parallel:result is None 2023-07-21T22:19:28.741 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:19:28.741 INFO:teuthology.orchestra.run.smithi190.stdout:Last metadata expiration check: 0:01:52 ago on Fri 21 Jul 2023 10:17:36 PM UTC. 2023-07-21T22:19:32.175 INFO:teuthology.orchestra.run.smithi190.stdout:Installed products updated. 2023-07-21T22:19:32.386 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:19:32.386 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2023-07-21T22:19:32.386 INFO:teuthology.orchestra.run.smithi190.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T22:19:32.386 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.386 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.387 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2023-07-21T22:19:32.387 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T22:19:32.387 INFO:teuthology.orchestra.run.smithi190.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T22:19:32.387 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T22:19:32.387 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.387 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.387 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.388 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.388 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:32.388 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.388 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.388 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.389 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:32.389 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:32.389 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:32.389 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:32.389 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:32.389 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.389 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.390 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:32.390 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.390 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.390 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.390 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:32.390 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:32.390 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.391 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:19:32.391 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T22:19:32.391 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T22:19:32.391 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T22:19:32.391 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T22:19:32.391 INFO:teuthology.orchestra.run.smithi190.stdout: jq-1.6-3.el8.x86_64 2023-07-21T22:19:32.391 INFO:teuthology.orchestra.run.smithi190.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T22:19:32.392 INFO:teuthology.orchestra.run.smithi190.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T22:19:32.392 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T22:19:32.392 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.392 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.392 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.392 INFO:teuthology.orchestra.run.smithi190.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T22:19:32.392 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:19:32.393 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T22:19:32.393 INFO:teuthology.orchestra.run.smithi190.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T22:19:32.393 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:19:32.393 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T22:19:32.393 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.393 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.393 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T22:19:32.394 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.394 INFO:teuthology.orchestra.run.smithi190.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T22:19:32.394 INFO:teuthology.orchestra.run.smithi190.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T22:19:32.394 INFO:teuthology.orchestra.run.smithi190.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T22:19:32.394 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T22:19:32.395 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T22:19:32.395 INFO:teuthology.orchestra.run.smithi190.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T22:19:32.395 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T22:19:32.395 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T22:19:32.395 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T22:19:32.395 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T22:19:32.395 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T22:19:32.395 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T22:19:32.396 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T22:19:32.396 INFO:teuthology.orchestra.run.smithi190.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T22:19:32.396 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:19:32.396 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:19:32.396 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:19:32.396 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T22:19:32.397 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T22:19:32.397 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T22:19:32.397 INFO:teuthology.orchestra.run.smithi190.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T22:19:32.397 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T22:19:32.397 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.397 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.397 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.397 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T22:19:32.398 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T22:19:32.398 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T22:19:32.398 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T22:19:32.398 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T22:19:32.398 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T22:19:32.398 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T22:19:32.398 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T22:19:32.399 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T22:19:32.399 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T22:19:32.399 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T22:19:32.399 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T22:19:32.399 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T22:19:32.399 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T22:19:32.399 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T22:19:32.399 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T22:19:32.400 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T22:19:32.400 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T22:19:32.400 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T22:19:32.400 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T22:19:32.400 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T22:19:32.400 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T22:19:32.400 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T22:19:32.400 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T22:19:32.401 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T22:19:32.401 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T22:19:32.401 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T22:19:32.401 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T22:19:32.401 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T22:19:32.401 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.401 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.402 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T22:19:32.402 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T22:19:32.402 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.402 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T22:19:32.402 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T22:19:32.402 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T22:19:32.402 INFO:teuthology.orchestra.run.smithi190.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T22:19:32.403 INFO:teuthology.orchestra.run.smithi190.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T22:19:32.403 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T22:19:32.403 INFO:teuthology.orchestra.run.smithi190.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T22:19:32.403 INFO:teuthology.orchestra.run.smithi190.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T22:19:32.403 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T22:19:32.403 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T22:19:32.404 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T22:19:32.404 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T22:19:32.404 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T22:19:32.404 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.404 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.404 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:19:32.404 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T22:19:32.404 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T22:19:32.405 INFO:teuthology.orchestra.run.smithi190.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T22:19:32.405 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T22:19:32.405 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T22:19:32.405 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T22:19:32.406 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:19:32.406 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2023-07-21T22:19:32.567 DEBUG:teuthology.parallel:result is None 2023-07-21T22:19:32.567 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:19:38.251 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T22:19:38.280 INFO:teuthology.orchestra.run.smithi107.stdout:18.0.0-4735.gbffe049d.el8 2023-07-21T22:19:38.281 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4735.gbffe049d.el8 2023-07-21T22:19:38.281 INFO:teuthology.task.install:The correct ceph version 18.0.0-4735.gbffe049d is installed. 2023-07-21T22:19:38.281 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:19:38.806 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T22:19:38.836 INFO:teuthology.orchestra.run.smithi132.stdout:18.0.0-4735.gbffe049d.el8 2023-07-21T22:19:38.836 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4735.gbffe049d.el8 2023-07-21T22:19:38.837 INFO:teuthology.task.install:The correct ceph version 18.0.0-4735.gbffe049d is installed. 2023-07-21T22:19:38.838 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:19:39.277 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T22:19:39.306 INFO:teuthology.orchestra.run.smithi190.stdout:18.0.0-4735.gbffe049d.el8 2023-07-21T22:19:39.307 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4735.gbffe049d.el8 2023-07-21T22:19:39.307 INFO:teuthology.task.install:The correct ceph version 18.0.0-4735.gbffe049d is installed. 2023-07-21T22:19:39.307 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-21T22:19:39.308 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:19:39.309 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T22:19:39.355 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-07-21T22:19:39.356 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T22:19:39.406 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-07-21T22:19:39.407 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T22:19:39.458 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-21T22:19:39.459 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:19:39.459 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T22:19:39.492 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T22:19:39.561 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-07-21T22:19:39.562 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T22:19:39.593 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T22:19:39.663 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-07-21T22:19:39.663 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T22:19:39.695 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T22:19:39.766 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-21T22:19:39.767 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:19:39.768 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T22:19:39.798 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T22:19:39.866 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-07-21T22:19:39.867 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T22:19:39.897 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T22:19:39.964 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-07-21T22:19:39.965 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T22:19:39.996 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T22:19:40.149 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-21T22:19:40.149 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:19:40.149 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T22:19:40.179 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T22:19:40.248 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-07-21T22:19:40.248 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T22:19:40.277 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T22:19:40.346 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-07-21T22:19:40.346 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T22:19:40.377 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T22:19:40.450 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-21T22:19:40.551 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-07-21T22:19:40.552 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:19:40.562 INFO:tasks.cephadm:Cluster fsid is af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:19:40.563 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-21T22:19:40.563 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.107', 'mon.b': '172.21.15.132', 'mon.c': '172.21.15.190'} 2023-07-21T22:19:40.564 INFO:tasks.cephadm:First mon is mon.a on smithi107 2023-07-21T22:19:40.564 INFO:tasks.cephadm:First mgr is x 2023-07-21T22:19:40.564 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-21T22:19:40.565 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2023-07-21T22:19:40.598 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname $(hostname -s) 2023-07-21T22:19:40.629 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname $(hostname -s) 2023-07-21T22:19:40.662 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-21T22:19:40.663 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:19:46.266 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71611/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4735-gbffe049d', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4735.gbffe049d'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-21 20:32:06.130547', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Jul13-base-2', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2023-07-21T22:19:51.787 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-rishabh-2023Jul13-base-2, sha1 bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:19:51.788 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/x86_64/flavors/default/cephadm 2023-07-21T22:19:51.789 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T22:19:53.110 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 22:19 /home/ubuntu/cephtest/cephadm 2023-07-21T22:19:53.110 DEBUG:teuthology.orchestra.run.smithi132:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T22:19:54.521 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 22:19 /home/ubuntu/cephtest/cephadm 2023-07-21T22:19:54.522 DEBUG:teuthology.orchestra.run.smithi190:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T22:19:55.926 INFO:teuthology.orchestra.run.smithi190.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 22:19 /home/ubuntu/cephtest/cephadm 2023-07-21T22:19:55.927 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T22:19:55.949 DEBUG:teuthology.orchestra.run.smithi132:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T22:19:55.971 DEBUG:teuthology.orchestra.run.smithi190:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T22:19:56.017 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea on all hosts... 2023-07-21T22:19:56.017 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea pull 2023-07-21T22:19:56.020 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea pull 2023-07-21T22:19:56.022 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea pull 2023-07-21T22:19:57.287 INFO:teuthology.orchestra.run.smithi132.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:19:57.343 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:19:57.372 INFO:teuthology.orchestra.run.smithi190.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:20:26.387 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2023-07-21T22:20:26.388 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_version": "ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev)", 2023-07-21T22:20:26.388 INFO:teuthology.orchestra.run.smithi132.stdout: "image_id": "fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c", 2023-07-21T22:20:26.388 INFO:teuthology.orchestra.run.smithi132.stdout: "repo_digests": [ 2023-07-21T22:20:26.388 INFO:teuthology.orchestra.run.smithi132.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e12d2e98545c41acbacae1458e8c994c4b4d7018583b915c7d66098e77144f43" 2023-07-21T22:20:26.388 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2023-07-21T22:20:26.388 INFO:teuthology.orchestra.run.smithi132.stdout:} 2023-07-21T22:20:46.747 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:20:46.748 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev)", 2023-07-21T22:20:46.748 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c", 2023-07-21T22:20:46.748 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2023-07-21T22:20:46.748 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e12d2e98545c41acbacae1458e8c994c4b4d7018583b915c7d66098e77144f43" 2023-07-21T22:20:46.748 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2023-07-21T22:20:46.749 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:20:49.373 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2023-07-21T22:20:49.374 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev)", 2023-07-21T22:20:49.374 INFO:teuthology.orchestra.run.smithi190.stdout: "image_id": "fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c", 2023-07-21T22:20:49.374 INFO:teuthology.orchestra.run.smithi190.stdout: "repo_digests": [ 2023-07-21T22:20:49.374 INFO:teuthology.orchestra.run.smithi190.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e12d2e98545c41acbacae1458e8c994c4b4d7018583b915c7d66098e77144f43" 2023-07-21T22:20:49.374 INFO:teuthology.orchestra.run.smithi190.stdout: ] 2023-07-21T22:20:49.374 INFO:teuthology.orchestra.run.smithi190.stdout:} 2023-07-21T22:20:49.393 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2023-07-21T22:20:49.431 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph 2023-07-21T22:20:49.469 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph 2023-07-21T22:20:49.500 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2023-07-21T22:20:49.530 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /etc/ceph 2023-07-21T22:20:49.561 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /etc/ceph 2023-07-21T22:20:49.587 INFO:tasks.cephadm:Writing seed config... 2023-07-21T22:20:49.589 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-21T22:20:49.589 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-21T22:20:49.589 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-21T22:20:49.589 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-21T22:20:49.590 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-21T22:20:49.590 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-07-21T22:20:49.590 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-21T22:20:49.590 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-21T22:20:49.590 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-21T22:20:49.591 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-21T22:20:49.591 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-21T22:20:49.591 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-21T22:20:49.591 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-21T22:20:49.591 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-21T22:20:49.592 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-21T22:20:49.592 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-21T22:20:49.592 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-21T22:20:49.592 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-21T22:20:49.592 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-21T22:20:49.593 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-21T22:20:49.593 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-21T22:20:49.593 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-21T22:20:49.593 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-21T22:20:49.593 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-21T22:20:49.593 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-21T22:20:49.594 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-21T22:20:49.594 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-21T22:20:49.594 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-07-21T22:20:49.594 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-07-21T22:20:49.594 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-07-21T22:20:49.595 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-21T22:20:49.595 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-21T22:20:49.595 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-07-21T22:20:49.595 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-07-21T22:20:49.595 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-21T22:20:49.596 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-21T22:20:49.596 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-07-21T22:20:49.596 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-21T22:20:49.596 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-21T22:20:49.596 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-21T22:20:49.597 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-21T22:20:49.597 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-21T22:20:49.597 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-21T22:20:49.597 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-21T22:20:49.597 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-07-21T22:20:49.598 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:20:49.599 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-21T22:20:49.617 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = af99f5d0-2814-11ee-9b34-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-21T22:20:49.617 DEBUG:teuthology.orchestra.run.smithi107:mon.a> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.a.service 2023-07-21T22:20:49.659 DEBUG:teuthology.orchestra.run.smithi107:mgr.x> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.x.service 2023-07-21T22:20:49.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:-- Logs begin at Fri 2023-07-21 22:03:40 UTC. -- 2023-07-21T22:20:49.690 INFO:tasks.cephadm:Bootstrapping... 2023-07-21T22:20:49.690 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea -v bootstrap --fsid af99f5d0-2814-11ee-9b34-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.107 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:20:49.718 INFO:journalctl@ceph.mgr.x.smithi107.stdout:-- Logs begin at Fri 2023-07-21 22:03:40 UTC. -- 2023-07-21T22:20:49.853 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:20:49.854 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', '-v', 'bootstrap', '--fsid', 'af99f5d0-2814-11ee-9b34-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.107', '--skip-admin-label'] 2023-07-21T22:20:49.998 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.1.1 2023-07-21T22:20:49.999 INFO:teuthology.orchestra.run.smithi107.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-21T22:20:49.999 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying podman|docker is present... 2023-07-21T22:20:50.141 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.1.1 2023-07-21T22:20:50.141 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying lvm2 is present... 2023-07-21T22:20:50.142 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying time synchronization is in place... 2023-07-21T22:20:50.148 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-21T22:20:50.148 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-21T22:20:50.154 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-21T22:20:50.154 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2023-07-21T22:20:50.160 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2023-07-21T22:20:50.166 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2023-07-21T22:20:50.166 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2023-07-21T22:20:50.166 INFO:teuthology.orchestra.run.smithi107.stdout:Repeating the final host check... 2023-07-21T22:20:50.307 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.1.1 2023-07-21T22:20:50.307 INFO:teuthology.orchestra.run.smithi107.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-21T22:20:50.308 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl is present 2023-07-21T22:20:50.308 INFO:teuthology.orchestra.run.smithi107.stdout:lvcreate is present 2023-07-21T22:20:50.312 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-21T22:20:50.313 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-21T22:20:50.318 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-21T22:20:50.318 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2023-07-21T22:20:50.323 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2023-07-21T22:20:50.330 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2023-07-21T22:20:50.330 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2023-07-21T22:20:50.331 INFO:teuthology.orchestra.run.smithi107.stdout:Host looks OK 2023-07-21T22:20:50.331 INFO:teuthology.orchestra.run.smithi107.stdout:Cluster fsid: af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:20:50.331 INFO:teuthology.orchestra.run.smithi107.stdout:Acquiring lock 139725115795440 on /run/cephadm/af99f5d0-2814-11ee-9b34-001a4aab830c.lock 2023-07-21T22:20:50.331 INFO:teuthology.orchestra.run.smithi107.stdout:Lock 139725115795440 acquired on /run/cephadm/af99f5d0-2814-11ee-9b34-001a4aab830c.lock 2023-07-21T22:20:50.331 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 3300 ... 2023-07-21T22:20:50.332 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 6789 ... 2023-07-21T22:20:50.332 INFO:teuthology.orchestra.run.smithi107.stdout:Base mon IP(s) is [172.21.15.107:3300, 172.21.15.107:6789], mon addrv is [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2023-07-21T22:20:50.334 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-07-21T22:20:50.334 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.107 metric 100 2023-07-21T22:20:50.337 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-21T22:20:50.337 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-07-21T22:20:50.337 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-07-21T22:20:50.339 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-21T22:20:50.339 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-21T22:20:50.340 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-21T22:20:50.340 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-07-21T22:20:50.340 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 fe80::a7c:b29c:9445:8548/64 scope link noprefixroute 2023-07-21T22:20:50.341 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-21T22:20:50.342 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2023-07-21T22:20:50.342 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2023-07-21T22:20:50.342 INFO:teuthology.orchestra.run.smithi107.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-21T22:20:50.343 INFO:teuthology.orchestra.run.smithi107.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-21T22:20:50.343 INFO:teuthology.orchestra.run.smithi107.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:20:50.785 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c 2023-07-21T22:20:50.785 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:20:50.786 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Getting image source signatures 2023-07-21T22:20:50.786 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:99ee8d5f86f7949582f207d2e584c4024ff3caa1cfdcb812f3da7de6a1ca5e74 2023-07-21T22:20:50.786 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-07-21T22:20:50.786 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying config sha256:fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c 2023-07-21T22:20:50.786 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-21T22:20:50.786 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Storing signatures 2023-07-21T22:20:51.636 INFO:teuthology.orchestra.run.smithi107.stdout:ceph: stdout ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev) 2023-07-21T22:20:51.637 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph version: ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev) 2023-07-21T22:20:51.637 INFO:teuthology.orchestra.run.smithi107.stdout:Extracting ceph user uid/gid from container image... 2023-07-21T22:20:52.281 INFO:teuthology.orchestra.run.smithi107.stdout:stat: stdout 167 167 2023-07-21T22:20:52.281 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial keys... 2023-07-21T22:20:52.939 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQDEBLtkGIsrIBAAacHxhpofaVq+fNkmywd+TA== 2023-07-21T22:20:53.588 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQDFBLtkElE/DxAAkNYJUADfys1IsSU+QYbl4Q== 2023-07-21T22:20:54.241 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQDFBLtkJ/ZRMxAAE1r1AImd5NSY7NlAhNmFTQ== 2023-07-21T22:20:54.242 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial monmap... 2023-07-21T22:20:54.988 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-21T22:20:54.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-21T22:20:54.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:20:54.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-21T22:20:54.989 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool for a [v2:172.21.15.107:3300,v1:172.21.15.107:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-21T22:20:54.989 INFO:teuthology.orchestra.run.smithi107.stdout:setting min_mon_release = pacific 2023-07-21T22:20:54.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: set fsid to af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:20:54.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-21T22:20:54.990 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:20:54.990 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mon... 2023-07-21T22:20:55.734 INFO:teuthology.orchestra.run.smithi107.stdout:create mon.a on 2023-07-21T22:20:55.859 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-21T22:20:55.943 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-21T22:20:56.052 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-af99f5d0-2814-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-af99f5d0-2814-11ee-9b34-001a4aab830c.target. 2023-07-21T22:20:56.053 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-af99f5d0-2814-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-af99f5d0-2814-11ee-9b34-001a4aab830c.target. 2023-07-21T22:20:56.209 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.a 2023-07-21T22:20:56.209 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.a.service: Unit ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.a.service not loaded. 2023-07-21T22:20:56.298 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-af99f5d0-2814-11ee-9b34-001a4aab830c.target.wants/ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@.service. 2023-07-21T22:20:56.597 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:56 smithi107 systemd[1]: Starting Ceph mon.a for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:20:56.848 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:56 smithi107 podman[96008]: 2023-07-21T22:20:56.848 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:56 smithi107 bash[96008]: ca29012fdf7bd1ada503e11bc5fc7ae109629291dc30ee38ffc8ece8e013945c 2023-07-21T22:20:56.849 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:56 smithi107 ceph-mon[96048]: mkfs af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:20:56.849 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:56 smithi107 ceph-mon[96048]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T22:20:56.900 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T22:20:56.901 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2023-07-21T22:20:56.907 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T22:20:56.909 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2023-07-21T22:20:56.909 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2023-07-21T22:20:56.910 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2023-07-21T22:20:56.910 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon to start... 2023-07-21T22:20:56.910 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon... 2023-07-21T22:20:57.101 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:56 smithi107 systemd[1]: Started Ceph mon.a for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:20:57.759 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout cluster: 2023-07-21T22:20:57.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout id: af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:20:57.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-21T22:20:57.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2023-07-21T22:20:57.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout services: 2023-07-21T22:20:57.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.588404s) 2023-07-21T22:20:57.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-21T22:20:57.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-21T22:20:57.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2023-07-21T22:20:57.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout data: 2023-07-21T22:20:57.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-21T22:20:57.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-21T22:20:57.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-21T22:20:57.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pgs: 2023-07-21T22:20:57.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2023-07-21T22:20:57.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.309+0000 7ff3110db700 1 Processor -- start 2023-07-21T22:20:57.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.310+0000 7ff3110db700 1 -- start start 2023-07-21T22:20:57.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.310+0000 7ff3110db700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 0x7ff30c104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:57.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.310+0000 7ff3110db700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c104bd0 con 0x7ff30c104280 2023-07-21T22:20:57.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.310+0000 7ff30ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 0x7ff30c104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:57.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.310+0000 7ff30ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 0x7ff30c104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45094/0 (socket says 172.21.15.107:45094) 2023-07-21T22:20:57.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.310+0000 7ff30ad9d700 1 -- 172.21.15.107:0/1940355400 learned_addr learned my addr 172.21.15.107:0/1940355400 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:20:57.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.310+0000 7ff30ad9d700 1 -- 172.21.15.107:0/1940355400 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c104d10 con 0x7ff30c104280 2023-07-21T22:20:57.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.310+0000 7ff30ad9d700 1 --2- 172.21.15.107:0/1940355400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 0x7ff30c104690 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0098a0 tx=0x7ff2fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7699e2438745b3e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:57.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.311+0000 7ff309d9b700 1 -- 172.21.15.107:0/1940355400 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2fc004160 con 0x7ff30c104280 2023-07-21T22:20:57.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.311+0000 7ff309d9b700 1 -- 172.21.15.107:0/1940355400 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc0042c0 con 0x7ff30c104280 2023-07-21T22:20:57.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.311+0000 7ff309d9b700 1 -- 172.21.15.107:0/1940355400 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2fc0045a0 con 0x7ff30c104280 2023-07-21T22:20:57.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.311+0000 7ff3110db700 1 -- 172.21.15.107:0/1940355400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 msgr2=0x7ff30c104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:57.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.311+0000 7ff3110db700 1 --2- 172.21.15.107:0/1940355400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 0x7ff30c104690 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0098a0 tx=0x7ff2fc009bb0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.764 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.311+0000 7ff3110db700 1 -- 172.21.15.107:0/1940355400 shutdown_connections 2023-07-21T22:20:57.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.311+0000 7ff3110db700 1 --2- 172.21.15.107:0/1940355400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 0x7ff30c104690 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.311+0000 7ff3110db700 1 -- 172.21.15.107:0/1940355400 >> 172.21.15.107:0/1940355400 conn(0x7ff30c0ff8d0 msgr2=0x7ff30c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:57.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.311+0000 7ff3110db700 1 -- 172.21.15.107:0/1940355400 shutdown_connections 2023-07-21T22:20:57.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.311+0000 7ff3110db700 1 -- 172.21.15.107:0/1940355400 wait complete. 2023-07-21T22:20:57.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.312+0000 7ff3110db700 1 Processor -- start 2023-07-21T22:20:57.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.312+0000 7ff3110db700 1 -- start start 2023-07-21T22:20:57.765 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.313+0000 7ff3110db700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 0x7ff30c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:57.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.313+0000 7ff3110db700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c197d20 con 0x7ff30c104280 2023-07-21T22:20:57.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.313+0000 7ff30ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 0x7ff30c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:57.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.313+0000 7ff30ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 0x7ff30c1977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45096/0 (socket says 172.21.15.107:45096) 2023-07-21T22:20:57.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.313+0000 7ff30ad9d700 1 -- 172.21.15.107:0/3985284574 learned_addr learned my addr 172.21.15.107:0/3985284574 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:20:57.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.313+0000 7ff30ad9d700 1 -- 172.21.15.107:0/3985284574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc009550 con 0x7ff30c104280 2023-07-21T22:20:57.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.313+0000 7ff30ad9d700 1 --2- 172.21.15.107:0/3985284574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 0x7ff30c1977e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff30c104fb0 tx=0x7ff2fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:57.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.313+0000 7ff2f3fff700 1 -- 172.21.15.107:0/3985284574 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2fc011630 con 0x7ff30c104280 2023-07-21T22:20:57.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.313+0000 7ff2f3fff700 1 -- 172.21.15.107:0/3985284574 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc011790 con 0x7ff30c104280 2023-07-21T22:20:57.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.313+0000 7ff3110db700 1 -- 172.21.15.107:0/3985284574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30c197f20 con 0x7ff30c104280 2023-07-21T22:20:57.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.313+0000 7ff2f3fff700 1 -- 172.21.15.107:0/3985284574 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2fc011a30 con 0x7ff30c104280 2023-07-21T22:20:57.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.313+0000 7ff3110db700 1 -- 172.21.15.107:0/3985284574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30c198340 con 0x7ff30c104280 2023-07-21T22:20:57.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.314+0000 7ff2f3fff700 1 -- 172.21.15.107:0/3985284574 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff2fc022070 con 0x7ff30c104280 2023-07-21T22:20:57.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.314+0000 7ff2f3fff700 1 -- 172.21.15.107:0/3985284574 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff2fc01c450 con 0x7ff30c104280 2023-07-21T22:20:57.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.314+0000 7ff3110db700 1 -- 172.21.15.107:0/3985284574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30c191010 con 0x7ff30c104280 2023-07-21T22:20:57.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.317+0000 7ff2f3fff700 1 -- 172.21.15.107:0/3985284574 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff2fc01b450 con 0x7ff30c104280 2023-07-21T22:20:57.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.385+0000 7ff3110db700 1 -- 172.21.15.107:0/3985284574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff30c02c850 con 0x7ff30c104280 2023-07-21T22:20:57.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.386+0000 7ff2f3fff700 1 -- 172.21.15.107:0/3985284574 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7ff2fc01b5f0 con 0x7ff30c104280 2023-07-21T22:20:57.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.386+0000 7ff3110db700 1 -- 172.21.15.107:0/3985284574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 msgr2=0x7ff30c1977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:57.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.386+0000 7ff3110db700 1 --2- 172.21.15.107:0/3985284574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 0x7ff30c1977e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff30c104fb0 tx=0x7ff2fc004750 comp rx=0 tx=0).stop 2023-07-21T22:20:57.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.386+0000 7ff3110db700 1 -- 172.21.15.107:0/3985284574 shutdown_connections 2023-07-21T22:20:57.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.386+0000 7ff3110db700 1 --2- 172.21.15.107:0/3985284574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104280 0x7ff30c1977e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.386+0000 7ff3110db700 1 -- 172.21.15.107:0/3985284574 >> 172.21.15.107:0/3985284574 conn(0x7ff30c0ff8d0 msgr2=0x7ff30c18bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:57.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.387+0000 7ff3110db700 1 -- 172.21.15.107:0/3985284574 shutdown_connections 2023-07-21T22:20:57.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:57.387+0000 7ff3110db700 1 -- 172.21.15.107:0/3985284574 wait complete. 2023-07-21T22:20:57.769 INFO:teuthology.orchestra.run.smithi107.stdout:mon is available 2023-07-21T22:20:57.769 INFO:teuthology.orchestra.run.smithi107.stdout:Assimilating anything we can from ceph.conf... 2023-07-21T22:20:58.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:57 smithi107 ceph-mon[96048]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T22:20:58.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:57 smithi107 ceph-mon[96048]: monmap e1: 1 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} 2023-07-21T22:20:58.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:57 smithi107 ceph-mon[96048]: fsmap 2023-07-21T22:20:58.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:57 smithi107 ceph-mon[96048]: osdmap e1: 0 total, 0 up, 0 in 2023-07-21T22:20:58.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:57 smithi107 ceph-mon[96048]: mgrmap e1: no daemons active 2023-07-21T22:20:58.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:57 smithi107 ceph-mon[96048]: from='client.? 172.21.15.107:0/3985284574' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-21T22:20:58.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2023-07-21T22:20:58.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2023-07-21T22:20:58.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:20:58.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2023-07-21T22:20:58.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-21T22:20:58.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-21T22:20:58.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-21T22:20:58.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-21T22:20:58.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2023-07-21T22:20:58.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-21T22:20:58.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-21T22:20:58.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2023-07-21T22:20:58.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2023-07-21T22:20:58.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-21T22:20:58.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-21T22:20:58.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-21T22:20:58.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-21T22:20:58.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-21T22:20:58.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.134+0000 7fcac7ea1700 1 Processor -- start 2023-07-21T22:20:58.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.134+0000 7fcac7ea1700 1 -- start start 2023-07-21T22:20:58.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.134+0000 7fcac7ea1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 0x7fcac0106da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:58.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.134+0000 7fcac7ea1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01072e0 con 0x7fcac0106990 2023-07-21T22:20:58.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.134+0000 7fcac5c3d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 0x7fcac0106da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:58.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.134+0000 7fcac5c3d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 0x7fcac0106da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45098/0 (socket says 172.21.15.107:45098) 2023-07-21T22:20:58.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.134+0000 7fcac5c3d700 1 -- 172.21.15.107:0/1249253109 learned_addr learned my addr 172.21.15.107:0/1249253109 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:20:58.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.135+0000 7fcac5c3d700 1 -- 172.21.15.107:0/1249253109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac0107420 con 0x7fcac0106990 2023-07-21T22:20:58.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.135+0000 7fcac5c3d700 1 --2- 172.21.15.107:0/1249253109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 0x7fcac0106da0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcab40098a0 tx=0x7fcab4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c2d76d4a9265cc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:58.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.135+0000 7fcac4c3b700 1 -- 172.21.15.107:0/1249253109 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcab4004160 con 0x7fcac0106990 2023-07-21T22:20:58.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.135+0000 7fcac4c3b700 1 -- 172.21.15.107:0/1249253109 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab40042c0 con 0x7fcac0106990 2023-07-21T22:20:58.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.135+0000 7fcac4c3b700 1 -- 172.21.15.107:0/1249253109 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcab40045a0 con 0x7fcac0106990 2023-07-21T22:20:58.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.135+0000 7fcac7ea1700 1 -- 172.21.15.107:0/1249253109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 msgr2=0x7fcac0106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:58.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.135+0000 7fcac7ea1700 1 --2- 172.21.15.107:0/1249253109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 0x7fcac0106da0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcab40098a0 tx=0x7fcab4009bb0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.135+0000 7fcac7ea1700 1 -- 172.21.15.107:0/1249253109 shutdown_connections 2023-07-21T22:20:58.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.135+0000 7fcac7ea1700 1 --2- 172.21.15.107:0/1249253109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 0x7fcac0106da0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.135+0000 7fcac7ea1700 1 -- 172.21.15.107:0/1249253109 >> 172.21.15.107:0/1249253109 conn(0x7fcac00ff040 msgr2=0x7fcac0101470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:58.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.136+0000 7fcac7ea1700 1 -- 172.21.15.107:0/1249253109 shutdown_connections 2023-07-21T22:20:58.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.136+0000 7fcac7ea1700 1 -- 172.21.15.107:0/1249253109 wait complete. 2023-07-21T22:20:58.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.136+0000 7fcac7ea1700 1 Processor -- start 2023-07-21T22:20:58.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.136+0000 7fcac7ea1700 1 -- start start 2023-07-21T22:20:58.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.136+0000 7fcac7ea1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 0x7fcac01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:58.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.136+0000 7fcac7ea1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac0198df0 con 0x7fcac0106990 2023-07-21T22:20:58.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.136+0000 7fcac5c3d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 0x7fcac01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:58.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.136+0000 7fcac5c3d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 0x7fcac01988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45100/0 (socket says 172.21.15.107:45100) 2023-07-21T22:20:58.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.136+0000 7fcac5c3d700 1 -- 172.21.15.107:0/3532522646 learned_addr learned my addr 172.21.15.107:0/3532522646 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:20:58.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.137+0000 7fcac5c3d700 1 -- 172.21.15.107:0/3532522646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab4009550 con 0x7fcac0106990 2023-07-21T22:20:58.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.137+0000 7fcac5c3d700 1 --2- 172.21.15.107:0/3532522646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 0x7fcac01988b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcac0104480 tx=0x7fcab4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.137+0000 7fcab2ffd700 1 -- 172.21.15.107:0/3532522646 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcab4011630 con 0x7fcac0106990 2023-07-21T22:20:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.137+0000 7fcab2ffd700 1 -- 172.21.15.107:0/3532522646 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab4011790 con 0x7fcac0106990 2023-07-21T22:20:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.137+0000 7fcab2ffd700 1 -- 172.21.15.107:0/3532522646 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcab4011a30 con 0x7fcac0106990 2023-07-21T22:20:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.137+0000 7fcac7ea1700 1 -- 172.21.15.107:0/3532522646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac0198ff0 con 0x7fcac0106990 2023-07-21T22:20:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.137+0000 7fcac7ea1700 1 -- 172.21.15.107:0/3532522646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac0199410 con 0x7fcac0106990 2023-07-21T22:20:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.137+0000 7fcab2ffd700 1 -- 172.21.15.107:0/3532522646 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcab4022070 con 0x7fcac0106990 2023-07-21T22:20:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.137+0000 7fcab2ffd700 1 -- 172.21.15.107:0/3532522646 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcab401c450 con 0x7fcac0106990 2023-07-21T22:20:58.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.138+0000 7fcac7ea1700 1 -- 172.21.15.107:0/3532522646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac01920c0 con 0x7fcac0106990 2023-07-21T22:20:58.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.140+0000 7fcab2ffd700 1 -- 172.21.15.107:0/3532522646 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fcab401b450 con 0x7fcac0106990 2023-07-21T22:20:58.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.207+0000 7fcac7ea1700 1 -- 172.21.15.107:0/3532522646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fcac002bea0 con 0x7fcac0106990 2023-07-21T22:20:58.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.211+0000 7fcab2ffd700 1 -- 172.21.15.107:0/3532522646 <== mon.0 v2:172.21.15.107:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcab401b5f0 con 0x7fcac0106990 2023-07-21T22:20:58.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.211+0000 7fcab2ffd700 1 -- 172.21.15.107:0/3532522646 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7fcab4024630 con 0x7fcac0106990 2023-07-21T22:20:58.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.211+0000 7fcac7ea1700 1 -- 172.21.15.107:0/3532522646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 msgr2=0x7fcac01988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:58.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.211+0000 7fcac7ea1700 1 --2- 172.21.15.107:0/3532522646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 0x7fcac01988b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcac0104480 tx=0x7fcab4004750 comp rx=0 tx=0).stop 2023-07-21T22:20:58.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.212+0000 7fcac7ea1700 1 -- 172.21.15.107:0/3532522646 shutdown_connections 2023-07-21T22:20:58.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.212+0000 7fcac7ea1700 1 --2- 172.21.15.107:0/3532522646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcac0106990 0x7fcac01988b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.212+0000 7fcac7ea1700 1 -- 172.21.15.107:0/3532522646 >> 172.21.15.107:0/3532522646 conn(0x7fcac00ff040 msgr2=0x7fcac018cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:58.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.212+0000 7fcac7ea1700 1 -- 172.21.15.107:0/3532522646 shutdown_connections 2023-07-21T22:20:58.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.212+0000 7fcac7ea1700 1 -- 172.21.15.107:0/3532522646 wait complete. 2023-07-21T22:20:58.611 INFO:teuthology.orchestra.run.smithi107.stdout:Generating new minimal ceph.conf... 2023-07-21T22:20:59.137 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:58 smithi107 ceph-mon[96048]: from='client.? 172.21.15.107:0/3532522646' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-21T22:20:59.137 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:58 smithi107 ceph-mon[96048]: from='client.? 172.21.15.107:0/3532522646' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-21T22:20:59.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.995+0000 7fa579bef700 1 Processor -- start 2023-07-21T22:20:59.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.995+0000 7fa579bef700 1 -- start start 2023-07-21T22:20:59.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.995+0000 7fa579bef700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 0x7fa5741082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:59.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.995+0000 7fa579bef700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa574073be0 con 0x7fa574105ed0 2023-07-21T22:20:59.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.995+0000 7fa5737fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 0x7fa5741082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:59.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.995+0000 7fa5737fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 0x7fa5741082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45102/0 (socket says 172.21.15.107:45102) 2023-07-21T22:20:59.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.995+0000 7fa5737fe700 1 -- 172.21.15.107:0/2189467051 learned_addr learned my addr 172.21.15.107:0/2189467051 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:20:59.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.996+0000 7fa5737fe700 1 -- 172.21.15.107:0/2189467051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa574073d20 con 0x7fa574105ed0 2023-07-21T22:20:59.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.996+0000 7fa5737fe700 1 --2- 172.21.15.107:0/2189467051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 0x7fa5741082b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa5640098a0 tx=0x7fa564009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d668c2c99676c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:59.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.996+0000 7fa5727fc700 1 -- 172.21.15.107:0/2189467051 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa564004160 con 0x7fa574105ed0 2023-07-21T22:20:59.543 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.996+0000 7fa5727fc700 1 -- 172.21.15.107:0/2189467051 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa5640042c0 con 0x7fa574105ed0 2023-07-21T22:20:59.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.996+0000 7fa5727fc700 1 -- 172.21.15.107:0/2189467051 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa564011a00 con 0x7fa574105ed0 2023-07-21T22:20:59.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.996+0000 7fa579bef700 1 -- 172.21.15.107:0/2189467051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 msgr2=0x7fa5741082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:59.544 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.996+0000 7fa579bef700 1 --2- 172.21.15.107:0/2189467051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 0x7fa5741082b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa5640098a0 tx=0x7fa564009bb0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.996+0000 7fa579bef700 1 -- 172.21.15.107:0/2189467051 shutdown_connections 2023-07-21T22:20:59.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.996+0000 7fa579bef700 1 --2- 172.21.15.107:0/2189467051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 0x7fa5741082b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.996+0000 7fa579bef700 1 -- 172.21.15.107:0/2189467051 >> 172.21.15.107:0/2189467051 conn(0x7fa5740ff8d0 msgr2=0x7fa574101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:59.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.996+0000 7fa579bef700 1 -- 172.21.15.107:0/2189467051 shutdown_connections 2023-07-21T22:20:59.545 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.997+0000 7fa579bef700 1 -- 172.21.15.107:0/2189467051 wait complete. 2023-07-21T22:20:59.546 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.997+0000 7fa579bef700 1 Processor -- start 2023-07-21T22:20:59.546 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.997+0000 7fa579bef700 1 -- start start 2023-07-21T22:20:59.546 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.997+0000 7fa579bef700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 0x7fa574105830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:59.546 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.997+0000 7fa579bef700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa574105d70 con 0x7fa574105ed0 2023-07-21T22:20:59.547 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.997+0000 7fa5737fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 0x7fa574105830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:59.547 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.997+0000 7fa5737fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 0x7fa574105830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45104/0 (socket says 172.21.15.107:45104) 2023-07-21T22:20:59.547 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.998+0000 7fa5737fe700 1 -- 172.21.15.107:0/369763227 learned_addr learned my addr 172.21.15.107:0/369763227 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:20:59.548 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.998+0000 7fa5737fe700 1 -- 172.21.15.107:0/369763227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa564009550 con 0x7fa574105ed0 2023-07-21T22:20:59.548 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.998+0000 7fa5737fe700 1 --2- 172.21.15.107:0/369763227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 0x7fa574105830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa564006af0 tx=0x7fa564011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:59.548 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.998+0000 7fa570ff9700 1 -- 172.21.15.107:0/369763227 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa564003a40 con 0x7fa574105ed0 2023-07-21T22:20:59.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.998+0000 7fa570ff9700 1 -- 172.21.15.107:0/369763227 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa564003ba0 con 0x7fa574105ed0 2023-07-21T22:20:59.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.998+0000 7fa579bef700 1 -- 172.21.15.107:0/369763227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa574103ee0 con 0x7fa574105ed0 2023-07-21T22:20:59.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.998+0000 7fa570ff9700 1 -- 172.21.15.107:0/369763227 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa56401a700 con 0x7fa574105ed0 2023-07-21T22:20:59.549 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.998+0000 7fa579bef700 1 -- 172.21.15.107:0/369763227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa574104360 con 0x7fa574105ed0 2023-07-21T22:20:59.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.998+0000 7fa570ff9700 1 -- 172.21.15.107:0/369763227 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa56401ac80 con 0x7fa574105ed0 2023-07-21T22:20:59.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.998+0000 7fa570ff9700 1 -- 172.21.15.107:0/369763227 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa564019700 con 0x7fa574105ed0 2023-07-21T22:20:59.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:58.999+0000 7fa579bef700 1 -- 172.21.15.107:0/369763227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5741093f0 con 0x7fa574105ed0 2023-07-21T22:20:59.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:59.001+0000 7fa570ff9700 1 -- 172.21.15.107:0/369763227 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fa56402de40 con 0x7fa574105ed0 2023-07-21T22:20:59.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:59.067+0000 7fa579bef700 1 -- 172.21.15.107:0/369763227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa5741049c0 con 0x7fa574105ed0 2023-07-21T22:20:59.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:59.067+0000 7fa570ff9700 1 -- 172.21.15.107:0/369763227 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fa564024ad0 con 0x7fa574105ed0 2023-07-21T22:20:59.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:59.068+0000 7fa579bef700 1 -- 172.21.15.107:0/369763227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 msgr2=0x7fa574105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:59.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:59.068+0000 7fa579bef700 1 --2- 172.21.15.107:0/369763227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 0x7fa574105830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa564006af0 tx=0x7fa564011ad0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:59.068+0000 7fa579bef700 1 -- 172.21.15.107:0/369763227 shutdown_connections 2023-07-21T22:20:59.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:59.068+0000 7fa579bef700 1 --2- 172.21.15.107:0/369763227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574105ed0 0x7fa574105830 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:59.068+0000 7fa579bef700 1 -- 172.21.15.107:0/369763227 >> 172.21.15.107:0/369763227 conn(0x7fa5740ff8d0 msgr2=0x7fa574100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:59.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:59.068+0000 7fa579bef700 1 -- 172.21.15.107:0/369763227 shutdown_connections 2023-07-21T22:20:59.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:20:59.068+0000 7fa579bef700 1 -- 172.21.15.107:0/369763227 wait complete. 2023-07-21T22:20:59.553 INFO:teuthology.orchestra.run.smithi107.stdout:Restarting the monitor... 2023-07-21T22:20:59.850 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:59 smithi107 systemd[1]: Stopping Ceph mon.a for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:21:00.116 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:59 smithi107 ceph-mon[96048]: from='client.? 172.21.15.107:0/369763227' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:21:00.116 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:59 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-a[96024]: 2023-07-21T22:20:59.823+0000 7f2c63793700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:21:00.117 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:20:59 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-a[96024]: 2023-07-21T22:20:59.823+0000 7f2c63793700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-21T22:21:00.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 bash[96379]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-a 2023-07-21T22:21:00.772 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 systemd[1]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.a.service: Succeeded. 2023-07-21T22:21:00.772 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 systemd[1]: Stopped Ceph mon.a for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:21:00.773 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 systemd[1]: Starting Ceph mon.a for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:21:00.993 INFO:teuthology.orchestra.run.smithi107.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-21T22:21:01.028 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 podman[96503]: 2023-07-21T22:21:01.029 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T22:21:01.029 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev), process ceph-mon, pid 7 2023-07-21T22:21:01.029 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: pidfile_write: ignore empty --pid-file 2023-07-21T22:21:01.029 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: load: jerasure load: lrc 2023-07-21T22:21:01.030 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 bash[96503]: 1b44ab1ea50546677d37156ca0e14f9af46db7c1678e68c64c8c88fbae7d0648 2023-07-21T22:21:01.030 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: RocksDB version: 7.9.2 2023-07-21T22:21:01.030 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Git sha 0 2023-07-21T22:21:01.030 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Compile date 2023-07-21 19:07:44 2023-07-21T22:21:01.030 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: DB SUMMARY 2023-07-21T22:21:01.030 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: DB Session ID: KAWMIJEGQ99732Y9RUGY 2023-07-21T22:21:01.030 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: CURRENT file: CURRENT 2023-07-21T22:21:01.031 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T22:21:01.031 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-21T22:21:01.031 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-21T22:21:01.031 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99075 ; 2023-07-21T22:21:01.031 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.error_if_exists: 0 2023-07-21T22:21:01.031 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.create_if_missing: 0 2023-07-21T22:21:01.031 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.paranoid_checks: 1 2023-07-21T22:21:01.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T22:21:01.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T22:21:01.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T22:21:01.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.env: 0x55833c754720 2023-07-21T22:21:01.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T22:21:01.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.info_log: 0x55833eab5260 2023-07-21T22:21:01.032 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T22:21:01.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.statistics: (nil) 2023-07-21T22:21:01.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.use_fsync: 0 2023-07-21T22:21:01.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_log_file_size: 0 2023-07-21T22:21:01.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T22:21:01.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T22:21:01.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T22:21:01.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T22:21:01.033 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.allow_fallocate: 1 2023-07-21T22:21:01.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T22:21:01.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T22:21:01.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.use_direct_reads: 0 2023-07-21T22:21:01.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T22:21:01.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T22:21:01.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.db_log_dir: 2023-07-21T22:21:01.034 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.wal_dir: 2023-07-21T22:21:01.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T22:21:01.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T22:21:01.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T22:21:01.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T22:21:01.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T22:21:01.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T22:21:01.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T22:21:01.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T22:21:01.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.write_buffer_manager: 0x55833e9d63c0 2023-07-21T22:21:01.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T22:21:01.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T22:21:01.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T22:21:01.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T22:21:01.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T22:21:01.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T22:21:01.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T22:21:01.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T22:21:01.037 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.unordered_write: 0 2023-07-21T22:21:01.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T22:21:01.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T22:21:01.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T22:21:01.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T22:21:01.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.row_cache: None 2023-07-21T22:21:01.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.wal_filter: None 2023-07-21T22:21:01.038 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T22:21:01.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T22:21:01.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.two_write_queues: 0 2023-07-21T22:21:01.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T22:21:01.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.wal_compression: 0 2023-07-21T22:21:01.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.atomic_flush: 0 2023-07-21T22:21:01.039 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T22:21:01.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T22:21:01.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T22:21:01.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.log_readahead_size: 0 2023-07-21T22:21:01.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T22:21:01.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T22:21:01.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T22:21:01.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T22:21:01.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T22:21:01.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T22:21:01.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T22:21:01.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_background_jobs: 2 2023-07-21T22:21:01.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_background_compactions: -1 2023-07-21T22:21:01.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_subcompactions: 1 2023-07-21T22:21:01.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T22:21:01.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T22:21:01.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T22:21:01.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T22:21:01.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T22:21:01.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T22:21:01.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T22:21:01.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T22:21:01.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_open_files: -1 2023-07-21T22:21:01.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T22:21:01.044 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T22:21:01.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T22:21:01.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T22:21:01.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_background_flushes: -1 2023-07-21T22:21:01.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Compression algorithms supported: 2023-07-21T22:21:01.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T22:21:01.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: kZSTD supported: 0 2023-07-21T22:21:01.045 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: kXpressCompression supported: 0 2023-07-21T22:21:01.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T22:21:01.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: kZlibCompression supported: 1 2023-07-21T22:21:01.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: kSnappyCompression supported: 1 2023-07-21T22:21:01.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: kLZ4Compression supported: 1 2023-07-21T22:21:01.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: kBZip2Compression supported: 0 2023-07-21T22:21:01.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T22:21:01.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T22:21:01.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-21T22:21:01.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T22:21:01.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T22:21:01.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.merge_operator: 2023-07-21T22:21:01.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compaction_filter: None 2023-07-21T22:21:01.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T22:21:01.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T22:21:01.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T22:21:01.048 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T22:21:01.048 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55833eab5360) 2023-07-21T22:21:01.048 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks: 1 2023-07-21T22:21:01.048 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T22:21:01.048 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T22:21:01.048 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_top_level_index_and_filter: 1 2023-07-21T22:21:01.048 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_type: 0 2023-07-21T22:21:01.049 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_index_type: 0 2023-07-21T22:21:01.049 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_shortening: 1 2023-07-21T22:21:01.049 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T22:21:01.049 INFO:journalctl@ceph.mon.a.smithi107.stdout: checksum: 4 2023-07-21T22:21:01.049 INFO:journalctl@ceph.mon.a.smithi107.stdout: no_block_cache: 0 2023-07-21T22:21:01.049 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache: 0x55833ea751f0 2023-07-21T22:21:01.049 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_name: BinnedLRUCache 2023-07-21T22:21:01.050 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_options: 2023-07-21T22:21:01.050 INFO:journalctl@ceph.mon.a.smithi107.stdout: capacity : 536870912 2023-07-21T22:21:01.050 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_shard_bits : 4 2023-07-21T22:21:01.050 INFO:journalctl@ceph.mon.a.smithi107.stdout: strict_capacity_limit : 0 2023-07-21T22:21:01.050 INFO:journalctl@ceph.mon.a.smithi107.stdout: high_pri_pool_ratio: 0.000 2023-07-21T22:21:01.050 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_compressed: (nil) 2023-07-21T22:21:01.051 INFO:journalctl@ceph.mon.a.smithi107.stdout: persistent_cache: (nil) 2023-07-21T22:21:01.051 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size: 4096 2023-07-21T22:21:01.051 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size_deviation: 10 2023-07-21T22:21:01.051 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_restart_interval: 16 2023-07-21T22:21:01.051 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_block_restart_interval: 1 2023-07-21T22:21:01.051 INFO:journalctl@ceph.mon.a.smithi107.stdout: metadata_block_size: 4096 2023-07-21T22:21:01.051 INFO:journalctl@ceph.mon.a.smithi107.stdout: partition_filters: 0 2023-07-21T22:21:01.052 INFO:journalctl@ceph.mon.a.smithi107.stdout: use_delta_encoding: 1 2023-07-21T22:21:01.052 INFO:journalctl@ceph.mon.a.smithi107.stdout: filter_policy: bloomfilter 2023-07-21T22:21:01.052 INFO:journalctl@ceph.mon.a.smithi107.stdout: whole_key_filtering: 1 2023-07-21T22:21:01.052 INFO:journalctl@ceph.mon.a.smithi107.stdout: verify_compression: 0 2023-07-21T22:21:01.052 INFO:journalctl@ceph.mon.a.smithi107.stdout: read_amp_bytes_per_bit: 0 2023-07-21T22:21:01.052 INFO:journalctl@ceph.mon.a.smithi107.stdout: format_version: 5 2023-07-21T22:21:01.052 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_index_compression: 1 2023-07-21T22:21:01.053 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_align: 0 2023-07-21T22:21:01.053 INFO:journalctl@ceph.mon.a.smithi107.stdout: max_auto_readahead_size: 262144 2023-07-21T22:21:01.053 INFO:journalctl@ceph.mon.a.smithi107.stdout: prepopulate_block_cache: 0 2023-07-21T22:21:01.053 INFO:journalctl@ceph.mon.a.smithi107.stdout: initial_auto_readahead_size: 8192 2023-07-21T22:21:01.053 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T22:21:01.053 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T22:21:01.053 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T22:21:01.053 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compression: NoCompression 2023-07-21T22:21:01.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T22:21:01.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T22:21:01.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T22:21:01.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.num_levels: 7 2023-07-21T22:21:01.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T22:21:01.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T22:21:01.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T22:21:01.055 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T22:21:01.055 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T22:21:01.055 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T22:21:01.055 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T22:21:01.055 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:21:01.055 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T22:21:01.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T22:21:01.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:21:01.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:21:01.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T22:21:01.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T22:21:01.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T22:21:01.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T22:21:01.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:21:01.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:21:01.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T22:21:01.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T22:21:01.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:21:01.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T22:21:01.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T22:21:01.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T22:21:01.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T22:21:01.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T22:21:01.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T22:21:01.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T22:21:01.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T22:21:01.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T22:21:01.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T22:21:01.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T22:21:01.059 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T22:21:01.059 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T22:21:01.059 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T22:21:01.059 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T22:21:01.059 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T22:21:01.059 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T22:21:01.059 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T22:21:01.060 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T22:21:01.060 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T22:21:01.060 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T22:21:01.060 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T22:21:01.060 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T22:21:01.061 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T22:21:01.061 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T22:21:01.061 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T22:21:01.061 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T22:21:01.061 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T22:21:01.061 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T22:21:01.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T22:21:01.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T22:21:01.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T22:21:01.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.table_properties_collectors: 2023-07-21T22:21:01.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.inplace_update_support: 0 2023-07-21T22:21:01.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T22:21:01.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T22:21:01.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T22:21:01.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T22:21:01.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.bloom_locality: 0 2023-07-21T22:21:01.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.max_successive_merges: 0 2023-07-21T22:21:01.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T22:21:01.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T22:21:01.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T22:21:01.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T22:21:01.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.ttl: 2592000 2023-07-21T22:21:01.064 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T22:21:01.064 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T22:21:01.064 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T22:21:01.064 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.enable_blob_files: false 2023-07-21T22:21:01.064 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.min_blob_size: 0 2023-07-21T22:21:01.064 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T22:21:01.064 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T22:21:01.065 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T22:21:01.065 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T22:21:01.065 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T22:21:01.065 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T22:21:01.065 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T22:21:01.065 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T22:21:01.065 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-21T22:21:01.065 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-21T22:21:01.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 07d5978b-a58e-4656-ad77-593f76bff9fe 2023-07-21T22:21:01.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689978060907467, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-21T22:21:01.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-21T22:21:01.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689978060908940, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94294, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92292, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13747, "raw_average_key_size": 49, "raw_value_size": 84789, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689978060, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "07d5978b-a58e-4656-ad77-593f76bff9fe", "db_session_id": "KAWMIJEGQ99732Y9RUGY", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-21T22:21:01.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689978060909045, "job": 1, "event": "recovery_finished"} 2023-07-21T22:21:01.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-21T22:21:01.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T22:21:01.067 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55833eb0e000 2023-07-21T22:21:01.067 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: DB pointer 0x55833eafa000 2023-07-21T22:21:01.067 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T22:21:01.067 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T22:21:01.067 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** DB Stats ** 2023-07-21T22:21:01.067 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:21:01.067 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T22:21:01.067 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:21:01.068 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:21:01.068 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T22:21:01.068 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:21:01.068 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:21:01.068 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2023-07-21T22:21:01.068 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2023-07-21T22:21:01.068 INFO:journalctl@ceph.mon.a.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:21:01.069 INFO:journalctl@ceph.mon.a.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T22:21:01.069 INFO:journalctl@ceph.mon.a.smithi107.stdout: L0 2/0 93.92 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 98.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:21:01.069 INFO:journalctl@ceph.mon.a.smithi107.stdout: Sum 2/0 93.92 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 98.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:21:01.069 INFO:journalctl@ceph.mon.a.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 98.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:21:01.069 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2023-07-21T22:21:01.069 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2023-07-21T22:21:01.069 INFO:journalctl@ceph.mon.a.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:21:01.070 INFO:journalctl@ceph.mon.a.smithi107.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T22:21:01.070 INFO:journalctl@ceph.mon.a.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 98.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:21:01.070 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2023-07-21T22:21:01.070 INFO:journalctl@ceph.mon.a.smithi107.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T22:21:01.070 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2023-07-21T22:21:01.070 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:21:01.070 INFO:journalctl@ceph.mon.a.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T22:21:01.070 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T22:21:01.071 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T22:21:01.071 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T22:21:01.071 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T22:21:01.071 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative compaction: 0.00 GB write, 19.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:21:01.071 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval compaction: 0.00 GB write, 19.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:21:01.072 INFO:journalctl@ceph.mon.a.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T22:21:01.072 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache BinnedLRUCache@0x55833ea751f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-07-21T22:21:01.072 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-07-21T22:21:01.072 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2023-07-21T22:21:01.072 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T22:21:01.073 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: starting mon.a rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:21:01.073 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: mon.a@-1(???) e1 preinit fsid af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:21:01.073 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: mon.a@-1(???).mds e1 new map 2023-07-21T22:21:01.073 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: mon.a@-1(???).mds e1 print_map 2023-07-21T22:21:01.073 INFO:journalctl@ceph.mon.a.smithi107.stdout: e1 2023-07-21T22:21:01.073 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T22:21:01.073 INFO:journalctl@ceph.mon.a.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:21:01.074 INFO:journalctl@ceph.mon.a.smithi107.stdout: legacy client fscid: -1 2023-07-21T22:21:01.074 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2023-07-21T22:21:01.074 INFO:journalctl@ceph.mon.a.smithi107.stdout: No filesystems configured 2023-07-21T22:21:01.074 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T22:21:01.074 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:21:01.074 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:21:01.074 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:21:01.075 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-21T22:21:01.075 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: expand_channel_meta expand map: {default=false} 2023-07-21T22:21:01.075 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: expand_channel_meta from 'false' to 'false' 2023-07-21T22:21:01.075 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: expand_channel_meta expanded map: {default=false} 2023-07-21T22:21:01.075 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: expand_channel_meta expand map: {default=info} 2023-07-21T22:21:01.075 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: expand_channel_meta from 'info' to 'info' 2023-07-21T22:21:01.075 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: expand_channel_meta expanded map: {default=info} 2023-07-21T22:21:01.075 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: expand_channel_meta expand map: {default=daemon} 2023-07-21T22:21:01.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-21T22:21:01.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: expand_channel_meta expanded map: {default=daemon} 2023-07-21T22:21:01.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: expand_channel_meta expand map: {default=debug} 2023-07-21T22:21:01.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: expand_channel_meta from 'debug' to 'debug' 2023-07-21T22:21:01.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: expand_channel_meta expanded map: {default=debug} 2023-07-21T22:21:01.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 systemd[1]: Started Ceph mon.a for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:21:01.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T22:21:01.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: monmap e1: 1 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} 2023-07-21T22:21:01.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: fsmap 2023-07-21T22:21:01.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: osdmap e1: 0 total, 0 up, 0 in 2023-07-21T22:21:01.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:00 smithi107 ceph-mon[96543]: mgrmap e1: no daemons active 2023-07-21T22:21:01.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.392+0000 7feedb6f3700 1 Processor -- start 2023-07-21T22:21:01.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.392+0000 7feedb6f3700 1 -- start start 2023-07-21T22:21:01.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.392+0000 7feedb6f3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 0x7feed4105790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:01.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.392+0000 7feedb6f3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed4105cd0 con 0x7feed4105380 2023-07-21T22:21:01.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.393+0000 7feed948f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 0x7feed4105790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:01.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.393+0000 7feed948f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 0x7feed4105790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45106/0 (socket says 172.21.15.107:45106) 2023-07-21T22:21:01.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.393+0000 7feed948f700 1 -- 172.21.15.107:0/2546308524 learned_addr learned my addr 172.21.15.107:0/2546308524 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:01.843 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.393+0000 7feed948f700 1 -- 172.21.15.107:0/2546308524 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed4105e10 con 0x7feed4105380 2023-07-21T22:21:01.843 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.393+0000 7feed948f700 1 --2- 172.21.15.107:0/2546308524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 0x7feed4105790 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7feed00098a0 tx=0x7feed0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e89141558ff3a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:01.843 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.393+0000 7feecbfff700 1 -- 172.21.15.107:0/2546308524 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feed0004160 con 0x7feed4105380 2023-07-21T22:21:01.843 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.393+0000 7feecbfff700 1 -- 172.21.15.107:0/2546308524 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feed00042c0 con 0x7feed4105380 2023-07-21T22:21:01.843 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.393+0000 7feecbfff700 1 -- 172.21.15.107:0/2546308524 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feed0011a00 con 0x7feed4105380 2023-07-21T22:21:01.843 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.394+0000 7feedb6f3700 1 -- 172.21.15.107:0/2546308524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 msgr2=0x7feed4105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:01.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.394+0000 7feedb6f3700 1 --2- 172.21.15.107:0/2546308524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 0x7feed4105790 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7feed00098a0 tx=0x7feed0009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.394+0000 7feedb6f3700 1 -- 172.21.15.107:0/2546308524 shutdown_connections 2023-07-21T22:21:01.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.394+0000 7feedb6f3700 1 --2- 172.21.15.107:0/2546308524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 0x7feed4105790 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.394+0000 7feedb6f3700 1 -- 172.21.15.107:0/2546308524 >> 172.21.15.107:0/2546308524 conn(0x7feed41009f0 msgr2=0x7feed4102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:01.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.394+0000 7feedb6f3700 1 -- 172.21.15.107:0/2546308524 shutdown_connections 2023-07-21T22:21:01.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.394+0000 7feedb6f3700 1 -- 172.21.15.107:0/2546308524 wait complete. 2023-07-21T22:21:01.844 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.395+0000 7feedb6f3700 1 Processor -- start 2023-07-21T22:21:01.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.395+0000 7feedb6f3700 1 -- start start 2023-07-21T22:21:01.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.395+0000 7feedb6f3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 0x7feed41990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:01.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.395+0000 7feedb6f3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed4199610 con 0x7feed4105380 2023-07-21T22:21:01.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.395+0000 7feed948f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 0x7feed41990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:01.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.395+0000 7feed948f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 0x7feed41990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45108/0 (socket says 172.21.15.107:45108) 2023-07-21T22:21:01.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.395+0000 7feed948f700 1 -- 172.21.15.107:0/276867076 learned_addr learned my addr 172.21.15.107:0/276867076 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:01.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.395+0000 7feed948f700 1 -- 172.21.15.107:0/276867076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed0009550 con 0x7feed4105380 2023-07-21T22:21:01.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.395+0000 7feed948f700 1 --2- 172.21.15.107:0/276867076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 0x7feed41990d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7feed0009fb0 tx=0x7feed0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:01.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.396+0000 7feeca7fc700 1 -- 172.21.15.107:0/276867076 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feed0003930 con 0x7feed4105380 2023-07-21T22:21:01.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.396+0000 7feeca7fc700 1 -- 172.21.15.107:0/276867076 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feed0003a90 con 0x7feed4105380 2023-07-21T22:21:01.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.396+0000 7feedb6f3700 1 -- 172.21.15.107:0/276867076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed4199810 con 0x7feed4105380 2023-07-21T22:21:01.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.396+0000 7feeca7fc700 1 -- 172.21.15.107:0/276867076 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feed0019610 con 0x7feed4105380 2023-07-21T22:21:01.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.396+0000 7feedb6f3700 1 -- 172.21.15.107:0/276867076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed4199c30 con 0x7feed4105380 2023-07-21T22:21:01.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.396+0000 7feeca7fc700 1 -- 172.21.15.107:0/276867076 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7feed0019ba0 con 0x7feed4105380 2023-07-21T22:21:01.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.396+0000 7feeca7fc700 1 -- 172.21.15.107:0/276867076 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7feed00243f0 con 0x7feed4105380 2023-07-21T22:21:01.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.397+0000 7feedb6f3700 1 -- 172.21.15.107:0/276867076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed4109810 con 0x7feed4105380 2023-07-21T22:21:01.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.399+0000 7feeca7fc700 1 -- 172.21.15.107:0/276867076 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7feed0003c00 con 0x7feed4105380 2023-07-21T22:21:01.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.464+0000 7feedb6f3700 1 -- 172.21.15.107:0/276867076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7feed4061b40 con 0x7feed4105380 2023-07-21T22:21:01.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.469+0000 7feeca7fc700 1 -- 172.21.15.107:0/276867076 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7feed001a9d0 con 0x7feed4105380 2023-07-21T22:21:01.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.470+0000 7feedb6f3700 1 -- 172.21.15.107:0/276867076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 msgr2=0x7feed41990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:01.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.470+0000 7feedb6f3700 1 --2- 172.21.15.107:0/276867076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 0x7feed41990d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7feed0009fb0 tx=0x7feed0011ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.470+0000 7feedb6f3700 1 -- 172.21.15.107:0/276867076 shutdown_connections 2023-07-21T22:21:01.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.470+0000 7feedb6f3700 1 --2- 172.21.15.107:0/276867076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed4105380 0x7feed41990d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.470+0000 7feedb6f3700 1 -- 172.21.15.107:0/276867076 >> 172.21.15.107:0/276867076 conn(0x7feed41009f0 msgr2=0x7feed4102050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:01.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.470+0000 7feedb6f3700 1 -- 172.21.15.107:0/276867076 shutdown_connections 2023-07-21T22:21:01.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:01.470+0000 7feedb6f3700 1 -- 172.21.15.107:0/276867076 wait complete. 2023-07-21T22:21:01.850 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-21T22:21:01.898 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:21:01.898 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mgr... 2023-07-21T22:21:01.899 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 9283 ... 2023-07-21T22:21:01.899 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 8765 ... 2023-07-21T22:21:02.096 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.x 2023-07-21T22:21:02.096 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.x.service: Unit ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.x.service not loaded. 2023-07-21T22:21:02.182 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-af99f5d0-2814-11ee-9b34-001a4aab830c.target.wants/ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@.service. 2023-07-21T22:21:02.436 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:02 smithi107 systemd[1]: Starting Ceph mgr.x for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:21:02.681 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T22:21:02.681 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2023-07-21T22:21:02.688 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T22:21:02.689 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2023-07-21T22:21:02.689 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2023-07-21T22:21:02.689 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2023-07-21T22:21:02.697 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T22:21:02.697 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2023-07-21T22:21:02.704 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T22:21:02.704 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2023-07-21T22:21:02.705 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2023-07-21T22:21:02.705 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-21T22:21:02.705 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr to start... 2023-07-21T22:21:02.705 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr... 2023-07-21T22:21:02.789 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:02 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/276867076' entity='client.admin' 2023-07-21T22:21:02.789 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:02 smithi107 podman[96770]: 2023-07-21T22:21:02.789 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:02 smithi107 bash[96770]: dcfdbbf70d283185198b3563069b276832a30897412c5a2636b55ff321857d36 2023-07-21T22:21:02.789 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:02 smithi107 systemd[1]: Started Ceph mgr.x for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:21:03.062 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:02 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:02.742+0000 7fbfe65b7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:21:03.062 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:02 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:02.823+0000 7fbfe65b7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:21:03.062 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:02 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:02.919+0000 7fbfe65b7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:21:03.062 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:03 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:03.016+0000 7fbfe65b7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:21:03.321 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:03 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:03.194+0000 7fbfe65b7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:21:03.607 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:03 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3349411042' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T22:21:03.607 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:03 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:03.524+0000 7fbfe65b7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:21:03.654 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2023-07-21T22:21:03.654 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:21:03.654 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "af99f5d0-2814-11ee-9b34-001a4aab830c", 2023-07-21T22:21:03.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T22:21:03.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T22:21:03.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T22:21:03.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T22:21:03.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:03.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T22:21:03.655 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T22:21:03.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2023-07-21T22:21:03.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:21:03.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T22:21:03.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T22:21:03.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:21:03.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-07-21T22:21:03.656 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T22:21:03.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:21:03.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T22:21:03.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T22:21:03.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:03.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T22:21:03.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:21:03.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T22:21:03.657 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T22:21:03.658 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T22:21:03.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T22:21:03.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T22:21:03.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T22:21:03.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:03.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T22:21:03.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T22:21:03.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T22:21:03.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T22:21:03.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T22:21:03.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T22:21:03.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T22:21:03.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T22:21:03.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T22:21:03.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:03.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T22:21:03.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:21:03.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T22:21:03.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T22:21:03.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:03.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T22:21:03.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-21T22:21:03.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T22:21:03.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T22:21:03.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T22:21:03.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T22:21:03.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T22:21:03.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:21:03.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:21:03.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:03.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T22:21:03.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:21:03.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T22:20:56.800109+0000", 2023-07-21T22:21:03.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:21:03.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:03.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T22:21:03.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:21:03.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.168+0000 7f6e59993700 1 Processor -- start 2023-07-21T22:21:03.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.168+0000 7f6e59993700 1 -- start start 2023-07-21T22:21:03.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.168+0000 7f6e59993700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 0x7f6e54106dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.168+0000 7f6e59993700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e54107300 con 0x7f6e541069b0 2023-07-21T22:21:03.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.169+0000 7f6e52ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 0x7f6e54106dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.169+0000 7f6e52ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 0x7f6e54106dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45114/0 (socket says 172.21.15.107:45114) 2023-07-21T22:21:03.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.169+0000 7f6e52ffd700 1 -- 172.21.15.107:0/2397982653 learned_addr learned my addr 172.21.15.107:0/2397982653 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:03.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.169+0000 7f6e52ffd700 1 -- 172.21.15.107:0/2397982653 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e54107440 con 0x7f6e541069b0 2023-07-21T22:21:03.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.169+0000 7f6e52ffd700 1 --2- 172.21.15.107:0/2397982653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 0x7f6e54106dc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6e440098a0 tx=0x7f6e44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b895337b347bdeba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:03.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.169+0000 7f6e51ffb700 1 -- 172.21.15.107:0/2397982653 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e44004160 con 0x7f6e541069b0 2023-07-21T22:21:03.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.169+0000 7f6e51ffb700 1 -- 172.21.15.107:0/2397982653 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e440042c0 con 0x7f6e541069b0 2023-07-21T22:21:03.669 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.169+0000 7f6e51ffb700 1 -- 172.21.15.107:0/2397982653 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e44011a00 con 0x7f6e541069b0 2023-07-21T22:21:03.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.170+0000 7f6e59993700 1 -- 172.21.15.107:0/2397982653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 msgr2=0x7f6e54106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:03.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.170+0000 7f6e59993700 1 --2- 172.21.15.107:0/2397982653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 0x7f6e54106dc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6e440098a0 tx=0x7f6e44009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.170+0000 7f6e59993700 1 -- 172.21.15.107:0/2397982653 shutdown_connections 2023-07-21T22:21:03.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.170+0000 7f6e59993700 1 --2- 172.21.15.107:0/2397982653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 0x7f6e54106dc0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.170+0000 7f6e59993700 1 -- 172.21.15.107:0/2397982653 >> 172.21.15.107:0/2397982653 conn(0x7f6e540ff040 msgr2=0x7f6e54101470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:03.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.170+0000 7f6e59993700 1 -- 172.21.15.107:0/2397982653 shutdown_connections 2023-07-21T22:21:03.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.170+0000 7f6e59993700 1 -- 172.21.15.107:0/2397982653 wait complete. 2023-07-21T22:21:03.670 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.171+0000 7f6e59993700 1 Processor -- start 2023-07-21T22:21:03.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.171+0000 7f6e59993700 1 -- start start 2023-07-21T22:21:03.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.171+0000 7f6e59993700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 0x7f6e5419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.171+0000 7f6e59993700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5419b870 con 0x7f6e541069b0 2023-07-21T22:21:03.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.171+0000 7f6e52ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 0x7f6e5419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.171+0000 7f6e52ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 0x7f6e5419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45116/0 (socket says 172.21.15.107:45116) 2023-07-21T22:21:03.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.171+0000 7f6e52ffd700 1 -- 172.21.15.107:0/3349411042 learned_addr learned my addr 172.21.15.107:0/3349411042 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:03.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.171+0000 7f6e52ffd700 1 -- 172.21.15.107:0/3349411042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e44009550 con 0x7f6e541069b0 2023-07-21T22:21:03.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.172+0000 7f6e52ffd700 1 --2- 172.21.15.107:0/3349411042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 0x7f6e5419b330 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009fb0 tx=0x7f6e44011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:03.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.172+0000 7f6e58991700 1 -- 172.21.15.107:0/3349411042 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e44003a40 con 0x7f6e541069b0 2023-07-21T22:21:03.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.172+0000 7f6e58991700 1 -- 172.21.15.107:0/3349411042 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e44003ba0 con 0x7f6e541069b0 2023-07-21T22:21:03.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.172+0000 7f6e59993700 1 -- 172.21.15.107:0/3349411042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5419ba70 con 0x7f6e541069b0 2023-07-21T22:21:03.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.172+0000 7f6e58991700 1 -- 172.21.15.107:0/3349411042 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e4401a750 con 0x7f6e541069b0 2023-07-21T22:21:03.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.172+0000 7f6e59993700 1 -- 172.21.15.107:0/3349411042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5419be90 con 0x7f6e541069b0 2023-07-21T22:21:03.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.172+0000 7f6e58991700 1 -- 172.21.15.107:0/3349411042 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6e4401ac80 con 0x7f6e541069b0 2023-07-21T22:21:03.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.172+0000 7f6e58991700 1 -- 172.21.15.107:0/3349411042 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6e440193c0 con 0x7f6e541069b0 2023-07-21T22:21:03.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.173+0000 7f6e3a7fc700 1 -- 172.21.15.107:0/3349411042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e340052f0 con 0x7f6e541069b0 2023-07-21T22:21:03.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.175+0000 7f6e58991700 1 -- 172.21.15.107:0/3349411042 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f6e44024b60 con 0x7f6e541069b0 2023-07-21T22:21:03.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.246+0000 7f6e3a7fc700 1 -- 172.21.15.107:0/3349411042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6e34005c90 con 0x7f6e541069b0 2023-07-21T22:21:03.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.247+0000 7f6e58991700 1 -- 172.21.15.107:0/3349411042 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f6e44024d00 con 0x7f6e541069b0 2023-07-21T22:21:03.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.248+0000 7f6e3a7fc700 1 -- 172.21.15.107:0/3349411042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 msgr2=0x7f6e5419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:03.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.248+0000 7f6e3a7fc700 1 --2- 172.21.15.107:0/3349411042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 0x7f6e5419b330 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009fb0 tx=0x7f6e44011ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.248+0000 7f6e3a7fc700 1 -- 172.21.15.107:0/3349411042 shutdown_connections 2023-07-21T22:21:03.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.248+0000 7f6e3a7fc700 1 --2- 172.21.15.107:0/3349411042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e541069b0 0x7f6e5419b330 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.248+0000 7f6e3a7fc700 1 -- 172.21.15.107:0/3349411042 >> 172.21.15.107:0/3349411042 conn(0x7f6e540ff040 msgr2=0x7f6e54100720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:03.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.248+0000 7f6e3a7fc700 1 -- 172.21.15.107:0/3349411042 shutdown_connections 2023-07-21T22:21:03.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:03.248+0000 7f6e3a7fc700 1 -- 172.21.15.107:0/3349411042 wait complete. 2023-07-21T22:21:03.674 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (1/15)... 2023-07-21T22:21:03.882 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:03 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:03.616+0000 7fbfe65b7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:21:03.882 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:03 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:03.743+0000 7fbfe65b7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:21:04.686 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:03 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:03.836+0000 7fbfe65b7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:21:04.686 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:04 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:04.249+0000 7fbfe65b7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:21:05.186 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:04 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:04.826+0000 7fbfe65b7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:21:05.186 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:04 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:04.992+0000 7fbfe65b7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:21:06.067 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:05 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:05.752+0000 7fbfe65b7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:21:06.067 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:05 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:05.846+0000 7fbfe65b7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:21:06.358 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:06 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:06.022+0000 7fbfe65b7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:21:06.613 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2023-07-21T22:21:06.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:21:06.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "af99f5d0-2814-11ee-9b34-001a4aab830c", 2023-07-21T22:21:06.614 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T22:21:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T22:21:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T22:21:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T22:21:06.615 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T22:21:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T22:21:06.616 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2023-07-21T22:21:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:21:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T22:21:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T22:21:06.617 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:21:06.618 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-07-21T22:21:06.618 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T22:21:06.618 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:21:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T22:21:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T22:21:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T22:21:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:21:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T22:21:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T22:21:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T22:21:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T22:21:06.620 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T22:21:06.621 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T22:21:06.621 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:06.621 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T22:21:06.621 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T22:21:06.621 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T22:21:06.621 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T22:21:06.621 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T22:21:06.622 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T22:21:06.622 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T22:21:06.622 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T22:21:06.622 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T22:21:06.622 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:06.622 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T22:21:06.622 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:21:06.622 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T22:21:06.623 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T22:21:06.623 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:06.623 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T22:21:06.623 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-21T22:21:06.623 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T22:21:06.623 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T22:21:06.624 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T22:21:06.624 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T22:21:06.624 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T22:21:06.624 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:21:06.624 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:21:06.625 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:06.625 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T22:21:06.625 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:21:06.625 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T22:20:56.800109+0000", 2023-07-21T22:21:06.625 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:21:06.625 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:06.625 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T22:21:06.626 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:21:06.626 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.116+0000 7ff1c02af700 1 Processor -- start 2023-07-21T22:21:06.626 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.116+0000 7ff1c02af700 1 -- start start 2023-07-21T22:21:06.626 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.117+0000 7ff1c02af700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 0x7ff1b81082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:06.626 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.117+0000 7ff1c02af700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b8073be0 con 0x7ff1b8105ed0 2023-07-21T22:21:06.626 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.117+0000 7ff1be04b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 0x7ff1b81082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:06.626 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.117+0000 7ff1be04b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 0x7ff1b81082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45118/0 (socket says 172.21.15.107:45118) 2023-07-21T22:21:06.626 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.117+0000 7ff1be04b700 1 -- 172.21.15.107:0/205071268 learned_addr learned my addr 172.21.15.107:0/205071268 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:06.627 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.118+0000 7ff1be04b700 1 -- 172.21.15.107:0/205071268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8073d20 con 0x7ff1b8105ed0 2023-07-21T22:21:06.627 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.118+0000 7ff1be04b700 1 --2- 172.21.15.107:0/205071268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 0x7ff1b81082b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40098a0 tx=0x7ff1b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e571acfe7e65fa8e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:06.627 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.118+0000 7ff1bd049700 1 -- 172.21.15.107:0/205071268 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1b4004160 con 0x7ff1b8105ed0 2023-07-21T22:21:06.627 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.118+0000 7ff1bd049700 1 -- 172.21.15.107:0/205071268 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1b40042c0 con 0x7ff1b8105ed0 2023-07-21T22:21:06.627 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.118+0000 7ff1bd049700 1 -- 172.21.15.107:0/205071268 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1b4011a00 con 0x7ff1b8105ed0 2023-07-21T22:21:06.627 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.118+0000 7ff1c02af700 1 -- 172.21.15.107:0/205071268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 msgr2=0x7ff1b81082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:06.627 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.118+0000 7ff1c02af700 1 --2- 172.21.15.107:0/205071268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 0x7ff1b81082b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40098a0 tx=0x7ff1b4009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:06.628 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.119+0000 7ff1c02af700 1 -- 172.21.15.107:0/205071268 shutdown_connections 2023-07-21T22:21:06.628 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.119+0000 7ff1c02af700 1 --2- 172.21.15.107:0/205071268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 0x7ff1b81082b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:06.628 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.119+0000 7ff1c02af700 1 -- 172.21.15.107:0/205071268 >> 172.21.15.107:0/205071268 conn(0x7ff1b80ff8d0 msgr2=0x7ff1b8101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:06.628 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.119+0000 7ff1c02af700 1 -- 172.21.15.107:0/205071268 shutdown_connections 2023-07-21T22:21:06.628 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.119+0000 7ff1c02af700 1 -- 172.21.15.107:0/205071268 wait complete. 2023-07-21T22:21:06.628 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.119+0000 7ff1c02af700 1 Processor -- start 2023-07-21T22:21:06.628 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.119+0000 7ff1c02af700 1 -- start start 2023-07-21T22:21:06.629 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.119+0000 7ff1c02af700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 0x7ff1b8199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:06.629 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.119+0000 7ff1c02af700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b81995c0 con 0x7ff1b8105ed0 2023-07-21T22:21:06.629 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.119+0000 7ff1be04b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 0x7ff1b8199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:06.629 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.119+0000 7ff1be04b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 0x7ff1b8199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45120/0 (socket says 172.21.15.107:45120) 2023-07-21T22:21:06.629 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.119+0000 7ff1be04b700 1 -- 172.21.15.107:0/1518732260 learned_addr learned my addr 172.21.15.107:0/1518732260 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:06.629 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.120+0000 7ff1be04b700 1 -- 172.21.15.107:0/1518732260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b4009550 con 0x7ff1b8105ed0 2023-07-21T22:21:06.629 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.120+0000 7ff1be04b700 1 --2- 172.21.15.107:0/1518732260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 0x7ff1b8199080 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4006af0 tx=0x7ff1b4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:06.629 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.120+0000 7ff1af7fe700 1 -- 172.21.15.107:0/1518732260 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1b4003a40 con 0x7ff1b8105ed0 2023-07-21T22:21:06.630 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.120+0000 7ff1af7fe700 1 -- 172.21.15.107:0/1518732260 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1b4003ba0 con 0x7ff1b8105ed0 2023-07-21T22:21:06.630 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.120+0000 7ff1c02af700 1 -- 172.21.15.107:0/1518732260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b8199820 con 0x7ff1b8105ed0 2023-07-21T22:21:06.630 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.120+0000 7ff1af7fe700 1 -- 172.21.15.107:0/1518732260 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1b401a700 con 0x7ff1b8105ed0 2023-07-21T22:21:06.630 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.120+0000 7ff1c02af700 1 -- 172.21.15.107:0/1518732260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b8199ca0 con 0x7ff1b8105ed0 2023-07-21T22:21:06.630 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.121+0000 7ff1af7fe700 1 -- 172.21.15.107:0/1518732260 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff1b401a930 con 0x7ff1b8105ed0 2023-07-21T22:21:06.630 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.121+0000 7ff1c02af700 1 -- 172.21.15.107:0/1518732260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b81093f0 con 0x7ff1b8105ed0 2023-07-21T22:21:06.630 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.121+0000 7ff1af7fe700 1 -- 172.21.15.107:0/1518732260 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff1b402dd00 con 0x7ff1b8105ed0 2023-07-21T22:21:06.631 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.123+0000 7ff1af7fe700 1 -- 172.21.15.107:0/1518732260 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff1b40248d0 con 0x7ff1b8105ed0 2023-07-21T22:21:06.631 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.195+0000 7ff1c02af700 1 -- 172.21.15.107:0/1518732260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff1b81095e0 con 0x7ff1b8105ed0 2023-07-21T22:21:06.631 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.196+0000 7ff1af7fe700 1 -- 172.21.15.107:0/1518732260 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7ff1b401ad20 con 0x7ff1b8105ed0 2023-07-21T22:21:06.631 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.197+0000 7ff1c02af700 1 -- 172.21.15.107:0/1518732260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 msgr2=0x7ff1b8199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:06.631 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.197+0000 7ff1c02af700 1 --2- 172.21.15.107:0/1518732260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 0x7ff1b8199080 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4006af0 tx=0x7ff1b4011ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:06.631 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.197+0000 7ff1c02af700 1 -- 172.21.15.107:0/1518732260 shutdown_connections 2023-07-21T22:21:06.631 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.197+0000 7ff1c02af700 1 --2- 172.21.15.107:0/1518732260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1b8105ed0 0x7ff1b8199080 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:06.631 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.197+0000 7ff1c02af700 1 -- 172.21.15.107:0/1518732260 >> 172.21.15.107:0/1518732260 conn(0x7ff1b80ff8d0 msgr2=0x7ff1b8100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:06.632 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.197+0000 7ff1c02af700 1 -- 172.21.15.107:0/1518732260 shutdown_connections 2023-07-21T22:21:06.632 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:06.197+0000 7ff1c02af700 1 -- 172.21.15.107:0/1518732260 wait complete. 2023-07-21T22:21:06.632 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (2/15)... 2023-07-21T22:21:06.686 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:06 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:06.312+0000 7fbfe65b7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:21:06.687 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:06 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:06.542+0000 7fbfe65b7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:21:06.992 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:06 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1518732260' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T22:21:06.992 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:06 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:06.691+0000 7fbfe65b7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:21:07.436 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:06 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:06.947+0000 7fbfe65b7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:21:07.936 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:07 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:07.631+0000 7fbfe65b7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:21:07.936 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:07 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:07.759+0000 7fbfe65b7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:21:08.315 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:07 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:07.947+0000 7fbfe65b7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:21:08.315 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:08 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:08.060+0000 7fbfe65b7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:21:09.436 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:08 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:08.270+0000 7fbfe65b7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:21:09.437 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:09.122+0000 7fbfe65b7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:21:09.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-mon[96543]: Activating manager daemon x 2023-07-21T22:21:09.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-mon[96543]: mgrmap e2: x(active, starting, since 0.0030797s) 2023-07-21T22:21:09.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-mon[96543]: from='mgr.14100 172.21.15.107:0/2579467567' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:21:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-mon[96543]: from='mgr.14100 172.21.15.107:0/2579467567' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:21:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-mon[96543]: from='mgr.14100 172.21.15.107:0/2579467567' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:21:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-mon[96543]: from='mgr.14100 172.21.15.107:0/2579467567' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:21:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-mon[96543]: from='mgr.14100 172.21.15.107:0/2579467567' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:21:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-mon[96543]: Manager daemon x is now available 2023-07-21T22:21:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-mon[96543]: from='mgr.14100 172.21.15.107:0/2579467567' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:21:09.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-mon[96543]: from='mgr.14100 172.21.15.107:0/2579467567' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:21:09.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-mon[96543]: from='mgr.14100 172.21.15.107:0/2579467567' entity='mgr.x' 2023-07-21T22:21:09.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-mon[96543]: from='mgr.14100 172.21.15.107:0/2579467567' entity='mgr.x' 2023-07-21T22:21:09.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:09 smithi107 ceph-mon[96543]: from='mgr.14100 172.21.15.107:0/2579467567' entity='mgr.x' 2023-07-21T22:21:11.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:11 smithi107 ceph-mon[96543]: mgrmap e3: x(active, since 1.00737s) 2023-07-21T22:21:12.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:12 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1522345227' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T22:21:13.686 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:13 smithi107 ceph-mon[96543]: mgrmap e4: x(active, since 3s) 2023-07-21T22:21:14.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2023-07-21T22:21:14.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:21:14.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "af99f5d0-2814-11ee-9b34-001a4aab830c", 2023-07-21T22:21:14.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T22:21:14.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T22:21:14.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T22:21:14.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T22:21:14.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:14.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T22:21:14.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T22:21:14.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2023-07-21T22:21:14.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:21:14.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T22:21:14.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T22:21:14.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:21:14.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2023-07-21T22:21:14.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T22:21:14.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:21:14.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T22:21:14.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T22:21:14.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:14.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T22:21:14.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:21:14.667 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T22:21:14.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T22:21:14.668 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T22:21:14.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T22:21:14.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T22:21:14.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T22:21:14.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:14.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T22:21:14.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T22:21:14.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T22:21:14.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T22:21:14.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T22:21:14.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T22:21:14.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T22:21:14.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T22:21:14.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T22:21:14.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:14.676 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T22:21:14.676 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:21:14.676 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T22:21:14.676 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T22:21:14.676 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:14.676 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T22:21:14.676 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T22:21:14.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T22:21:14.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T22:21:14.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T22:21:14.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T22:21:14.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T22:21:14.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:21:14.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:21:14.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:14.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T22:21:14.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:21:14.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T22:20:56.800109+0000", 2023-07-21T22:21:14.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:21:14.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:21:14.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T22:21:14.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:21:14.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.310+0000 7fbefccc3700 1 Processor -- start 2023-07-21T22:21:14.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.310+0000 7fbefccc3700 1 -- start start 2023-07-21T22:21:14.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.311+0000 7fbefccc3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 0x7fbef81082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:14.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.311+0000 7fbefccc3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef8073be0 con 0x7fbef8105ed0 2023-07-21T22:21:14.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.311+0000 7fbef659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 0x7fbef81082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:14.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.311+0000 7fbef659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 0x7fbef81082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45146/0 (socket says 172.21.15.107:45146) 2023-07-21T22:21:14.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.311+0000 7fbef659c700 1 -- 172.21.15.107:0/2087458062 learned_addr learned my addr 172.21.15.107:0/2087458062 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:14.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.311+0000 7fbef659c700 1 -- 172.21.15.107:0/2087458062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef8073d20 con 0x7fbef8105ed0 2023-07-21T22:21:14.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.311+0000 7fbef659c700 1 --2- 172.21.15.107:0/2087458062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 0x7fbef81082b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbee00098a0 tx=0x7fbee0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f315360a9eccca03 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:14.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.311+0000 7fbef559a700 1 -- 172.21.15.107:0/2087458062 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbee0004160 con 0x7fbef8105ed0 2023-07-21T22:21:14.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.311+0000 7fbef559a700 1 -- 172.21.15.107:0/2087458062 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbee00042c0 con 0x7fbef8105ed0 2023-07-21T22:21:14.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.311+0000 7fbef559a700 1 -- 172.21.15.107:0/2087458062 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbee0011a00 con 0x7fbef8105ed0 2023-07-21T22:21:14.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.312+0000 7fbefccc3700 1 -- 172.21.15.107:0/2087458062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 msgr2=0x7fbef81082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:14.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.312+0000 7fbefccc3700 1 --2- 172.21.15.107:0/2087458062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 0x7fbef81082b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbee00098a0 tx=0x7fbee0009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.681 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.312+0000 7fbefccc3700 1 -- 172.21.15.107:0/2087458062 shutdown_connections 2023-07-21T22:21:14.681 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.312+0000 7fbefccc3700 1 --2- 172.21.15.107:0/2087458062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 0x7fbef81082b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.681 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.312+0000 7fbefccc3700 1 -- 172.21.15.107:0/2087458062 >> 172.21.15.107:0/2087458062 conn(0x7fbef80ff8d0 msgr2=0x7fbef8101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:14.681 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.312+0000 7fbefccc3700 1 -- 172.21.15.107:0/2087458062 shutdown_connections 2023-07-21T22:21:14.681 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.312+0000 7fbefccc3700 1 -- 172.21.15.107:0/2087458062 wait complete. 2023-07-21T22:21:14.681 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.313+0000 7fbefccc3700 1 Processor -- start 2023-07-21T22:21:14.681 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.313+0000 7fbefccc3700 1 -- start start 2023-07-21T22:21:14.681 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.313+0000 7fbefccc3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 0x7fbef8199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:14.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.313+0000 7fbefccc3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef81995c0 con 0x7fbef8105ed0 2023-07-21T22:21:14.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.313+0000 7fbef659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 0x7fbef8199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:14.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.313+0000 7fbef659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 0x7fbef8199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45148/0 (socket says 172.21.15.107:45148) 2023-07-21T22:21:14.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.313+0000 7fbef659c700 1 -- 172.21.15.107:0/1522345227 learned_addr learned my addr 172.21.15.107:0/1522345227 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:14.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.313+0000 7fbef659c700 1 -- 172.21.15.107:0/1522345227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee0009550 con 0x7fbef8105ed0 2023-07-21T22:21:14.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.314+0000 7fbef659c700 1 --2- 172.21.15.107:0/1522345227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 0x7fbef8199080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009fb0 tx=0x7fbee0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:14.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.314+0000 7fbeef7fe700 1 -- 172.21.15.107:0/1522345227 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbee0003930 con 0x7fbef8105ed0 2023-07-21T22:21:14.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.314+0000 7fbefccc3700 1 -- 172.21.15.107:0/1522345227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef8199820 con 0x7fbef8105ed0 2023-07-21T22:21:14.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.314+0000 7fbeef7fe700 1 -- 172.21.15.107:0/1522345227 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbee0003a90 con 0x7fbef8105ed0 2023-07-21T22:21:14.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.314+0000 7fbeef7fe700 1 -- 172.21.15.107:0/1522345227 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbee00195e0 con 0x7fbef8105ed0 2023-07-21T22:21:14.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.314+0000 7fbefccc3700 1 -- 172.21.15.107:0/1522345227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef8199ca0 con 0x7fbef8105ed0 2023-07-21T22:21:14.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.315+0000 7fbeef7fe700 1 -- 172.21.15.107:0/1522345227 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 3) v1 ==== 45366+0+0 (secure 0 0 0) 0x7fbee00198a0 con 0x7fbef8105ed0 2023-07-21T22:21:14.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.315+0000 7fbeef7fe700 1 --2- 172.21.15.107:0/1522345227 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7fbee4038550 0x7fbee403aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:14.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.315+0000 7fbeef7fe700 1 -- 172.21.15.107:0/1522345227 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbee004c3a0 con 0x7fbef8105ed0 2023-07-21T22:21:14.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.315+0000 7fbefccc3700 1 -- 172.21.15.107:0/1522345227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef8061b40 con 0x7fbef8105ed0 2023-07-21T22:21:14.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.315+0000 7fbef5d9b700 1 --2- 172.21.15.107:0/1522345227 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7fbee4038550 0x7fbee403aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:14.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.315+0000 7fbef5d9b700 1 --2- 172.21.15.107:0/1522345227 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7fbee4038550 0x7fbee403aa00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbee8006fd0 tx=0x7fbee8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:14.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.320+0000 7fbeef7fe700 1 -- 172.21.15.107:0/1522345227 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbee0019b50 con 0x7fbef8105ed0 2023-07-21T22:21:14.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.582+0000 7fbefccc3700 1 -- 172.21.15.107:0/1522345227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbef819c680 con 0x7fbef8105ed0 2023-07-21T22:21:14.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.583+0000 7fbeef7fe700 1 -- 172.21.15.107:0/1522345227 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fbee0018350 con 0x7fbef8105ed0 2023-07-21T22:21:14.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.585+0000 7fbefccc3700 1 -- 172.21.15.107:0/1522345227 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7fbee4038550 msgr2=0x7fbee403aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:14.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.585+0000 7fbefccc3700 1 --2- 172.21.15.107:0/1522345227 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7fbee4038550 0x7fbee403aa00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbee8006fd0 tx=0x7fbee8006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:14.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.585+0000 7fbefccc3700 1 -- 172.21.15.107:0/1522345227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 msgr2=0x7fbef8199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:14.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.585+0000 7fbefccc3700 1 --2- 172.21.15.107:0/1522345227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 0x7fbef8199080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009fb0 tx=0x7fbee0011ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.585+0000 7fbefccc3700 1 -- 172.21.15.107:0/1522345227 shutdown_connections 2023-07-21T22:21:14.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.585+0000 7fbefccc3700 1 --2- 172.21.15.107:0/1522345227 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7fbee4038550 0x7fbee403aa00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.585+0000 7fbefccc3700 1 --2- 172.21.15.107:0/1522345227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbef8105ed0 0x7fbef8199080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.585+0000 7fbefccc3700 1 -- 172.21.15.107:0/1522345227 >> 172.21.15.107:0/1522345227 conn(0x7fbef80ff8d0 msgr2=0x7fbef8100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:14.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.585+0000 7fbefccc3700 1 -- 172.21.15.107:0/1522345227 shutdown_connections 2023-07-21T22:21:14.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:11.585+0000 7fbefccc3700 1 -- 172.21.15.107:0/1522345227 wait complete. 2023-07-21T22:21:14.686 INFO:teuthology.orchestra.run.smithi107.stdout:mgr is available 2023-07-21T22:21:16.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2023-07-21T22:21:16.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2023-07-21T22:21:16.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:21:16.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-21T22:21:16.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-21T22:21:16.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-21T22:21:16.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-21T22:21:16.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2023-07-21T22:21:16.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-21T22:21:16.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-21T22:21:16.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2023-07-21T22:21:16.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2023-07-21T22:21:16.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-21T22:21:16.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-21T22:21:16.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-21T22:21:16.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-21T22:21:16.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-21T22:21:16.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.279+0000 7fe591d52700 1 Processor -- start 2023-07-21T22:21:16.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.279+0000 7fe591d52700 1 -- start start 2023-07-21T22:21:16.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.279+0000 7fe591d52700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 0x7fe58c108290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:16.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.279+0000 7fe591d52700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c079d20 con 0x7fe58c105eb0 2023-07-21T22:21:16.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.279+0000 7fe58b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 0x7fe58c108290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:16.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.279+0000 7fe58b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 0x7fe58c108290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45152/0 (socket says 172.21.15.107:45152) 2023-07-21T22:21:16.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.279+0000 7fe58b7fe700 1 -- 172.21.15.107:0/3862202683 learned_addr learned my addr 172.21.15.107:0/3862202683 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:16.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.279+0000 7fe58b7fe700 1 -- 172.21.15.107:0/3862202683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c079e60 con 0x7fe58c105eb0 2023-07-21T22:21:16.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.280+0000 7fe58b7fe700 1 --2- 172.21.15.107:0/3862202683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 0x7fe58c108290 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0098a0 tx=0x7fe57c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e040127d1bff074b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:16.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.280+0000 7fe58a7fc700 1 -- 172.21.15.107:0/3862202683 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe57c004160 con 0x7fe58c105eb0 2023-07-21T22:21:16.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.280+0000 7fe58a7fc700 1 -- 172.21.15.107:0/3862202683 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe57c0042c0 con 0x7fe58c105eb0 2023-07-21T22:21:16.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.280+0000 7fe58a7fc700 1 -- 172.21.15.107:0/3862202683 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe57c011a00 con 0x7fe58c105eb0 2023-07-21T22:21:16.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.280+0000 7fe591d52700 1 -- 172.21.15.107:0/3862202683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 msgr2=0x7fe58c108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:16.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.280+0000 7fe591d52700 1 --2- 172.21.15.107:0/3862202683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 0x7fe58c108290 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0098a0 tx=0x7fe57c009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.280+0000 7fe591d52700 1 -- 172.21.15.107:0/3862202683 shutdown_connections 2023-07-21T22:21:16.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.280+0000 7fe591d52700 1 --2- 172.21.15.107:0/3862202683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 0x7fe58c108290 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.280+0000 7fe591d52700 1 -- 172.21.15.107:0/3862202683 >> 172.21.15.107:0/3862202683 conn(0x7fe58c0ff8d0 msgr2=0x7fe58c101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:16.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.280+0000 7fe591d52700 1 -- 172.21.15.107:0/3862202683 shutdown_connections 2023-07-21T22:21:16.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.280+0000 7fe591d52700 1 -- 172.21.15.107:0/3862202683 wait complete. 2023-07-21T22:21:16.832 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.281+0000 7fe591d52700 1 Processor -- start 2023-07-21T22:21:16.832 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.281+0000 7fe591d52700 1 -- start start 2023-07-21T22:21:16.832 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.281+0000 7fe591d52700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 0x7fe58c1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:16.832 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.281+0000 7fe591d52700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c1995f0 con 0x7fe58c105eb0 2023-07-21T22:21:16.832 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.281+0000 7fe58b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 0x7fe58c1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:16.832 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.281+0000 7fe58b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 0x7fe58c1990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45154/0 (socket says 172.21.15.107:45154) 2023-07-21T22:21:16.832 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.281+0000 7fe58b7fe700 1 -- 172.21.15.107:0/320997758 learned_addr learned my addr 172.21.15.107:0/320997758 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:16.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.282+0000 7fe58b7fe700 1 -- 172.21.15.107:0/320997758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c009550 con 0x7fe58c105eb0 2023-07-21T22:21:16.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.282+0000 7fe58b7fe700 1 --2- 172.21.15.107:0/320997758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 0x7fe58c1990b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe57c006af0 tx=0x7fe57c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:16.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.282+0000 7fe588ff9700 1 -- 172.21.15.107:0/320997758 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe57c003a40 con 0x7fe58c105eb0 2023-07-21T22:21:16.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.282+0000 7fe588ff9700 1 -- 172.21.15.107:0/320997758 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe57c003ba0 con 0x7fe58c105eb0 2023-07-21T22:21:16.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.282+0000 7fe591d52700 1 -- 172.21.15.107:0/320997758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58c199850 con 0x7fe58c105eb0 2023-07-21T22:21:16.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.282+0000 7fe588ff9700 1 -- 172.21.15.107:0/320997758 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe57c01a700 con 0x7fe58c105eb0 2023-07-21T22:21:16.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.282+0000 7fe591d52700 1 -- 172.21.15.107:0/320997758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58c199cd0 con 0x7fe58c105eb0 2023-07-21T22:21:16.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.283+0000 7fe588ff9700 1 -- 172.21.15.107:0/320997758 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fe57c01a980 con 0x7fe58c105eb0 2023-07-21T22:21:16.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.283+0000 7fe588ff9700 1 --2- 172.21.15.107:0/320997758 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7fe5740386c0 0x7fe57403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:16.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.283+0000 7fe591d52700 1 -- 172.21.15.107:0/320997758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58c1093d0 con 0x7fe58c105eb0 2023-07-21T22:21:16.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.283+0000 7fe588ff9700 1 -- 172.21.15.107:0/320997758 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe57c04c650 con 0x7fe58c105eb0 2023-07-21T22:21:16.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.283+0000 7fe58affd700 1 --2- 172.21.15.107:0/320997758 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7fe5740386c0 0x7fe57403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:16.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.283+0000 7fe58affd700 1 --2- 172.21.15.107:0/320997758 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7fe5740386c0 0x7fe57403ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe580006fd0 tx=0x7fe580006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:16.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.288+0000 7fe588ff9700 1 -- 172.21.15.107:0/320997758 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe57c018b80 con 0x7fe58c105eb0 2023-07-21T22:21:16.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.470+0000 7fe591d52700 1 -- 172.21.15.107:0/320997758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe58c1095c0 con 0x7fe58c105eb0 2023-07-21T22:21:16.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.471+0000 7fe588ff9700 1 -- 172.21.15.107:0/320997758 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fe57c018350 con 0x7fe58c105eb0 2023-07-21T22:21:16.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.474+0000 7fe591d52700 1 -- 172.21.15.107:0/320997758 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7fe5740386c0 msgr2=0x7fe57403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:16.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.474+0000 7fe591d52700 1 --2- 172.21.15.107:0/320997758 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7fe5740386c0 0x7fe57403ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe580006fd0 tx=0x7fe580006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:16.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.474+0000 7fe591d52700 1 -- 172.21.15.107:0/320997758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 msgr2=0x7fe58c1990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:16.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.474+0000 7fe591d52700 1 --2- 172.21.15.107:0/320997758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 0x7fe58c1990b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe57c006af0 tx=0x7fe57c011ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.474+0000 7fe591d52700 1 -- 172.21.15.107:0/320997758 shutdown_connections 2023-07-21T22:21:16.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.474+0000 7fe591d52700 1 --2- 172.21.15.107:0/320997758 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7fe5740386c0 0x7fe57403ab70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.474+0000 7fe591d52700 1 --2- 172.21.15.107:0/320997758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe58c105eb0 0x7fe58c1990b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.474+0000 7fe591d52700 1 -- 172.21.15.107:0/320997758 >> 172.21.15.107:0/320997758 conn(0x7fe58c0ff8d0 msgr2=0x7fe58c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:16.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.474+0000 7fe591d52700 1 -- 172.21.15.107:0/320997758 shutdown_connections 2023-07-21T22:21:16.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:16.474+0000 7fe591d52700 1 -- 172.21.15.107:0/320997758 wait complete. 2023-07-21T22:21:16.836 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling cephadm module... 2023-07-21T22:21:17.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:16 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/320997758' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-21T22:21:18.298 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:17 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/4179755810' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-21T22:21:18.299 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:17 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: ignoring --setuser ceph since I am not root 2023-07-21T22:21:18.299 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:17 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: ignoring --setgroup ceph since I am not root 2023-07-21T22:21:18.299 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:18.084+0000 7fd002c7f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:21:18.299 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:18.165+0000 7fd002c7f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:21:18.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.244+0000 7f9554157700 1 Processor -- start 2023-07-21T22:21:18.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.244+0000 7f9554157700 1 -- start start 2023-07-21T22:21:18.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.244+0000 7f9554157700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 0x7f954c104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.244+0000 7f9554157700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c104bd0 con 0x7f954c104280 2023-07-21T22:21:18.371 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.244+0000 7f9551ef3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 0x7f954c104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.244+0000 7f9551ef3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 0x7f954c104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45158/0 (socket says 172.21.15.107:45158) 2023-07-21T22:21:18.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.244+0000 7f9551ef3700 1 -- 172.21.15.107:0/1090031235 learned_addr learned my addr 172.21.15.107:0/1090031235 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:18.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.244+0000 7f9551ef3700 1 -- 172.21.15.107:0/1090031235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c104d10 con 0x7f954c104280 2023-07-21T22:21:18.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.245+0000 7f9551ef3700 1 --2- 172.21.15.107:0/1090031235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 0x7f954c104690 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f954800bb40 tx=0x7f954800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bedd21bc56cb7d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:18.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.245+0000 7f9550ef1700 1 -- 172.21.15.107:0/1090031235 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f954800dbe0 con 0x7f954c104280 2023-07-21T22:21:18.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.245+0000 7f9550ef1700 1 -- 172.21.15.107:0/1090031235 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f95480044d0 con 0x7f954c104280 2023-07-21T22:21:18.372 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.245+0000 7f9550ef1700 1 -- 172.21.15.107:0/1090031235 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9548010470 con 0x7f954c104280 2023-07-21T22:21:18.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.245+0000 7f9554157700 1 -- 172.21.15.107:0/1090031235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 msgr2=0x7f954c104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.245+0000 7f9554157700 1 --2- 172.21.15.107:0/1090031235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 0x7f954c104690 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f954800bb40 tx=0x7f954800be50 comp rx=0 tx=0).stop 2023-07-21T22:21:18.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.245+0000 7f9554157700 1 -- 172.21.15.107:0/1090031235 shutdown_connections 2023-07-21T22:21:18.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.245+0000 7f9554157700 1 --2- 172.21.15.107:0/1090031235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 0x7f954c104690 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.245+0000 7f9554157700 1 -- 172.21.15.107:0/1090031235 >> 172.21.15.107:0/1090031235 conn(0x7f954c0ff8d0 msgr2=0x7f954c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:18.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.245+0000 7f9554157700 1 -- 172.21.15.107:0/1090031235 shutdown_connections 2023-07-21T22:21:18.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.245+0000 7f9554157700 1 -- 172.21.15.107:0/1090031235 wait complete. 2023-07-21T22:21:18.373 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.246+0000 7f9554157700 1 Processor -- start 2023-07-21T22:21:18.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.246+0000 7f9554157700 1 -- start start 2023-07-21T22:21:18.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.246+0000 7f9554157700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 0x7f954c196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.246+0000 7f9554157700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c1973d0 con 0x7f954c104280 2023-07-21T22:21:18.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.246+0000 7f9551ef3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 0x7f954c196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.246+0000 7f9551ef3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 0x7f954c196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45160/0 (socket says 172.21.15.107:45160) 2023-07-21T22:21:18.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.246+0000 7f9551ef3700 1 -- 172.21.15.107:0/4179755810 learned_addr learned my addr 172.21.15.107:0/4179755810 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:18.374 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.246+0000 7f9551ef3700 1 -- 172.21.15.107:0/4179755810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954800b7f0 con 0x7f954c104280 2023-07-21T22:21:18.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.247+0000 7f9551ef3700 1 --2- 172.21.15.107:0/4179755810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 0x7f954c196e90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9548003a80 tx=0x7f9548003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:18.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.247+0000 7f9542ffd700 1 -- 172.21.15.107:0/4179755810 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f954801e070 con 0x7f954c104280 2023-07-21T22:21:18.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.247+0000 7f9542ffd700 1 -- 172.21.15.107:0/4179755810 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9548004360 con 0x7f954c104280 2023-07-21T22:21:18.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.247+0000 7f9554157700 1 -- 172.21.15.107:0/4179755810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c1975d0 con 0x7f954c104280 2023-07-21T22:21:18.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.247+0000 7f9542ffd700 1 -- 172.21.15.107:0/4179755810 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9548022a50 con 0x7f954c104280 2023-07-21T22:21:18.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.247+0000 7f9554157700 1 -- 172.21.15.107:0/4179755810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954c1979f0 con 0x7f954c104280 2023-07-21T22:21:18.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.248+0000 7f9542ffd700 1 -- 172.21.15.107:0/4179755810 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f954802c440 con 0x7f954c104280 2023-07-21T22:21:18.375 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.248+0000 7f9542ffd700 1 --2- 172.21.15.107:0/4179755810 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f9538038710 0x7f953803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.248+0000 7f9542ffd700 1 -- 172.21.15.107:0/4179755810 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f954804ec70 con 0x7f954c104280 2023-07-21T22:21:18.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.248+0000 7f95516f2700 1 --2- 172.21.15.107:0/4179755810 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f9538038710 0x7f953803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.248+0000 7f9554157700 1 -- 172.21.15.107:0/4179755810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954c190bb0 con 0x7f954c104280 2023-07-21T22:21:18.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.248+0000 7f95516f2700 1 --2- 172.21.15.107:0/4179755810 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f9538038710 0x7f953803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f953c006fd0 tx=0x7f953c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:18.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.253+0000 7f9542ffd700 1 -- 172.21.15.107:0/4179755810 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9548027030 con 0x7f954c104280 2023-07-21T22:21:18.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.490+0000 7f9554157700 1 -- 172.21.15.107:0/4179755810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f954c1061e0 con 0x7f954c104280 2023-07-21T22:21:18.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.925+0000 7f9542ffd700 1 -- 172.21.15.107:0/4179755810 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f954802c740 con 0x7f954c104280 2023-07-21T22:21:18.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.926+0000 7f9542ffd700 1 -- 172.21.15.107:0/4179755810 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f954802a930 con 0x7f954c104280 2023-07-21T22:21:18.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.928+0000 7f9554157700 1 -- 172.21.15.107:0/4179755810 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f9538038710 msgr2=0x7f953803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.928+0000 7f9554157700 1 --2- 172.21.15.107:0/4179755810 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f9538038710 0x7f953803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f953c006fd0 tx=0x7f953c006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:18.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.928+0000 7f9554157700 1 -- 172.21.15.107:0/4179755810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 msgr2=0x7f954c196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.928+0000 7f9554157700 1 --2- 172.21.15.107:0/4179755810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 0x7f954c196e90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9548003a80 tx=0x7f9548003df0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.928+0000 7f9554157700 1 -- 172.21.15.107:0/4179755810 shutdown_connections 2023-07-21T22:21:18.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.928+0000 7f9554157700 1 --2- 172.21.15.107:0/4179755810 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f9538038710 0x7f953803abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.928+0000 7f9554157700 1 --2- 172.21.15.107:0/4179755810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f954c104280 0x7f954c196e90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.928+0000 7f9554157700 1 -- 172.21.15.107:0/4179755810 >> 172.21.15.107:0/4179755810 conn(0x7f954c0ff8d0 msgr2=0x7f954c18dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:18.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.929+0000 7f9554157700 1 -- 172.21.15.107:0/4179755810 shutdown_connections 2023-07-21T22:21:18.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:17.929+0000 7f9554157700 1 -- 172.21.15.107:0/4179755810 wait complete. 2023-07-21T22:21:18.559 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:18.253+0000 7fd002c7f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:21:18.559 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:18.351+0000 7fd002c7f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:21:18.890 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:18.527+0000 7fd002c7f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:21:19.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:18 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/4179755810' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-21T22:21:19.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:18 smithi107 ceph-mon[96543]: mgrmap e5: x(active, since 8s) 2023-07-21T22:21:19.187 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:18.844+0000 7fd002c7f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:21:19.187 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:18.934+0000 7fd002c7f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:21:19.187 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:19 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:19.066+0000 7fd002c7f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:21:19.414 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:21:19.414 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-21T22:21:19.414 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T22:21:19.415 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-21T22:21:19.415 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-21T22:21:19.415 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:21:19.415 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.764+0000 7eff87856700 1 Processor -- start 2023-07-21T22:21:19.415 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.764+0000 7eff87856700 1 -- start start 2023-07-21T22:21:19.415 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.765+0000 7eff87856700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 0x7eff80108290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.415 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.765+0000 7eff87856700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff80073c00 con 0x7eff80105eb0 2023-07-21T22:21:19.415 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.765+0000 7eff855f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 0x7eff80108290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.416 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.765+0000 7eff855f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 0x7eff80108290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45174/0 (socket says 172.21.15.107:45174) 2023-07-21T22:21:19.416 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.765+0000 7eff855f2700 1 -- 172.21.15.107:0/432640638 learned_addr learned my addr 172.21.15.107:0/432640638 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:19.416 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.765+0000 7eff855f2700 1 -- 172.21.15.107:0/432640638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80073d40 con 0x7eff80105eb0 2023-07-21T22:21:19.416 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.765+0000 7eff855f2700 1 --2- 172.21.15.107:0/432640638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 0x7eff80108290 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0098a0 tx=0x7eff7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf04591510ffd15 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.416 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.765+0000 7eff77fff700 1 -- 172.21.15.107:0/432640638 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff7c004160 con 0x7eff80105eb0 2023-07-21T22:21:19.416 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.765+0000 7eff77fff700 1 -- 172.21.15.107:0/432640638 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7eff7c0042c0 con 0x7eff80105eb0 2023-07-21T22:21:19.416 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.765+0000 7eff77fff700 1 -- 172.21.15.107:0/432640638 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff7c011a00 con 0x7eff80105eb0 2023-07-21T22:21:19.417 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.766+0000 7eff87856700 1 -- 172.21.15.107:0/432640638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 msgr2=0x7eff80108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.417 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.766+0000 7eff87856700 1 --2- 172.21.15.107:0/432640638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 0x7eff80108290 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0098a0 tx=0x7eff7c009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.417 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.766+0000 7eff87856700 1 -- 172.21.15.107:0/432640638 shutdown_connections 2023-07-21T22:21:19.417 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.766+0000 7eff87856700 1 --2- 172.21.15.107:0/432640638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 0x7eff80108290 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.417 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.766+0000 7eff87856700 1 -- 172.21.15.107:0/432640638 >> 172.21.15.107:0/432640638 conn(0x7eff800ff8d0 msgr2=0x7eff80101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.417 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.766+0000 7eff87856700 1 -- 172.21.15.107:0/432640638 shutdown_connections 2023-07-21T22:21:19.417 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.766+0000 7eff87856700 1 -- 172.21.15.107:0/432640638 wait complete. 2023-07-21T22:21:19.417 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.767+0000 7eff87856700 1 Processor -- start 2023-07-21T22:21:19.418 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.767+0000 7eff87856700 1 -- start start 2023-07-21T22:21:19.418 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.767+0000 7eff87856700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 0x7eff801990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.418 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.767+0000 7eff87856700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff80199620 con 0x7eff80105eb0 2023-07-21T22:21:19.418 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.767+0000 7eff855f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 0x7eff801990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.418 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.767+0000 7eff855f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 0x7eff801990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45176/0 (socket says 172.21.15.107:45176) 2023-07-21T22:21:19.418 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.767+0000 7eff855f2700 1 -- 172.21.15.107:0/4016238608 learned_addr learned my addr 172.21.15.107:0/4016238608 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:19.418 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.767+0000 7eff855f2700 1 -- 172.21.15.107:0/4016238608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7c009550 con 0x7eff80105eb0 2023-07-21T22:21:19.419 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.767+0000 7eff855f2700 1 --2- 172.21.15.107:0/4016238608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 0x7eff801990e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7eff7c006af0 tx=0x7eff7c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.419 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.768+0000 7eff767fc700 1 -- 172.21.15.107:0/4016238608 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff7c003a40 con 0x7eff80105eb0 2023-07-21T22:21:19.419 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.768+0000 7eff767fc700 1 -- 172.21.15.107:0/4016238608 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7eff7c003ba0 con 0x7eff80105eb0 2023-07-21T22:21:19.419 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.768+0000 7eff87856700 1 -- 172.21.15.107:0/4016238608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff80199820 con 0x7eff80105eb0 2023-07-21T22:21:19.419 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.768+0000 7eff767fc700 1 -- 172.21.15.107:0/4016238608 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff7c01a700 con 0x7eff80105eb0 2023-07-21T22:21:19.419 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.768+0000 7eff87856700 1 -- 172.21.15.107:0/4016238608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff80199c40 con 0x7eff80105eb0 2023-07-21T22:21:19.419 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.768+0000 7eff767fc700 1 -- 172.21.15.107:0/4016238608 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7eff7c01a980 con 0x7eff80105eb0 2023-07-21T22:21:19.419 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.769+0000 7eff87856700 1 -- 172.21.15.107:0/4016238608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff80061b40 con 0x7eff80105eb0 2023-07-21T22:21:19.420 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.769+0000 7eff767fc700 1 --2- 172.21.15.107:0/4016238608 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7eff6c038710 0x7eff6c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.420 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.769+0000 7eff84df1700 1 -- 172.21.15.107:0/4016238608 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7eff6c038710 msgr2=0x7eff6c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3802710476 2023-07-21T22:21:19.420 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.769+0000 7eff767fc700 1 -- 172.21.15.107:0/4016238608 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7eff7c04d6a0 con 0x7eff80105eb0 2023-07-21T22:21:19.420 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.769+0000 7eff84df1700 1 --2- 172.21.15.107:0/4016238608 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7eff6c038710 0x7eff6c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:21:19.420 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.773+0000 7eff767fc700 1 -- 172.21.15.107:0/4016238608 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7eff7c018db0 con 0x7eff80105eb0 2023-07-21T22:21:19.420 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.970+0000 7eff84df1700 1 -- 172.21.15.107:0/4016238608 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7eff6c038710 msgr2=0x7eff6c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3802710476 2023-07-21T22:21:19.420 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:18.970+0000 7eff84df1700 1 --2- 172.21.15.107:0/4016238608 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7eff6c038710 0x7eff6c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:21:19.421 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.018+0000 7eff87856700 1 -- 172.21.15.107:0/4016238608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7eff8019cd50 con 0x7eff80105eb0 2023-07-21T22:21:19.421 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.019+0000 7eff767fc700 1 -- 172.21.15.107:0/4016238608 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7eff7c018350 con 0x7eff80105eb0 2023-07-21T22:21:19.421 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.021+0000 7eff87856700 1 -- 172.21.15.107:0/4016238608 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7eff6c038710 msgr2=0x7eff6c03abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:21:19.421 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.021+0000 7eff87856700 1 --2- 172.21.15.107:0/4016238608 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7eff6c038710 0x7eff6c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.421 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.021+0000 7eff87856700 1 -- 172.21.15.107:0/4016238608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 msgr2=0x7eff801990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.421 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.021+0000 7eff87856700 1 --2- 172.21.15.107:0/4016238608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 0x7eff801990e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7eff7c006af0 tx=0x7eff7c011ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.421 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.021+0000 7eff87856700 1 -- 172.21.15.107:0/4016238608 shutdown_connections 2023-07-21T22:21:19.421 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.021+0000 7eff87856700 1 --2- 172.21.15.107:0/4016238608 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7eff6c038710 0x7eff6c03abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.422 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.021+0000 7eff87856700 1 --2- 172.21.15.107:0/4016238608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff80105eb0 0x7eff801990e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.422 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.021+0000 7eff87856700 1 -- 172.21.15.107:0/4016238608 >> 172.21.15.107:0/4016238608 conn(0x7eff800ff8d0 msgr2=0x7eff80100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.422 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.021+0000 7eff87856700 1 -- 172.21.15.107:0/4016238608 shutdown_connections 2023-07-21T22:21:19.422 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.021+0000 7eff87856700 1 -- 172.21.15.107:0/4016238608 wait complete. 2023-07-21T22:21:19.422 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2023-07-21T22:21:19.422 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 5... 2023-07-21T22:21:19.643 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:19 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:19.170+0000 7fd002c7f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:21:19.936 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:19 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:19.598+0000 7fd002c7f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:21:20.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:19 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/4016238608' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-21T22:21:20.686 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:20.180+0000 7fd002c7f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:21:20.686 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:20.347+0000 7fd002c7f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:21:21.408 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:21 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:21.102+0000 7fd002c7f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:21:21.408 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:21 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:21.193+0000 7fd002c7f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:21:21.685 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:21 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:21.363+0000 7fd002c7f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:21:21.936 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:21 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:21.640+0000 7fd002c7f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:21:21.937 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:21 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:21.862+0000 7fd002c7f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:21:22.310 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:22 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:22.008+0000 7fd002c7f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:21:22.686 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:22 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:22.265+0000 7fd002c7f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:21:23.316 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:22 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:22.954+0000 7fd002c7f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:21:23.316 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:23 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:23.083+0000 7fd002c7f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:21:23.640 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:23 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:23.271+0000 7fd002c7f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:21:23.640 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:23 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:23.383+0000 7fd002c7f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:21:23.936 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:23 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:23.594+0000 7fd002c7f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:21:24.936 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:24.446+0000 7fd002c7f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:21:24.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: Active manager daemon x restarted 2023-07-21T22:21:24.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: Activating manager daemon x 2023-07-21T22:21:24.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: osdmap e2: 0 total, 0 up, 0 in 2023-07-21T22:21:24.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: mgrmap e6: x(active, starting, since 0.00344756s) 2023-07-21T22:21:24.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:21:24.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:21:24.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:21:24.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:21:24.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:21:24.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: Manager daemon x is now available 2023-07-21T22:21:24.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:24.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:24.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:24.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:24.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:24 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:25.853 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:25 smithi107 ceph-mon[96543]: Found migration_current of "None". Setting to last migration. 2023-07-21T22:21:25.853 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:25 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:21:25.853 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:25 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:21:25.853 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:25 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:25.853 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:25 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:25.854 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:25 smithi107 ceph-mon[96543]: mgrmap e7: x(active, since 1.00539s) 2023-07-21T22:21:25.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:21:25.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-21T22:21:25.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-21T22:21:25.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:21:25.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.898+0000 7f4be7c60700 1 Processor -- start 2023-07-21T22:21:25.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.898+0000 7f4be7c60700 1 -- start start 2023-07-21T22:21:25.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.898+0000 7f4be7c60700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 0x7f4be00791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:25.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.898+0000 7f4be7c60700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be0079710 con 0x7f4be007acd0 2023-07-21T22:21:25.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.898+0000 7f4be59fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 0x7f4be00791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:25.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.898+0000 7f4be59fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 0x7f4be00791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45184/0 (socket says 172.21.15.107:45184) 2023-07-21T22:21:25.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.898+0000 7f4be59fc700 1 -- 172.21.15.107:0/3919390745 learned_addr learned my addr 172.21.15.107:0/3919390745 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:25.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.899+0000 7f4be59fc700 1 -- 172.21.15.107:0/3919390745 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be0079850 con 0x7f4be007acd0 2023-07-21T22:21:25.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.899+0000 7f4be59fc700 1 --2- 172.21.15.107:0/3919390745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 0x7f4be00791d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0098a0 tx=0x7f4bdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b077b9d246b5308d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:25.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.899+0000 7f4be49fa700 1 -- 172.21.15.107:0/3919390745 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bdc004160 con 0x7f4be007acd0 2023-07-21T22:21:25.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.899+0000 7f4be49fa700 1 -- 172.21.15.107:0/3919390745 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4bdc0042c0 con 0x7f4be007acd0 2023-07-21T22:21:25.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.899+0000 7f4be49fa700 1 -- 172.21.15.107:0/3919390745 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bdc011a00 con 0x7f4be007acd0 2023-07-21T22:21:25.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.900+0000 7f4be7c60700 1 -- 172.21.15.107:0/3919390745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 msgr2=0x7f4be00791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:25.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.900+0000 7f4be7c60700 1 --2- 172.21.15.107:0/3919390745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 0x7f4be00791d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0098a0 tx=0x7f4bdc009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:25.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.900+0000 7f4be7c60700 1 -- 172.21.15.107:0/3919390745 shutdown_connections 2023-07-21T22:21:25.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.900+0000 7f4be7c60700 1 --2- 172.21.15.107:0/3919390745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 0x7f4be00791d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:25.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.900+0000 7f4be7c60700 1 -- 172.21.15.107:0/3919390745 >> 172.21.15.107:0/3919390745 conn(0x7f4be01009f0 msgr2=0x7f4be0102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:25.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.900+0000 7f4be7c60700 1 -- 172.21.15.107:0/3919390745 shutdown_connections 2023-07-21T22:21:25.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.900+0000 7f4be7c60700 1 -- 172.21.15.107:0/3919390745 wait complete. 2023-07-21T22:21:25.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.901+0000 7f4be7c60700 1 Processor -- start 2023-07-21T22:21:25.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.901+0000 7f4be7c60700 1 -- start start 2023-07-21T22:21:25.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.901+0000 7f4be7c60700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 0x7f4be019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:25.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.901+0000 7f4be7c60700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be019b8b0 con 0x7f4be007acd0 2023-07-21T22:21:25.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.901+0000 7f4be59fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 0x7f4be019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:25.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.901+0000 7f4be59fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 0x7f4be019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45186/0 (socket says 172.21.15.107:45186) 2023-07-21T22:21:25.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.901+0000 7f4be59fc700 1 -- 172.21.15.107:0/895802085 learned_addr learned my addr 172.21.15.107:0/895802085 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:25.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.901+0000 7f4be59fc700 1 -- 172.21.15.107:0/895802085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc009550 con 0x7f4be007acd0 2023-07-21T22:21:25.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.901+0000 7f4be59fc700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 0x7f4be019b370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc006b20 tx=0x7f4bdc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:25.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.902+0000 7f4bd6ffd700 1 -- 172.21.15.107:0/895802085 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bdc003b70 con 0x7f4be007acd0 2023-07-21T22:21:25.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.902+0000 7f4be7c60700 1 -- 172.21.15.107:0/895802085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be019bab0 con 0x7f4be007acd0 2023-07-21T22:21:25.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.902+0000 7f4bd6ffd700 1 -- 172.21.15.107:0/895802085 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4bdc003cd0 con 0x7f4be007acd0 2023-07-21T22:21:25.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.902+0000 7f4bd6ffd700 1 -- 172.21.15.107:0/895802085 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bdc0247c0 con 0x7f4be007acd0 2023-07-21T22:21:25.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.902+0000 7f4be7c60700 1 -- 172.21.15.107:0/895802085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be019bed0 con 0x7f4be007acd0 2023-07-21T22:21:25.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.903+0000 7f4bd6ffd700 1 -- 172.21.15.107:0/895802085 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f4bdc01ab40 con 0x7f4be007acd0 2023-07-21T22:21:25.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.903+0000 7f4bd6ffd700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 0x7f4bcc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:25.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.903+0000 7f4bd6ffd700 1 -- 172.21.15.107:0/895802085 --> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4bcc03b2d0 con 0x7f4bcc038710 2023-07-21T22:21:25.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.903+0000 7f4bd6ffd700 1 -- 172.21.15.107:0/895802085 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4bdc04c600 con 0x7f4be007acd0 2023-07-21T22:21:25.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.903+0000 7f4be51fb700 1 -- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 msgr2=0x7f4bcc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3802710476 2023-07-21T22:21:25.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:19.903+0000 7f4be51fb700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 0x7f4bcc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:21:25.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:20.104+0000 7f4be51fb700 1 -- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 msgr2=0x7f4bcc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3802710476 2023-07-21T22:21:25.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:20.104+0000 7f4be51fb700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 0x7f4bcc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:21:25.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:20.505+0000 7f4be51fb700 1 -- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 msgr2=0x7f4bcc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3802710476 2023-07-21T22:21:25.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:20.505+0000 7f4be51fb700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 0x7f4bcc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-21T22:21:25.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:21.306+0000 7f4be51fb700 1 -- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 msgr2=0x7f4bcc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3802710476 2023-07-21T22:21:25.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:21.306+0000 7f4be51fb700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 0x7f4bcc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-21T22:21:25.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:22.907+0000 7f4be51fb700 1 -- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 msgr2=0x7f4bcc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3802710476 2023-07-21T22:21:25.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:22.907+0000 7f4be51fb700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 0x7f4bcc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-21T22:21:25.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:24.462+0000 7f4bd6ffd700 1 -- 172.21.15.107:0/895802085 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f4bdc01a430 con 0x7f4be007acd0 2023-07-21T22:21:25.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:24.462+0000 7f4bd6ffd700 1 -- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 msgr2=0x7f4bcc03abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:21:25.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:24.462+0000 7f4bd6ffd700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 0x7f4bcc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:25.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.463+0000 7f4bd6ffd700 1 -- 172.21.15.107:0/895802085 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f4bdc022c30 con 0x7f4be007acd0 2023-07-21T22:21:25.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.463+0000 7f4bd6ffd700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f4bcc03bfe0 0x7f4bcc03e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:25.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.463+0000 7f4bd6ffd700 1 -- 172.21.15.107:0/895802085 --> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4bcc03b2d0 con 0x7f4bcc03bfe0 2023-07-21T22:21:25.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.464+0000 7f4be51fb700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f4bcc03bfe0 0x7f4bcc03e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:25.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.464+0000 7f4be51fb700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f4bcc03bfe0 0x7f4bcc03e3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0003a10 tx=0x7f4bd0009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:25.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.465+0000 7f4bd6ffd700 1 -- 172.21.15.107:0/895802085 <== mgr.14118 v2:172.21.15.107:6800/659042147 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f4bcc03b2d0 con 0x7f4bcc03bfe0 2023-07-21T22:21:25.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.471+0000 7f4bd4ff9700 1 -- 172.21.15.107:0/895802085 --> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4bc8002600 con 0x7f4bcc03bfe0 2023-07-21T22:21:25.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.471+0000 7f4bd6ffd700 1 -- 172.21.15.107:0/895802085 <== mgr.14118 v2:172.21.15.107:6800/659042147 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4bc8002600 con 0x7f4bcc03bfe0 2023-07-21T22:21:25.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.471+0000 7f4bd4ff9700 1 -- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f4bcc03bfe0 msgr2=0x7f4bcc03e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:25.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.471+0000 7f4bd4ff9700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f4bcc03bfe0 0x7f4bcc03e3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0003a10 tx=0x7f4bd0009290 comp rx=0 tx=0).stop 2023-07-21T22:21:25.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.471+0000 7f4bd4ff9700 1 -- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 msgr2=0x7f4be019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:25.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.471+0000 7f4bd4ff9700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 0x7f4be019b370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc006b20 tx=0x7f4bdc011ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:25.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.471+0000 7f4bd4ff9700 1 -- 172.21.15.107:0/895802085 shutdown_connections 2023-07-21T22:21:25.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.471+0000 7f4bd4ff9700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f4bcc03bfe0 0x7f4bcc03e3c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:25.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.471+0000 7f4bd4ff9700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be007acd0 0x7f4be019b370 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:25.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.471+0000 7f4bd4ff9700 1 --2- 172.21.15.107:0/895802085 >> [v2:172.21.15.107:6800/3802710476,v1:172.21.15.107:6801/3802710476] conn(0x7f4bcc038710 0x7f4bcc03abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:25.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.471+0000 7f4bd4ff9700 1 -- 172.21.15.107:0/895802085 >> 172.21.15.107:0/895802085 conn(0x7f4be01009f0 msgr2=0x7f4be01020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:25.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.471+0000 7f4bd4ff9700 1 -- 172.21.15.107:0/895802085 shutdown_connections 2023-07-21T22:21:25.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:25.472+0000 7f4bd4ff9700 1 -- 172.21.15.107:0/895802085 wait complete. 2023-07-21T22:21:25.883 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 5 is available 2023-07-21T22:21:25.883 INFO:teuthology.orchestra.run.smithi107.stdout:Setting orchestrator backend to cephadm... 2023-07-21T22:21:26.915 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.289+0000 7f60ff698700 1 Processor -- start 2023-07-21T22:21:26.915 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.289+0000 7f60ff698700 1 -- start start 2023-07-21T22:21:26.916 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.289+0000 7f60ff698700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 0x7f60f8104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:26.916 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.289+0000 7f60ff698700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f8104bd0 con 0x7f60f8104280 2023-07-21T22:21:26.916 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.289+0000 7f60fd434700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 0x7f60f8104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:26.916 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.289+0000 7f60fd434700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 0x7f60f8104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45236/0 (socket says 172.21.15.107:45236) 2023-07-21T22:21:26.916 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.289+0000 7f60fd434700 1 -- 172.21.15.107:0/2904040065 learned_addr learned my addr 172.21.15.107:0/2904040065 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:26.916 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.289+0000 7f60fd434700 1 -- 172.21.15.107:0/2904040065 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f8104d10 con 0x7f60f8104280 2023-07-21T22:21:26.917 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.290+0000 7f60fd434700 1 --2- 172.21.15.107:0/2904040065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 0x7f60f8104690 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60e80098a0 tx=0x7f60e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef952ed0983efc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:26.917 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.290+0000 7f60effff700 1 -- 172.21.15.107:0/2904040065 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60e8004160 con 0x7f60f8104280 2023-07-21T22:21:26.917 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.290+0000 7f60effff700 1 -- 172.21.15.107:0/2904040065 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60e80042c0 con 0x7f60f8104280 2023-07-21T22:21:26.917 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.290+0000 7f60effff700 1 -- 172.21.15.107:0/2904040065 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60e8011a00 con 0x7f60f8104280 2023-07-21T22:21:26.917 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.290+0000 7f60ff698700 1 -- 172.21.15.107:0/2904040065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 msgr2=0x7f60f8104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:26.917 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.290+0000 7f60ff698700 1 --2- 172.21.15.107:0/2904040065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 0x7f60f8104690 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f60e80098a0 tx=0x7f60e8009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:26.917 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.290+0000 7f60ff698700 1 -- 172.21.15.107:0/2904040065 shutdown_connections 2023-07-21T22:21:26.918 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.290+0000 7f60ff698700 1 --2- 172.21.15.107:0/2904040065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 0x7f60f8104690 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:26.918 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.290+0000 7f60ff698700 1 -- 172.21.15.107:0/2904040065 >> 172.21.15.107:0/2904040065 conn(0x7f60f80ff8d0 msgr2=0x7f60f8101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:26.918 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.290+0000 7f60ff698700 1 -- 172.21.15.107:0/2904040065 shutdown_connections 2023-07-21T22:21:26.918 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.290+0000 7f60ff698700 1 -- 172.21.15.107:0/2904040065 wait complete. 2023-07-21T22:21:26.918 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.291+0000 7f60ff698700 1 Processor -- start 2023-07-21T22:21:26.918 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.291+0000 7f60ff698700 1 -- start start 2023-07-21T22:21:26.918 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.291+0000 7f60ff698700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 0x7f60f80739d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:26.918 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.291+0000 7f60ff698700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f819bdb0 con 0x7f60f8104280 2023-07-21T22:21:26.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.291+0000 7f60fd434700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 0x7f60f80739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:26.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.291+0000 7f60fd434700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 0x7f60f80739d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45238/0 (socket says 172.21.15.107:45238) 2023-07-21T22:21:26.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.291+0000 7f60fd434700 1 -- 172.21.15.107:0/3109509736 learned_addr learned my addr 172.21.15.107:0/3109509736 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:26.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.291+0000 7f60fd434700 1 -- 172.21.15.107:0/3109509736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e8009550 con 0x7f60f8104280 2023-07-21T22:21:26.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.292+0000 7f60fd434700 1 --2- 172.21.15.107:0/3109509736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 0x7f60f80739d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009fb0 tx=0x7f60e8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:26.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.292+0000 7f60ee7fc700 1 -- 172.21.15.107:0/3109509736 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60e8003930 con 0x7f60f8104280 2023-07-21T22:21:26.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.292+0000 7f60ee7fc700 1 -- 172.21.15.107:0/3109509736 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60e8003a90 con 0x7f60f8104280 2023-07-21T22:21:26.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.292+0000 7f60ff698700 1 -- 172.21.15.107:0/3109509736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f819bef0 con 0x7f60f8104280 2023-07-21T22:21:26.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.292+0000 7f60ee7fc700 1 -- 172.21.15.107:0/3109509736 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60e8019610 con 0x7f60f8104280 2023-07-21T22:21:26.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.292+0000 7f60ff698700 1 -- 172.21.15.107:0/3109509736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f819c2b0 con 0x7f60f8104280 2023-07-21T22:21:26.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.293+0000 7f60ee7fc700 1 -- 172.21.15.107:0/3109509736 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f60e8019880 con 0x7f60f8104280 2023-07-21T22:21:26.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.293+0000 7f60ee7fc700 1 --2- 172.21.15.107:0/3109509736 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f60e40385f0 0x7f60e403aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:26.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.293+0000 7f60ff698700 1 -- 172.21.15.107:0/3109509736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f8061b40 con 0x7f60f8104280 2023-07-21T22:21:26.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.293+0000 7f60ee7fc700 1 -- 172.21.15.107:0/3109509736 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f60e804c810 con 0x7f60f8104280 2023-07-21T22:21:26.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.293+0000 7f60fcc33700 1 --2- 172.21.15.107:0/3109509736 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f60e40385f0 0x7f60e403aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:26.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.293+0000 7f60fcc33700 1 --2- 172.21.15.107:0/3109509736 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f60e40385f0 0x7f60e403aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f60f4006fd0 tx=0x7f60f4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:26.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.298+0000 7f60ee7fc700 1 -- 172.21.15.107:0/3109509736 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f60e8018530 con 0x7f60f8104280 2023-07-21T22:21:26.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.498+0000 7f60ff698700 1 -- 172.21.15.107:0/3109509736 --> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f60f8101bb0 con 0x7f60e40385f0 2023-07-21T22:21:26.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.503+0000 7f60ee7fc700 1 -- 172.21.15.107:0/3109509736 <== mgr.14118 v2:172.21.15.107:6800/659042147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f60f8101bb0 con 0x7f60e40385f0 2023-07-21T22:21:26.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.506+0000 7f60ff698700 1 -- 172.21.15.107:0/3109509736 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f60e40385f0 msgr2=0x7f60e403aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:26.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.506+0000 7f60ff698700 1 --2- 172.21.15.107:0/3109509736 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f60e40385f0 0x7f60e403aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f60f4006fd0 tx=0x7f60f4006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:26.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.506+0000 7f60ff698700 1 -- 172.21.15.107:0/3109509736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 msgr2=0x7f60f80739d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:26.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.506+0000 7f60ff698700 1 --2- 172.21.15.107:0/3109509736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 0x7f60f80739d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009fb0 tx=0x7f60e8011ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:26.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.506+0000 7f60ff698700 1 -- 172.21.15.107:0/3109509736 shutdown_connections 2023-07-21T22:21:26.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.506+0000 7f60ff698700 1 --2- 172.21.15.107:0/3109509736 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f60e40385f0 0x7f60e403aaa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:26.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.506+0000 7f60ff698700 1 --2- 172.21.15.107:0/3109509736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60f8104280 0x7f60f80739d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:26.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.506+0000 7f60ff698700 1 -- 172.21.15.107:0/3109509736 >> 172.21.15.107:0/3109509736 conn(0x7f60f80ff8d0 msgr2=0x7f60f8100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:26.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.506+0000 7f60ff698700 1 -- 172.21.15.107:0/3109509736 shutdown_connections 2023-07-21T22:21:26.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:26.506+0000 7f60ff698700 1 -- 172.21.15.107:0/3109509736 wait complete. 2023-07-21T22:21:27.143 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:26 smithi107 ceph-mon[96543]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-21T22:21:27.143 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:26 smithi107 ceph-mon[96543]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-21T22:21:27.144 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:26 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:27.144 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:26 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:27.144 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:26 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:27.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-21T22:21:27.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.316+0000 7fc6fbe89700 1 Processor -- start 2023-07-21T22:21:27.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.316+0000 7fc6fbe89700 1 -- start start 2023-07-21T22:21:27.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.316+0000 7fc6fbe89700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 0x7fc6f4108260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:27.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.316+0000 7fc6fbe89700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f4079d60 con 0x7fc6f4105e80 2023-07-21T22:21:27.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.317+0000 7fc6f9c25700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 0x7fc6f4108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:27.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.317+0000 7fc6f9c25700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 0x7fc6f4108260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45244/0 (socket says 172.21.15.107:45244) 2023-07-21T22:21:27.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.317+0000 7fc6f9c25700 1 -- 172.21.15.107:0/792647636 learned_addr learned my addr 172.21.15.107:0/792647636 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:27.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.317+0000 7fc6f9c25700 1 -- 172.21.15.107:0/792647636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f4079ea0 con 0x7fc6f4105e80 2023-07-21T22:21:27.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.317+0000 7fc6f9c25700 1 --2- 172.21.15.107:0/792647636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 0x7fc6f4108260 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40098a0 tx=0x7fc6e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a6a70fe7abe2c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:27.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.317+0000 7fc6f8c23700 1 -- 172.21.15.107:0/792647636 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6e4004160 con 0x7fc6f4105e80 2023-07-21T22:21:27.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.317+0000 7fc6f8c23700 1 -- 172.21.15.107:0/792647636 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6e40042c0 con 0x7fc6f4105e80 2023-07-21T22:21:27.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.317+0000 7fc6f8c23700 1 -- 172.21.15.107:0/792647636 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6e4011a00 con 0x7fc6f4105e80 2023-07-21T22:21:27.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.318+0000 7fc6fbe89700 1 -- 172.21.15.107:0/792647636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 msgr2=0x7fc6f4108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:27.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.318+0000 7fc6fbe89700 1 --2- 172.21.15.107:0/792647636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 0x7fc6f4108260 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc6e40098a0 tx=0x7fc6e4009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:27.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.318+0000 7fc6fbe89700 1 -- 172.21.15.107:0/792647636 shutdown_connections 2023-07-21T22:21:27.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.318+0000 7fc6fbe89700 1 --2- 172.21.15.107:0/792647636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 0x7fc6f4108260 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:27.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.318+0000 7fc6fbe89700 1 -- 172.21.15.107:0/792647636 >> 172.21.15.107:0/792647636 conn(0x7fc6f40ff900 msgr2=0x7fc6f4101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:27.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.318+0000 7fc6fbe89700 1 -- 172.21.15.107:0/792647636 shutdown_connections 2023-07-21T22:21:27.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.318+0000 7fc6fbe89700 1 -- 172.21.15.107:0/792647636 wait complete. 2023-07-21T22:21:27.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.319+0000 7fc6fbe89700 1 Processor -- start 2023-07-21T22:21:27.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.319+0000 7fc6fbe89700 1 -- start start 2023-07-21T22:21:27.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.319+0000 7fc6fbe89700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 0x7fc6f4199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:27.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.319+0000 7fc6fbe89700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f41996c0 con 0x7fc6f4105e80 2023-07-21T22:21:27.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.319+0000 7fc6f9c25700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 0x7fc6f4199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:27.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.319+0000 7fc6f9c25700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 0x7fc6f4199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45246/0 (socket says 172.21.15.107:45246) 2023-07-21T22:21:27.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.319+0000 7fc6f9c25700 1 -- 172.21.15.107:0/257903033 learned_addr learned my addr 172.21.15.107:0/257903033 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:27.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.319+0000 7fc6f9c25700 1 -- 172.21.15.107:0/257903033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e4009550 con 0x7fc6f4105e80 2023-07-21T22:21:27.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.319+0000 7fc6f9c25700 1 --2- 172.21.15.107:0/257903033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 0x7fc6f4199180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009130 tx=0x7fc6e4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:27.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.320+0000 7fc6eaffd700 1 -- 172.21.15.107:0/257903033 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6e4003ab0 con 0x7fc6f4105e80 2023-07-21T22:21:27.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.320+0000 7fc6eaffd700 1 -- 172.21.15.107:0/257903033 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6e4003c10 con 0x7fc6f4105e80 2023-07-21T22:21:27.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.320+0000 7fc6fbe89700 1 -- 172.21.15.107:0/257903033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f41998c0 con 0x7fc6f4105e80 2023-07-21T22:21:27.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.320+0000 7fc6eaffd700 1 -- 172.21.15.107:0/257903033 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6e40247d0 con 0x7fc6f4105e80 2023-07-21T22:21:27.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.320+0000 7fc6fbe89700 1 -- 172.21.15.107:0/257903033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f4199ce0 con 0x7fc6f4105e80 2023-07-21T22:21:27.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.320+0000 7fc6eaffd700 1 -- 172.21.15.107:0/257903033 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fc6e402d3f0 con 0x7fc6f4105e80 2023-07-21T22:21:27.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.320+0000 7fc6eaffd700 1 --2- 172.21.15.107:0/257903033 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fc6e0038710 0x7fc6e003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:27.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.321+0000 7fc6eaffd700 1 -- 172.21.15.107:0/257903033 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc6e404c830 con 0x7fc6f4105e80 2023-07-21T22:21:27.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.321+0000 7fc6f9424700 1 --2- 172.21.15.107:0/257903033 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fc6e0038710 0x7fc6e003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:27.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.321+0000 7fc6fbe89700 1 -- 172.21.15.107:0/257903033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f4193130 con 0x7fc6f4105e80 2023-07-21T22:21:27.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.321+0000 7fc6f9424700 1 --2- 172.21.15.107:0/257903033 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fc6e0038710 0x7fc6e003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0006fd0 tx=0x7fc6f0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:27.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.325+0000 7fc6eaffd700 1 -- 172.21.15.107:0/257903033 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc6e4010970 con 0x7fc6f4105e80 2023-07-21T22:21:27.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.511+0000 7fc6fbe89700 1 -- 172.21.15.107:0/257903033 --> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6f4060950 con 0x7fc6e0038710 2023-07-21T22:21:27.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.511+0000 7fc6eaffd700 1 -- 172.21.15.107:0/257903033 <== mgr.14118 v2:172.21.15.107:6800/659042147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fc6f4060950 con 0x7fc6e0038710 2023-07-21T22:21:27.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.513+0000 7fc6fbe89700 1 -- 172.21.15.107:0/257903033 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fc6e0038710 msgr2=0x7fc6e003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:27.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.513+0000 7fc6fbe89700 1 --2- 172.21.15.107:0/257903033 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fc6e0038710 0x7fc6e003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0006fd0 tx=0x7fc6f0006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:27.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.513+0000 7fc6fbe89700 1 -- 172.21.15.107:0/257903033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 msgr2=0x7fc6f4199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:27.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.513+0000 7fc6fbe89700 1 --2- 172.21.15.107:0/257903033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 0x7fc6f4199180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009130 tx=0x7fc6e4011ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:27.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.513+0000 7fc6fbe89700 1 -- 172.21.15.107:0/257903033 shutdown_connections 2023-07-21T22:21:27.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.513+0000 7fc6fbe89700 1 --2- 172.21.15.107:0/257903033 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fc6e0038710 0x7fc6e003abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:27.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.513+0000 7fc6fbe89700 1 --2- 172.21.15.107:0/257903033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6f4105e80 0x7fc6f4199180 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:27.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.513+0000 7fc6fbe89700 1 -- 172.21.15.107:0/257903033 >> 172.21.15.107:0/257903033 conn(0x7fc6f40ff900 msgr2=0x7fc6f4101010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:27.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.514+0000 7fc6fbe89700 1 -- 172.21.15.107:0/257903033 shutdown_connections 2023-07-21T22:21:27.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:27.514+0000 7fc6fbe89700 1 -- 172.21.15.107:0/257903033 wait complete. 2023-07-21T22:21:27.866 INFO:teuthology.orchestra.run.smithi107.stdout:Generating ssh key... 2023-07-21T22:21:28.135 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:27 smithi107 ceph-mon[96543]: [21/Jul/2023:22:21:26] ENGINE Bus STARTING 2023-07-21T22:21:28.135 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:27 smithi107 ceph-mon[96543]: [21/Jul/2023:22:21:26] ENGINE Serving on http://172.21.15.107:8765 2023-07-21T22:21:28.136 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:27 smithi107 ceph-mon[96543]: [21/Jul/2023:22:21:26] ENGINE Serving on https://172.21.15.107:7150 2023-07-21T22:21:28.136 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:27 smithi107 ceph-mon[96543]: [21/Jul/2023:22:21:26] ENGINE Bus STARTED 2023-07-21T22:21:28.136 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:27 smithi107 ceph-mon[96543]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:28.136 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:27 smithi107 ceph-mon[96543]: mgrmap e8: x(active, since 2s) 2023-07-21T22:21:28.936 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: Generating public/private rsa key pair. 2023-07-21T22:21:28.937 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: Your identification has been saved in /tmp/tmpihsmdl4d/key. 2023-07-21T22:21:28.937 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: Your public key has been saved in /tmp/tmpihsmdl4d/key.pub. 2023-07-21T22:21:28.937 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: The key fingerprint is: 2023-07-21T22:21:28.937 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: SHA256:fUuftYexD00B36wsJbD/SAoz/w3cECKC/+3Mny2PN2c ceph-af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:21:28.937 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: The key's randomart image is: 2023-07-21T22:21:28.937 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: +---[RSA 3072]----+ 2023-07-21T22:21:28.937 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: | . . | 2023-07-21T22:21:28.938 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: | . o o..| 2023-07-21T22:21:28.938 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: | . . . o o .oo| 2023-07-21T22:21:28.938 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: | . . o o = ..| 2023-07-21T22:21:28.938 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: | . S . B + o| 2023-07-21T22:21:28.938 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: | . * * O O.| 2023-07-21T22:21:28.938 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: | . + = O.o| 2023-07-21T22:21:28.938 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: | + ..*ooE| 2023-07-21T22:21:28.939 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: | +.=+++.| 2023-07-21T22:21:28.939 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: +----[SHA256]-----+ 2023-07-21T22:21:29.005 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.301+0000 7ff6df9e6700 1 Processor -- start 2023-07-21T22:21:29.006 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.301+0000 7ff6df9e6700 1 -- start start 2023-07-21T22:21:29.006 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.301+0000 7ff6df9e6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 0x7ff6d810a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:29.007 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.301+0000 7ff6df9e6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d8105d30 con 0x7ff6d8073b80 2023-07-21T22:21:29.007 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.301+0000 7ff6dd782700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 0x7ff6d810a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:29.007 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.301+0000 7ff6dd782700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 0x7ff6d810a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45250/0 (socket says 172.21.15.107:45250) 2023-07-21T22:21:29.008 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.301+0000 7ff6dd782700 1 -- 172.21.15.107:0/2366910719 learned_addr learned my addr 172.21.15.107:0/2366910719 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:29.008 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.301+0000 7ff6dd782700 1 -- 172.21.15.107:0/2366910719 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d810a880 con 0x7ff6d8073b80 2023-07-21T22:21:29.009 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.302+0000 7ff6dd782700 1 --2- 172.21.15.107:0/2366910719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 0x7ff6d810a340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40098a0 tx=0x7ff6d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a823958e4fd39eb4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:29.009 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.302+0000 7ff6cffff700 1 -- 172.21.15.107:0/2366910719 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4004160 con 0x7ff6d8073b80 2023-07-21T22:21:29.009 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.302+0000 7ff6cffff700 1 -- 172.21.15.107:0/2366910719 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6d40042c0 con 0x7ff6d8073b80 2023-07-21T22:21:29.009 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.302+0000 7ff6cffff700 1 -- 172.21.15.107:0/2366910719 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4011a00 con 0x7ff6d8073b80 2023-07-21T22:21:29.009 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.302+0000 7ff6df9e6700 1 -- 172.21.15.107:0/2366910719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 msgr2=0x7ff6d810a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:29.009 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.302+0000 7ff6df9e6700 1 --2- 172.21.15.107:0/2366910719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 0x7ff6d810a340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40098a0 tx=0x7ff6d4009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:29.009 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.302+0000 7ff6df9e6700 1 -- 172.21.15.107:0/2366910719 shutdown_connections 2023-07-21T22:21:29.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.302+0000 7ff6df9e6700 1 --2- 172.21.15.107:0/2366910719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 0x7ff6d810a340 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:29.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.302+0000 7ff6df9e6700 1 -- 172.21.15.107:0/2366910719 >> 172.21.15.107:0/2366910719 conn(0x7ff6d80ff8d0 msgr2=0x7ff6d8101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:29.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.302+0000 7ff6df9e6700 1 -- 172.21.15.107:0/2366910719 shutdown_connections 2023-07-21T22:21:29.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.302+0000 7ff6df9e6700 1 -- 172.21.15.107:0/2366910719 wait complete. 2023-07-21T22:21:29.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.303+0000 7ff6df9e6700 1 Processor -- start 2023-07-21T22:21:29.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.303+0000 7ff6df9e6700 1 -- start start 2023-07-21T22:21:29.010 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.303+0000 7ff6df9e6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 0x7ff6d819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:29.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.303+0000 7ff6df9e6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d819b880 con 0x7ff6d8073b80 2023-07-21T22:21:29.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.303+0000 7ff6dd782700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 0x7ff6d819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:29.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.303+0000 7ff6dd782700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 0x7ff6d819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45252/0 (socket says 172.21.15.107:45252) 2023-07-21T22:21:29.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.303+0000 7ff6dd782700 1 -- 172.21.15.107:0/219991529 learned_addr learned my addr 172.21.15.107:0/219991529 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:29.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.303+0000 7ff6dd782700 1 -- 172.21.15.107:0/219991529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4009550 con 0x7ff6d8073b80 2023-07-21T22:21:29.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.304+0000 7ff6dd782700 1 --2- 172.21.15.107:0/219991529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 0x7ff6d819b340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009130 tx=0x7ff6d4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:29.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.304+0000 7ff6ce7fc700 1 -- 172.21.15.107:0/219991529 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4003b70 con 0x7ff6d8073b80 2023-07-21T22:21:29.011 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.304+0000 7ff6df9e6700 1 -- 172.21.15.107:0/219991529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d819ba80 con 0x7ff6d8073b80 2023-07-21T22:21:29.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.304+0000 7ff6df9e6700 1 -- 172.21.15.107:0/219991529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d819bea0 con 0x7ff6d8073b80 2023-07-21T22:21:29.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.304+0000 7ff6ce7fc700 1 -- 172.21.15.107:0/219991529 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6d4003cd0 con 0x7ff6d8073b80 2023-07-21T22:21:29.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.304+0000 7ff6ce7fc700 1 -- 172.21.15.107:0/219991529 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6d40197e0 con 0x7ff6d8073b80 2023-07-21T22:21:29.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.304+0000 7ff6ce7fc700 1 -- 172.21.15.107:0/219991529 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ff6d4019a70 con 0x7ff6d8073b80 2023-07-21T22:21:29.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.305+0000 7ff6df9e6700 1 -- 172.21.15.107:0/219991529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6bc005320 con 0x7ff6d8073b80 2023-07-21T22:21:29.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.305+0000 7ff6ce7fc700 1 --2- 172.21.15.107:0/219991529 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7ff6c4038760 0x7ff6c403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:29.012 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.305+0000 7ff6ce7fc700 1 -- 172.21.15.107:0/219991529 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff6d404c860 con 0x7ff6d8073b80 2023-07-21T22:21:29.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.305+0000 7ff6dcf81700 1 --2- 172.21.15.107:0/219991529 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7ff6c4038760 0x7ff6c403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:29.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.305+0000 7ff6dcf81700 1 --2- 172.21.15.107:0/219991529 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7ff6c4038760 0x7ff6c403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8006fd0 tx=0x7ff6c8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:29.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.309+0000 7ff6ce7fc700 1 -- 172.21.15.107:0/219991529 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff6d4018db0 con 0x7ff6d8073b80 2023-07-21T22:21:29.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.494+0000 7ff6df9e6700 1 -- 172.21.15.107:0/219991529 --> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6bc000bf0 con 0x7ff6c4038760 2023-07-21T22:21:29.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.634+0000 7ff6ce7fc700 1 -- 172.21.15.107:0/219991529 <== mgr.14118 v2:172.21.15.107:6800/659042147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff6bc000bf0 con 0x7ff6c4038760 2023-07-21T22:21:29.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.636+0000 7ff6df9e6700 1 -- 172.21.15.107:0/219991529 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7ff6c4038760 msgr2=0x7ff6c403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:29.013 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.636+0000 7ff6df9e6700 1 --2- 172.21.15.107:0/219991529 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7ff6c4038760 0x7ff6c403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8006fd0 tx=0x7ff6c8006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:29.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.636+0000 7ff6df9e6700 1 -- 172.21.15.107:0/219991529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 msgr2=0x7ff6d819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:29.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.636+0000 7ff6df9e6700 1 --2- 172.21.15.107:0/219991529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 0x7ff6d819b340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009130 tx=0x7ff6d4011ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:29.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.636+0000 7ff6df9e6700 1 -- 172.21.15.107:0/219991529 shutdown_connections 2023-07-21T22:21:29.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.636+0000 7ff6df9e6700 1 --2- 172.21.15.107:0/219991529 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7ff6c4038760 0x7ff6c403ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:29.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.636+0000 7ff6df9e6700 1 --2- 172.21.15.107:0/219991529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6d8073b80 0x7ff6d819b340 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:29.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.636+0000 7ff6df9e6700 1 -- 172.21.15.107:0/219991529 >> 172.21.15.107:0/219991529 conn(0x7ff6d80ff8d0 msgr2=0x7ff6d8100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:29.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.636+0000 7ff6df9e6700 1 -- 172.21.15.107:0/219991529 shutdown_connections 2023-07-21T22:21:29.014 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:28.636+0000 7ff6df9e6700 1 -- 172.21.15.107:0/219991529 wait complete. 2023-07-21T22:21:29.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-mon[96543]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:29.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:29.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:28 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:30.045 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfWxRL+/ISZ4SJNsKH4cSZlYy4IO1VMxX3jALPtwymDUlUNNS4Kqpql8iEWOWc5IyOGBGjrP/D80WPp4MqnkUtfXFxM+9ke2eScz1c2hrAL8F4LaSThG5NE9HFE6gLbZ8VucKkzvdK3lmyB94ZYGlKD9vDr7p8qwTLX6y2t+Ate07THz4w6AmD/U19JRn50pjZ5LZ2egN9Eiuf4waQN1HfolIU7V1brjkmNKwQX6839xX1IqQnGYXiehw+X/lr/TOZiwCP/fHEzTbcmtmQscW2R4MnhGrHweLLpSxxOPfZi1Np4rmP3H7zGTKLRgHK/IIKLeBcyPDLj9j7Sz9jTtXBIaHTEvWTYwCvQuyBror7CHG4jJYoEagANiiDvHMQC2Z5wTFQC0V9YtF0ETZZreM0P33nBnfvyx1h8E1cDDsws19xnRty6wVwPRAgQykw7Ys0bgg9obVije8FXfRQMh/IP5pzNprrziTtpPaSxGmwBGr6AIhHhzpUBCwjQSz+QKc= ceph-af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:21:30.046 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.410+0000 7f789e1a1700 1 Processor -- start 2023-07-21T22:21:30.046 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.410+0000 7f789e1a1700 1 -- start start 2023-07-21T22:21:30.046 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.410+0000 7f789e1a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 0x7f7898103c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:30.046 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.410+0000 7f789e1a1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7898104170 con 0x7f7898103820 2023-07-21T22:21:30.046 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.411+0000 7f789d19f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 0x7f7898103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:30.046 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.411+0000 7f789d19f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 0x7f7898103c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45256/0 (socket says 172.21.15.107:45256) 2023-07-21T22:21:30.046 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.411+0000 7f789d19f700 1 -- 172.21.15.107:0/1575339091 learned_addr learned my addr 172.21.15.107:0/1575339091 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:30.047 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.411+0000 7f789d19f700 1 -- 172.21.15.107:0/1575339091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78981042b0 con 0x7f7898103820 2023-07-21T22:21:30.047 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.411+0000 7f789d19f700 1 --2- 172.21.15.107:0/1575339091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 0x7f7898103c30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f788c00aa40 tx=0x7f788c00ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a81125255b9af4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:30.047 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.411+0000 7f788bfff700 1 -- 172.21.15.107:0/1575339091 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f788c004160 con 0x7f7898103820 2023-07-21T22:21:30.047 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.411+0000 7f788bfff700 1 -- 172.21.15.107:0/1575339091 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f788c00cd90 con 0x7f7898103820 2023-07-21T22:21:30.047 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.411+0000 7f788bfff700 1 -- 172.21.15.107:0/1575339091 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f788c003c80 con 0x7f7898103820 2023-07-21T22:21:30.047 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.412+0000 7f789e1a1700 1 -- 172.21.15.107:0/1575339091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 msgr2=0x7f7898103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:30.047 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.412+0000 7f789e1a1700 1 --2- 172.21.15.107:0/1575339091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 0x7f7898103c30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f788c00aa40 tx=0x7f788c00ad50 comp rx=0 tx=0).stop 2023-07-21T22:21:30.048 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.412+0000 7f789e1a1700 1 -- 172.21.15.107:0/1575339091 shutdown_connections 2023-07-21T22:21:30.048 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.412+0000 7f789e1a1700 1 --2- 172.21.15.107:0/1575339091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 0x7f7898103c30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.048 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.412+0000 7f789e1a1700 1 -- 172.21.15.107:0/1575339091 >> 172.21.15.107:0/1575339091 conn(0x7f78980fee90 msgr2=0x7f78981012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:30.048 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.412+0000 7f789e1a1700 1 -- 172.21.15.107:0/1575339091 shutdown_connections 2023-07-21T22:21:30.048 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.412+0000 7f789e1a1700 1 -- 172.21.15.107:0/1575339091 wait complete. 2023-07-21T22:21:30.048 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.412+0000 7f789e1a1700 1 Processor -- start 2023-07-21T22:21:30.048 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.413+0000 7f789e1a1700 1 -- start start 2023-07-21T22:21:30.048 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.413+0000 7f789e1a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 0x7f7898198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:30.049 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.413+0000 7f789e1a1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78981994c0 con 0x7f7898103820 2023-07-21T22:21:30.049 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.413+0000 7f789d19f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 0x7f7898198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:30.049 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.413+0000 7f789d19f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 0x7f7898198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45258/0 (socket says 172.21.15.107:45258) 2023-07-21T22:21:30.049 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.413+0000 7f789d19f700 1 -- 172.21.15.107:0/1906651596 learned_addr learned my addr 172.21.15.107:0/1906651596 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:30.049 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.413+0000 7f789d19f700 1 -- 172.21.15.107:0/1906651596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c00a6f0 con 0x7f7898103820 2023-07-21T22:21:30.049 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.413+0000 7f789d19f700 1 --2- 172.21.15.107:0/1906651596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 0x7f7898198f80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f788c003880 tx=0x7f788c0041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:30.049 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.413+0000 7f788a7fc700 1 -- 172.21.15.107:0/1906651596 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f788c020400 con 0x7f7898103820 2023-07-21T22:21:30.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.413+0000 7f789e1a1700 1 -- 172.21.15.107:0/1906651596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78981996c0 con 0x7f7898103820 2023-07-21T22:21:30.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.413+0000 7f788a7fc700 1 -- 172.21.15.107:0/1906651596 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f788c020a00 con 0x7f7898103820 2023-07-21T22:21:30.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.413+0000 7f788a7fc700 1 -- 172.21.15.107:0/1906651596 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f788c0108e0 con 0x7f7898103820 2023-07-21T22:21:30.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.413+0000 7f789e1a1700 1 -- 172.21.15.107:0/1906651596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7898199ae0 con 0x7f7898103820 2023-07-21T22:21:30.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.414+0000 7f788a7fc700 1 -- 172.21.15.107:0/1906651596 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f788c020b70 con 0x7f7898103820 2023-07-21T22:21:30.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.414+0000 7f789e1a1700 1 -- 172.21.15.107:0/1906651596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7898192790 con 0x7f7898103820 2023-07-21T22:21:30.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.414+0000 7f788a7fc700 1 --2- 172.21.15.107:0/1906651596 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f78840386f0 0x7f788403aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:30.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.414+0000 7f788a7fc700 1 -- 172.21.15.107:0/1906651596 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f788c04c600 con 0x7f7898103820 2023-07-21T22:21:30.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.415+0000 7f789c99e700 1 --2- 172.21.15.107:0/1906651596 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f78840386f0 0x7f788403aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:30.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.415+0000 7f789c99e700 1 --2- 172.21.15.107:0/1906651596 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f78840386f0 0x7f788403aba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7894006fd0 tx=0x7f7894006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:30.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.419+0000 7f788a7fc700 1 -- 172.21.15.107:0/1906651596 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f788c020600 con 0x7f7898103820 2023-07-21T22:21:30.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.608+0000 7f789e1a1700 1 -- 172.21.15.107:0/1906651596 --> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f7898060950 con 0x7f78840386f0 2023-07-21T22:21:30.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.608+0000 7f788a7fc700 1 -- 172.21.15.107:0/1906651596 <== mgr.14118 v2:172.21.15.107:6800/659042147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f7898060950 con 0x7f78840386f0 2023-07-21T22:21:30.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.610+0000 7f789e1a1700 1 -- 172.21.15.107:0/1906651596 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f78840386f0 msgr2=0x7f788403aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:30.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.610+0000 7f789e1a1700 1 --2- 172.21.15.107:0/1906651596 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f78840386f0 0x7f788403aba0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7894006fd0 tx=0x7f7894006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:30.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.610+0000 7f789e1a1700 1 -- 172.21.15.107:0/1906651596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 msgr2=0x7f7898198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:30.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.610+0000 7f789e1a1700 1 --2- 172.21.15.107:0/1906651596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 0x7f7898198f80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f788c003880 tx=0x7f788c0041d0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.610+0000 7f789e1a1700 1 -- 172.21.15.107:0/1906651596 shutdown_connections 2023-07-21T22:21:30.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.610+0000 7f789e1a1700 1 --2- 172.21.15.107:0/1906651596 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f78840386f0 0x7f788403aba0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.610+0000 7f789e1a1700 1 --2- 172.21.15.107:0/1906651596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7898103820 0x7f7898198f80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.610+0000 7f789e1a1700 1 -- 172.21.15.107:0/1906651596 >> 172.21.15.107:0/1906651596 conn(0x7f78980fee90 msgr2=0x7f789818d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:30.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.611+0000 7f789e1a1700 1 -- 172.21.15.107:0/1906651596 shutdown_connections 2023-07-21T22:21:30.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:29.611+0000 7f789e1a1700 1 -- 172.21.15.107:0/1906651596 wait complete. 2023-07-21T22:21:30.053 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-21T22:21:30.053 INFO:teuthology.orchestra.run.smithi107.stdout:Adding key to root@localhost authorized_keys... 2023-07-21T22:21:30.053 INFO:teuthology.orchestra.run.smithi107.stdout:Adding host smithi107... 2023-07-21T22:21:30.249 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:29 smithi107 ceph-mon[96543]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:30.250 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:29 smithi107 ceph-mon[96543]: Generating ssh key... 2023-07-21T22:21:31.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:30 smithi107 ceph-mon[96543]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:32.353 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:31 smithi107 ceph-mon[96543]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:33.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Added host 'smithi107' with addr '172.21.15.107' 2023-07-21T22:21:33.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.451+0000 7f31a637e700 1 Processor -- start 2023-07-21T22:21:33.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.451+0000 7f31a637e700 1 -- start start 2023-07-21T22:21:33.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.451+0000 7f31a637e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0073ba0 0x7f31a0107f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:33.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.451+0000 7f31a637e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a0108460 con 0x7f31a0073ba0 2023-07-21T22:21:33.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.451+0000 7f319ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0073ba0 0x7f31a0107f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:33.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.451+0000 7f319ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0073ba0 0x7f31a0107f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45262/0 (socket says 172.21.15.107:45262) 2023-07-21T22:21:33.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.451+0000 7f319ffff700 1 -- 172.21.15.107:0/1252035774 learned_addr learned my addr 172.21.15.107:0/1252035774 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:33.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.451+0000 7f319ffff700 1 -- 172.21.15.107:0/1252035774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a01085a0 con 0x7f31a0073ba0 2023-07-21T22:21:33.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.451+0000 7f319ffff700 1 --2- 172.21.15.107:0/1252035774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0073ba0 0x7f31a0107f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f31940098a0 tx=0x7f3194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d6ddd8b1a54060 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:33.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.452+0000 7f319effd700 1 -- 172.21.15.107:0/1252035774 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3194004160 con 0x7f31a0073ba0 2023-07-21T22:21:33.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.452+0000 7f319effd700 1 -- 172.21.15.107:0/1252035774 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f31940042c0 con 0x7f31a0073ba0 2023-07-21T22:21:33.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.452+0000 7f319effd700 1 -- 172.21.15.107:0/1252035774 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3194011a00 con 0x7f31a0073ba0 2023-07-21T22:21:33.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.452+0000 7f31a637e700 1 -- 172.21.15.107:0/1252035774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0073ba0 msgr2=0x7f31a0107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:33.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.452+0000 7f31a637e700 1 --2- 172.21.15.107:0/1252035774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0073ba0 0x7f31a0107f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f31940098a0 tx=0x7f3194009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:33.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.452+0000 7f31a637e700 1 -- 172.21.15.107:0/1252035774 shutdown_connections 2023-07-21T22:21:33.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.452+0000 7f31a637e700 1 --2- 172.21.15.107:0/1252035774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a0073ba0 0x7f31a0107f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:33.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.452+0000 7f31a637e700 1 -- 172.21.15.107:0/1252035774 >> 172.21.15.107:0/1252035774 conn(0x7f31a00ff8d0 msgr2=0x7f31a0101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:33.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.452+0000 7f31a637e700 1 -- 172.21.15.107:0/1252035774 shutdown_connections 2023-07-21T22:21:33.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.452+0000 7f31a637e700 1 -- 172.21.15.107:0/1252035774 wait complete. 2023-07-21T22:21:33.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.453+0000 7f31a637e700 1 Processor -- start 2023-07-21T22:21:33.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.453+0000 7f31a637e700 1 -- start start 2023-07-21T22:21:33.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.453+0000 7f31a637e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a01990b0 0x7f31a01994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:33.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.453+0000 7f31a637e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a0199a00 con 0x7f31a01990b0 2023-07-21T22:21:33.302 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.453+0000 7f319ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a01990b0 0x7f31a01994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:33.303 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.453+0000 7f319ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a01990b0 0x7f31a01994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45264/0 (socket says 172.21.15.107:45264) 2023-07-21T22:21:33.303 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.453+0000 7f319ffff700 1 -- 172.21.15.107:0/3939565833 learned_addr learned my addr 172.21.15.107:0/3939565833 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:33.303 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.454+0000 7f319ffff700 1 -- 172.21.15.107:0/3939565833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3194009550 con 0x7f31a01990b0 2023-07-21T22:21:33.303 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.454+0000 7f319ffff700 1 --2- 172.21.15.107:0/3939565833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a01990b0 0x7f31a01994c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3194006af0 tx=0x7f3194011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:33.304 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.454+0000 7f319d7fa700 1 -- 172.21.15.107:0/3939565833 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3194003a40 con 0x7f31a01990b0 2023-07-21T22:21:33.304 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.454+0000 7f319d7fa700 1 -- 172.21.15.107:0/3939565833 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3194003ba0 con 0x7f31a01990b0 2023-07-21T22:21:33.304 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.454+0000 7f31a637e700 1 -- 172.21.15.107:0/3939565833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a0199c00 con 0x7f31a01990b0 2023-07-21T22:21:33.304 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.454+0000 7f319d7fa700 1 -- 172.21.15.107:0/3939565833 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f319401a700 con 0x7f31a01990b0 2023-07-21T22:21:33.305 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.454+0000 7f31a637e700 1 -- 172.21.15.107:0/3939565833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a019c860 con 0x7f31a01990b0 2023-07-21T22:21:33.305 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.455+0000 7f319d7fa700 1 -- 172.21.15.107:0/3939565833 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f319401a980 con 0x7f31a01990b0 2023-07-21T22:21:33.305 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.455+0000 7f31a637e700 1 -- 172.21.15.107:0/3939565833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a0073ba0 con 0x7f31a01990b0 2023-07-21T22:21:33.305 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.455+0000 7f319d7fa700 1 --2- 172.21.15.107:0/3939565833 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f3188038710 0x7f318803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:33.306 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.455+0000 7f319d7fa700 1 -- 172.21.15.107:0/3939565833 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f319404c740 con 0x7f31a01990b0 2023-07-21T22:21:33.306 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.455+0000 7f319f7fe700 1 --2- 172.21.15.107:0/3939565833 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f3188038710 0x7f318803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:33.306 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.455+0000 7f319f7fe700 1 --2- 172.21.15.107:0/3939565833 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f3188038710 0x7f318803abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3190006fd0 tx=0x7f3190006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:33.306 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.460+0000 7f319d7fa700 1 -- 172.21.15.107:0/3939565833 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f319402d960 con 0x7f31a01990b0 2023-07-21T22:21:33.306 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:30.647+0000 7f31a637e700 1 -- 172.21.15.107:0/3939565833 --> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}) v1 -- 0x7f31a0101bc0 con 0x7f3188038710 2023-07-21T22:21:33.307 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:32.908+0000 7f319d7fa700 1 -- 172.21.15.107:0/3939565833 <== mgr.14118 v2:172.21.15.107:6800/659042147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f31a0101bc0 con 0x7f3188038710 2023-07-21T22:21:33.307 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:32.911+0000 7f31a637e700 1 -- 172.21.15.107:0/3939565833 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f3188038710 msgr2=0x7f318803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:33.307 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:32.911+0000 7f31a637e700 1 --2- 172.21.15.107:0/3939565833 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f3188038710 0x7f318803abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3190006fd0 tx=0x7f3190006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:33.307 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:32.911+0000 7f31a637e700 1 -- 172.21.15.107:0/3939565833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a01990b0 msgr2=0x7f31a01994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:33.307 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:32.911+0000 7f31a637e700 1 --2- 172.21.15.107:0/3939565833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a01990b0 0x7f31a01994c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3194006af0 tx=0x7f3194011ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:33.307 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:32.911+0000 7f31a637e700 1 -- 172.21.15.107:0/3939565833 shutdown_connections 2023-07-21T22:21:33.307 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:32.911+0000 7f31a637e700 1 --2- 172.21.15.107:0/3939565833 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f3188038710 0x7f318803abc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:33.308 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:32.911+0000 7f31a637e700 1 --2- 172.21.15.107:0/3939565833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a01990b0 0x7f31a01994c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:33.308 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:32.911+0000 7f31a637e700 1 -- 172.21.15.107:0/3939565833 >> 172.21.15.107:0/3939565833 conn(0x7f31a00ff8d0 msgr2=0x7f31a0100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:33.308 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:32.911+0000 7f31a637e700 1 -- 172.21.15.107:0/3939565833 shutdown_connections 2023-07-21T22:21:33.308 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:32.911+0000 7f31a637e700 1 -- 172.21.15.107:0/3939565833 wait complete. 2023-07-21T22:21:33.308 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mon service... 2023-07-21T22:21:33.309 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:32 smithi107 ceph-mon[96543]: Deploying cephadm binary to smithi107 2023-07-21T22:21:33.309 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:32 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:33.309 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:32 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:34.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:33 smithi107 ceph-mon[96543]: Added host smithi107 2023-07-21T22:21:34.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:33 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:34.240 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-21T22:21:34.240 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.674+0000 7f5f42eeb700 1 Processor -- start 2023-07-21T22:21:34.240 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.674+0000 7f5f42eeb700 1 -- start start 2023-07-21T22:21:34.240 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.675+0000 7f5f42eeb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 0x7f5f3c1082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:34.241 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.675+0000 7f5f42eeb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c079d20 con 0x7f5f3c105ec0 2023-07-21T22:21:34.241 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.675+0000 7f5f40c87700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 0x7f5f3c1082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:34.241 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.675+0000 7f5f40c87700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 0x7f5f3c1082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45298/0 (socket says 172.21.15.107:45298) 2023-07-21T22:21:34.241 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.675+0000 7f5f40c87700 1 -- 172.21.15.107:0/1012933819 learned_addr learned my addr 172.21.15.107:0/1012933819 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:34.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.675+0000 7f5f40c87700 1 -- 172.21.15.107:0/1012933819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c079e60 con 0x7f5f3c105ec0 2023-07-21T22:21:34.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.675+0000 7f5f40c87700 1 --2- 172.21.15.107:0/1012933819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 0x7f5f3c1082a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f300098a0 tx=0x7f5f30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0a6fa208278b96 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:34.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.675+0000 7f5f3b7fe700 1 -- 172.21.15.107:0/1012933819 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f30004160 con 0x7f5f3c105ec0 2023-07-21T22:21:34.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.675+0000 7f5f3b7fe700 1 -- 172.21.15.107:0/1012933819 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f300042c0 con 0x7f5f3c105ec0 2023-07-21T22:21:34.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.675+0000 7f5f3b7fe700 1 -- 172.21.15.107:0/1012933819 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f30011a00 con 0x7f5f3c105ec0 2023-07-21T22:21:34.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.676+0000 7f5f42eeb700 1 -- 172.21.15.107:0/1012933819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 msgr2=0x7f5f3c1082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:34.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.676+0000 7f5f42eeb700 1 --2- 172.21.15.107:0/1012933819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 0x7f5f3c1082a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f300098a0 tx=0x7f5f30009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:34.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.676+0000 7f5f42eeb700 1 -- 172.21.15.107:0/1012933819 shutdown_connections 2023-07-21T22:21:34.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.676+0000 7f5f42eeb700 1 --2- 172.21.15.107:0/1012933819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 0x7f5f3c1082a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:34.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.676+0000 7f5f42eeb700 1 -- 172.21.15.107:0/1012933819 >> 172.21.15.107:0/1012933819 conn(0x7f5f3c0ff900 msgr2=0x7f5f3c101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:34.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.676+0000 7f5f42eeb700 1 -- 172.21.15.107:0/1012933819 shutdown_connections 2023-07-21T22:21:34.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.676+0000 7f5f42eeb700 1 -- 172.21.15.107:0/1012933819 wait complete. 2023-07-21T22:21:34.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.676+0000 7f5f42eeb700 1 Processor -- start 2023-07-21T22:21:34.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.677+0000 7f5f42eeb700 1 -- start start 2023-07-21T22:21:34.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.677+0000 7f5f42eeb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 0x7f5f3c105820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:34.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.677+0000 7f5f42eeb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c105d60 con 0x7f5f3c105ec0 2023-07-21T22:21:34.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.677+0000 7f5f40c87700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 0x7f5f3c105820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:34.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.677+0000 7f5f40c87700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 0x7f5f3c105820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45300/0 (socket says 172.21.15.107:45300) 2023-07-21T22:21:34.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.677+0000 7f5f40c87700 1 -- 172.21.15.107:0/2435479551 learned_addr learned my addr 172.21.15.107:0/2435479551 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:34.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.677+0000 7f5f40c87700 1 -- 172.21.15.107:0/2435479551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f30009550 con 0x7f5f3c105ec0 2023-07-21T22:21:34.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.677+0000 7f5f40c87700 1 --2- 172.21.15.107:0/2435479551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 0x7f5f3c105820 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009130 tx=0x7f5f30011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:34.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.677+0000 7f5f39ffb700 1 -- 172.21.15.107:0/2435479551 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f30003ab0 con 0x7f5f3c105ec0 2023-07-21T22:21:34.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.677+0000 7f5f39ffb700 1 -- 172.21.15.107:0/2435479551 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f30003c10 con 0x7f5f3c105ec0 2023-07-21T22:21:34.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.677+0000 7f5f42eeb700 1 -- 172.21.15.107:0/2435479551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3c103ed0 con 0x7f5f3c105ec0 2023-07-21T22:21:34.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.677+0000 7f5f39ffb700 1 -- 172.21.15.107:0/2435479551 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f300247d0 con 0x7f5f3c105ec0 2023-07-21T22:21:34.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.678+0000 7f5f42eeb700 1 -- 172.21.15.107:0/2435479551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c104350 con 0x7f5f3c105ec0 2023-07-21T22:21:34.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.678+0000 7f5f39ffb700 1 -- 172.21.15.107:0/2435479551 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f5f3002d3f0 con 0x7f5f3c105ec0 2023-07-21T22:21:34.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.678+0000 7f5f42eeb700 1 -- 172.21.15.107:0/2435479551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f3c061b40 con 0x7f5f3c105ec0 2023-07-21T22:21:34.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.678+0000 7f5f39ffb700 1 --2- 172.21.15.107:0/2435479551 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f5f2c038760 0x7f5f2c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:34.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.679+0000 7f5f39ffb700 1 -- 172.21.15.107:0/2435479551 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5f3004c960 con 0x7f5f3c105ec0 2023-07-21T22:21:34.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.679+0000 7f5f3bfff700 1 --2- 172.21.15.107:0/2435479551 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f5f2c038760 0x7f5f2c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:34.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.679+0000 7f5f3bfff700 1 --2- 172.21.15.107:0/2435479551 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f5f2c038760 0x7f5f2c03ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5f28006fd0 tx=0x7f5f28006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:34.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.683+0000 7f5f39ffb700 1 -- 172.21.15.107:0/2435479551 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5f30010970 con 0x7f5f3c105ec0 2023-07-21T22:21:34.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.887+0000 7f5f42eeb700 1 -- 172.21.15.107:0/2435479551 --> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5f3c104590 con 0x7f5f2c038760 2023-07-21T22:21:34.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.891+0000 7f5f39ffb700 1 -- 172.21.15.107:0/2435479551 <== mgr.14118 v2:172.21.15.107:6800/659042147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5f3c104590 con 0x7f5f2c038760 2023-07-21T22:21:34.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.893+0000 7f5f42eeb700 1 -- 172.21.15.107:0/2435479551 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f5f2c038760 msgr2=0x7f5f2c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:34.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.893+0000 7f5f42eeb700 1 --2- 172.21.15.107:0/2435479551 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f5f2c038760 0x7f5f2c03ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5f28006fd0 tx=0x7f5f28006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:34.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.893+0000 7f5f42eeb700 1 -- 172.21.15.107:0/2435479551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 msgr2=0x7f5f3c105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:34.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.893+0000 7f5f42eeb700 1 --2- 172.21.15.107:0/2435479551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 0x7f5f3c105820 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009130 tx=0x7f5f30011ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:34.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.894+0000 7f5f42eeb700 1 -- 172.21.15.107:0/2435479551 shutdown_connections 2023-07-21T22:21:34.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.894+0000 7f5f42eeb700 1 --2- 172.21.15.107:0/2435479551 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f5f2c038760 0x7f5f2c03ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:34.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.894+0000 7f5f42eeb700 1 --2- 172.21.15.107:0/2435479551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c105ec0 0x7f5f3c105820 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:34.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.894+0000 7f5f42eeb700 1 -- 172.21.15.107:0/2435479551 >> 172.21.15.107:0/2435479551 conn(0x7f5f3c0ff900 msgr2=0x7f5f3c101010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:34.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.894+0000 7f5f42eeb700 1 -- 172.21.15.107:0/2435479551 shutdown_connections 2023-07-21T22:21:34.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:33.894+0000 7f5f42eeb700 1 -- 172.21.15.107:0/2435479551 wait complete. 2023-07-21T22:21:34.248 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mgr service... 2023-07-21T22:21:35.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:34 smithi107 ceph-mon[96543]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:35.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:34 smithi107 ceph-mon[96543]: Saving service mon spec with placement count:5 2023-07-21T22:21:35.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-21T22:21:35.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.773+0000 7f755c4f6700 1 Processor -- start 2023-07-21T22:21:35.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.773+0000 7f755c4f6700 1 -- start start 2023-07-21T22:21:35.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.773+0000 7f755c4f6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 0x7f75540791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:35.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.773+0000 7f755c4f6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7554079730 con 0x7f755407acf0 2023-07-21T22:21:35.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.773+0000 7f755a292700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 0x7f75540791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:35.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.773+0000 7f755a292700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 0x7f75540791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45304/0 (socket says 172.21.15.107:45304) 2023-07-21T22:21:35.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.773+0000 7f755a292700 1 -- 172.21.15.107:0/3916293555 learned_addr learned my addr 172.21.15.107:0/3916293555 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:35.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.773+0000 7f755a292700 1 -- 172.21.15.107:0/3916293555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7554079870 con 0x7f755407acf0 2023-07-21T22:21:35.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.774+0000 7f755a292700 1 --2- 172.21.15.107:0/3916293555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 0x7f75540791f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f754c0098a0 tx=0x7f754c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14aefb3b052426ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:35.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.774+0000 7f7559290700 1 -- 172.21.15.107:0/3916293555 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f754c004160 con 0x7f755407acf0 2023-07-21T22:21:35.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.774+0000 7f7559290700 1 -- 172.21.15.107:0/3916293555 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f754c0042c0 con 0x7f755407acf0 2023-07-21T22:21:35.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.774+0000 7f7559290700 1 -- 172.21.15.107:0/3916293555 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f754c011a00 con 0x7f755407acf0 2023-07-21T22:21:35.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.774+0000 7f755c4f6700 1 -- 172.21.15.107:0/3916293555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 msgr2=0x7f75540791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:35.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.774+0000 7f755c4f6700 1 --2- 172.21.15.107:0/3916293555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 0x7f75540791f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f754c0098a0 tx=0x7f754c009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:35.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.774+0000 7f755c4f6700 1 -- 172.21.15.107:0/3916293555 shutdown_connections 2023-07-21T22:21:35.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.774+0000 7f755c4f6700 1 --2- 172.21.15.107:0/3916293555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 0x7f75540791f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:35.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.774+0000 7f755c4f6700 1 -- 172.21.15.107:0/3916293555 >> 172.21.15.107:0/3916293555 conn(0x7f75541009f0 msgr2=0x7f7554102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:35.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.774+0000 7f755c4f6700 1 -- 172.21.15.107:0/3916293555 shutdown_connections 2023-07-21T22:21:35.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.774+0000 7f755c4f6700 1 -- 172.21.15.107:0/3916293555 wait complete. 2023-07-21T22:21:35.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.775+0000 7f755c4f6700 1 Processor -- start 2023-07-21T22:21:35.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.775+0000 7f755c4f6700 1 -- start start 2023-07-21T22:21:35.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.775+0000 7f755c4f6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 0x7f755419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:35.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.775+0000 7f755c4f6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755419b860 con 0x7f755407acf0 2023-07-21T22:21:35.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.775+0000 7f755a292700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 0x7f755419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:35.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.775+0000 7f755a292700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 0x7f755419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45306/0 (socket says 172.21.15.107:45306) 2023-07-21T22:21:35.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.775+0000 7f755a292700 1 -- 172.21.15.107:0/4133683623 learned_addr learned my addr 172.21.15.107:0/4133683623 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:35.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.776+0000 7f755a292700 1 -- 172.21.15.107:0/4133683623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c009550 con 0x7f755407acf0 2023-07-21T22:21:35.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.776+0000 7f755a292700 1 --2- 172.21.15.107:0/4133683623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 0x7f755419b320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f754c009fb0 tx=0x7f754c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:35.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.776+0000 7f75537fe700 1 -- 172.21.15.107:0/4133683623 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f754c003930 con 0x7f755407acf0 2023-07-21T22:21:35.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.776+0000 7f75537fe700 1 -- 172.21.15.107:0/4133683623 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f754c003a90 con 0x7f755407acf0 2023-07-21T22:21:35.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.776+0000 7f755c4f6700 1 -- 172.21.15.107:0/4133683623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755419ba60 con 0x7f755407acf0 2023-07-21T22:21:35.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.776+0000 7f75537fe700 1 -- 172.21.15.107:0/4133683623 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f754c0195e0 con 0x7f755407acf0 2023-07-21T22:21:35.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.776+0000 7f755c4f6700 1 -- 172.21.15.107:0/4133683623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755419be80 con 0x7f755407acf0 2023-07-21T22:21:35.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.777+0000 7f75537fe700 1 -- 172.21.15.107:0/4133683623 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f754c0198a0 con 0x7f755407acf0 2023-07-21T22:21:35.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.777+0000 7f755c4f6700 1 -- 172.21.15.107:0/4133683623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7554061b40 con 0x7f755407acf0 2023-07-21T22:21:35.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.777+0000 7f75537fe700 1 --2- 172.21.15.107:0/4133683623 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f753c0386c0 0x7f753c03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:35.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.777+0000 7f7559a91700 1 --2- 172.21.15.107:0/4133683623 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f753c0386c0 0x7f753c03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:35.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.777+0000 7f75537fe700 1 -- 172.21.15.107:0/4133683623 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f754c04c9e0 con 0x7f755407acf0 2023-07-21T22:21:35.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.777+0000 7f7559a91700 1 --2- 172.21.15.107:0/4133683623 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f753c0386c0 0x7f753c03ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7548006fd0 tx=0x7f7548006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:35.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.782+0000 7f75537fe700 1 -- 172.21.15.107:0/4133683623 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f754c02b340 con 0x7f755407acf0 2023-07-21T22:21:35.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.983+0000 7f755c4f6700 1 -- 172.21.15.107:0/4133683623 --> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f755419e980 con 0x7f753c0386c0 2023-07-21T22:21:35.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.990+0000 7f75537fe700 1 -- 172.21.15.107:0/4133683623 <== mgr.14118 v2:172.21.15.107:6800/659042147 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f755419e980 con 0x7f753c0386c0 2023-07-21T22:21:35.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.993+0000 7f755c4f6700 1 -- 172.21.15.107:0/4133683623 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f753c0386c0 msgr2=0x7f753c03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:35.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.993+0000 7f755c4f6700 1 --2- 172.21.15.107:0/4133683623 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f753c0386c0 0x7f753c03ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7548006fd0 tx=0x7f7548006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:35.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.993+0000 7f755c4f6700 1 -- 172.21.15.107:0/4133683623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 msgr2=0x7f755419b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:35.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.993+0000 7f755c4f6700 1 --2- 172.21.15.107:0/4133683623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 0x7f755419b320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f754c009fb0 tx=0x7f754c011ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:35.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.993+0000 7f755c4f6700 1 -- 172.21.15.107:0/4133683623 shutdown_connections 2023-07-21T22:21:35.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.993+0000 7f755c4f6700 1 --2- 172.21.15.107:0/4133683623 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f753c0386c0 0x7f753c03ab70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:35.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.993+0000 7f755c4f6700 1 --2- 172.21.15.107:0/4133683623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f755407acf0 0x7f755419b320 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:35.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.993+0000 7f755c4f6700 1 -- 172.21.15.107:0/4133683623 >> 172.21.15.107:0/4133683623 conn(0x7f75541009f0 msgr2=0x7f75541020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:35.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.993+0000 7f755c4f6700 1 -- 172.21.15.107:0/4133683623 shutdown_connections 2023-07-21T22:21:35.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:34.993+0000 7f755c4f6700 1 -- 172.21.15.107:0/4133683623 wait complete. 2023-07-21T22:21:36.427 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:35 smithi107 ceph-mon[96543]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:36.427 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:35 smithi107 ceph-mon[96543]: Saving service mgr spec with placement count:2 2023-07-21T22:21:36.427 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:35 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:36.427 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:35 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:36.427 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:35 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:36.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.881+0000 7fcc69f0a700 1 Processor -- start 2023-07-21T22:21:36.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.881+0000 7fcc69f0a700 1 -- start start 2023-07-21T22:21:36.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.881+0000 7fcc69f0a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 0x7fcc64105cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:36.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.881+0000 7fcc69f0a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc64106230 con 0x7fcc641058e0 2023-07-21T22:21:36.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.881+0000 7fcc637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 0x7fcc64105cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:36.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.881+0000 7fcc637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 0x7fcc64105cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45310/0 (socket says 172.21.15.107:45310) 2023-07-21T22:21:36.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.881+0000 7fcc637fe700 1 -- 172.21.15.107:0/3779514232 learned_addr learned my addr 172.21.15.107:0/3779514232 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:36.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.882+0000 7fcc637fe700 1 -- 172.21.15.107:0/3779514232 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc64106370 con 0x7fcc641058e0 2023-07-21T22:21:36.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.882+0000 7fcc637fe700 1 --2- 172.21.15.107:0/3779514232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 0x7fcc64105cf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc540098a0 tx=0x7fcc54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8d39847943cb91 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:36.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.882+0000 7fcc627fc700 1 -- 172.21.15.107:0/3779514232 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc54004160 con 0x7fcc641058e0 2023-07-21T22:21:36.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.882+0000 7fcc627fc700 1 -- 172.21.15.107:0/3779514232 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc540042c0 con 0x7fcc641058e0 2023-07-21T22:21:36.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.882+0000 7fcc627fc700 1 -- 172.21.15.107:0/3779514232 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc540119b0 con 0x7fcc641058e0 2023-07-21T22:21:36.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.882+0000 7fcc69f0a700 1 -- 172.21.15.107:0/3779514232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 msgr2=0x7fcc64105cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:36.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.882+0000 7fcc69f0a700 1 --2- 172.21.15.107:0/3779514232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 0x7fcc64105cf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc540098a0 tx=0x7fcc54009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:36.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.882+0000 7fcc69f0a700 1 -- 172.21.15.107:0/3779514232 shutdown_connections 2023-07-21T22:21:36.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.882+0000 7fcc69f0a700 1 --2- 172.21.15.107:0/3779514232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 0x7fcc64105cf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:36.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.882+0000 7fcc69f0a700 1 -- 172.21.15.107:0/3779514232 >> 172.21.15.107:0/3779514232 conn(0x7fcc640ff040 msgr2=0x7fcc64101470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:36.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.883+0000 7fcc69f0a700 1 -- 172.21.15.107:0/3779514232 shutdown_connections 2023-07-21T22:21:36.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.883+0000 7fcc69f0a700 1 -- 172.21.15.107:0/3779514232 wait complete. 2023-07-21T22:21:36.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.883+0000 7fcc69f0a700 1 Processor -- start 2023-07-21T22:21:36.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.883+0000 7fcc69f0a700 1 -- start start 2023-07-21T22:21:36.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.883+0000 7fcc69f0a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 0x7fcc6419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:36.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.883+0000 7fcc69f0a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6419b840 con 0x7fcc641058e0 2023-07-21T22:21:36.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.884+0000 7fcc637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 0x7fcc6419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:36.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.884+0000 7fcc637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 0x7fcc6419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45312/0 (socket says 172.21.15.107:45312) 2023-07-21T22:21:36.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.884+0000 7fcc637fe700 1 -- 172.21.15.107:0/1073447450 learned_addr learned my addr 172.21.15.107:0/1073447450 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:36.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.884+0000 7fcc637fe700 1 -- 172.21.15.107:0/1073447450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc54009550 con 0x7fcc641058e0 2023-07-21T22:21:36.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.884+0000 7fcc637fe700 1 --2- 172.21.15.107:0/1073447450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 0x7fcc6419b300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc54003e60 tx=0x7fcc54011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:36.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.884+0000 7fcc60ff9700 1 -- 172.21.15.107:0/1073447450 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc54003970 con 0x7fcc641058e0 2023-07-21T22:21:36.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.884+0000 7fcc69f0a700 1 -- 172.21.15.107:0/1073447450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6419ba40 con 0x7fcc641058e0 2023-07-21T22:21:36.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.884+0000 7fcc60ff9700 1 -- 172.21.15.107:0/1073447450 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc54003ad0 con 0x7fcc641058e0 2023-07-21T22:21:36.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.884+0000 7fcc60ff9700 1 -- 172.21.15.107:0/1073447450 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc540195e0 con 0x7fcc641058e0 2023-07-21T22:21:36.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.884+0000 7fcc69f0a700 1 -- 172.21.15.107:0/1073447450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6419be60 con 0x7fcc641058e0 2023-07-21T22:21:36.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.885+0000 7fcc60ff9700 1 -- 172.21.15.107:0/1073447450 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fcc540198a0 con 0x7fcc641058e0 2023-07-21T22:21:36.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.885+0000 7fcc69f0a700 1 -- 172.21.15.107:0/1073447450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc64061b40 con 0x7fcc641058e0 2023-07-21T22:21:36.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.885+0000 7fcc60ff9700 1 --2- 172.21.15.107:0/1073447450 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fcc4c0386c0 0x7fcc4c03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:36.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.885+0000 7fcc60ff9700 1 -- 172.21.15.107:0/1073447450 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcc5404cab0 con 0x7fcc641058e0 2023-07-21T22:21:36.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.885+0000 7fcc62ffd700 1 --2- 172.21.15.107:0/1073447450 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fcc4c0386c0 0x7fcc4c03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:36.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.885+0000 7fcc62ffd700 1 --2- 172.21.15.107:0/1073447450 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fcc4c0386c0 0x7fcc4c03ab70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcc58006fd0 tx=0x7fcc58006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:36.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:35.890+0000 7fcc60ff9700 1 -- 172.21.15.107:0/1073447450 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcc5402b340 con 0x7fcc641058e0 2023-07-21T22:21:36.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:36.089+0000 7fcc69f0a700 1 -- 172.21.15.107:0/1073447450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fcc64061b40 con 0x7fcc641058e0 2023-07-21T22:21:36.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:36.095+0000 7fcc60ff9700 1 -- 172.21.15.107:0/1073447450 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fcc54018b40 con 0x7fcc641058e0 2023-07-21T22:21:36.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:36.097+0000 7fcc69f0a700 1 -- 172.21.15.107:0/1073447450 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fcc4c0386c0 msgr2=0x7fcc4c03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:36.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:36.097+0000 7fcc69f0a700 1 --2- 172.21.15.107:0/1073447450 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fcc4c0386c0 0x7fcc4c03ab70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcc58006fd0 tx=0x7fcc58006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:36.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:36.097+0000 7fcc69f0a700 1 -- 172.21.15.107:0/1073447450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 msgr2=0x7fcc6419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:36.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:36.097+0000 7fcc69f0a700 1 --2- 172.21.15.107:0/1073447450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 0x7fcc6419b300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc54003e60 tx=0x7fcc54011aa0 comp rx=0 tx=0).stop 2023-07-21T22:21:36.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:36.097+0000 7fcc69f0a700 1 -- 172.21.15.107:0/1073447450 shutdown_connections 2023-07-21T22:21:36.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:36.097+0000 7fcc69f0a700 1 --2- 172.21.15.107:0/1073447450 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fcc4c0386c0 0x7fcc4c03ab70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:36.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:36.097+0000 7fcc69f0a700 1 --2- 172.21.15.107:0/1073447450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc641058e0 0x7fcc6419b300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:36.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:36.097+0000 7fcc69f0a700 1 -- 172.21.15.107:0/1073447450 >> 172.21.15.107:0/1073447450 conn(0x7fcc640ff040 msgr2=0x7fcc641006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:36.601 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:36.098+0000 7fcc69f0a700 1 -- 172.21.15.107:0/1073447450 shutdown_connections 2023-07-21T22:21:36.602 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:36.098+0000 7fcc69f0a700 1 -- 172.21.15.107:0/1073447450 wait complete. 2023-07-21T22:21:37.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:37 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1073447450' entity='client.admin' 2023-07-21T22:21:37.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.222+0000 7f1620ea0700 1 Processor -- start 2023-07-21T22:21:37.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.222+0000 7f1620ea0700 1 -- start start 2023-07-21T22:21:37.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.223+0000 7f1620ea0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 0x7f161c108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:37.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.223+0000 7f1620ea0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c073c20 con 0x7f161c105e90 2023-07-21T22:21:37.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.223+0000 7f161a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 0x7f161c108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:37.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.223+0000 7f161a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 0x7f161c108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45316/0 (socket says 172.21.15.107:45316) 2023-07-21T22:21:37.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.223+0000 7f161a59c700 1 -- 172.21.15.107:0/480241696 learned_addr learned my addr 172.21.15.107:0/480241696 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:37.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.223+0000 7f161a59c700 1 -- 172.21.15.107:0/480241696 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c073d60 con 0x7f161c105e90 2023-07-21T22:21:37.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.223+0000 7f161a59c700 1 --2- 172.21.15.107:0/480241696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 0x7f161c108270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f160c0098a0 tx=0x7f160c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7225df2e61c4e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:37.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.224+0000 7f161959a700 1 -- 172.21.15.107:0/480241696 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f160c004160 con 0x7f161c105e90 2023-07-21T22:21:37.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.224+0000 7f161959a700 1 -- 172.21.15.107:0/480241696 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f160c0042c0 con 0x7f161c105e90 2023-07-21T22:21:37.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.224+0000 7f161959a700 1 -- 172.21.15.107:0/480241696 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f160c0119b0 con 0x7f161c105e90 2023-07-21T22:21:37.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.224+0000 7f1620ea0700 1 -- 172.21.15.107:0/480241696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 msgr2=0x7f161c108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:37.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.224+0000 7f1620ea0700 1 --2- 172.21.15.107:0/480241696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 0x7f161c108270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f160c0098a0 tx=0x7f160c009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:37.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.224+0000 7f1620ea0700 1 -- 172.21.15.107:0/480241696 shutdown_connections 2023-07-21T22:21:37.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.224+0000 7f1620ea0700 1 --2- 172.21.15.107:0/480241696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 0x7f161c108270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:37.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.224+0000 7f1620ea0700 1 -- 172.21.15.107:0/480241696 >> 172.21.15.107:0/480241696 conn(0x7f161c0ff8d0 msgr2=0x7f161c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:37.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.224+0000 7f1620ea0700 1 -- 172.21.15.107:0/480241696 shutdown_connections 2023-07-21T22:21:37.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.224+0000 7f1620ea0700 1 -- 172.21.15.107:0/480241696 wait complete. 2023-07-21T22:21:37.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.225+0000 7f1620ea0700 1 Processor -- start 2023-07-21T22:21:37.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.225+0000 7f1620ea0700 1 -- start start 2023-07-21T22:21:37.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.225+0000 7f1620ea0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 0x7f161c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:37.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.225+0000 7f1620ea0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c199640 con 0x7f161c105e90 2023-07-21T22:21:37.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.225+0000 7f161a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 0x7f161c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:37.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.225+0000 7f161a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 0x7f161c199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45318/0 (socket says 172.21.15.107:45318) 2023-07-21T22:21:37.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.225+0000 7f161a59c700 1 -- 172.21.15.107:0/1941190504 learned_addr learned my addr 172.21.15.107:0/1941190504 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:37.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.226+0000 7f161a59c700 1 -- 172.21.15.107:0/1941190504 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c009550 con 0x7f161c105e90 2023-07-21T22:21:37.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.226+0000 7f161a59c700 1 --2- 172.21.15.107:0/1941190504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 0x7f161c199100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f160c003e60 tx=0x7f160c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:37.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.226+0000 7f160b7fe700 1 -- 172.21.15.107:0/1941190504 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f160c003970 con 0x7f161c105e90 2023-07-21T22:21:37.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.226+0000 7f1620ea0700 1 -- 172.21.15.107:0/1941190504 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161c199840 con 0x7f161c105e90 2023-07-21T22:21:37.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.226+0000 7f160b7fe700 1 -- 172.21.15.107:0/1941190504 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f160c003ad0 con 0x7f161c105e90 2023-07-21T22:21:37.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.226+0000 7f160b7fe700 1 -- 172.21.15.107:0/1941190504 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f160c0195e0 con 0x7f161c105e90 2023-07-21T22:21:37.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.226+0000 7f1620ea0700 1 -- 172.21.15.107:0/1941190504 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161c199c60 con 0x7f161c105e90 2023-07-21T22:21:37.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.227+0000 7f160b7fe700 1 -- 172.21.15.107:0/1941190504 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f160c0198c0 con 0x7f161c105e90 2023-07-21T22:21:37.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.227+0000 7f1620ea0700 1 -- 172.21.15.107:0/1941190504 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161c1094c0 con 0x7f161c105e90 2023-07-21T22:21:37.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.227+0000 7f160b7fe700 1 --2- 172.21.15.107:0/1941190504 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f16040386c0 0x7f160403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:37.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.227+0000 7f160b7fe700 1 -- 172.21.15.107:0/1941190504 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f160c04c870 con 0x7f161c105e90 2023-07-21T22:21:37.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.227+0000 7f1619d9b700 1 --2- 172.21.15.107:0/1941190504 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f16040386c0 0x7f160403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:37.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.228+0000 7f1619d9b700 1 --2- 172.21.15.107:0/1941190504 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f16040386c0 0x7f160403ab70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1610006fd0 tx=0x7f1610006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:37.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.232+0000 7f160b7fe700 1 -- 172.21.15.107:0/1941190504 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f160c02b340 con 0x7f161c105e90 2023-07-21T22:21:37.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.428+0000 7f1620ea0700 1 -- 172.21.15.107:0/1941190504 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f161c1096b0 con 0x7f161c105e90 2023-07-21T22:21:37.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.433+0000 7f160b7fe700 1 -- 172.21.15.107:0/1941190504 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f160c018b40 con 0x7f161c105e90 2023-07-21T22:21:37.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.435+0000 7f1620ea0700 1 -- 172.21.15.107:0/1941190504 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f16040386c0 msgr2=0x7f160403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:37.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.435+0000 7f1620ea0700 1 --2- 172.21.15.107:0/1941190504 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f16040386c0 0x7f160403ab70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1610006fd0 tx=0x7f1610006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:37.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.435+0000 7f1620ea0700 1 -- 172.21.15.107:0/1941190504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 msgr2=0x7f161c199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:37.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.435+0000 7f1620ea0700 1 --2- 172.21.15.107:0/1941190504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 0x7f161c199100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f160c003e60 tx=0x7f160c011aa0 comp rx=0 tx=0).stop 2023-07-21T22:21:37.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.435+0000 7f1620ea0700 1 -- 172.21.15.107:0/1941190504 shutdown_connections 2023-07-21T22:21:37.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.435+0000 7f1620ea0700 1 --2- 172.21.15.107:0/1941190504 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f16040386c0 0x7f160403ab70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:37.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.435+0000 7f1620ea0700 1 --2- 172.21.15.107:0/1941190504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f161c105e90 0x7f161c199100 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:37.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.435+0000 7f1620ea0700 1 -- 172.21.15.107:0/1941190504 >> 172.21.15.107:0/1941190504 conn(0x7f161c0ff8d0 msgr2=0x7f161c100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:37.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.436+0000 7f1620ea0700 1 -- 172.21.15.107:0/1941190504 shutdown_connections 2023-07-21T22:21:37.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:37.436+0000 7f1620ea0700 1 -- 172.21.15.107:0/1941190504 wait complete. 2023-07-21T22:21:37.899 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling the dashboard module... 2023-07-21T22:21:38.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:38 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:38.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:38 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1941190504' entity='client.admin' 2023-07-21T22:21:38.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:38 smithi107 ceph-mon[96543]: from='mgr.14118 172.21.15.107:0/316279734' entity='mgr.x' 2023-07-21T22:21:39.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:39 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2171876503' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-21T22:21:39.437 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:39 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: ignoring --setuser ceph since I am not root 2023-07-21T22:21:39.437 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:39 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: ignoring --setgroup ceph since I am not root 2023-07-21T22:21:39.437 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:39 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:39.318+0000 7f93757c6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:21:39.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.296+0000 7fd795fa2700 1 Processor -- start 2023-07-21T22:21:39.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.296+0000 7fd795fa2700 1 -- start start 2023-07-21T22:21:39.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.296+0000 7fd795fa2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 0x7fd790107f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:39.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.296+0000 7fd795fa2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd790108480 con 0x7fd790079ca0 2023-07-21T22:21:39.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.296+0000 7fd78f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 0x7fd790107f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:39.738 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.297+0000 7fd78f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 0x7fd790107f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45322/0 (socket says 172.21.15.107:45322) 2023-07-21T22:21:39.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.297+0000 7fd78f7fe700 1 -- 172.21.15.107:0/1989957688 learned_addr learned my addr 172.21.15.107:0/1989957688 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:39.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.297+0000 7fd78f7fe700 1 -- 172.21.15.107:0/1989957688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7901085c0 con 0x7fd790079ca0 2023-07-21T22:21:39.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.297+0000 7fd78f7fe700 1 --2- 172.21.15.107:0/1989957688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 0x7fd790107f40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd7880098a0 tx=0x7fd788009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4863a5cbee347b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:39.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.297+0000 7fd78e7fc700 1 -- 172.21.15.107:0/1989957688 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd788004160 con 0x7fd790079ca0 2023-07-21T22:21:39.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.297+0000 7fd78e7fc700 1 -- 172.21.15.107:0/1989957688 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7880042c0 con 0x7fd790079ca0 2023-07-21T22:21:39.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.297+0000 7fd78e7fc700 1 -- 172.21.15.107:0/1989957688 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7880119b0 con 0x7fd790079ca0 2023-07-21T22:21:39.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.298+0000 7fd795fa2700 1 -- 172.21.15.107:0/1989957688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 msgr2=0x7fd790107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:39.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.298+0000 7fd795fa2700 1 --2- 172.21.15.107:0/1989957688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 0x7fd790107f40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd7880098a0 tx=0x7fd788009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:39.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.298+0000 7fd795fa2700 1 -- 172.21.15.107:0/1989957688 shutdown_connections 2023-07-21T22:21:39.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.298+0000 7fd795fa2700 1 --2- 172.21.15.107:0/1989957688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 0x7fd790107f40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:39.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.298+0000 7fd795fa2700 1 -- 172.21.15.107:0/1989957688 >> 172.21.15.107:0/1989957688 conn(0x7fd7900ff8d0 msgr2=0x7fd790101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:39.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.298+0000 7fd795fa2700 1 -- 172.21.15.107:0/1989957688 shutdown_connections 2023-07-21T22:21:39.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.298+0000 7fd795fa2700 1 -- 172.21.15.107:0/1989957688 wait complete. 2023-07-21T22:21:39.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.299+0000 7fd795fa2700 1 Processor -- start 2023-07-21T22:21:39.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.299+0000 7fd795fa2700 1 -- start start 2023-07-21T22:21:39.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.299+0000 7fd795fa2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 0x7fd790199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:39.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.299+0000 7fd795fa2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd790199680 con 0x7fd790079ca0 2023-07-21T22:21:39.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.299+0000 7fd78f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 0x7fd790199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:39.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.299+0000 7fd78f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 0x7fd790199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45324/0 (socket says 172.21.15.107:45324) 2023-07-21T22:21:39.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.299+0000 7fd78f7fe700 1 -- 172.21.15.107:0/2171876503 learned_addr learned my addr 172.21.15.107:0/2171876503 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:39.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.299+0000 7fd78f7fe700 1 -- 172.21.15.107:0/2171876503 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd788009550 con 0x7fd790079ca0 2023-07-21T22:21:39.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.300+0000 7fd78f7fe700 1 --2- 172.21.15.107:0/2171876503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 0x7fd790199140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd788000c00 tx=0x7fd788011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:39.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.300+0000 7fd78cff9700 1 -- 172.21.15.107:0/2171876503 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd788003b20 con 0x7fd790079ca0 2023-07-21T22:21:39.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.300+0000 7fd795fa2700 1 -- 172.21.15.107:0/2171876503 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd790199880 con 0x7fd790079ca0 2023-07-21T22:21:39.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.300+0000 7fd78cff9700 1 -- 172.21.15.107:0/2171876503 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd788003c80 con 0x7fd790079ca0 2023-07-21T22:21:39.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.300+0000 7fd78cff9700 1 -- 172.21.15.107:0/2171876503 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd788019770 con 0x7fd790079ca0 2023-07-21T22:21:39.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.300+0000 7fd795fa2700 1 -- 172.21.15.107:0/2171876503 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd790199ca0 con 0x7fd790079ca0 2023-07-21T22:21:39.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.301+0000 7fd78cff9700 1 -- 172.21.15.107:0/2171876503 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fd788019a30 con 0x7fd790079ca0 2023-07-21T22:21:39.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.301+0000 7fd795fa2700 1 -- 172.21.15.107:0/2171876503 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd790061b40 con 0x7fd790079ca0 2023-07-21T22:21:39.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.301+0000 7fd78cff9700 1 --2- 172.21.15.107:0/2171876503 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd778038710 0x7fd77803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:39.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.301+0000 7fd78cff9700 1 -- 172.21.15.107:0/2171876503 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd78804cb60 con 0x7fd790079ca0 2023-07-21T22:21:39.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.301+0000 7fd78effd700 1 --2- 172.21.15.107:0/2171876503 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd778038710 0x7fd77803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:39.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.301+0000 7fd78effd700 1 --2- 172.21.15.107:0/2171876503 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd778038710 0x7fd77803abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd784006fd0 tx=0x7fd784006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:39.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.305+0000 7fd78cff9700 1 -- 172.21.15.107:0/2171876503 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd78800b4d0 con 0x7fd790079ca0 2023-07-21T22:21:39.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:38.566+0000 7fd795fa2700 1 -- 172.21.15.107:0/2171876503 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fd79019c7d0 con 0x7fd790079ca0 2023-07-21T22:21:39.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:39.148+0000 7fd78cff9700 1 -- 172.21.15.107:0/2171876503 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fd788019ce0 con 0x7fd790079ca0 2023-07-21T22:21:39.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:39.150+0000 7fd78cff9700 1 -- 172.21.15.107:0/2171876503 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fd788018740 con 0x7fd790079ca0 2023-07-21T22:21:39.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:39.152+0000 7fd795fa2700 1 -- 172.21.15.107:0/2171876503 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd778038710 msgr2=0x7fd77803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:39.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:39.152+0000 7fd795fa2700 1 --2- 172.21.15.107:0/2171876503 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd778038710 0x7fd77803abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd784006fd0 tx=0x7fd784006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:39.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:39.152+0000 7fd795fa2700 1 -- 172.21.15.107:0/2171876503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 msgr2=0x7fd790199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:39.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:39.152+0000 7fd795fa2700 1 --2- 172.21.15.107:0/2171876503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 0x7fd790199140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd788000c00 tx=0x7fd788011a30 comp rx=0 tx=0).stop 2023-07-21T22:21:39.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:39.152+0000 7fd795fa2700 1 -- 172.21.15.107:0/2171876503 shutdown_connections 2023-07-21T22:21:39.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:39.152+0000 7fd795fa2700 1 --2- 172.21.15.107:0/2171876503 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd778038710 0x7fd77803abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:39.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:39.152+0000 7fd795fa2700 1 --2- 172.21.15.107:0/2171876503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd790079ca0 0x7fd790199140 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:39.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:39.152+0000 7fd795fa2700 1 -- 172.21.15.107:0/2171876503 >> 172.21.15.107:0/2171876503 conn(0x7fd7900ff8d0 msgr2=0x7fd790100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:39.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:39.152+0000 7fd795fa2700 1 -- 172.21.15.107:0/2171876503 shutdown_connections 2023-07-21T22:21:39.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:39.152+0000 7fd795fa2700 1 -- 172.21.15.107:0/2171876503 wait complete. 2023-07-21T22:21:39.751 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:39 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:39.401+0000 7f93757c6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:21:39.751 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:39 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:39.491+0000 7f93757c6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:21:39.752 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:39 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:39.586+0000 7f93757c6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:21:40.010 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:39 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:39.763+0000 7f93757c6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:21:40.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:40 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2171876503' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-21T22:21:40.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:40 smithi107 ceph-mon[96543]: mgrmap e9: x(active, since 14s) 2023-07-21T22:21:40.437 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:40 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:40.082+0000 7f93757c6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:21:40.437 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:40 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:40.173+0000 7f93757c6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:21:40.437 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:40 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:40.306+0000 7f93757c6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:21:40.878 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:40 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:40.402+0000 7f93757c6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:21:40.888 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:21:40.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-21T22:21:40.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T22:21:40.889 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-21T22:21:40.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-21T22:21:40.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:21:40.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.186+0000 7fd77b8ef700 1 Processor -- start 2023-07-21T22:21:40.890 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.186+0000 7fd77b8ef700 1 -- start start 2023-07-21T22:21:40.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.186+0000 7fd77b8ef700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 0x7fd774104680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:40.891 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.186+0000 7fd77b8ef700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd774104bc0 con 0x7fd774104270 2023-07-21T22:21:40.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.187+0000 7fd77968b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 0x7fd774104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:40.892 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.187+0000 7fd77968b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 0x7fd774104680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45338/0 (socket says 172.21.15.107:45338) 2023-07-21T22:21:40.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.187+0000 7fd77968b700 1 -- 172.21.15.107:0/2069565959 learned_addr learned my addr 172.21.15.107:0/2069565959 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:40.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.187+0000 7fd77968b700 1 -- 172.21.15.107:0/2069565959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774104d00 con 0x7fd774104270 2023-07-21T22:21:40.893 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.187+0000 7fd77968b700 1 --2- 172.21.15.107:0/2069565959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 0x7fd774104680 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd7700098a0 tx=0x7fd770009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60878a7ec30e6f53 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:40.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.187+0000 7fd76bfff700 1 -- 172.21.15.107:0/2069565959 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd770004160 con 0x7fd774104270 2023-07-21T22:21:40.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.187+0000 7fd76bfff700 1 -- 172.21.15.107:0/2069565959 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7700042c0 con 0x7fd774104270 2023-07-21T22:21:40.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.187+0000 7fd76bfff700 1 -- 172.21.15.107:0/2069565959 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7700119b0 con 0x7fd774104270 2023-07-21T22:21:40.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.188+0000 7fd77b8ef700 1 -- 172.21.15.107:0/2069565959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 msgr2=0x7fd774104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:40.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.188+0000 7fd77b8ef700 1 --2- 172.21.15.107:0/2069565959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 0x7fd774104680 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd7700098a0 tx=0x7fd770009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:40.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.188+0000 7fd77b8ef700 1 -- 172.21.15.107:0/2069565959 shutdown_connections 2023-07-21T22:21:40.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.188+0000 7fd77b8ef700 1 --2- 172.21.15.107:0/2069565959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 0x7fd774104680 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:40.895 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.188+0000 7fd77b8ef700 1 -- 172.21.15.107:0/2069565959 >> 172.21.15.107:0/2069565959 conn(0x7fd7740ff900 msgr2=0x7fd774101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:40.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.188+0000 7fd77b8ef700 1 -- 172.21.15.107:0/2069565959 shutdown_connections 2023-07-21T22:21:40.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.188+0000 7fd77b8ef700 1 -- 172.21.15.107:0/2069565959 wait complete. 2023-07-21T22:21:40.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.189+0000 7fd77b8ef700 1 Processor -- start 2023-07-21T22:21:40.896 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.189+0000 7fd77b8ef700 1 -- start start 2023-07-21T22:21:40.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.189+0000 7fd77b8ef700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 0x7fd77418f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:40.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.189+0000 7fd77b8ef700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77418f950 con 0x7fd774104270 2023-07-21T22:21:40.897 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.189+0000 7fd77968b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 0x7fd77418f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:40.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.189+0000 7fd77968b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 0x7fd77418f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45340/0 (socket says 172.21.15.107:45340) 2023-07-21T22:21:40.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.189+0000 7fd77968b700 1 -- 172.21.15.107:0/1450825817 learned_addr learned my addr 172.21.15.107:0/1450825817 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:40.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.189+0000 7fd77968b700 1 -- 172.21.15.107:0/1450825817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd770009550 con 0x7fd774104270 2023-07-21T22:21:40.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.189+0000 7fd77968b700 1 --2- 172.21.15.107:0/1450825817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 0x7fd77418f410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd770006af0 tx=0x7fd770011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:40.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.189+0000 7fd76a7fc700 1 -- 172.21.15.107:0/1450825817 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd770003a40 con 0x7fd774104270 2023-07-21T22:21:40.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.189+0000 7fd76a7fc700 1 -- 172.21.15.107:0/1450825817 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd770003ba0 con 0x7fd774104270 2023-07-21T22:21:40.898 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.189+0000 7fd77b8ef700 1 -- 172.21.15.107:0/1450825817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77418da60 con 0x7fd774104270 2023-07-21T22:21:40.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.190+0000 7fd76a7fc700 1 -- 172.21.15.107:0/1450825817 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7700196c0 con 0x7fd774104270 2023-07-21T22:21:40.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.190+0000 7fd77b8ef700 1 -- 172.21.15.107:0/1450825817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77418de80 con 0x7fd774104270 2023-07-21T22:21:40.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.190+0000 7fd76a7fc700 1 -- 172.21.15.107:0/1450825817 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fd770019980 con 0x7fd774104270 2023-07-21T22:21:40.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.190+0000 7fd76a7fc700 1 --2- 172.21.15.107:0/1450825817 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd760038710 0x7fd76003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:40.899 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.190+0000 7fd76a7fc700 1 -- 172.21.15.107:0/1450825817 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd77004c6e0 con 0x7fd774104270 2023-07-21T22:21:40.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.191+0000 7fd778e8a700 1 -- 172.21.15.107:0/1450825817 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd760038710 msgr2=0x7fd76003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/659042147 2023-07-21T22:21:40.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.191+0000 7fd778e8a700 1 --2- 172.21.15.107:0/1450825817 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd760038710 0x7fd76003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:21:40.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.191+0000 7fd77b8ef700 1 -- 172.21.15.107:0/1450825817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd774061b40 con 0x7fd774104270 2023-07-21T22:21:40.900 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.196+0000 7fd76a7fc700 1 -- 172.21.15.107:0/1450825817 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd7700243f0 con 0x7fd774104270 2023-07-21T22:21:40.901 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.392+0000 7fd778e8a700 1 -- 172.21.15.107:0/1450825817 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd760038710 msgr2=0x7fd76003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/659042147 2023-07-21T22:21:40.901 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.392+0000 7fd778e8a700 1 --2- 172.21.15.107:0/1450825817 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd760038710 0x7fd76003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:21:40.901 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.441+0000 7fd77b8ef700 1 -- 172.21.15.107:0/1450825817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fd77418e6b0 con 0x7fd774104270 2023-07-21T22:21:40.901 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.441+0000 7fd76a7fc700 1 -- 172.21.15.107:0/1450825817 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fd77004a030 con 0x7fd774104270 2023-07-21T22:21:40.901 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.443+0000 7fd77b8ef700 1 -- 172.21.15.107:0/1450825817 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd760038710 msgr2=0x7fd76003abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:21:40.901 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.443+0000 7fd77b8ef700 1 --2- 172.21.15.107:0/1450825817 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd760038710 0x7fd76003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:40.902 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.443+0000 7fd77b8ef700 1 -- 172.21.15.107:0/1450825817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 msgr2=0x7fd77418f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:40.902 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.443+0000 7fd77b8ef700 1 --2- 172.21.15.107:0/1450825817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 0x7fd77418f410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd770006af0 tx=0x7fd770011a10 comp rx=0 tx=0).stop 2023-07-21T22:21:40.902 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.443+0000 7fd77b8ef700 1 -- 172.21.15.107:0/1450825817 shutdown_connections 2023-07-21T22:21:40.902 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.443+0000 7fd77b8ef700 1 --2- 172.21.15.107:0/1450825817 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7fd760038710 0x7fd76003abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:40.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.443+0000 7fd77b8ef700 1 --2- 172.21.15.107:0/1450825817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd774104270 0x7fd77418f410 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:40.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.443+0000 7fd77b8ef700 1 -- 172.21.15.107:0/1450825817 >> 172.21.15.107:0/1450825817 conn(0x7fd7740ff900 msgr2=0x7fd774101030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:40.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.443+0000 7fd77b8ef700 1 -- 172.21.15.107:0/1450825817 shutdown_connections 2023-07-21T22:21:40.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:40.443+0000 7fd77b8ef700 1 -- 172.21.15.107:0/1450825817 wait complete. 2023-07-21T22:21:40.903 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2023-07-21T22:21:40.904 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 9... 2023-07-21T22:21:41.186 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:40 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:40.837+0000 7f93757c6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:21:41.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:41 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1450825817' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-21T22:21:41.937 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:41 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:41.427+0000 7f93757c6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:21:41.937 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:41 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:41.595+0000 7f93757c6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:21:42.659 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:42 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:42.352+0000 7f93757c6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:21:42.659 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:42 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:42.443+0000 7f93757c6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:21:42.659 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:42 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:42.613+0000 7f93757c6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:21:43.312 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:42 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:42.891+0000 7f93757c6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:21:43.312 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:43 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:43.118+0000 7f93757c6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:21:43.936 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:43 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:43.267+0000 7f93757c6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:21:43.937 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:43 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:43.528+0000 7f93757c6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:21:44.579 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:44 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:44.215+0000 7f93757c6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:21:44.579 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:44 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:44.344+0000 7f93757c6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:21:44.904 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:44 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:44.533+0000 7f93757c6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:21:44.905 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:44 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:44.646+0000 7f93757c6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:21:46.186 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:44 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:44.859+0000 7f93757c6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:21:46.187 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:21:45 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:21:45.700+0000 7f93757c6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:21:46.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:45 smithi107 ceph-mon[96543]: Active manager daemon x restarted 2023-07-21T22:21:46.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:45 smithi107 ceph-mon[96543]: Activating manager daemon x 2023-07-21T22:21:46.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:45 smithi107 ceph-mon[96543]: osdmap e3: 0 total, 0 up, 0 in 2023-07-21T22:21:46.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:45 smithi107 ceph-mon[96543]: mgrmap e10: x(active, starting, since 0.00319759s) 2023-07-21T22:21:46.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:45 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:21:46.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:45 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:21:46.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:45 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:21:46.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:45 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:21:46.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:45 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:21:46.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:45 smithi107 ceph-mon[96543]: Manager daemon x is now available 2023-07-21T22:21:46.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:45 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:46.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:45 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:47.166 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:21:47.167 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:21:47.167 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:46 smithi107 ceph-mon[96543]: mgrmap e11: x(active, since 1.00537s) 2023-07-21T22:21:47.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:21:47.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-21T22:21:47.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-21T22:21:47.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:21:47.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.285+0000 7f49df63f700 1 Processor -- start 2023-07-21T22:21:47.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.286+0000 7f49df63f700 1 -- start start 2023-07-21T22:21:47.251 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.286+0000 7f49df63f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d8103660 0x7f49d8103a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:47.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.286+0000 7f49df63f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8103fb0 con 0x7f49d8103660 2023-07-21T22:21:47.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.286+0000 7f49dd3db700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d8103660 0x7f49d8103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:47.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.286+0000 7f49dd3db700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d8103660 0x7f49d8103a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45348/0 (socket says 172.21.15.107:45348) 2023-07-21T22:21:47.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.286+0000 7f49dd3db700 1 -- 172.21.15.107:0/3264873384 learned_addr learned my addr 172.21.15.107:0/3264873384 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:47.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.287+0000 7f49dd3db700 1 -- 172.21.15.107:0/3264873384 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d81040f0 con 0x7f49d8103660 2023-07-21T22:21:47.252 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.287+0000 7f49dd3db700 1 --2- 172.21.15.107:0/3264873384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d8103660 0x7f49d8103a70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49d40098a0 tx=0x7f49d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a3fa5858841767 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:47.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.287+0000 7f49cffff700 1 -- 172.21.15.107:0/3264873384 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d4004160 con 0x7f49d8103660 2023-07-21T22:21:47.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.287+0000 7f49cffff700 1 -- 172.21.15.107:0/3264873384 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49d40042c0 con 0x7f49d8103660 2023-07-21T22:21:47.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.287+0000 7f49cffff700 1 -- 172.21.15.107:0/3264873384 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d4011960 con 0x7f49d8103660 2023-07-21T22:21:47.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.288+0000 7f49df63f700 1 -- 172.21.15.107:0/3264873384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d8103660 msgr2=0x7f49d8103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:47.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.288+0000 7f49df63f700 1 --2- 172.21.15.107:0/3264873384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d8103660 0x7f49d8103a70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49d40098a0 tx=0x7f49d4009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:47.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.288+0000 7f49df63f700 1 -- 172.21.15.107:0/3264873384 shutdown_connections 2023-07-21T22:21:47.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.288+0000 7f49df63f700 1 --2- 172.21.15.107:0/3264873384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d8103660 0x7f49d8103a70 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49d40098a0 tx=0x7f49d4009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:47.253 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.288+0000 7f49df63f700 1 -- 172.21.15.107:0/3264873384 >> 172.21.15.107:0/3264873384 conn(0x7f49d80ff040 msgr2=0x7f49d8101470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:47.254 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.289+0000 7f49df63f700 1 -- 172.21.15.107:0/3264873384 shutdown_connections 2023-07-21T22:21:47.254 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.289+0000 7f49df63f700 1 -- 172.21.15.107:0/3264873384 wait complete. 2023-07-21T22:21:47.254 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.289+0000 7f49df63f700 1 Processor -- start 2023-07-21T22:21:47.254 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.289+0000 7f49df63f700 1 -- start start 2023-07-21T22:21:47.254 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.290+0000 7f49df63f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d819f950 0x7f49d819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:47.255 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.290+0000 7f49df63f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d81a02a0 con 0x7f49d819f950 2023-07-21T22:21:47.255 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.290+0000 7f49dd3db700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d819f950 0x7f49d819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:47.255 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.290+0000 7f49dd3db700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d819f950 0x7f49d819fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45350/0 (socket says 172.21.15.107:45350) 2023-07-21T22:21:47.255 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.290+0000 7f49dd3db700 1 -- 172.21.15.107:0/1494430900 learned_addr learned my addr 172.21.15.107:0/1494430900 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:47.255 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.290+0000 7f49dd3db700 1 -- 172.21.15.107:0/1494430900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d4009550 con 0x7f49d819f950 2023-07-21T22:21:47.255 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.290+0000 7f49dd3db700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d819f950 0x7f49d819fd60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f49d4000c00 tx=0x7f49d4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:47.255 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.290+0000 7f49ce7fc700 1 -- 172.21.15.107:0/1494430900 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d40036a0 con 0x7f49d819f950 2023-07-21T22:21:47.256 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.290+0000 7f49ce7fc700 1 -- 172.21.15.107:0/1494430900 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49d401f070 con 0x7f49d819f950 2023-07-21T22:21:47.256 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.290+0000 7f49ce7fc700 1 -- 172.21.15.107:0/1494430900 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d4026df0 con 0x7f49d819f950 2023-07-21T22:21:47.256 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.291+0000 7f49df63f700 1 -- 172.21.15.107:0/1494430900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d81a04a0 con 0x7f49d819f950 2023-07-21T22:21:47.256 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.291+0000 7f49df63f700 1 -- 172.21.15.107:0/1494430900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d81a3100 con 0x7f49d819f950 2023-07-21T22:21:47.256 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.293+0000 7f49ce7fc700 1 -- 172.21.15.107:0/1494430900 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f49d40265f0 con 0x7f49d819f950 2023-07-21T22:21:47.256 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.293+0000 7f49ce7fc700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 0x7f49c403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:47.256 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.293+0000 7f49ce7fc700 1 -- 172.21.15.107:0/1494430900 --> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49c403b3c0 con 0x7f49c4038800 2023-07-21T22:21:47.257 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.293+0000 7f49ce7fc700 1 -- 172.21.15.107:0/1494430900 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f49d4021070 con 0x7f49d819f950 2023-07-21T22:21:47.257 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.294+0000 7f49dcbda700 1 -- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 msgr2=0x7f49c403acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/659042147 2023-07-21T22:21:47.257 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.294+0000 7f49dcbda700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 0x7f49c403acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:21:47.257 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.495+0000 7f49dcbda700 1 -- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 msgr2=0x7f49c403acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/659042147 2023-07-21T22:21:47.257 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.495+0000 7f49dcbda700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 0x7f49c403acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:21:47.257 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.896+0000 7f49dcbda700 1 -- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 msgr2=0x7f49c403acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/659042147 2023-07-21T22:21:47.257 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:41.896+0000 7f49dcbda700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 0x7f49c403acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-21T22:21:47.257 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:42.697+0000 7f49dcbda700 1 -- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 msgr2=0x7f49c403acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/659042147 2023-07-21T22:21:47.258 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:42.697+0000 7f49dcbda700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 0x7f49c403acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-21T22:21:47.258 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:44.298+0000 7f49dcbda700 1 -- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 msgr2=0x7f49c403acb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/659042147 2023-07-21T22:21:47.258 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:44.298+0000 7f49dcbda700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 0x7f49c403acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-21T22:21:47.258 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:45.715+0000 7f49ce7fc700 1 -- 172.21.15.107:0/1494430900 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7f49d40039e0 con 0x7f49d819f950 2023-07-21T22:21:47.258 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:45.715+0000 7f49ce7fc700 1 -- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 msgr2=0x7f49c403acb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:21:47.258 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:45.715+0000 7f49ce7fc700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 0x7f49c403acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:47.258 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.716+0000 7f49ce7fc700 1 -- 172.21.15.107:0/1494430900 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f49d404dd20 con 0x7f49d819f950 2023-07-21T22:21:47.258 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.716+0000 7f49ce7fc700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f49c403bff0 0x7f49c403e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:47.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.717+0000 7f49ce7fc700 1 -- 172.21.15.107:0/1494430900 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49c403b3c0 con 0x7f49c403bff0 2023-07-21T22:21:47.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.717+0000 7f49dcbda700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f49c403bff0 0x7f49c403e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:47.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.718+0000 7f49dcbda700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f49c403bff0 0x7f49c403e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f49c8003a10 tx=0x7f49c80092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:47.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.718+0000 7f49ce7fc700 1 -- 172.21.15.107:0/1494430900 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f49c403b3c0 con 0x7f49c403bff0 2023-07-21T22:21:47.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.724+0000 7f49df63f700 1 -- 172.21.15.107:0/1494430900 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f49d81a0cd0 con 0x7f49c403bff0 2023-07-21T22:21:47.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.725+0000 7f49ce7fc700 1 -- 172.21.15.107:0/1494430900 <== mgr.14150 v2:172.21.15.107:6800/1657248266 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f49d81a0cd0 con 0x7f49c403bff0 2023-07-21T22:21:47.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.725+0000 7f49df63f700 1 -- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f49c403bff0 msgr2=0x7f49c403e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:47.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.725+0000 7f49df63f700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f49c403bff0 0x7f49c403e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f49c8003a10 tx=0x7f49c80092b0 comp rx=0 tx=0).stop 2023-07-21T22:21:47.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.725+0000 7f49df63f700 1 -- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d819f950 msgr2=0x7f49d819fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:47.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.725+0000 7f49df63f700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d819f950 0x7f49d819fd60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f49d4000c00 tx=0x7f49d4011ac0 comp rx=0 tx=0).stop 2023-07-21T22:21:47.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.725+0000 7f49df63f700 1 -- 172.21.15.107:0/1494430900 shutdown_connections 2023-07-21T22:21:47.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.725+0000 7f49df63f700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f49c403bff0 0x7f49c403e3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:47.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.725+0000 7f49df63f700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49d819f950 0x7f49d819fd60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:47.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.725+0000 7f49df63f700 1 --2- 172.21.15.107:0/1494430900 >> [v2:172.21.15.107:6800/659042147,v1:172.21.15.107:6801/659042147] conn(0x7f49c4038800 0x7f49c403acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:47.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.725+0000 7f49df63f700 1 -- 172.21.15.107:0/1494430900 >> 172.21.15.107:0/1494430900 conn(0x7f49d80ff040 msgr2=0x7f49d8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:47.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.725+0000 7f49df63f700 1 -- 172.21.15.107:0/1494430900 shutdown_connections 2023-07-21T22:21:47.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:46.725+0000 7f49df63f700 1 -- 172.21.15.107:0/1494430900 wait complete. 2023-07-21T22:21:47.261 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 9 is available 2023-07-21T22:21:47.261 INFO:teuthology.orchestra.run.smithi107.stdout:Generating a dashboard self-signed certificate... 2023-07-21T22:21:48.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:47 smithi107 ceph-mon[96543]: [21/Jul/2023:22:21:47] ENGINE Bus STARTING 2023-07-21T22:21:48.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:21:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-21T22:21:48.395 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.649+0000 7f63eb665700 1 Processor -- start 2023-07-21T22:21:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.649+0000 7f63eb665700 1 -- start start 2023-07-21T22:21:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.649+0000 7f63eb665700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 0x7f63e4104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.649+0000 7f63eb665700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4104bd0 con 0x7f63e4104280 2023-07-21T22:21:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.650+0000 7f63e9401700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 0x7f63e4104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.650+0000 7f63e9401700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 0x7f63e4104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45406/0 (socket says 172.21.15.107:45406) 2023-07-21T22:21:48.396 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.650+0000 7f63e9401700 1 -- 172.21.15.107:0/2631990586 learned_addr learned my addr 172.21.15.107:0/2631990586 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.650+0000 7f63e9401700 1 -- 172.21.15.107:0/2631990586 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e4104d10 con 0x7f63e4104280 2023-07-21T22:21:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.650+0000 7f63e9401700 1 --2- 172.21.15.107:0/2631990586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 0x7f63e4104690 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63e00098a0 tx=0x7f63e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7de805a8219cc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.650+0000 7f63dbfff700 1 -- 172.21.15.107:0/2631990586 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63e0004160 con 0x7f63e4104280 2023-07-21T22:21:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.651+0000 7f63dbfff700 1 -- 172.21.15.107:0/2631990586 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63e00042c0 con 0x7f63e4104280 2023-07-21T22:21:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.651+0000 7f63dbfff700 1 -- 172.21.15.107:0/2631990586 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63e00119b0 con 0x7f63e4104280 2023-07-21T22:21:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.651+0000 7f63eb665700 1 -- 172.21.15.107:0/2631990586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 msgr2=0x7f63e4104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.651+0000 7f63eb665700 1 --2- 172.21.15.107:0/2631990586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 0x7f63e4104690 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63e00098a0 tx=0x7f63e0009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.651+0000 7f63eb665700 1 -- 172.21.15.107:0/2631990586 shutdown_connections 2023-07-21T22:21:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.651+0000 7f63eb665700 1 --2- 172.21.15.107:0/2631990586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 0x7f63e4104690 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.651+0000 7f63eb665700 1 -- 172.21.15.107:0/2631990586 >> 172.21.15.107:0/2631990586 conn(0x7f63e40ff8d0 msgr2=0x7f63e4101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.651+0000 7f63eb665700 1 -- 172.21.15.107:0/2631990586 shutdown_connections 2023-07-21T22:21:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.651+0000 7f63eb665700 1 -- 172.21.15.107:0/2631990586 wait complete. 2023-07-21T22:21:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.652+0000 7f63eb665700 1 Processor -- start 2023-07-21T22:21:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.652+0000 7f63eb665700 1 -- start start 2023-07-21T22:21:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.652+0000 7f63eb665700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 0x7f63e4199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.652+0000 7f63eb665700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e41995d0 con 0x7f63e4104280 2023-07-21T22:21:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.652+0000 7f63e9401700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 0x7f63e4199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.652+0000 7f63e9401700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 0x7f63e4199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45408/0 (socket says 172.21.15.107:45408) 2023-07-21T22:21:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.652+0000 7f63e9401700 1 -- 172.21.15.107:0/1415490562 learned_addr learned my addr 172.21.15.107:0/1415490562 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.653+0000 7f63e9401700 1 -- 172.21.15.107:0/1415490562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e0009550 con 0x7f63e4104280 2023-07-21T22:21:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.653+0000 7f63e9401700 1 --2- 172.21.15.107:0/1415490562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 0x7f63e4199090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63e0011980 tx=0x7f63e0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:48.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.653+0000 7f63da7fc700 1 -- 172.21.15.107:0/1415490562 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63e0003a60 con 0x7f63e4104280 2023-07-21T22:21:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.653+0000 7f63da7fc700 1 -- 172.21.15.107:0/1415490562 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63e0003bc0 con 0x7f63e4104280 2023-07-21T22:21:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.653+0000 7f63eb665700 1 -- 172.21.15.107:0/1415490562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e41997d0 con 0x7f63e4104280 2023-07-21T22:21:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.653+0000 7f63da7fc700 1 -- 172.21.15.107:0/1415490562 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63e001a670 con 0x7f63e4104280 2023-07-21T22:21:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.653+0000 7f63eb665700 1 -- 172.21.15.107:0/1415490562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e4199bf0 con 0x7f63e4104280 2023-07-21T22:21:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.654+0000 7f63da7fc700 1 -- 172.21.15.107:0/1415490562 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f63e001a930 con 0x7f63e4104280 2023-07-21T22:21:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.654+0000 7f63eb665700 1 -- 172.21.15.107:0/1415490562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e4061b40 con 0x7f63e4104280 2023-07-21T22:21:48.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.654+0000 7f63da7fc700 1 --2- 172.21.15.107:0/1415490562 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f63d0038630 0x7f63d003aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.654+0000 7f63da7fc700 1 -- 172.21.15.107:0/1415490562 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f63e004c770 con 0x7f63e4104280 2023-07-21T22:21:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.654+0000 7f63e8c00700 1 --2- 172.21.15.107:0/1415490562 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f63d0038630 0x7f63d003aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.655+0000 7f63e8c00700 1 --2- 172.21.15.107:0/1415490562 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f63d0038630 0x7f63d003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f63d4006fd0 tx=0x7f63d4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.663+0000 7f63da7fc700 1 -- 172.21.15.107:0/1415490562 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f63e0018d90 con 0x7f63e4104280 2023-07-21T22:21:48.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:47.860+0000 7f63eb665700 1 -- 172.21.15.107:0/1415490562 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f63e40736c0 con 0x7f63d0038630 2023-07-21T22:21:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:48.037+0000 7f63da7fc700 1 -- 172.21.15.107:0/1415490562 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f63e40736c0 con 0x7f63d0038630 2023-07-21T22:21:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:48.039+0000 7f63eb665700 1 -- 172.21.15.107:0/1415490562 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f63d0038630 msgr2=0x7f63d003aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:48.039+0000 7f63eb665700 1 --2- 172.21.15.107:0/1415490562 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f63d0038630 0x7f63d003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f63d4006fd0 tx=0x7f63d4006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:48.039+0000 7f63eb665700 1 -- 172.21.15.107:0/1415490562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 msgr2=0x7f63e4199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:48.039+0000 7f63eb665700 1 --2- 172.21.15.107:0/1415490562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 0x7f63e4199090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63e0011980 tx=0x7f63e0011a60 comp rx=0 tx=0).stop 2023-07-21T22:21:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:48.039+0000 7f63eb665700 1 -- 172.21.15.107:0/1415490562 shutdown_connections 2023-07-21T22:21:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:48.039+0000 7f63eb665700 1 --2- 172.21.15.107:0/1415490562 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f63d0038630 0x7f63d003aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:48.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:48.039+0000 7f63eb665700 1 --2- 172.21.15.107:0/1415490562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63e4104280 0x7f63e4199090 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:48.039+0000 7f63eb665700 1 -- 172.21.15.107:0/1415490562 >> 172.21.15.107:0/1415490562 conn(0x7f63e40ff8d0 msgr2=0x7f63e4072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:48.039+0000 7f63eb665700 1 -- 172.21.15.107:0/1415490562 shutdown_connections 2023-07-21T22:21:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:48.039+0000 7f63eb665700 1 -- 172.21.15.107:0/1415490562 wait complete. 2023-07-21T22:21:48.403 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial admin user... 2023-07-21T22:21:49.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:49 smithi107 ceph-mon[96543]: [21/Jul/2023:22:21:47] ENGINE Serving on https://172.21.15.107:7150 2023-07-21T22:21:49.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:49 smithi107 ceph-mon[96543]: [21/Jul/2023:22:21:47] ENGINE Serving on http://172.21.15.107:8765 2023-07-21T22:21:49.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:49 smithi107 ceph-mon[96543]: [21/Jul/2023:22:21:47] ENGINE Bus STARTED 2023-07-21T22:21:49.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:49 smithi107 ceph-mon[96543]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:49.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:49 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:21:49.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:49 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:21:49.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:49 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:21:49.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:49 smithi107 ceph-mon[96543]: mgrmap e12: x(active, since 2s) 2023-07-21T22:21:50.338 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$iU4l1cBjvEC4XTdt7jNri.9MDOopi3WJ.LqOYCyB23kqC7UlEGR5a", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1689978109, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-21T22:21:50.338 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.227+0000 7f9b4204c700 1 Processor -- start 2023-07-21T22:21:50.338 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.227+0000 7f9b4204c700 1 -- start start 2023-07-21T22:21:50.339 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.227+0000 7f9b4204c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 0x7f9b3c107f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.339 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.227+0000 7f9b4204c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c1084a0 con 0x7f9b3c073b60 2023-07-21T22:21:50.339 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.227+0000 7f9b3b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 0x7f9b3c107f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.339 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.227+0000 7f9b3b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 0x7f9b3c107f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45412/0 (socket says 172.21.15.107:45412) 2023-07-21T22:21:50.339 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.227+0000 7f9b3b7fe700 1 -- 172.21.15.107:0/2186631306 learned_addr learned my addr 172.21.15.107:0/2186631306 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:50.339 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.228+0000 7f9b3b7fe700 1 -- 172.21.15.107:0/2186631306 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c1085e0 con 0x7f9b3c073b60 2023-07-21T22:21:50.339 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.228+0000 7f9b3b7fe700 1 --2- 172.21.15.107:0/2186631306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 0x7f9b3c107f60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0098a0 tx=0x7f9b2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8805d2844ca15c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.339 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.228+0000 7f9b3a7fc700 1 -- 172.21.15.107:0/2186631306 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b2c004160 con 0x7f9b3c073b60 2023-07-21T22:21:50.340 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.228+0000 7f9b3a7fc700 1 -- 172.21.15.107:0/2186631306 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2c0042c0 con 0x7f9b3c073b60 2023-07-21T22:21:50.340 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.228+0000 7f9b3a7fc700 1 -- 172.21.15.107:0/2186631306 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b2c0119b0 con 0x7f9b3c073b60 2023-07-21T22:21:50.340 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.229+0000 7f9b4204c700 1 -- 172.21.15.107:0/2186631306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 msgr2=0x7f9b3c107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.340 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.229+0000 7f9b4204c700 1 --2- 172.21.15.107:0/2186631306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 0x7f9b3c107f60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0098a0 tx=0x7f9b2c009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.340 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.229+0000 7f9b4204c700 1 -- 172.21.15.107:0/2186631306 shutdown_connections 2023-07-21T22:21:50.340 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.229+0000 7f9b4204c700 1 --2- 172.21.15.107:0/2186631306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 0x7f9b3c107f60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.340 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.229+0000 7f9b4204c700 1 -- 172.21.15.107:0/2186631306 >> 172.21.15.107:0/2186631306 conn(0x7f9b3c0ff8d0 msgr2=0x7f9b3c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.341 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.229+0000 7f9b4204c700 1 -- 172.21.15.107:0/2186631306 shutdown_connections 2023-07-21T22:21:50.341 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.229+0000 7f9b4204c700 1 -- 172.21.15.107:0/2186631306 wait complete. 2023-07-21T22:21:50.341 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.229+0000 7f9b4204c700 1 Processor -- start 2023-07-21T22:21:50.341 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.229+0000 7f9b4204c700 1 -- start start 2023-07-21T22:21:50.341 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.230+0000 7f9b4204c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 0x7f9b3c1058b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.341 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.230+0000 7f9b4204c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c105df0 con 0x7f9b3c073b60 2023-07-21T22:21:50.341 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.230+0000 7f9b3b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 0x7f9b3c1058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.342 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.230+0000 7f9b3b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 0x7f9b3c1058b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45414/0 (socket says 172.21.15.107:45414) 2023-07-21T22:21:50.342 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.230+0000 7f9b3b7fe700 1 -- 172.21.15.107:0/2630797018 learned_addr learned my addr 172.21.15.107:0/2630797018 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:50.342 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.230+0000 7f9b3b7fe700 1 -- 172.21.15.107:0/2630797018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c009550 con 0x7f9b3c073b60 2023-07-21T22:21:50.342 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.230+0000 7f9b3b7fe700 1 --2- 172.21.15.107:0/2630797018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 0x7f9b3c1058b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c000c00 tx=0x7f9b2c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.342 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.231+0000 7f9b38ff9700 1 -- 172.21.15.107:0/2630797018 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b2c003b20 con 0x7f9b3c073b60 2023-07-21T22:21:50.342 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.231+0000 7f9b38ff9700 1 -- 172.21.15.107:0/2630797018 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2c003c80 con 0x7f9b3c073b60 2023-07-21T22:21:50.342 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.231+0000 7f9b4204c700 1 -- 172.21.15.107:0/2630797018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3c103f00 con 0x7f9b3c073b60 2023-07-21T22:21:50.342 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.231+0000 7f9b38ff9700 1 -- 172.21.15.107:0/2630797018 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b2c019720 con 0x7f9b3c073b60 2023-07-21T22:21:50.343 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.232+0000 7f9b4204c700 1 -- 172.21.15.107:0/2630797018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3c104320 con 0x7f9b3c073b60 2023-07-21T22:21:50.343 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.232+0000 7f9b38ff9700 1 -- 172.21.15.107:0/2630797018 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f9b2c019990 con 0x7f9b3c073b60 2023-07-21T22:21:50.343 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.233+0000 7f9b38ff9700 1 --2- 172.21.15.107:0/2630797018 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9b24038800 0x7f9b2403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.343 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.233+0000 7f9b4204c700 1 -- 172.21.15.107:0/2630797018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3c1094e0 con 0x7f9b3c073b60 2023-07-21T22:21:50.343 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.233+0000 7f9b38ff9700 1 -- 172.21.15.107:0/2630797018 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9b2c04ca00 con 0x7f9b3c073b60 2023-07-21T22:21:50.343 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.233+0000 7f9b3affd700 1 --2- 172.21.15.107:0/2630797018 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9b24038800 0x7f9b2403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.343 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.233+0000 7f9b3affd700 1 --2- 172.21.15.107:0/2630797018 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9b24038800 0x7f9b2403acb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b30006fd0 tx=0x7f9b30006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.344 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.238+0000 7f9b38ff9700 1 -- 172.21.15.107:0/2630797018 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9b2c018db0 con 0x7f9b3c073b60 2023-07-21T22:21:50.344 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.459+0000 7f9b4204c700 1 -- 172.21.15.107:0/2630797018 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f9b3c061960 con 0x7f9b24038800 2023-07-21T22:21:50.344 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.685+0000 7f9b38ff9700 1 -- 172.21.15.107:0/2630797018 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f9b3c061960 con 0x7f9b24038800 2023-07-21T22:21:50.344 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.687+0000 7f9b4204c700 1 -- 172.21.15.107:0/2630797018 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9b24038800 msgr2=0x7f9b2403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.344 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.687+0000 7f9b4204c700 1 --2- 172.21.15.107:0/2630797018 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9b24038800 0x7f9b2403acb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b30006fd0 tx=0x7f9b30006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:50.344 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.687+0000 7f9b4204c700 1 -- 172.21.15.107:0/2630797018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 msgr2=0x7f9b3c1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.344 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.687+0000 7f9b4204c700 1 --2- 172.21.15.107:0/2630797018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 0x7f9b3c1058b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c000c00 tx=0x7f9b2c011a30 comp rx=0 tx=0).stop 2023-07-21T22:21:50.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.687+0000 7f9b4204c700 1 -- 172.21.15.107:0/2630797018 shutdown_connections 2023-07-21T22:21:50.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.687+0000 7f9b4204c700 1 --2- 172.21.15.107:0/2630797018 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9b24038800 0x7f9b2403acb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.687+0000 7f9b4204c700 1 --2- 172.21.15.107:0/2630797018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b3c073b60 0x7f9b3c1058b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.687+0000 7f9b4204c700 1 -- 172.21.15.107:0/2630797018 >> 172.21.15.107:0/2630797018 conn(0x7f9b3c0ff8d0 msgr2=0x7f9b3c100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.688+0000 7f9b4204c700 1 -- 172.21.15.107:0/2630797018 shutdown_connections 2023-07-21T22:21:50.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:49.688+0000 7f9b4204c700 1 -- 172.21.15.107:0/2630797018 wait complete. 2023-07-21T22:21:50.345 INFO:teuthology.orchestra.run.smithi107.stdout:Fetching dashboard port number... 2023-07-21T22:21:50.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:50 smithi107 ceph-mon[96543]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:50.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:21:51.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 8443 2023-07-21T22:21:51.382 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.746+0000 7efdc3033700 1 Processor -- start 2023-07-21T22:21:51.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.746+0000 7efdc3033700 1 -- start start 2023-07-21T22:21:51.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.746+0000 7efdc3033700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 0x7efdbc106dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.746+0000 7efdc3033700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc107300 con 0x7efdbc1069b0 2023-07-21T22:21:51.383 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.746+0000 7efdc0dcf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 0x7efdbc106dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.746+0000 7efdc0dcf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 0x7efdbc106dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45418/0 (socket says 172.21.15.107:45418) 2023-07-21T22:21:51.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.746+0000 7efdc0dcf700 1 -- 172.21.15.107:0/1906259704 learned_addr learned my addr 172.21.15.107:0/1906259704 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:51.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.747+0000 7efdc0dcf700 1 -- 172.21.15.107:0/1906259704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc107440 con 0x7efdbc1069b0 2023-07-21T22:21:51.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.747+0000 7efdc0dcf700 1 --2- 172.21.15.107:0/1906259704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 0x7efdbc106dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efdb00098a0 tx=0x7efdb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfa55cca446cfca server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:51.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.747+0000 7efdbb7fe700 1 -- 172.21.15.107:0/1906259704 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdb0004160 con 0x7efdbc1069b0 2023-07-21T22:21:51.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.747+0000 7efdbb7fe700 1 -- 172.21.15.107:0/1906259704 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb00042c0 con 0x7efdbc1069b0 2023-07-21T22:21:51.384 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.747+0000 7efdbb7fe700 1 -- 172.21.15.107:0/1906259704 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdb00119b0 con 0x7efdbc1069b0 2023-07-21T22:21:51.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.747+0000 7efdc3033700 1 -- 172.21.15.107:0/1906259704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 msgr2=0x7efdbc106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:51.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.747+0000 7efdc3033700 1 --2- 172.21.15.107:0/1906259704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 0x7efdbc106dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efdb00098a0 tx=0x7efdb0009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.747+0000 7efdc3033700 1 -- 172.21.15.107:0/1906259704 shutdown_connections 2023-07-21T22:21:51.385 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.747+0000 7efdc3033700 1 --2- 172.21.15.107:0/1906259704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 0x7efdbc106dc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.747+0000 7efdc3033700 1 -- 172.21.15.107:0/1906259704 >> 172.21.15.107:0/1906259704 conn(0x7efdbc0ff040 msgr2=0x7efdbc101470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:51.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.747+0000 7efdc3033700 1 -- 172.21.15.107:0/1906259704 shutdown_connections 2023-07-21T22:21:51.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.748+0000 7efdc3033700 1 -- 172.21.15.107:0/1906259704 wait complete. 2023-07-21T22:21:51.386 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.748+0000 7efdc3033700 1 Processor -- start 2023-07-21T22:21:51.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.748+0000 7efdc3033700 1 -- start start 2023-07-21T22:21:51.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.748+0000 7efdc3033700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 0x7efdbc199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.748+0000 7efdc3033700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc199660 con 0x7efdbc1069b0 2023-07-21T22:21:51.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.748+0000 7efdc0dcf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 0x7efdbc199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.748+0000 7efdc0dcf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 0x7efdbc199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45420/0 (socket says 172.21.15.107:45420) 2023-07-21T22:21:51.387 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.748+0000 7efdc0dcf700 1 -- 172.21.15.107:0/2159207796 learned_addr learned my addr 172.21.15.107:0/2159207796 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:51.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.749+0000 7efdc0dcf700 1 -- 172.21.15.107:0/2159207796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb0009550 con 0x7efdbc1069b0 2023-07-21T22:21:51.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.749+0000 7efdc0dcf700 1 --2- 172.21.15.107:0/2159207796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 0x7efdbc199120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efdb0011980 tx=0x7efdb0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:51.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.749+0000 7efdb9ffb700 1 -- 172.21.15.107:0/2159207796 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdb0003a60 con 0x7efdbc1069b0 2023-07-21T22:21:51.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.749+0000 7efdb9ffb700 1 -- 172.21.15.107:0/2159207796 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb0003bc0 con 0x7efdbc1069b0 2023-07-21T22:21:51.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.749+0000 7efdc3033700 1 -- 172.21.15.107:0/2159207796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc199860 con 0x7efdbc1069b0 2023-07-21T22:21:51.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.749+0000 7efdb9ffb700 1 -- 172.21.15.107:0/2159207796 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdb001a670 con 0x7efdbc1069b0 2023-07-21T22:21:51.388 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.749+0000 7efdc3033700 1 -- 172.21.15.107:0/2159207796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc199c80 con 0x7efdbc1069b0 2023-07-21T22:21:51.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.750+0000 7efdb9ffb700 1 -- 172.21.15.107:0/2159207796 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7efdb0024440 con 0x7efdbc1069b0 2023-07-21T22:21:51.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.750+0000 7efdb9ffb700 1 --2- 172.21.15.107:0/2159207796 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efdac038760 0x7efdac03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.750+0000 7efdc3033700 1 -- 172.21.15.107:0/2159207796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdbc061b40 con 0x7efdbc1069b0 2023-07-21T22:21:51.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.750+0000 7efdb9ffb700 1 -- 172.21.15.107:0/2159207796 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efdb004ca10 con 0x7efdbc1069b0 2023-07-21T22:21:51.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.750+0000 7efdbbfff700 1 --2- 172.21.15.107:0/2159207796 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efdac038760 0x7efdac03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.751+0000 7efdbbfff700 1 --2- 172.21.15.107:0/2159207796 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efdac038760 0x7efdac03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efda8006fd0 tx=0x7efda8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:51.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.755+0000 7efdb9ffb700 1 -- 172.21.15.107:0/2159207796 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7efdb00189a0 con 0x7efdbc1069b0 2023-07-21T22:21:51.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.951+0000 7efdc3033700 1 -- 172.21.15.107:0/2159207796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7efdbc19c6c0 con 0x7efdbc1069b0 2023-07-21T22:21:51.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.951+0000 7efdb9ffb700 1 -- 172.21.15.107:0/2159207796 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7efdb0018350 con 0x7efdbc1069b0 2023-07-21T22:21:51.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.953+0000 7efdc3033700 1 -- 172.21.15.107:0/2159207796 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efdac038760 msgr2=0x7efdac03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:51.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.953+0000 7efdc3033700 1 --2- 172.21.15.107:0/2159207796 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efdac038760 0x7efdac03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efda8006fd0 tx=0x7efda8006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:51.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.954+0000 7efdc3033700 1 -- 172.21.15.107:0/2159207796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 msgr2=0x7efdbc199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:51.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.954+0000 7efdc3033700 1 --2- 172.21.15.107:0/2159207796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 0x7efdbc199120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efdb0011980 tx=0x7efdb0011a60 comp rx=0 tx=0).stop 2023-07-21T22:21:51.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.954+0000 7efdc3033700 1 -- 172.21.15.107:0/2159207796 shutdown_connections 2023-07-21T22:21:51.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.954+0000 7efdc3033700 1 --2- 172.21.15.107:0/2159207796 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efdac038760 0x7efdac03ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.954+0000 7efdc3033700 1 --2- 172.21.15.107:0/2159207796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdbc1069b0 0x7efdbc199120 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.954+0000 7efdc3033700 1 -- 172.21.15.107:0/2159207796 >> 172.21.15.107:0/2159207796 conn(0x7efdbc0ff040 msgr2=0x7efdbc100700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:51.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.954+0000 7efdc3033700 1 -- 172.21.15.107:0/2159207796 shutdown_connections 2023-07-21T22:21:51.391 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:50.954+0000 7efdc3033700 1 -- 172.21.15.107:0/2159207796 wait complete. 2023-07-21T22:21:51.392 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T22:21:51.392 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2023-07-21T22:21:51.398 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T22:21:51.398 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2023-07-21T22:21:51.398 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2023-07-21T22:21:51.398 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-21T22:21:51.399 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph Dashboard is now available at: 2023-07-21T22:21:51.400 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:21:51.400 INFO:teuthology.orchestra.run.smithi107.stdout: URL: https://smithi107:8443/ 2023-07-21T22:21:51.400 INFO:teuthology.orchestra.run.smithi107.stdout: User: admin 2023-07-21T22:21:51.400 INFO:teuthology.orchestra.run.smithi107.stdout: Password: y510u4ndgs 2023-07-21T22:21:51.400 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:21:51.401 INFO:teuthology.orchestra.run.smithi107.stdout:Saving cluster configuration to /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config directory 2023-07-21T22:21:51.401 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling autotune for osd_memory_target 2023-07-21T22:21:52.427 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:51 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2159207796' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-21T22:21:52.446 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.830+0000 7f099d7af700 1 Processor -- start 2023-07-21T22:21:52.446 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.830+0000 7f099d7af700 1 -- start start 2023-07-21T22:21:52.446 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.830+0000 7f099d7af700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 0x7f0998105d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.447 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.830+0000 7f099d7af700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0998106270 con 0x7f0998105920 2023-07-21T22:21:52.447 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.830+0000 7f0996ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 0x7f0998105d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.447 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.830+0000 7f0996ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 0x7f0998105d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45428/0 (socket says 172.21.15.107:45428) 2023-07-21T22:21:52.447 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.830+0000 7f0996ffd700 1 -- 172.21.15.107:0/3897860099 learned_addr learned my addr 172.21.15.107:0/3897860099 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:52.447 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.831+0000 7f0996ffd700 1 -- 172.21.15.107:0/3897860099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09981063b0 con 0x7f0998105920 2023-07-21T22:21:52.447 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.831+0000 7f0996ffd700 1 --2- 172.21.15.107:0/3897860099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 0x7f0998105d30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f09880098a0 tx=0x7f0988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6320b679e4d6998b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.447 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.831+0000 7f0995ffb700 1 -- 172.21.15.107:0/3897860099 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0988004160 con 0x7f0998105920 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.831+0000 7f0995ffb700 1 -- 172.21.15.107:0/3897860099 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09880042c0 con 0x7f0998105920 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.831+0000 7f0995ffb700 1 -- 172.21.15.107:0/3897860099 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09880119b0 con 0x7f0998105920 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.831+0000 7f099d7af700 1 -- 172.21.15.107:0/3897860099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 msgr2=0x7f0998105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.831+0000 7f099d7af700 1 --2- 172.21.15.107:0/3897860099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 0x7f0998105d30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f09880098a0 tx=0x7f0988009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.831+0000 7f099d7af700 1 -- 172.21.15.107:0/3897860099 shutdown_connections 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.831+0000 7f099d7af700 1 --2- 172.21.15.107:0/3897860099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 0x7f0998105d30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.831+0000 7f099d7af700 1 -- 172.21.15.107:0/3897860099 >> 172.21.15.107:0/3897860099 conn(0x7f09980ff040 msgr2=0x7f0998101470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.832+0000 7f099d7af700 1 -- 172.21.15.107:0/3897860099 shutdown_connections 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.832+0000 7f099d7af700 1 -- 172.21.15.107:0/3897860099 wait complete. 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.832+0000 7f099d7af700 1 Processor -- start 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.832+0000 7f099d7af700 1 -- start start 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.832+0000 7f099d7af700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 0x7f099819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.832+0000 7f099d7af700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099819b8d0 con 0x7f0998105920 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.833+0000 7f0996ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 0x7f099819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.833+0000 7f0996ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 0x7f099819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45430/0 (socket says 172.21.15.107:45430) 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.833+0000 7f0996ffd700 1 -- 172.21.15.107:0/853806431 learned_addr learned my addr 172.21.15.107:0/853806431 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.833+0000 7f0996ffd700 1 -- 172.21.15.107:0/853806431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0988009550 con 0x7f0998105920 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.833+0000 7f0996ffd700 1 --2- 172.21.15.107:0/853806431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 0x7f099819b390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0988006af0 tx=0x7f0988011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.833+0000 7f097ffff700 1 -- 172.21.15.107:0/853806431 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0988003a40 con 0x7f0998105920 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.833+0000 7f099d7af700 1 -- 172.21.15.107:0/853806431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099819bad0 con 0x7f0998105920 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.833+0000 7f097ffff700 1 -- 172.21.15.107:0/853806431 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0988003ba0 con 0x7f0998105920 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.833+0000 7f097ffff700 1 -- 172.21.15.107:0/853806431 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09880196c0 con 0x7f0998105920 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.833+0000 7f099d7af700 1 -- 172.21.15.107:0/853806431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099819bef0 con 0x7f0998105920 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.834+0000 7f097ffff700 1 -- 172.21.15.107:0/853806431 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f098801aa80 con 0x7f0998105920 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.834+0000 7f097ffff700 1 --2- 172.21.15.107:0/853806431 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0980038710 0x7f098003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.834+0000 7f099d7af700 1 -- 172.21.15.107:0/853806431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0998061b40 con 0x7f0998105920 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.834+0000 7f097ffff700 1 -- 172.21.15.107:0/853806431 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f098804ccd0 con 0x7f0998105920 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.834+0000 7f09967fc700 1 --2- 172.21.15.107:0/853806431 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0980038710 0x7f098003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.835+0000 7f09967fc700 1 --2- 172.21.15.107:0/853806431 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0980038710 0x7f098003abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f098c006fd0 tx=0x7f098c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.839+0000 7f097ffff700 1 -- 172.21.15.107:0/853806431 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0988023750 con 0x7f0998105920 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:51.928+0000 7f097ffff700 1 -- 172.21.15.107:0/853806431 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f098802d430 con 0x7f0998105920 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.025+0000 7f099d7af700 1 -- 172.21.15.107:0/853806431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f099819e930 con 0x7f0998105920 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.025+0000 7f097ffff700 1 -- 172.21.15.107:0/853806431 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f0988023bb0 con 0x7f0998105920 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.027+0000 7f099d7af700 1 -- 172.21.15.107:0/853806431 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0980038710 msgr2=0x7f098003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.027+0000 7f099d7af700 1 --2- 172.21.15.107:0/853806431 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0980038710 0x7f098003abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f098c006fd0 tx=0x7f098c006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.027+0000 7f099d7af700 1 -- 172.21.15.107:0/853806431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 msgr2=0x7f099819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.027+0000 7f099d7af700 1 --2- 172.21.15.107:0/853806431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 0x7f099819b390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0988006af0 tx=0x7f0988011a10 comp rx=0 tx=0).stop 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.027+0000 7f099d7af700 1 -- 172.21.15.107:0/853806431 shutdown_connections 2023-07-21T22:21:52.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.027+0000 7f099d7af700 1 --2- 172.21.15.107:0/853806431 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0980038710 0x7f098003abc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.027+0000 7f099d7af700 1 --2- 172.21.15.107:0/853806431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0998105920 0x7f099819b390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.027+0000 7f099d7af700 1 -- 172.21.15.107:0/853806431 >> 172.21.15.107:0/853806431 conn(0x7f09980ff040 msgr2=0x7f0998100770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.027+0000 7f099d7af700 1 -- 172.21.15.107:0/853806431 shutdown_connections 2023-07-21T22:21:52.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.027+0000 7f099d7af700 1 -- 172.21.15.107:0/853806431 wait complete. 2023-07-21T22:21:53.339 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:52 smithi107 ceph-mon[96543]: mgrmap e13: x(active, since 6s) 2023-07-21T22:21:53.586 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.848+0000 7f52c8d00700 1 Processor -- start 2023-07-21T22:21:53.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.848+0000 7f52c8d00700 1 -- start start 2023-07-21T22:21:53.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.848+0000 7f52c8d00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 0x7f52c41082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:53.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.848+0000 7f52c8d00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4073bc0 con 0x7f52c4105ef0 2023-07-21T22:21:53.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.849+0000 7f52c259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 0x7f52c41082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:53.591 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.849+0000 7f52c259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 0x7f52c41082d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45434/0 (socket says 172.21.15.107:45434) 2023-07-21T22:21:53.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.849+0000 7f52c259c700 1 -- 172.21.15.107:0/736598536 learned_addr learned my addr 172.21.15.107:0/736598536 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:53.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.849+0000 7f52c259c700 1 -- 172.21.15.107:0/736598536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4073d00 con 0x7f52c4105ef0 2023-07-21T22:21:53.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.849+0000 7f52c259c700 1 --2- 172.21.15.107:0/736598536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 0x7f52c41082d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f52b80098a0 tx=0x7f52b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8299cddd74cfea63 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:53.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.849+0000 7f52c159a700 1 -- 172.21.15.107:0/736598536 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52b8004160 con 0x7f52c4105ef0 2023-07-21T22:21:53.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.849+0000 7f52c159a700 1 -- 172.21.15.107:0/736598536 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b80042c0 con 0x7f52c4105ef0 2023-07-21T22:21:53.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.849+0000 7f52c159a700 1 -- 172.21.15.107:0/736598536 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52b80119b0 con 0x7f52c4105ef0 2023-07-21T22:21:53.592 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.850+0000 7f52c8d00700 1 -- 172.21.15.107:0/736598536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 msgr2=0x7f52c41082d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:53.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.850+0000 7f52c8d00700 1 --2- 172.21.15.107:0/736598536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 0x7f52c41082d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f52b80098a0 tx=0x7f52b8009bb0 comp rx=0 tx=0).stop 2023-07-21T22:21:53.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.850+0000 7f52c8d00700 1 -- 172.21.15.107:0/736598536 shutdown_connections 2023-07-21T22:21:53.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.850+0000 7f52c8d00700 1 --2- 172.21.15.107:0/736598536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 0x7f52c41082d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:53.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.850+0000 7f52c8d00700 1 -- 172.21.15.107:0/736598536 >> 172.21.15.107:0/736598536 conn(0x7f52c40ff8d0 msgr2=0x7f52c4101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:53.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.850+0000 7f52c8d00700 1 -- 172.21.15.107:0/736598536 shutdown_connections 2023-07-21T22:21:53.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.850+0000 7f52c8d00700 1 -- 172.21.15.107:0/736598536 wait complete. 2023-07-21T22:21:53.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.851+0000 7f52c8d00700 1 Processor -- start 2023-07-21T22:21:53.593 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.851+0000 7f52c8d00700 1 -- start start 2023-07-21T22:21:53.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.851+0000 7f52c8d00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 0x7f52c4105850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:53.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.851+0000 7f52c8d00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4105d90 con 0x7f52c4105ef0 2023-07-21T22:21:53.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.851+0000 7f52c259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 0x7f52c4105850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:53.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.851+0000 7f52c259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 0x7f52c4105850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45436/0 (socket says 172.21.15.107:45436) 2023-07-21T22:21:53.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.851+0000 7f52c259c700 1 -- 172.21.15.107:0/3316134100 learned_addr learned my addr 172.21.15.107:0/3316134100 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:53.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.851+0000 7f52c259c700 1 -- 172.21.15.107:0/3316134100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b8009550 con 0x7f52c4105ef0 2023-07-21T22:21:53.594 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.851+0000 7f52c259c700 1 --2- 172.21.15.107:0/3316134100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 0x7f52c4105850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f52b8003e60 tx=0x7f52b8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:53.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.851+0000 7f52b37fe700 1 -- 172.21.15.107:0/3316134100 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52b8003970 con 0x7f52c4105ef0 2023-07-21T22:21:53.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.852+0000 7f52b37fe700 1 -- 172.21.15.107:0/3316134100 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b8003ad0 con 0x7f52c4105ef0 2023-07-21T22:21:53.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.852+0000 7f52c8d00700 1 -- 172.21.15.107:0/3316134100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c4103ea0 con 0x7f52c4105ef0 2023-07-21T22:21:53.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.852+0000 7f52b37fe700 1 -- 172.21.15.107:0/3316134100 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52b80195e0 con 0x7f52c4105ef0 2023-07-21T22:21:53.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.852+0000 7f52c8d00700 1 -- 172.21.15.107:0/3316134100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c41042c0 con 0x7f52c4105ef0 2023-07-21T22:21:53.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.852+0000 7f52b37fe700 1 -- 172.21.15.107:0/3316134100 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f52b801a430 con 0x7f52c4105ef0 2023-07-21T22:21:53.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.852+0000 7f52c8d00700 1 -- 172.21.15.107:0/3316134100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c41094c0 con 0x7f52c4105ef0 2023-07-21T22:21:53.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.853+0000 7f52b37fe700 1 --2- 172.21.15.107:0/3316134100 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f52ac0387b0 0x7f52ac03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:53.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.853+0000 7f52b37fe700 1 -- 172.21.15.107:0/3316134100 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f52b804ca30 con 0x7f52c4105ef0 2023-07-21T22:21:53.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.853+0000 7f52c1d9b700 1 --2- 172.21.15.107:0/3316134100 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f52ac0387b0 0x7f52ac03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:53.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.853+0000 7f52c1d9b700 1 --2- 172.21.15.107:0/3316134100 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f52ac0387b0 0x7f52ac03ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f52b4006fd0 tx=0x7f52b4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:53.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:52.857+0000 7f52b37fe700 1 -- 172.21.15.107:0/3316134100 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f52b8003c40 con 0x7f52c4105ef0 2023-07-21T22:21:53.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:53.147+0000 7f52c8d00700 1 -- 172.21.15.107:0/3316134100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f52c41096e0 con 0x7f52c4105ef0 2023-07-21T22:21:53.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:53.152+0000 7f52b37fe700 1 -- 172.21.15.107:0/3316134100 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f52b8018350 con 0x7f52c4105ef0 2023-07-21T22:21:53.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:53.154+0000 7f52c8d00700 1 -- 172.21.15.107:0/3316134100 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f52ac0387b0 msgr2=0x7f52ac03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:53.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:53.154+0000 7f52c8d00700 1 --2- 172.21.15.107:0/3316134100 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f52ac0387b0 0x7f52ac03ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f52b4006fd0 tx=0x7f52b4006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:53.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:53.154+0000 7f52c8d00700 1 -- 172.21.15.107:0/3316134100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 msgr2=0x7f52c4105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:53.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:53.154+0000 7f52c8d00700 1 --2- 172.21.15.107:0/3316134100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 0x7f52c4105850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f52b8003e60 tx=0x7f52b8011aa0 comp rx=0 tx=0).stop 2023-07-21T22:21:53.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:53.154+0000 7f52c8d00700 1 -- 172.21.15.107:0/3316134100 shutdown_connections 2023-07-21T22:21:53.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:53.154+0000 7f52c8d00700 1 --2- 172.21.15.107:0/3316134100 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f52ac0387b0 0x7f52ac03ac60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:53.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:53.154+0000 7f52c8d00700 1 --2- 172.21.15.107:0/3316134100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c4105ef0 0x7f52c4105850 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:53.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:53.154+0000 7f52c8d00700 1 -- 172.21.15.107:0/3316134100 >> 172.21.15.107:0/3316134100 conn(0x7f52c40ff8d0 msgr2=0x7f52c4100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:53.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:53.154+0000 7f52c8d00700 1 -- 172.21.15.107:0/3316134100 shutdown_connections 2023-07-21T22:21:53.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2023-07-21T22:21:53.154+0000 7f52c8d00700 1 -- 172.21.15.107:0/3316134100 wait complete. 2023-07-21T22:21:53.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-21T22:21:53.598 INFO:teuthology.orchestra.run.smithi107.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-21T22:21:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:21:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:21:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:21:53.598 INFO:teuthology.orchestra.run.smithi107.stdout:Or, if you are only running a single cluster on this host: 2023-07-21T22:21:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:21:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-21T22:21:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:21:53.599 INFO:teuthology.orchestra.run.smithi107.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-21T22:21:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:21:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: ceph telemetry on 2023-07-21T22:21:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:21:53.599 INFO:teuthology.orchestra.run.smithi107.stdout:For more information see: 2023-07-21T22:21:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:21:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-21T22:21:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:21:53.600 INFO:teuthology.orchestra.run.smithi107.stdout:Bootstrap complete. 2023-07-21T22:21:53.625 INFO:tasks.cephadm:Fetching config... 2023-07-21T22:21:53.626 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:21:53.626 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-21T22:21:53.643 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-21T22:21:53.643 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:21:53.643 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-21T22:21:53.701 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-21T22:21:53.702 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:21:53.702 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-21T22:21:53.768 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-21T22:21:53.768 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:21:53.769 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-21T22:21:53.825 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-21T22:21:53.825 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfWxRL+/ISZ4SJNsKH4cSZlYy4IO1VMxX3jALPtwymDUlUNNS4Kqpql8iEWOWc5IyOGBGjrP/D80WPp4MqnkUtfXFxM+9ke2eScz1c2hrAL8F4LaSThG5NE9HFE6gLbZ8VucKkzvdK3lmyB94ZYGlKD9vDr7p8qwTLX6y2t+Ate07THz4w6AmD/U19JRn50pjZ5LZ2egN9Eiuf4waQN1HfolIU7V1brjkmNKwQX6839xX1IqQnGYXiehw+X/lr/TOZiwCP/fHEzTbcmtmQscW2R4MnhGrHweLLpSxxOPfZi1Np4rmP3H7zGTKLRgHK/IIKLeBcyPDLj9j7Sz9jTtXBIaHTEvWTYwCvQuyBror7CHG4jJYoEagANiiDvHMQC2Z5wTFQC0V9YtF0ETZZreM0P33nBnfvyx1h8E1cDDsws19xnRty6wVwPRAgQykw7Ys0bgg9obVije8FXfRQMh/IP5pzNprrziTtpPaSxGmwBGr6AIhHhzpUBCwjQSz+QKc= ceph-af99f5d0-2814-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T22:21:53.903 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfWxRL+/ISZ4SJNsKH4cSZlYy4IO1VMxX3jALPtwymDUlUNNS4Kqpql8iEWOWc5IyOGBGjrP/D80WPp4MqnkUtfXFxM+9ke2eScz1c2hrAL8F4LaSThG5NE9HFE6gLbZ8VucKkzvdK3lmyB94ZYGlKD9vDr7p8qwTLX6y2t+Ate07THz4w6AmD/U19JRn50pjZ5LZ2egN9Eiuf4waQN1HfolIU7V1brjkmNKwQX6839xX1IqQnGYXiehw+X/lr/TOZiwCP/fHEzTbcmtmQscW2R4MnhGrHweLLpSxxOPfZi1Np4rmP3H7zGTKLRgHK/IIKLeBcyPDLj9j7Sz9jTtXBIaHTEvWTYwCvQuyBror7CHG4jJYoEagANiiDvHMQC2Z5wTFQC0V9YtF0ETZZreM0P33nBnfvyx1h8E1cDDsws19xnRty6wVwPRAgQykw7Ys0bgg9obVije8FXfRQMh/IP5pzNprrziTtpPaSxGmwBGr6AIhHhzpUBCwjQSz+QKc= ceph-af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:21:53.926 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfWxRL+/ISZ4SJNsKH4cSZlYy4IO1VMxX3jALPtwymDUlUNNS4Kqpql8iEWOWc5IyOGBGjrP/D80WPp4MqnkUtfXFxM+9ke2eScz1c2hrAL8F4LaSThG5NE9HFE6gLbZ8VucKkzvdK3lmyB94ZYGlKD9vDr7p8qwTLX6y2t+Ate07THz4w6AmD/U19JRn50pjZ5LZ2egN9Eiuf4waQN1HfolIU7V1brjkmNKwQX6839xX1IqQnGYXiehw+X/lr/TOZiwCP/fHEzTbcmtmQscW2R4MnhGrHweLLpSxxOPfZi1Np4rmP3H7zGTKLRgHK/IIKLeBcyPDLj9j7Sz9jTtXBIaHTEvWTYwCvQuyBror7CHG4jJYoEagANiiDvHMQC2Z5wTFQC0V9YtF0ETZZreM0P33nBnfvyx1h8E1cDDsws19xnRty6wVwPRAgQykw7Ys0bgg9obVije8FXfRQMh/IP5pzNprrziTtpPaSxGmwBGr6AIhHhzpUBCwjQSz+QKc= ceph-af99f5d0-2814-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T22:21:53.970 INFO:teuthology.orchestra.run.smithi132.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfWxRL+/ISZ4SJNsKH4cSZlYy4IO1VMxX3jALPtwymDUlUNNS4Kqpql8iEWOWc5IyOGBGjrP/D80WPp4MqnkUtfXFxM+9ke2eScz1c2hrAL8F4LaSThG5NE9HFE6gLbZ8VucKkzvdK3lmyB94ZYGlKD9vDr7p8qwTLX6y2t+Ate07THz4w6AmD/U19JRn50pjZ5LZ2egN9Eiuf4waQN1HfolIU7V1brjkmNKwQX6839xX1IqQnGYXiehw+X/lr/TOZiwCP/fHEzTbcmtmQscW2R4MnhGrHweLLpSxxOPfZi1Np4rmP3H7zGTKLRgHK/IIKLeBcyPDLj9j7Sz9jTtXBIaHTEvWTYwCvQuyBror7CHG4jJYoEagANiiDvHMQC2Z5wTFQC0V9YtF0ETZZreM0P33nBnfvyx1h8E1cDDsws19xnRty6wVwPRAgQykw7Ys0bgg9obVije8FXfRQMh/IP5pzNprrziTtpPaSxGmwBGr6AIhHhzpUBCwjQSz+QKc= ceph-af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:21:53.982 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfWxRL+/ISZ4SJNsKH4cSZlYy4IO1VMxX3jALPtwymDUlUNNS4Kqpql8iEWOWc5IyOGBGjrP/D80WPp4MqnkUtfXFxM+9ke2eScz1c2hrAL8F4LaSThG5NE9HFE6gLbZ8VucKkzvdK3lmyB94ZYGlKD9vDr7p8qwTLX6y2t+Ate07THz4w6AmD/U19JRn50pjZ5LZ2egN9Eiuf4waQN1HfolIU7V1brjkmNKwQX6839xX1IqQnGYXiehw+X/lr/TOZiwCP/fHEzTbcmtmQscW2R4MnhGrHweLLpSxxOPfZi1Np4rmP3H7zGTKLRgHK/IIKLeBcyPDLj9j7Sz9jTtXBIaHTEvWTYwCvQuyBror7CHG4jJYoEagANiiDvHMQC2Z5wTFQC0V9YtF0ETZZreM0P33nBnfvyx1h8E1cDDsws19xnRty6wVwPRAgQykw7Ys0bgg9obVije8FXfRQMh/IP5pzNprrziTtpPaSxGmwBGr6AIhHhzpUBCwjQSz+QKc= ceph-af99f5d0-2814-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T22:21:54.028 INFO:teuthology.orchestra.run.smithi190.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfWxRL+/ISZ4SJNsKH4cSZlYy4IO1VMxX3jALPtwymDUlUNNS4Kqpql8iEWOWc5IyOGBGjrP/D80WPp4MqnkUtfXFxM+9ke2eScz1c2hrAL8F4LaSThG5NE9HFE6gLbZ8VucKkzvdK3lmyB94ZYGlKD9vDr7p8qwTLX6y2t+Ate07THz4w6AmD/U19JRn50pjZ5LZ2egN9Eiuf4waQN1HfolIU7V1brjkmNKwQX6839xX1IqQnGYXiehw+X/lr/TOZiwCP/fHEzTbcmtmQscW2R4MnhGrHweLLpSxxOPfZi1Np4rmP3H7zGTKLRgHK/IIKLeBcyPDLj9j7Sz9jTtXBIaHTEvWTYwCvQuyBror7CHG4jJYoEagANiiDvHMQC2Z5wTFQC0V9YtF0ETZZreM0P33nBnfvyx1h8E1cDDsws19xnRty6wVwPRAgQykw7Ys0bgg9obVije8FXfRQMh/IP5pzNprrziTtpPaSxGmwBGr6AIhHhzpUBCwjQSz+QKc= ceph-af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:21:54.040 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-21T22:21:54.461 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:54 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3316134100' entity='client.admin' 2023-07-21T22:21:54.825 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:21:56.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.117+0000 7f026e6ba700 1 -- 172.21.15.107:0/3656601461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02680ffa80 msgr2=0x7f02680ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:56.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.117+0000 7f026e6ba700 1 --2- 172.21.15.107:0/3656601461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02680ffa80 0x7f02680ffe90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0258009910 tx=0x7f0258009c20 comp rx=0 tx=0).stop 2023-07-21T22:21:56.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.118+0000 7f026e6ba700 1 -- 172.21.15.107:0/3656601461 shutdown_connections 2023-07-21T22:21:56.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.118+0000 7f026e6ba700 1 --2- 172.21.15.107:0/3656601461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02680ffa80 0x7f02680ffe90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:56.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.118+0000 7f026e6ba700 1 -- 172.21.15.107:0/3656601461 >> 172.21.15.107:0/3656601461 conn(0x7f02680f9220 msgr2=0x7f02680fb650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:56.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.118+0000 7f026e6ba700 1 -- 172.21.15.107:0/3656601461 shutdown_connections 2023-07-21T22:21:56.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.118+0000 7f026e6ba700 1 -- 172.21.15.107:0/3656601461 wait complete. 2023-07-21T22:21:56.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.118+0000 7f026e6ba700 1 Processor -- start 2023-07-21T22:21:56.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.118+0000 7f026e6ba700 1 -- start start 2023-07-21T22:21:56.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.118+0000 7f026e6ba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02680ffa80 0x7f02680fe210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:56.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.118+0000 7f026e6ba700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0268197480 con 0x7f02680ffa80 2023-07-21T22:21:56.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.119+0000 7f0267fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02680ffa80 0x7f02680fe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:56.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.119+0000 7f0267fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02680ffa80 0x7f02680fe210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45442/0 (socket says 172.21.15.107:45442) 2023-07-21T22:21:56.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.119+0000 7f0267fff700 1 -- 172.21.15.107:0/1120301430 learned_addr learned my addr 172.21.15.107:0/1120301430 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:56.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.119+0000 7f0267fff700 1 -- 172.21.15.107:0/1120301430 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02580095f0 con 0x7f02680ffa80 2023-07-21T22:21:56.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.119+0000 7f0267fff700 1 --2- 172.21.15.107:0/1120301430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02680ffa80 0x7f02680fe210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0258004f40 tx=0x7f02580057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:56.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.119+0000 7f02657fa700 1 -- 172.21.15.107:0/1120301430 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f025801d070 con 0x7f02680ffa80 2023-07-21T22:21:56.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.119+0000 7f026e6ba700 1 -- 172.21.15.107:0/1120301430 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02681975c0 con 0x7f02680ffa80 2023-07-21T22:21:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.119+0000 7f02657fa700 1 -- 172.21.15.107:0/1120301430 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f025800bc10 con 0x7f02680ffa80 2023-07-21T22:21:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.119+0000 7f02657fa700 1 -- 172.21.15.107:0/1120301430 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0258003a70 con 0x7f02680ffa80 2023-07-21T22:21:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.119+0000 7f026e6ba700 1 -- 172.21.15.107:0/1120301430 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0268197a60 con 0x7f02680ffa80 2023-07-21T22:21:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.120+0000 7f02657fa700 1 -- 172.21.15.107:0/1120301430 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f0258003d30 con 0x7f02680ffa80 2023-07-21T22:21:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.120+0000 7f02657fa700 1 --2- 172.21.15.107:0/1120301430 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f02500387b0 0x7f025003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.120+0000 7f026e6ba700 1 -- 172.21.15.107:0/1120301430 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026818d0c0 con 0x7f02680ffa80 2023-07-21T22:21:56.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.120+0000 7f02657fa700 1 -- 172.21.15.107:0/1120301430 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f025804c750 con 0x7f02680ffa80 2023-07-21T22:21:56.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.120+0000 7f02677fe700 1 --2- 172.21.15.107:0/1120301430 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f02500387b0 0x7f025003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:56.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.121+0000 7f02677fe700 1 --2- 172.21.15.107:0/1120301430 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f02500387b0 0x7f025003ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f025c006fd0 tx=0x7f025c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:56.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.125+0000 7f02657fa700 1 -- 172.21.15.107:0/1120301430 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0258026070 con 0x7f02680ffa80 2023-07-21T22:21:56.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:21:56.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:21:56.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:56.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:21:56.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:56.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.315+0000 7f026e6ba700 1 -- 172.21.15.107:0/1120301430 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f026802c850 con 0x7f02680ffa80 2023-07-21T22:21:56.364 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.319+0000 7f02657fa700 1 -- 172.21.15.107:0/1120301430 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0258029930 con 0x7f02680ffa80 2023-07-21T22:21:56.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.321+0000 7f026e6ba700 1 -- 172.21.15.107:0/1120301430 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f02500387b0 msgr2=0x7f025003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:56.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.321+0000 7f026e6ba700 1 --2- 172.21.15.107:0/1120301430 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f02500387b0 0x7f025003ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f025c006fd0 tx=0x7f025c006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:56.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.321+0000 7f026e6ba700 1 -- 172.21.15.107:0/1120301430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02680ffa80 msgr2=0x7f02680fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:56.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.321+0000 7f026e6ba700 1 --2- 172.21.15.107:0/1120301430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02680ffa80 0x7f02680fe210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0258004f40 tx=0x7f02580057c0 comp rx=0 tx=0).stop 2023-07-21T22:21:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.321+0000 7f026e6ba700 1 -- 172.21.15.107:0/1120301430 shutdown_connections 2023-07-21T22:21:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.321+0000 7f026e6ba700 1 --2- 172.21.15.107:0/1120301430 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f02500387b0 0x7f025003ac60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.321+0000 7f026e6ba700 1 --2- 172.21.15.107:0/1120301430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02680ffa80 0x7f02680fe210 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.321+0000 7f026e6ba700 1 -- 172.21.15.107:0/1120301430 >> 172.21.15.107:0/1120301430 conn(0x7f02680f9220 msgr2=0x7f02680f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.321+0000 7f026e6ba700 1 -- 172.21.15.107:0/1120301430 shutdown_connections 2023-07-21T22:21:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:56.321+0000 7f026e6ba700 1 -- 172.21.15.107:0/1120301430 wait complete. 2023-07-21T22:21:56.785 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-21T22:21:56.786 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-21T22:21:57.334 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:21:57.686 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:57 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1120301430' entity='client.admin' 2023-07-21T22:21:57.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:57 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:57.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:57 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:21:58.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.482+0000 7f64d6a44700 1 -- 172.21.15.107:0/1540652451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d00fdc70 msgr2=0x7f64d00fe080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:58.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.482+0000 7f64d6a44700 1 --2- 172.21.15.107:0/1540652451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d00fdc70 0x7f64d00fe080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009910 tx=0x7f64b8009c20 comp rx=0 tx=0).stop 2023-07-21T22:21:58.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.482+0000 7f64d6a44700 1 -- 172.21.15.107:0/1540652451 shutdown_connections 2023-07-21T22:21:58.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.482+0000 7f64d6a44700 1 --2- 172.21.15.107:0/1540652451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d00fdc70 0x7f64d00fe080 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.482+0000 7f64d6a44700 1 -- 172.21.15.107:0/1540652451 >> 172.21.15.107:0/1540652451 conn(0x7f64d00f9220 msgr2=0x7f64d00fb650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:58.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.482+0000 7f64d6a44700 1 -- 172.21.15.107:0/1540652451 shutdown_connections 2023-07-21T22:21:58.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.482+0000 7f64d6a44700 1 -- 172.21.15.107:0/1540652451 wait complete. 2023-07-21T22:21:58.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.482+0000 7f64d6a44700 1 Processor -- start 2023-07-21T22:21:58.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.483+0000 7f64d6a44700 1 -- start start 2023-07-21T22:21:58.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.483+0000 7f64d6a44700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d00fdc70 0x7f64d00fa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.483+0000 7f64d6a44700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d0197430 con 0x7f64d00fdc70 2023-07-21T22:21:58.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.483+0000 7f64cffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d00fdc70 0x7f64d00fa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.483+0000 7f64cffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d00fdc70 0x7f64d00fa1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45448/0 (socket says 172.21.15.107:45448) 2023-07-21T22:21:58.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.483+0000 7f64cffff700 1 -- 172.21.15.107:0/251710856 learned_addr learned my addr 172.21.15.107:0/251710856 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:21:58.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.483+0000 7f64cffff700 1 -- 172.21.15.107:0/251710856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b80095f0 con 0x7f64d00fdc70 2023-07-21T22:21:58.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.483+0000 7f64cffff700 1 --2- 172.21.15.107:0/251710856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d00fdc70 0x7f64d00fa1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f64b8006010 tx=0x7f64b80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:58.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.483+0000 7f64cd7fa700 1 -- 172.21.15.107:0/251710856 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64b801d070 con 0x7f64d00fdc70 2023-07-21T22:21:58.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.484+0000 7f64d6a44700 1 -- 172.21.15.107:0/251710856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d0197570 con 0x7f64d00fdc70 2023-07-21T22:21:58.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.484+0000 7f64cd7fa700 1 -- 172.21.15.107:0/251710856 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64b80044e0 con 0x7f64d00fdc70 2023-07-21T22:21:58.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.484+0000 7f64cd7fa700 1 -- 172.21.15.107:0/251710856 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64b80039b0 con 0x7f64d00fdc70 2023-07-21T22:21:58.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.484+0000 7f64d6a44700 1 -- 172.21.15.107:0/251710856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d01979b0 con 0x7f64d00fdc70 2023-07-21T22:21:58.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.484+0000 7f64cd7fa700 1 -- 172.21.15.107:0/251710856 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f64b8003c70 con 0x7f64d00fdc70 2023-07-21T22:21:58.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.484+0000 7f64cd7fa700 1 --2- 172.21.15.107:0/251710856 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64bc0387b0 0x7f64bc03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:58.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.485+0000 7f64cd7fa700 1 -- 172.21.15.107:0/251710856 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f64b804c830 con 0x7f64d00fdc70 2023-07-21T22:21:58.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.485+0000 7f64cf7fe700 1 --2- 172.21.15.107:0/251710856 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64bc0387b0 0x7f64bc03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:58.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.485+0000 7f64d6a44700 1 -- 172.21.15.107:0/251710856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d01039b0 con 0x7f64d00fdc70 2023-07-21T22:21:58.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.485+0000 7f64cf7fe700 1 --2- 172.21.15.107:0/251710856 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64bc0387b0 0x7f64bc03ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f64c0006fd0 tx=0x7f64c0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.489+0000 7f64cd7fa700 1 -- 172.21.15.107:0/251710856 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f64b800f460 con 0x7f64d00fdc70 2023-07-21T22:21:58.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.679+0000 7f64d6a44700 1 -- 172.21.15.107:0/251710856 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f64d0066ef0 con 0x7f64bc0387b0 2023-07-21T22:21:58.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.682+0000 7f64cd7fa700 1 -- 172.21.15.107:0/251710856 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f64d0066ef0 con 0x7f64bc0387b0 2023-07-21T22:21:58.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.684+0000 7f64d6a44700 1 -- 172.21.15.107:0/251710856 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64bc0387b0 msgr2=0x7f64bc03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:58.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.684+0000 7f64d6a44700 1 --2- 172.21.15.107:0/251710856 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64bc0387b0 0x7f64bc03ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f64c0006fd0 tx=0x7f64c0006e40 comp rx=0 tx=0).stop 2023-07-21T22:21:58.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.684+0000 7f64d6a44700 1 -- 172.21.15.107:0/251710856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d00fdc70 msgr2=0x7f64d00fa1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:58.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.684+0000 7f64d6a44700 1 --2- 172.21.15.107:0/251710856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d00fdc70 0x7f64d00fa1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f64b8006010 tx=0x7f64b80057c0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.685+0000 7f64d6a44700 1 -- 172.21.15.107:0/251710856 shutdown_connections 2023-07-21T22:21:58.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.685+0000 7f64d6a44700 1 --2- 172.21.15.107:0/251710856 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64bc0387b0 0x7f64bc03ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.685+0000 7f64d6a44700 1 --2- 172.21.15.107:0/251710856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64d00fdc70 0x7f64d00fa1a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:58.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.685+0000 7f64d6a44700 1 -- 172.21.15.107:0/251710856 >> 172.21.15.107:0/251710856 conn(0x7f64d00f9220 msgr2=0x7f64d00fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:58.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.685+0000 7f64d6a44700 1 -- 172.21.15.107:0/251710856 shutdown_connections 2023-07-21T22:21:58.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:21:58.685+0000 7f64d6a44700 1 -- 172.21.15.107:0/251710856 wait complete. 2023-07-21T22:21:59.179 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi132 2023-07-21T22:21:59.180 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-07-21T22:21:59.181 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/etc/ceph/ceph.conf 2023-07-21T22:21:59.202 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-07-21T22:21:59.202 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:21:59.261 INFO:tasks.cephadm:Adding host smithi132 to orchestrator... 2023-07-21T22:21:59.262 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch host add smithi132 2023-07-21T22:21:59.824 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:22:00.061 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:59 smithi107 ceph-mon[96543]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:00.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:59 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:00.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:59 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:00.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:59 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:00.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:59 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:00.062 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:59 smithi107 ceph-mon[96543]: Updating smithi107:/etc/ceph/ceph.conf 2023-07-21T22:22:00.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:59 smithi107 ceph-mon[96543]: Updating smithi107:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:00.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:59 smithi107 ceph-mon[96543]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:22:00.063 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:59 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:00.064 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:59 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:00.064 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:21:59 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:00.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:00 smithi107 ceph-mon[96543]: Updating smithi107:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:22:01.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.062+0000 7f5da29fa700 1 -- 172.21.15.107:0/415574017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c104540 msgr2=0x7f5d9c104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:01.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.062+0000 7f5da29fa700 1 --2- 172.21.15.107:0/415574017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c104540 0x7f5d9c104910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009910 tx=0x7f5d8c009c20 comp rx=0 tx=0).stop 2023-07-21T22:22:01.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.062+0000 7f5da29fa700 1 -- 172.21.15.107:0/415574017 shutdown_connections 2023-07-21T22:22:01.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.062+0000 7f5da29fa700 1 --2- 172.21.15.107:0/415574017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c104540 0x7f5d9c104910 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:01.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.062+0000 7f5da29fa700 1 -- 172.21.15.107:0/415574017 >> 172.21.15.107:0/415574017 conn(0x7f5d9c0fa020 msgr2=0x7f5d9c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:01.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.062+0000 7f5da29fa700 1 -- 172.21.15.107:0/415574017 shutdown_connections 2023-07-21T22:22:01.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.062+0000 7f5da29fa700 1 -- 172.21.15.107:0/415574017 wait complete. 2023-07-21T22:22:01.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.063+0000 7f5da29fa700 1 Processor -- start 2023-07-21T22:22:01.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.063+0000 7f5da29fa700 1 -- start start 2023-07-21T22:22:01.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.063+0000 7f5da29fa700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c104540 0x7f5d9c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:01.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.063+0000 7f5da29fa700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c199b50 con 0x7f5d9c104540 2023-07-21T22:22:01.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.063+0000 7f5d9bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c104540 0x7f5d9c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:01.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.063+0000 7f5d9bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c104540 0x7f5d9c199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45454/0 (socket says 172.21.15.107:45454) 2023-07-21T22:22:01.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.063+0000 7f5d9bfff700 1 -- 172.21.15.107:0/1395210416 learned_addr learned my addr 172.21.15.107:0/1395210416 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:22:01.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.064+0000 7f5d9bfff700 1 -- 172.21.15.107:0/1395210416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c0095f0 con 0x7f5d9c104540 2023-07-21T22:22:01.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.064+0000 7f5d9bfff700 1 --2- 172.21.15.107:0/1395210416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c104540 0x7f5d9c199610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c004f40 tx=0x7f5d8c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:01.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.064+0000 7f5d997fa700 1 -- 172.21.15.107:0/1395210416 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d8c01f070 con 0x7f5d9c104540 2023-07-21T22:22:01.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.064+0000 7f5d997fa700 1 -- 172.21.15.107:0/1395210416 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d8c004370 con 0x7f5d9c104540 2023-07-21T22:22:01.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.064+0000 7f5da29fa700 1 -- 172.21.15.107:0/1395210416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c199d50 con 0x7f5d9c104540 2023-07-21T22:22:01.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.064+0000 7f5d997fa700 1 -- 172.21.15.107:0/1395210416 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d8c0039b0 con 0x7f5d9c104540 2023-07-21T22:22:01.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.064+0000 7f5da29fa700 1 -- 172.21.15.107:0/1395210416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c19a1f0 con 0x7f5d9c104540 2023-07-21T22:22:01.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.065+0000 7f5d997fa700 1 -- 172.21.15.107:0/1395210416 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f5d8c003c70 con 0x7f5d9c104540 2023-07-21T22:22:01.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.065+0000 7f5da29fa700 1 -- 172.21.15.107:0/1395210416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c061b40 con 0x7f5d9c104540 2023-07-21T22:22:01.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.065+0000 7f5d997fa700 1 --2- 172.21.15.107:0/1395210416 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5d840387b0 0x7f5d8403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:01.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.065+0000 7f5d997fa700 1 -- 172.21.15.107:0/1395210416 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5d8c04bfc0 con 0x7f5d9c104540 2023-07-21T22:22:01.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.065+0000 7f5d9b7fe700 1 --2- 172.21.15.107:0/1395210416 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5d840387b0 0x7f5d8403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:01.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.065+0000 7f5d9b7fe700 1 --2- 172.21.15.107:0/1395210416 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5d840387b0 0x7f5d8403ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5d90006fd0 tx=0x7f5d90006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:01.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.070+0000 7f5d997fa700 1 -- 172.21.15.107:0/1395210416 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5d8c07e770 con 0x7f5d9c104540 2023-07-21T22:22:01.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:01.258+0000 7f5da29fa700 1 -- 172.21.15.107:0/1395210416 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi132", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d9c103460 con 0x7f5d840387b0 2023-07-21T22:22:02.686 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:02 smithi107 ceph-mon[96543]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi132", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:02.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:02 smithi107 ceph-mon[96543]: Deploying cephadm binary to smithi132 2023-07-21T22:22:03.565 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:03.520+0000 7f5d997fa700 1 -- 172.21.15.107:0/1395210416 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5d9c103460 con 0x7f5d840387b0 2023-07-21T22:22:03.566 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi132' with addr '172.21.15.132' 2023-07-21T22:22:03.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:03.522+0000 7f5da29fa700 1 -- 172.21.15.107:0/1395210416 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5d840387b0 msgr2=0x7f5d8403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:03.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:03.522+0000 7f5da29fa700 1 --2- 172.21.15.107:0/1395210416 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5d840387b0 0x7f5d8403ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5d90006fd0 tx=0x7f5d90006e40 comp rx=0 tx=0).stop 2023-07-21T22:22:03.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:03.522+0000 7f5da29fa700 1 -- 172.21.15.107:0/1395210416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c104540 msgr2=0x7f5d9c199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:03.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:03.522+0000 7f5da29fa700 1 --2- 172.21.15.107:0/1395210416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c104540 0x7f5d9c199610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c004f40 tx=0x7f5d8c005020 comp rx=0 tx=0).stop 2023-07-21T22:22:03.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:03.522+0000 7f5da29fa700 1 -- 172.21.15.107:0/1395210416 shutdown_connections 2023-07-21T22:22:03.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:03.522+0000 7f5da29fa700 1 --2- 172.21.15.107:0/1395210416 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5d840387b0 0x7f5d8403ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:03.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:03.522+0000 7f5da29fa700 1 --2- 172.21.15.107:0/1395210416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c104540 0x7f5d9c199610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:03.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:03.522+0000 7f5da29fa700 1 -- 172.21.15.107:0/1395210416 >> 172.21.15.107:0/1395210416 conn(0x7f5d9c0fa020 msgr2=0x7f5d9c0fbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:03.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:03.522+0000 7f5da29fa700 1 -- 172.21.15.107:0/1395210416 shutdown_connections 2023-07-21T22:22:03.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:03.522+0000 7f5da29fa700 1 -- 172.21.15.107:0/1395210416 wait complete. 2023-07-21T22:22:03.977 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-21T22:22:04.527 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:22:04.916 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:04 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:04.916 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:04 smithi107 ceph-mon[96543]: Added host smithi132 2023-07-21T22:22:04.916 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:04 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:04.916 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:04 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:05.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.693+0000 7ff1d89f4700 1 -- 172.21.15.107:0/4276989746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1d4104540 msgr2=0x7ff1d4104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.693+0000 7ff1d89f4700 1 --2- 172.21.15.107:0/4276989746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1d4104540 0x7ff1d4104910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009910 tx=0x7ff1bc009c20 comp rx=0 tx=0).stop 2023-07-21T22:22:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.693+0000 7ff1d89f4700 1 -- 172.21.15.107:0/4276989746 shutdown_connections 2023-07-21T22:22:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.693+0000 7ff1d89f4700 1 --2- 172.21.15.107:0/4276989746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1d4104540 0x7ff1d4104910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.693+0000 7ff1d89f4700 1 -- 172.21.15.107:0/4276989746 >> 172.21.15.107:0/4276989746 conn(0x7ff1d40fa020 msgr2=0x7ff1d40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:05.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.694+0000 7ff1d89f4700 1 -- 172.21.15.107:0/4276989746 shutdown_connections 2023-07-21T22:22:05.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.694+0000 7ff1d89f4700 1 -- 172.21.15.107:0/4276989746 wait complete. 2023-07-21T22:22:05.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.694+0000 7ff1d89f4700 1 Processor -- start 2023-07-21T22:22:05.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.694+0000 7ff1d89f4700 1 -- start start 2023-07-21T22:22:05.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.695+0000 7ff1d89f4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1d4104540 0x7ff1d41957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:05.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.695+0000 7ff1d89f4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4195d30 con 0x7ff1d4104540 2023-07-21T22:22:05.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.695+0000 7ff1d259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1d4104540 0x7ff1d41957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:05.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.695+0000 7ff1d259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1d4104540 0x7ff1d41957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45480/0 (socket says 172.21.15.107:45480) 2023-07-21T22:22:05.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.695+0000 7ff1d259c700 1 -- 172.21.15.107:0/1684416088 learned_addr learned my addr 172.21.15.107:0/1684416088 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:22:05.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.695+0000 7ff1d259c700 1 -- 172.21.15.107:0/1684416088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc0095f0 con 0x7ff1d4104540 2023-07-21T22:22:05.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.695+0000 7ff1d259c700 1 --2- 172.21.15.107:0/1684416088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1d4104540 0x7ff1d41957f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc004f40 tx=0x7ff1bc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:05.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.695+0000 7ff1cb7fe700 1 -- 172.21.15.107:0/1684416088 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1bc01d070 con 0x7ff1d4104540 2023-07-21T22:22:05.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.695+0000 7ff1d89f4700 1 -- 172.21.15.107:0/1684416088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d4195f90 con 0x7ff1d4104540 2023-07-21T22:22:05.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.695+0000 7ff1cb7fe700 1 -- 172.21.15.107:0/1684416088 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1bc004370 con 0x7ff1d4104540 2023-07-21T22:22:05.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.695+0000 7ff1cb7fe700 1 -- 172.21.15.107:0/1684416088 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1bc0039b0 con 0x7ff1d4104540 2023-07-21T22:22:05.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.695+0000 7ff1d89f4700 1 -- 172.21.15.107:0/1684416088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d4196410 con 0x7ff1d4104540 2023-07-21T22:22:05.744 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.696+0000 7ff1cb7fe700 1 -- 172.21.15.107:0/1684416088 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7ff1bc003c70 con 0x7ff1d4104540 2023-07-21T22:22:05.744 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.696+0000 7ff1d89f4700 1 -- 172.21.15.107:0/1684416088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d4196660 con 0x7ff1d4104540 2023-07-21T22:22:05.745 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.696+0000 7ff1cb7fe700 1 --2- 172.21.15.107:0/1684416088 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff1c00387b0 0x7ff1c003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:05.745 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.696+0000 7ff1cb7fe700 1 -- 172.21.15.107:0/1684416088 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff1bc04c7c0 con 0x7ff1d4104540 2023-07-21T22:22:05.745 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.696+0000 7ff1d1d9b700 1 --2- 172.21.15.107:0/1684416088 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff1c00387b0 0x7ff1c003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:05.745 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.697+0000 7ff1d1d9b700 1 --2- 172.21.15.107:0/1684416088 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff1c00387b0 0x7ff1c003ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4006fd0 tx=0x7ff1c4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:05.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.701+0000 7ff1cb7fe700 1 -- 172.21.15.107:0/1684416088 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff1d4196660 con 0x7ff1d4104540 2023-07-21T22:22:05.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:05 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:05.939 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.894+0000 7ff1d89f4700 1 -- 172.21.15.107:0/1684416088 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1d4196660 con 0x7ff1c00387b0 2023-07-21T22:22:05.940 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.895+0000 7ff1cb7fe700 1 -- 172.21.15.107:0/1684416088 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7ff1d4196660 con 0x7ff1c00387b0 2023-07-21T22:22:05.940 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:22:05.940 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.132", "hostname": "smithi132", "labels": [], "status": ""}] 2023-07-21T22:22:05.942 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.897+0000 7ff1d89f4700 1 -- 172.21.15.107:0/1684416088 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff1c00387b0 msgr2=0x7ff1c003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:05.942 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.897+0000 7ff1d89f4700 1 --2- 172.21.15.107:0/1684416088 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff1c00387b0 0x7ff1c003ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4006fd0 tx=0x7ff1c4006e40 comp rx=0 tx=0).stop 2023-07-21T22:22:05.942 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.897+0000 7ff1d89f4700 1 -- 172.21.15.107:0/1684416088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1d4104540 msgr2=0x7ff1d41957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:05.942 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.897+0000 7ff1d89f4700 1 --2- 172.21.15.107:0/1684416088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1d4104540 0x7ff1d41957f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc004f40 tx=0x7ff1bc005020 comp rx=0 tx=0).stop 2023-07-21T22:22:05.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.897+0000 7ff1d89f4700 1 -- 172.21.15.107:0/1684416088 shutdown_connections 2023-07-21T22:22:05.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.897+0000 7ff1d89f4700 1 --2- 172.21.15.107:0/1684416088 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff1c00387b0 0x7ff1c003ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:05.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.897+0000 7ff1d89f4700 1 --2- 172.21.15.107:0/1684416088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1d4104540 0x7ff1d41957f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:05.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.897+0000 7ff1d89f4700 1 -- 172.21.15.107:0/1684416088 >> 172.21.15.107:0/1684416088 conn(0x7ff1d40fa020 msgr2=0x7ff1d40fbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:05.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.897+0000 7ff1d89f4700 1 -- 172.21.15.107:0/1684416088 shutdown_connections 2023-07-21T22:22:05.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:05.897+0000 7ff1d89f4700 1 -- 172.21.15.107:0/1684416088 wait complete. 2023-07-21T22:22:06.283 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi190 2023-07-21T22:22:06.283 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-07-21T22:22:06.284 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.conf 2023-07-21T22:22:06.305 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-07-21T22:22:06.305 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:22:06.364 INFO:tasks.cephadm:Adding host smithi190 to orchestrator... 2023-07-21T22:22:06.365 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch host add smithi190 2023-07-21T22:22:06.913 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:22:07.178 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:06 smithi107 ceph-mon[96543]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:07.178 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:06 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:07.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:06 smithi107 ceph-mon[96543]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:22:08.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.072+0000 7f4629305700 1 -- 172.21.15.107:0/4278796557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4624104540 msgr2=0x7f4624104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:08.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.072+0000 7f4629305700 1 --2- 172.21.15.107:0/4278796557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4624104540 0x7f4624104910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4614009910 tx=0x7f4614009c20 comp rx=0 tx=0).stop 2023-07-21T22:22:08.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.072+0000 7f4629305700 1 -- 172.21.15.107:0/4278796557 shutdown_connections 2023-07-21T22:22:08.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.072+0000 7f4629305700 1 --2- 172.21.15.107:0/4278796557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4624104540 0x7f4624104910 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:08.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.072+0000 7f4629305700 1 -- 172.21.15.107:0/4278796557 >> 172.21.15.107:0/4278796557 conn(0x7f46240fa020 msgr2=0x7f46240fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:08.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.072+0000 7f4629305700 1 -- 172.21.15.107:0/4278796557 shutdown_connections 2023-07-21T22:22:08.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.072+0000 7f4629305700 1 -- 172.21.15.107:0/4278796557 wait complete. 2023-07-21T22:22:08.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.073+0000 7f4629305700 1 Processor -- start 2023-07-21T22:22:08.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.073+0000 7f4629305700 1 -- start start 2023-07-21T22:22:08.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.073+0000 7f4629305700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4624104540 0x7f4624194990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:08.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.073+0000 7f4629305700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4624194ed0 con 0x7f4624104540 2023-07-21T22:22:08.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.073+0000 7f4622ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4624104540 0x7f4624194990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:08.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.073+0000 7f4622ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4624104540 0x7f4624194990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45486/0 (socket says 172.21.15.107:45486) 2023-07-21T22:22:08.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.073+0000 7f4622ffd700 1 -- 172.21.15.107:0/1701167597 learned_addr learned my addr 172.21.15.107:0/1701167597 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:22:08.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.074+0000 7f4622ffd700 1 -- 172.21.15.107:0/1701167597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46140095f0 con 0x7f4624104540 2023-07-21T22:22:08.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.074+0000 7f4622ffd700 1 --2- 172.21.15.107:0/1701167597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4624104540 0x7f4624194990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4614004f40 tx=0x7f4614005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:08.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.074+0000 7f460bfff700 1 -- 172.21.15.107:0/1701167597 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f461401d070 con 0x7f4624104540 2023-07-21T22:22:08.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.074+0000 7f460bfff700 1 -- 172.21.15.107:0/1701167597 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4614004370 con 0x7f4624104540 2023-07-21T22:22:08.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.074+0000 7f4629305700 1 -- 172.21.15.107:0/1701167597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4624195130 con 0x7f4624104540 2023-07-21T22:22:08.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.074+0000 7f460bfff700 1 -- 172.21.15.107:0/1701167597 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46140039b0 con 0x7f4624104540 2023-07-21T22:22:08.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.075+0000 7f4629305700 1 -- 172.21.15.107:0/1701167597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4624195630 con 0x7f4624104540 2023-07-21T22:22:08.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.075+0000 7f460bfff700 1 -- 172.21.15.107:0/1701167597 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f4614003c70 con 0x7f4624104540 2023-07-21T22:22:08.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.075+0000 7f460bfff700 1 --2- 172.21.15.107:0/1701167597 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f460c0387b0 0x7f460c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:08.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.075+0000 7f460bfff700 1 -- 172.21.15.107:0/1701167597 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f461404de40 con 0x7f4624104540 2023-07-21T22:22:08.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.075+0000 7f46227fc700 1 --2- 172.21.15.107:0/1701167597 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f460c0387b0 0x7f460c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:08.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.076+0000 7f46227fc700 1 --2- 172.21.15.107:0/1701167597 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f460c0387b0 0x7f460c03ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4618006fd0 tx=0x7f4618006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:08.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.076+0000 7f4629305700 1 -- 172.21.15.107:0/1701167597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4624061b40 con 0x7f4624104540 2023-07-21T22:22:08.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.080+0000 7f460bfff700 1 -- 172.21.15.107:0/1701167597 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4614026070 con 0x7f4624104540 2023-07-21T22:22:08.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:08.271+0000 7f4629305700 1 -- 172.21.15.107:0/1701167597 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}) v1 -- 0x7f4624103050 con 0x7f460c0387b0 2023-07-21T22:22:09.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:08 smithi107 ceph-mon[96543]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi132", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:22:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:10.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:09 smithi107 ceph-mon[96543]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:10.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:09 smithi107 ceph-mon[96543]: Updating smithi132:/etc/ceph/ceph.conf 2023-07-21T22:22:10.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:09 smithi107 ceph-mon[96543]: Updating smithi132:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:10.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:09 smithi107 ceph-mon[96543]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:22:10.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:09 smithi107 ceph-mon[96543]: Deploying cephadm binary to smithi190 2023-07-21T22:22:10.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:09 smithi107 ceph-mon[96543]: Updating smithi132:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:22:10.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:09 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:10.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:09 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:10.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:09 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:10.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:10.691+0000 7f460bfff700 1 -- 172.21.15.107:0/1701167597 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4624103050 con 0x7f460c0387b0 2023-07-21T22:22:10.736 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi190' with addr '172.21.15.190' 2023-07-21T22:22:10.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:10.693+0000 7f4629305700 1 -- 172.21.15.107:0/1701167597 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f460c0387b0 msgr2=0x7f460c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:10.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:10.693+0000 7f4629305700 1 --2- 172.21.15.107:0/1701167597 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f460c0387b0 0x7f460c03ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4618006fd0 tx=0x7f4618006e40 comp rx=0 tx=0).stop 2023-07-21T22:22:10.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:10.693+0000 7f4629305700 1 -- 172.21.15.107:0/1701167597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4624104540 msgr2=0x7f4624194990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:10.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:10.693+0000 7f4629305700 1 --2- 172.21.15.107:0/1701167597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4624104540 0x7f4624194990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4614004f40 tx=0x7f4614005020 comp rx=0 tx=0).stop 2023-07-21T22:22:10.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:10.693+0000 7f4629305700 1 -- 172.21.15.107:0/1701167597 shutdown_connections 2023-07-21T22:22:10.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:10.693+0000 7f4629305700 1 --2- 172.21.15.107:0/1701167597 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f460c0387b0 0x7f460c03ac60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:10.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:10.693+0000 7f4629305700 1 --2- 172.21.15.107:0/1701167597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4624104540 0x7f4624194990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:10.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:10.693+0000 7f4629305700 1 -- 172.21.15.107:0/1701167597 >> 172.21.15.107:0/1701167597 conn(0x7f46240fa020 msgr2=0x7f4624068840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:10.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:10.693+0000 7f4629305700 1 -- 172.21.15.107:0/1701167597 shutdown_connections 2023-07-21T22:22:10.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:10.693+0000 7f4629305700 1 -- 172.21.15.107:0/1701167597 wait complete. 2023-07-21T22:22:11.137 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:10 smithi107 ceph-mon[96543]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:11.138 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:10 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:11.138 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:10 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:11.163 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-21T22:22:11.714 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:22:12.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:12 smithi107 ceph-mon[96543]: Added host smithi190 2023-07-21T22:22:12.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:12 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:12.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:12 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:12.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.841+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3831651646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8104540 msgr2=0x7fb4c8104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:12.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.841+0000 7fb4cdf44700 1 --2- 172.21.15.107:0/3831651646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8104540 0x7fb4c8104910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009910 tx=0x7fb4b8009c20 comp rx=0 tx=0).stop 2023-07-21T22:22:12.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.841+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3831651646 shutdown_connections 2023-07-21T22:22:12.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.841+0000 7fb4cdf44700 1 --2- 172.21.15.107:0/3831651646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8104540 0x7fb4c8104910 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:12.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.841+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3831651646 >> 172.21.15.107:0/3831651646 conn(0x7fb4c80fa020 msgr2=0x7fb4c80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:12.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.841+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3831651646 shutdown_connections 2023-07-21T22:22:12.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.841+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3831651646 wait complete. 2023-07-21T22:22:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.842+0000 7fb4cdf44700 1 Processor -- start 2023-07-21T22:22:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.842+0000 7fb4cdf44700 1 -- start start 2023-07-21T22:22:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.843+0000 7fb4cdf44700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8104540 0x7fb4c8199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.843+0000 7fb4cdf44700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c8199ba0 con 0x7fb4c8104540 2023-07-21T22:22:12.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.843+0000 7fb4c77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8104540 0x7fb4c8199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:12.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.843+0000 7fb4c77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8104540 0x7fb4c8199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45512/0 (socket says 172.21.15.107:45512) 2023-07-21T22:22:12.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.843+0000 7fb4c77fe700 1 -- 172.21.15.107:0/3157051834 learned_addr learned my addr 172.21.15.107:0/3157051834 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:22:12.893 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.843+0000 7fb4c77fe700 1 -- 172.21.15.107:0/3157051834 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b80095f0 con 0x7fb4c8104540 2023-07-21T22:22:12.893 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.843+0000 7fb4c77fe700 1 --2- 172.21.15.107:0/3157051834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8104540 0x7fb4c8199660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8006010 tx=0x7fb4b8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:12.893 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.843+0000 7fb4c4ff9700 1 -- 172.21.15.107:0/3157051834 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4b801d070 con 0x7fb4c8104540 2023-07-21T22:22:12.894 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.843+0000 7fb4c4ff9700 1 -- 172.21.15.107:0/3157051834 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4b800bc10 con 0x7fb4c8104540 2023-07-21T22:22:12.894 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.843+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3157051834 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c8199e00 con 0x7fb4c8104540 2023-07-21T22:22:12.894 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.843+0000 7fb4c4ff9700 1 -- 172.21.15.107:0/3157051834 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4b8003910 con 0x7fb4c8104540 2023-07-21T22:22:12.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.843+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3157051834 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c8193650 con 0x7fb4c8104540 2023-07-21T22:22:12.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.844+0000 7fb4c4ff9700 1 -- 172.21.15.107:0/3157051834 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fb4b8003bd0 con 0x7fb4c8104540 2023-07-21T22:22:12.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.844+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3157051834 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c8061b40 con 0x7fb4c8104540 2023-07-21T22:22:12.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.844+0000 7fb4c4ff9700 1 --2- 172.21.15.107:0/3157051834 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb4b00387b0 0x7fb4b003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:12.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.844+0000 7fb4c4ff9700 1 -- 172.21.15.107:0/3157051834 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb4b804c530 con 0x7fb4c8104540 2023-07-21T22:22:12.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.844+0000 7fb4c6ffd700 1 --2- 172.21.15.107:0/3157051834 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb4b00387b0 0x7fb4b003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.845+0000 7fb4c6ffd700 1 --2- 172.21.15.107:0/3157051834 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb4b00387b0 0x7fb4b003ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc006fd0 tx=0x7fb4bc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:12.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:12.849+0000 7fb4c4ff9700 1 -- 172.21.15.107:0/3157051834 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb4b8029950 con 0x7fb4c8104540 2023-07-21T22:22:13.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:13.040+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3157051834 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4c8194180 con 0x7fb4b00387b0 2023-07-21T22:22:13.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:13.041+0000 7fb4c4ff9700 1 -- 172.21.15.107:0/3157051834 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7fb4c8194180 con 0x7fb4b00387b0 2023-07-21T22:22:13.086 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:22:13.086 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.132", "hostname": "smithi132", "labels": [], "status": ""}, {"addr": "172.21.15.190", "hostname": "smithi190", "labels": [], "status": ""}] 2023-07-21T22:22:13.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:13.043+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3157051834 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb4b00387b0 msgr2=0x7fb4b003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:13.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:13.043+0000 7fb4cdf44700 1 --2- 172.21.15.107:0/3157051834 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb4b00387b0 0x7fb4b003ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc006fd0 tx=0x7fb4bc006e40 comp rx=0 tx=0).stop 2023-07-21T22:22:13.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:13.043+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3157051834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8104540 msgr2=0x7fb4c8199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:13.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:13.043+0000 7fb4cdf44700 1 --2- 172.21.15.107:0/3157051834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8104540 0x7fb4c8199660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8006010 tx=0x7fb4b8004ff0 comp rx=0 tx=0).stop 2023-07-21T22:22:13.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:13.043+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3157051834 shutdown_connections 2023-07-21T22:22:13.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:13.043+0000 7fb4cdf44700 1 --2- 172.21.15.107:0/3157051834 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb4b00387b0 0x7fb4b003ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:13.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:13.043+0000 7fb4cdf44700 1 --2- 172.21.15.107:0/3157051834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8104540 0x7fb4c8199660 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:13.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:13.043+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3157051834 >> 172.21.15.107:0/3157051834 conn(0x7fb4c80fa020 msgr2=0x7fb4c80fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:13.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:13.043+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3157051834 shutdown_connections 2023-07-21T22:22:13.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:13.043+0000 7fb4cdf44700 1 -- 172.21.15.107:0/3157051834 wait complete. 2023-07-21T22:22:13.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:13 smithi107 ceph-mon[96543]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:13.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:13.466 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-21T22:22:13.467 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd crush tunables default 2023-07-21T22:22:14.015 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:22:14.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:14 smithi107 ceph-mon[96543]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:22:15.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.110+0000 7f4966e50700 1 -- 172.21.15.107:0/3757207146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960100820 msgr2=0x7f4960100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.110+0000 7f4966e50700 1 --2- 172.21.15.107:0/3757207146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960100820 0x7f4960100bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4954009910 tx=0x7f4954009c20 comp rx=0 tx=0).stop 2023-07-21T22:22:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.110+0000 7f4966e50700 1 -- 172.21.15.107:0/3757207146 shutdown_connections 2023-07-21T22:22:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.110+0000 7f4966e50700 1 --2- 172.21.15.107:0/3757207146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960100820 0x7f4960100bf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.110+0000 7f4966e50700 1 -- 172.21.15.107:0/3757207146 >> 172.21.15.107:0/3757207146 conn(0x7f49600fa020 msgr2=0x7f49600fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.110+0000 7f4966e50700 1 -- 172.21.15.107:0/3757207146 shutdown_connections 2023-07-21T22:22:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.110+0000 7f4966e50700 1 -- 172.21.15.107:0/3757207146 wait complete. 2023-07-21T22:22:15.157 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.111+0000 7f4966e50700 1 Processor -- start 2023-07-21T22:22:15.157 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.111+0000 7f4966e50700 1 -- start start 2023-07-21T22:22:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.111+0000 7f4966e50700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960100820 0x7f496019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.111+0000 7f4966e50700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496019aea0 con 0x7f4960100820 2023-07-21T22:22:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.111+0000 7f4964bec700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960100820 0x7f496019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.111+0000 7f4964bec700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960100820 0x7f496019a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45518/0 (socket says 172.21.15.107:45518) 2023-07-21T22:22:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.111+0000 7f4964bec700 1 -- 172.21.15.107:0/181226069 learned_addr learned my addr 172.21.15.107:0/181226069 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:22:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.111+0000 7f4964bec700 1 -- 172.21.15.107:0/181226069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49540095f0 con 0x7f4960100820 2023-07-21T22:22:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.111+0000 7f4964bec700 1 --2- 172.21.15.107:0/181226069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960100820 0x7f496019a960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4954005850 tx=0x7f4954004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.111+0000 7f495dffb700 1 -- 172.21.15.107:0/181226069 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f495401d070 con 0x7f4960100820 2023-07-21T22:22:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.111+0000 7f495dffb700 1 -- 172.21.15.107:0/181226069 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4954004370 con 0x7f4960100820 2023-07-21T22:22:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.111+0000 7f4966e50700 1 -- 172.21.15.107:0/181226069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496019b0a0 con 0x7f4960100820 2023-07-21T22:22:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.111+0000 7f495dffb700 1 -- 172.21.15.107:0/181226069 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4954003ab0 con 0x7f4960100820 2023-07-21T22:22:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.112+0000 7f4966e50700 1 -- 172.21.15.107:0/181226069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4960194950 con 0x7f4960100820 2023-07-21T22:22:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.112+0000 7f495dffb700 1 -- 172.21.15.107:0/181226069 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f495400f460 con 0x7f4960100820 2023-07-21T22:22:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.113+0000 7f495dffb700 1 --2- 172.21.15.107:0/181226069 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f49500387b0 0x7f495003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.113+0000 7f495dffb700 1 -- 172.21.15.107:0/181226069 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f495404c570 con 0x7f4960100820 2023-07-21T22:22:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.113+0000 7f4966e50700 1 -- 172.21.15.107:0/181226069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f496019b4c0 con 0x7f4960100820 2023-07-21T22:22:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.113+0000 7f495ffff700 1 --2- 172.21.15.107:0/181226069 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f49500387b0 0x7f495003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.113+0000 7f495ffff700 1 --2- 172.21.15.107:0/181226069 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f49500387b0 0x7f495003ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f494c006fd0 tx=0x7f494c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:15.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.117+0000 7f495dffb700 1 -- 172.21.15.107:0/181226069 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4954026020 con 0x7f4960100820 2023-07-21T22:22:15.358 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:15.313+0000 7f4966e50700 1 -- 172.21.15.107:0/181226069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4960195f20 con 0x7f4960100820 2023-07-21T22:22:15.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:15 smithi107 ceph-mon[96543]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:16.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:16.159+0000 7f495dffb700 1 -- 172.21.15.107:0/181226069 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4954029950 con 0x7f4960100820 2023-07-21T22:22:16.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:16.161+0000 7f4966e50700 1 -- 172.21.15.107:0/181226069 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f49500387b0 msgr2=0x7f495003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:16.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:16.161+0000 7f4966e50700 1 --2- 172.21.15.107:0/181226069 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f49500387b0 0x7f495003ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f494c006fd0 tx=0x7f494c006e40 comp rx=0 tx=0).stop 2023-07-21T22:22:16.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:16.161+0000 7f4966e50700 1 -- 172.21.15.107:0/181226069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960100820 msgr2=0x7f496019a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:16.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:16.161+0000 7f4966e50700 1 --2- 172.21.15.107:0/181226069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960100820 0x7f496019a960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4954005850 tx=0x7f4954004ff0 comp rx=0 tx=0).stop 2023-07-21T22:22:16.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:16.161+0000 7f4966e50700 1 -- 172.21.15.107:0/181226069 shutdown_connections 2023-07-21T22:22:16.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:16.161+0000 7f4966e50700 1 --2- 172.21.15.107:0/181226069 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f49500387b0 0x7f495003ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:16.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:16.161+0000 7f4966e50700 1 --2- 172.21.15.107:0/181226069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4960100820 0x7f496019a960 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:16.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:16.161+0000 7f4966e50700 1 -- 172.21.15.107:0/181226069 >> 172.21.15.107:0/181226069 conn(0x7f49600fa020 msgr2=0x7f4960103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:16.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:16.161+0000 7f4966e50700 1 -- 172.21.15.107:0/181226069 shutdown_connections 2023-07-21T22:22:16.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:16.161+0000 7f4966e50700 1 -- 172.21.15.107:0/181226069 wait complete. 2023-07-21T22:22:16.208 INFO:teuthology.orchestra.run.smithi107.stderr:adjusted tunables profile to default 2023-07-21T22:22:16.557 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:16.557 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:16.557 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:16.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:16.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:22:16.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:16.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:16.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:16 smithi107 ceph-mon[96543]: Updating smithi190:/etc/ceph/ceph.conf 2023-07-21T22:22:16.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:16 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/181226069' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T22:22:16.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:16.559 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:16.559 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:16.560 INFO:tasks.cephadm:Adding mon.a on smithi107 2023-07-21T22:22:16.560 INFO:tasks.cephadm:Adding mon.b on smithi132 2023-07-21T22:22:16.560 INFO:tasks.cephadm:Adding mon.c on smithi190 2023-07-21T22:22:16.561 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch apply mon '3;smithi107:172.21.15.107=a;smithi132:172.21.15.132=b;smithi190:172.21.15.190=c' 2023-07-21T22:22:17.137 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:17.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:17 smithi107 ceph-mon[96543]: Updating smithi190:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:17.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:17 smithi107 ceph-mon[96543]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:22:17.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:17 smithi107 ceph-mon[96543]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:17.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:17 smithi107 ceph-mon[96543]: Updating smithi190:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:22:17.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:17 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/181226069' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T22:22:17.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:17 smithi107 ceph-mon[96543]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:22:18.382 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.335+0000 7f5730264700 1 -- 172.21.15.190:0/3111346128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57280fe450 msgr2=0x7f57280fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:18.382 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.335+0000 7f5730264700 1 --2- 172.21.15.190:0/3111346128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57280fe450 0x7f57280fe820 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5724009910 tx=0x7f5724009c20 comp rx=0 tx=0).stop 2023-07-21T22:22:18.383 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.335+0000 7f5730264700 1 -- 172.21.15.190:0/3111346128 shutdown_connections 2023-07-21T22:22:18.383 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.335+0000 7f5730264700 1 --2- 172.21.15.190:0/3111346128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57280fe450 0x7f57280fe820 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:18.383 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.335+0000 7f5730264700 1 -- 172.21.15.190:0/3111346128 >> 172.21.15.190:0/3111346128 conn(0x7f5728074f40 msgr2=0x7f5728075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:18.383 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.336+0000 7f5730264700 1 -- 172.21.15.190:0/3111346128 shutdown_connections 2023-07-21T22:22:18.383 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.336+0000 7f5730264700 1 -- 172.21.15.190:0/3111346128 wait complete. 2023-07-21T22:22:18.383 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.336+0000 7f5730264700 1 Processor -- start 2023-07-21T22:22:18.383 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.336+0000 7f5730264700 1 -- start start 2023-07-21T22:22:18.384 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.337+0000 7f5730264700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57280fe450 0x7f57281997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:18.384 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.337+0000 7f5730264700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5728199d30 con 0x7f57280fe450 2023-07-21T22:22:18.385 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.337+0000 7f572e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57280fe450 0x7f57281997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:18.385 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.337+0000 7f572e000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57280fe450 0x7f57281997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.190:57282/0 (socket says 172.21.15.190:57282) 2023-07-21T22:22:18.385 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.337+0000 7f572e000700 1 -- 172.21.15.190:0/945848922 learned_addr learned my addr 172.21.15.190:0/945848922 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-07-21T22:22:18.385 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.337+0000 7f572e000700 1 -- 172.21.15.190:0/945848922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57240095f0 con 0x7f57280fe450 2023-07-21T22:22:18.386 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.338+0000 7f572e000700 1 --2- 172.21.15.190:0/945848922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57280fe450 0x7f57281997f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5724004f40 tx=0x7f5724005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:18.386 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.338+0000 7f571effd700 1 -- 172.21.15.190:0/945848922 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f572401d070 con 0x7f57280fe450 2023-07-21T22:22:18.386 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.338+0000 7f5730264700 1 -- 172.21.15.190:0/945848922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5728199f30 con 0x7f57280fe450 2023-07-21T22:22:18.386 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.338+0000 7f571effd700 1 -- 172.21.15.190:0/945848922 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5724004370 con 0x7f57280fe450 2023-07-21T22:22:18.386 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.338+0000 7f571effd700 1 -- 172.21.15.190:0/945848922 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57240039b0 con 0x7f57280fe450 2023-07-21T22:22:18.386 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.338+0000 7f5730264700 1 -- 172.21.15.190:0/945848922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57281937e0 con 0x7f57280fe450 2023-07-21T22:22:18.386 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.339+0000 7f571effd700 1 -- 172.21.15.190:0/945848922 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f5724003c70 con 0x7f57280fe450 2023-07-21T22:22:18.386 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.339+0000 7f5730264700 1 -- 172.21.15.190:0/945848922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572819a350 con 0x7f57280fe450 2023-07-21T22:22:18.387 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.339+0000 7f571effd700 1 --2- 172.21.15.190:0/945848922 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57140387b0 0x7f571403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:18.387 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.339+0000 7f571effd700 1 -- 172.21.15.190:0/945848922 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f572404c8a0 con 0x7f57280fe450 2023-07-21T22:22:18.387 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.339+0000 7f572d7ff700 1 --2- 172.21.15.190:0/945848922 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57140387b0 0x7f571403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:18.387 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.340+0000 7f572d7ff700 1 --2- 172.21.15.190:0/945848922 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57140387b0 0x7f571403ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5718006fd0 tx=0x7f5718006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:18.389 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.344+0000 7f571effd700 1 -- 172.21.15.190:0/945848922 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5724029360 con 0x7f57280fe450 2023-07-21T22:22:18.579 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.533+0000 7f5730264700 1 -- 172.21.15.190:0/945848922 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi132:172.21.15.132=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f5728101f90 con 0x7f57140387b0 2023-07-21T22:22:18.589 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.543+0000 7f571effd700 1 -- 172.21.15.190:0/945848922 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5728101f90 con 0x7f57140387b0 2023-07-21T22:22:18.589 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mon update... 2023-07-21T22:22:18.591 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.545+0000 7f5730264700 1 -- 172.21.15.190:0/945848922 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57140387b0 msgr2=0x7f571403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:18.591 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.545+0000 7f5730264700 1 --2- 172.21.15.190:0/945848922 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57140387b0 0x7f571403ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5718006fd0 tx=0x7f5718006e40 comp rx=0 tx=0).stop 2023-07-21T22:22:18.591 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.545+0000 7f5730264700 1 -- 172.21.15.190:0/945848922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57280fe450 msgr2=0x7f57281997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:18.592 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.545+0000 7f5730264700 1 --2- 172.21.15.190:0/945848922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57280fe450 0x7f57281997f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5724004f40 tx=0x7f5724005020 comp rx=0 tx=0).stop 2023-07-21T22:22:18.592 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.545+0000 7f5730264700 1 -- 172.21.15.190:0/945848922 shutdown_connections 2023-07-21T22:22:18.592 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.545+0000 7f5730264700 1 --2- 172.21.15.190:0/945848922 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57140387b0 0x7f571403ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:18.592 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.545+0000 7f5730264700 1 --2- 172.21.15.190:0/945848922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57280fe450 0x7f57281997f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:18.593 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.545+0000 7f5730264700 1 -- 172.21.15.190:0/945848922 >> 172.21.15.190:0/945848922 conn(0x7f5728074f40 msgr2=0x7f5728101880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:18.593 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.545+0000 7f5730264700 1 -- 172.21.15.190:0/945848922 shutdown_connections 2023-07-21T22:22:18.593 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:18.545+0000 7f5730264700 1 -- 172.21.15.190:0/945848922 wait complete. 2023-07-21T22:22:19.007 DEBUG:teuthology.orchestra.run.smithi132:mon.b> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.b.service 2023-07-21T22:22:19.010 DEBUG:teuthology.orchestra.run.smithi190:mon.c> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.c.service 2023-07-21T22:22:19.013 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T22:22:19.014 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T22:22:19.043 INFO:journalctl@ceph.mon.c.smithi190.stdout:-- Logs begin at Fri 2023-07-21 22:03:29 UTC. -- 2023-07-21T22:22:19.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:-- Logs begin at Fri 2023-07-21 22:03:19 UTC. -- 2023-07-21T22:22:19.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:19 smithi107 ceph-mon[96543]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:19.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:19.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:19.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:19.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:19.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:19.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:19.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:19.802 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:20.686 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:20 smithi107 ceph-mon[96543]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi132:172.21.15.132=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:20.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:20 smithi107 ceph-mon[96543]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi132:172.21.15.132=b;smithi190:172.21.15.190=c;count:3 2023-07-21T22:22:20.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:20 smithi107 ceph-mon[96543]: Deploying daemon mon.c on smithi190 2023-07-21T22:22:21.481 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.435+0000 7fcb7b36a700 1 -- 172.21.15.190:0/1166511325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb74104560 msgr2=0x7fcb74104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:21.481 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.435+0000 7fcb7b36a700 1 --2- 172.21.15.190:0/1166511325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb74104560 0x7fcb74104930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009910 tx=0x7fcb70009c20 comp rx=0 tx=0).stop 2023-07-21T22:22:21.482 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.435+0000 7fcb7b36a700 1 -- 172.21.15.190:0/1166511325 shutdown_connections 2023-07-21T22:22:21.482 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.435+0000 7fcb7b36a700 1 --2- 172.21.15.190:0/1166511325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb74104560 0x7fcb74104930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:21.482 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.435+0000 7fcb7b36a700 1 -- 172.21.15.190:0/1166511325 >> 172.21.15.190:0/1166511325 conn(0x7fcb740742b0 msgr2=0x7fcb740746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:21.482 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.435+0000 7fcb7b36a700 1 -- 172.21.15.190:0/1166511325 shutdown_connections 2023-07-21T22:22:21.482 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.435+0000 7fcb7b36a700 1 -- 172.21.15.190:0/1166511325 wait complete. 2023-07-21T22:22:21.483 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.436+0000 7fcb7b36a700 1 Processor -- start 2023-07-21T22:22:21.483 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.436+0000 7fcb7b36a700 1 -- start start 2023-07-21T22:22:21.484 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.436+0000 7fcb7b36a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb74104560 0x7fcb74195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:21.484 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.436+0000 7fcb7b36a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb74195cc0 con 0x7fcb74104560 2023-07-21T22:22:21.485 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.436+0000 7fcb79106700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb74104560 0x7fcb74195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:21.485 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.436+0000 7fcb79106700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb74104560 0x7fcb74195780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.190:57288/0 (socket says 172.21.15.190:57288) 2023-07-21T22:22:21.485 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.436+0000 7fcb79106700 1 -- 172.21.15.190:0/652758046 learned_addr learned my addr 172.21.15.190:0/652758046 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-07-21T22:22:21.485 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.436+0000 7fcb79106700 1 -- 172.21.15.190:0/652758046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb700095f0 con 0x7fcb74104560 2023-07-21T22:22:21.485 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.437+0000 7fcb79106700 1 --2- 172.21.15.190:0/652758046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb74104560 0x7fcb74195780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb70004f40 tx=0x7fcb70005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:21.485 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.437+0000 7fcb6a7fc700 1 -- 172.21.15.190:0/652758046 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb7001d070 con 0x7fcb74104560 2023-07-21T22:22:21.485 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.437+0000 7fcb7b36a700 1 -- 172.21.15.190:0/652758046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb74195f20 con 0x7fcb74104560 2023-07-21T22:22:21.486 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.437+0000 7fcb6a7fc700 1 -- 172.21.15.190:0/652758046 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb70004370 con 0x7fcb74104560 2023-07-21T22:22:21.486 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.437+0000 7fcb6a7fc700 1 -- 172.21.15.190:0/652758046 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb700039b0 con 0x7fcb74104560 2023-07-21T22:22:21.486 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.437+0000 7fcb7b36a700 1 -- 172.21.15.190:0/652758046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb741963a0 con 0x7fcb74104560 2023-07-21T22:22:21.486 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.438+0000 7fcb7b36a700 1 -- 172.21.15.190:0/652758046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb741965f0 con 0x7fcb74104560 2023-07-21T22:22:21.486 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.438+0000 7fcb6a7fc700 1 -- 172.21.15.190:0/652758046 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fcb70003c70 con 0x7fcb74104560 2023-07-21T22:22:21.487 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.438+0000 7fcb6a7fc700 1 --2- 172.21.15.190:0/652758046 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcb600387b0 0x7fcb6003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:21.487 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.438+0000 7fcb6a7fc700 1 -- 172.21.15.190:0/652758046 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcb7004c580 con 0x7fcb74104560 2023-07-21T22:22:21.487 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.438+0000 7fcb78905700 1 --2- 172.21.15.190:0/652758046 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcb600387b0 0x7fcb6003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:21.487 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.440+0000 7fcb78905700 1 --2- 172.21.15.190:0/652758046 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcb600387b0 0x7fcb6003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb64006fd0 tx=0x7fcb64006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:21.488 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.443+0000 7fcb6a7fc700 1 -- 172.21.15.190:0/652758046 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcb741965f0 con 0x7fcb74104560 2023-07-21T22:22:21.639 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:21 smithi190 systemd[1]: Starting Ceph mon.c for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:22:21.686 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:21 smithi107 ceph-mon[96543]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:21.826 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.779+0000 7fcb7b36a700 1 -- 172.21.15.190:0/652758046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcb74103800 con 0x7fcb74104560 2023-07-21T22:22:21.826 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.780+0000 7fcb6a7fc700 1 -- 172.21.15.190:0/652758046 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7fcb70026030 con 0x7fcb74104560 2023-07-21T22:22:21.827 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:22:21.827 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":1,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","modified":"2023-07-21T22:20:54.562258Z","created":"2023-07-21T22:20:54.562258Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-21T22:22:21.829 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.782+0000 7fcb7b36a700 1 -- 172.21.15.190:0/652758046 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcb600387b0 msgr2=0x7fcb6003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:21.829 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.782+0000 7fcb7b36a700 1 --2- 172.21.15.190:0/652758046 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcb600387b0 0x7fcb6003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb64006fd0 tx=0x7fcb64006e40 comp rx=0 tx=0).stop 2023-07-21T22:22:21.829 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.782+0000 7fcb7b36a700 1 -- 172.21.15.190:0/652758046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb74104560 msgr2=0x7fcb74195780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:21.830 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.782+0000 7fcb7b36a700 1 --2- 172.21.15.190:0/652758046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb74104560 0x7fcb74195780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb70004f40 tx=0x7fcb70005020 comp rx=0 tx=0).stop 2023-07-21T22:22:21.830 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.782+0000 7fcb7b36a700 1 -- 172.21.15.190:0/652758046 shutdown_connections 2023-07-21T22:22:21.830 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.782+0000 7fcb7b36a700 1 --2- 172.21.15.190:0/652758046 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcb600387b0 0x7fcb6003ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:21.830 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.782+0000 7fcb7b36a700 1 --2- 172.21.15.190:0/652758046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb74104560 0x7fcb74195780 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:21.831 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.782+0000 7fcb7b36a700 1 -- 172.21.15.190:0/652758046 >> 172.21.15.190:0/652758046 conn(0x7fcb740742b0 msgr2=0x7fcb74105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:21.831 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.782+0000 7fcb7b36a700 1 -- 172.21.15.190:0/652758046 shutdown_connections 2023-07-21T22:22:21.831 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:21.782+0000 7fcb7b36a700 1 -- 172.21.15.190:0/652758046 wait complete. 2023-07-21T22:22:21.831 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 1 2023-07-21T22:22:22.111 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:21 smithi190 podman[98273]: 2023-07-21T22:22:22.378 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T22:22:22.378 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev), process ceph-mon, pid 7 2023-07-21T22:22:22.378 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: pidfile_write: ignore empty --pid-file 2023-07-21T22:22:22.378 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 bash[98273]: 799a149ed230cbc43f7492ffd2c4370260ef396f8af8bac2cd16d56c92ea7ed4 2023-07-21T22:22:22.379 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: load: jerasure load: lrc 2023-07-21T22:22:22.379 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: RocksDB version: 7.9.2 2023-07-21T22:22:22.379 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Git sha 0 2023-07-21T22:22:22.379 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Compile date 2023-07-21 19:07:44 2023-07-21T22:22:22.379 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: DB SUMMARY 2023-07-21T22:22:22.379 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: DB Session ID: Y8FTLAZN3XWRZVKIE7ZV 2023-07-21T22:22:22.379 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: CURRENT file: CURRENT 2023-07-21T22:22:22.379 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T22:22:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-21T22:22:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-07-21T22:22:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-07-21T22:22:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.error_if_exists: 0 2023-07-21T22:22:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.create_if_missing: 0 2023-07-21T22:22:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.paranoid_checks: 1 2023-07-21T22:22:22.380 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T22:22:22.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T22:22:22.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T22:22:22.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.env: 0x5592d452d720 2023-07-21T22:22:22.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T22:22:22.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.info_log: 0x5592d66c1260 2023-07-21T22:22:22.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T22:22:22.381 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.statistics: (nil) 2023-07-21T22:22:22.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.use_fsync: 0 2023-07-21T22:22:22.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_log_file_size: 0 2023-07-21T22:22:22.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T22:22:22.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T22:22:22.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T22:22:22.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T22:22:22.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.allow_fallocate: 1 2023-07-21T22:22:22.382 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T22:22:22.383 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T22:22:22.383 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.use_direct_reads: 0 2023-07-21T22:22:22.383 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T22:22:22.383 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T22:22:22.383 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.db_log_dir: 2023-07-21T22:22:22.383 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.wal_dir: 2023-07-21T22:22:22.385 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T22:22:22.385 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T22:22:22.385 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T22:22:22.385 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T22:22:22.386 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T22:22:22.386 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T22:22:22.386 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T22:22:22.386 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T22:22:22.386 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.write_buffer_manager: 0x5592d65e23c0 2023-07-21T22:22:22.386 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T22:22:22.387 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T22:22:22.387 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T22:22:22.387 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T22:22:22.387 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T22:22:22.388 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T22:22:22.388 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T22:22:22.388 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T22:22:22.388 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.unordered_write: 0 2023-07-21T22:22:22.388 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T22:22:22.388 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T22:22:22.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T22:22:22.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T22:22:22.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.row_cache: None 2023-07-21T22:22:22.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.wal_filter: None 2023-07-21T22:22:22.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T22:22:22.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T22:22:22.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.two_write_queues: 0 2023-07-21T22:22:22.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T22:22:22.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.wal_compression: 0 2023-07-21T22:22:22.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.atomic_flush: 0 2023-07-21T22:22:22.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T22:22:22.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T22:22:22.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T22:22:22.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.log_readahead_size: 0 2023-07-21T22:22:22.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T22:22:22.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T22:22:22.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T22:22:22.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T22:22:22.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T22:22:22.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T22:22:22.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T22:22:22.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_background_jobs: 2 2023-07-21T22:22:22.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_background_compactions: -1 2023-07-21T22:22:22.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_subcompactions: 1 2023-07-21T22:22:22.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T22:22:22.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T22:22:22.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T22:22:22.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T22:22:22.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T22:22:22.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T22:22:22.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T22:22:22.393 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T22:22:22.393 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_open_files: -1 2023-07-21T22:22:22.393 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T22:22:22.393 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T22:22:22.393 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T22:22:22.393 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T22:22:22.394 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_background_flushes: -1 2023-07-21T22:22:22.394 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Compression algorithms supported: 2023-07-21T22:22:22.394 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T22:22:22.394 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: kZSTD supported: 0 2023-07-21T22:22:22.394 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: kXpressCompression supported: 0 2023-07-21T22:22:22.394 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T22:22:22.394 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: kZlibCompression supported: 1 2023-07-21T22:22:22.395 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: kSnappyCompression supported: 1 2023-07-21T22:22:22.395 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: kLZ4Compression supported: 1 2023-07-21T22:22:22.395 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: kBZip2Compression supported: 0 2023-07-21T22:22:22.395 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T22:22:22.395 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T22:22:22.395 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-07-21T22:22:22.395 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T22:22:22.395 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T22:22:22.396 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.merge_operator: 2023-07-21T22:22:22.396 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compaction_filter: None 2023-07-21T22:22:22.396 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T22:22:22.396 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T22:22:22.396 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T22:22:22.396 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T22:22:22.396 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5592d66c1360) 2023-07-21T22:22:22.397 INFO:journalctl@ceph.mon.c.smithi190.stdout: cache_index_and_filter_blocks: 1 2023-07-21T22:22:22.397 INFO:journalctl@ceph.mon.c.smithi190.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T22:22:22.397 INFO:journalctl@ceph.mon.c.smithi190.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T22:22:22.397 INFO:journalctl@ceph.mon.c.smithi190.stdout: pin_top_level_index_and_filter: 1 2023-07-21T22:22:22.397 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_type: 0 2023-07-21T22:22:22.398 INFO:journalctl@ceph.mon.c.smithi190.stdout: data_block_index_type: 0 2023-07-21T22:22:22.398 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_shortening: 1 2023-07-21T22:22:22.398 INFO:journalctl@ceph.mon.c.smithi190.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T22:22:22.398 INFO:journalctl@ceph.mon.c.smithi190.stdout: checksum: 4 2023-07-21T22:22:22.398 INFO:journalctl@ceph.mon.c.smithi190.stdout: no_block_cache: 0 2023-07-21T22:22:22.399 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache: 0x5592d66811f0 2023-07-21T22:22:22.399 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_name: BinnedLRUCache 2023-07-21T22:22:22.399 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_options: 2023-07-21T22:22:22.399 INFO:journalctl@ceph.mon.c.smithi190.stdout: capacity : 536870912 2023-07-21T22:22:22.399 INFO:journalctl@ceph.mon.c.smithi190.stdout: num_shard_bits : 4 2023-07-21T22:22:22.399 INFO:journalctl@ceph.mon.c.smithi190.stdout: strict_capacity_limit : 0 2023-07-21T22:22:22.399 INFO:journalctl@ceph.mon.c.smithi190.stdout: high_pri_pool_ratio: 0.000 2023-07-21T22:22:22.400 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_cache_compressed: (nil) 2023-07-21T22:22:22.400 INFO:journalctl@ceph.mon.c.smithi190.stdout: persistent_cache: (nil) 2023-07-21T22:22:22.400 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_size: 4096 2023-07-21T22:22:22.400 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_size_deviation: 10 2023-07-21T22:22:22.400 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_restart_interval: 16 2023-07-21T22:22:22.400 INFO:journalctl@ceph.mon.c.smithi190.stdout: index_block_restart_interval: 1 2023-07-21T22:22:22.400 INFO:journalctl@ceph.mon.c.smithi190.stdout: metadata_block_size: 4096 2023-07-21T22:22:22.401 INFO:journalctl@ceph.mon.c.smithi190.stdout: partition_filters: 0 2023-07-21T22:22:22.401 INFO:journalctl@ceph.mon.c.smithi190.stdout: use_delta_encoding: 1 2023-07-21T22:22:22.401 INFO:journalctl@ceph.mon.c.smithi190.stdout: filter_policy: bloomfilter 2023-07-21T22:22:22.401 INFO:journalctl@ceph.mon.c.smithi190.stdout: whole_key_filtering: 1 2023-07-21T22:22:22.401 INFO:journalctl@ceph.mon.c.smithi190.stdout: verify_compression: 0 2023-07-21T22:22:22.401 INFO:journalctl@ceph.mon.c.smithi190.stdout: read_amp_bytes_per_bit: 0 2023-07-21T22:22:22.401 INFO:journalctl@ceph.mon.c.smithi190.stdout: format_version: 5 2023-07-21T22:22:22.401 INFO:journalctl@ceph.mon.c.smithi190.stdout: enable_index_compression: 1 2023-07-21T22:22:22.402 INFO:journalctl@ceph.mon.c.smithi190.stdout: block_align: 0 2023-07-21T22:22:22.402 INFO:journalctl@ceph.mon.c.smithi190.stdout: max_auto_readahead_size: 262144 2023-07-21T22:22:22.402 INFO:journalctl@ceph.mon.c.smithi190.stdout: prepopulate_block_cache: 0 2023-07-21T22:22:22.402 INFO:journalctl@ceph.mon.c.smithi190.stdout: initial_auto_readahead_size: 8192 2023-07-21T22:22:22.402 INFO:journalctl@ceph.mon.c.smithi190.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T22:22:22.402 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T22:22:22.402 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T22:22:22.402 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compression: NoCompression 2023-07-21T22:22:22.403 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T22:22:22.403 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T22:22:22.403 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T22:22:22.403 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.num_levels: 7 2023-07-21T22:22:22.403 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T22:22:22.403 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T22:22:22.403 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T22:22:22.403 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T22:22:22.404 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T22:22:22.404 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T22:22:22.404 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T22:22:22.404 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:22:22.404 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T22:22:22.405 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T22:22:22.405 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:22:22.405 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:22:22.405 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T22:22:22.405 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T22:22:22.405 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T22:22:22.405 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T22:22:22.405 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:22:22.406 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:22:22.406 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T22:22:22.406 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T22:22:22.406 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:22:22.406 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T22:22:22.406 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T22:22:22.406 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T22:22:22.407 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T22:22:22.407 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T22:22:22.407 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T22:22:22.407 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T22:22:22.407 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T22:22:22.407 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T22:22:22.407 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T22:22:22.408 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T22:22:22.408 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T22:22:22.408 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T22:22:22.408 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T22:22:22.408 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T22:22:22.409 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T22:22:22.409 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T22:22:22.409 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T22:22:22.409 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T22:22:22.410 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T22:22:22.410 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T22:22:22.410 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T22:22:22.410 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T22:22:22.410 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T22:22:22.410 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T22:22:22.410 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T22:22:22.411 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T22:22:22.411 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T22:22:22.411 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T22:22:22.411 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T22:22:22.411 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T22:22:22.411 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T22:22:22.411 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.table_properties_collectors: 2023-07-21T22:22:22.412 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.inplace_update_support: 0 2023-07-21T22:22:22.412 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T22:22:22.412 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T22:22:22.412 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T22:22:22.412 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T22:22:22.412 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.bloom_locality: 0 2023-07-21T22:22:22.412 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.max_successive_merges: 0 2023-07-21T22:22:22.413 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T22:22:22.413 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T22:22:22.413 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T22:22:22.413 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T22:22:22.413 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.ttl: 2592000 2023-07-21T22:22:22.414 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T22:22:22.414 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T22:22:22.414 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T22:22:22.414 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.enable_blob_files: false 2023-07-21T22:22:22.414 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.min_blob_size: 0 2023-07-21T22:22:22.414 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T22:22:22.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T22:22:22.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T22:22:22.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T22:22:22.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T22:22:22.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T22:22:22.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T22:22:22.415 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T22:22:22.416 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-21T22:22:22.416 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-21T22:22:22.416 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 38844d9c-2d64-4f76-8a6b-60a283658ccf 2023-07-21T22:22:22.416 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689978142277316, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-21T22:22:22.416 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-21T22:22:22.416 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689978142277874, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689978142, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "38844d9c-2d64-4f76-8a6b-60a283658ccf", "db_session_id": "Y8FTLAZN3XWRZVKIE7ZV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-21T22:22:22.416 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689978142277963, "job": 1, "event": "recovery_finished"} 2023-07-21T22:22:22.417 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-21T22:22:22.417 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T22:22:22.417 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5592d671a000 2023-07-21T22:22:22.417 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: DB pointer 0x5592d6706000 2023-07-21T22:22:22.417 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T22:22:22.417 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T22:22:22.417 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** DB Stats ** 2023-07-21T22:22:22.417 INFO:journalctl@ceph.mon.c.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:22:22.418 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T22:22:22.418 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:22:22.418 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:22:22.418 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T22:22:22.418 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:22:22.418 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:22:22.418 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2023-07-21T22:22:22.419 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** Compaction Stats [default] ** 2023-07-21T22:22:22.419 INFO:journalctl@ceph.mon.c.smithi190.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:22:22.419 INFO:journalctl@ceph.mon.c.smithi190.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T22:22:22.419 INFO:journalctl@ceph.mon.c.smithi190.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:22:22.419 INFO:journalctl@ceph.mon.c.smithi190.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:22:22.419 INFO:journalctl@ceph.mon.c.smithi190.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:22:22.420 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2023-07-21T22:22:22.420 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** Compaction Stats [default] ** 2023-07-21T22:22:22.420 INFO:journalctl@ceph.mon.c.smithi190.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:22:22.420 INFO:journalctl@ceph.mon.c.smithi190.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T22:22:22.420 INFO:journalctl@ceph.mon.c.smithi190.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:22:22.421 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2023-07-21T22:22:22.421 INFO:journalctl@ceph.mon.c.smithi190.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T22:22:22.421 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2023-07-21T22:22:22.421 INFO:journalctl@ceph.mon.c.smithi190.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:22:22.421 INFO:journalctl@ceph.mon.c.smithi190.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T22:22:22.421 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T22:22:22.421 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T22:22:22.422 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T22:22:22.422 INFO:journalctl@ceph.mon.c.smithi190.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T22:22:22.422 INFO:journalctl@ceph.mon.c.smithi190.stdout: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:22:22.422 INFO:journalctl@ceph.mon.c.smithi190.stdout: Interval compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:22:22.422 INFO:journalctl@ceph.mon.c.smithi190.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T22:22:22.422 INFO:journalctl@ceph.mon.c.smithi190.stdout: Block cache BinnedLRUCache@0x5592d66811f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-07-21T22:22:22.422 INFO:journalctl@ceph.mon.c.smithi190.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-21T22:22:22.422 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2023-07-21T22:22:22.423 INFO:journalctl@ceph.mon.c.smithi190.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T22:22:22.423 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-07-21T22:22:22.423 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: using public_addr v2:172.21.15.190:0/0 -> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2023-07-21T22:22:22.423 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 systemd[1]: Started Ceph mon.c for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:22:22.423 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: starting mon.c rank -1 at public addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] at bind addrs [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:22:22.423 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(???) e0 preinit fsid af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:22:22.423 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing).mds e1 new map 2023-07-21T22:22:22.424 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing).mds e1 print_map 2023-07-21T22:22:22.424 INFO:journalctl@ceph.mon.c.smithi190.stdout: e1 2023-07-21T22:22:22.424 INFO:journalctl@ceph.mon.c.smithi190.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T22:22:22.424 INFO:journalctl@ceph.mon.c.smithi190.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:22:22.424 INFO:journalctl@ceph.mon.c.smithi190.stdout: legacy client fscid: -1 2023-07-21T22:22:22.424 INFO:journalctl@ceph.mon.c.smithi190.stdout: 2023-07-21T22:22:22.424 INFO:journalctl@ceph.mon.c.smithi190.stdout: No filesystems configured 2023-07-21T22:22:22.424 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-21T22:22:22.425 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-21T22:22:22.425 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-21T22:22:22.425 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-21T22:22:22.425 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-21T22:22:22.425 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-21T22:22:22.426 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T22:22:22.426 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:22:22.426 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:22:22.426 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:22:22.426 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:22.426 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: Updating smithi132:/etc/ceph/ceph.conf 2023-07-21T22:22:22.426 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: Updating smithi132:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:22.426 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:22:22.427 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: Deploying cephadm binary to smithi190 2023-07-21T22:22:22.427 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: Updating smithi132:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:22:22.427 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.427 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.427 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.427 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:22.427 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.427 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:22.428 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: Added host smithi190 2023-07-21T22:22:22.428 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.428 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.428 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:22.428 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.428 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:22:22.428 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:22.429 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.429 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.429 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.429 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.429 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:22:22.429 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:22.430 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:22.430 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: Updating smithi190:/etc/ceph/ceph.conf 2023-07-21T22:22:22.430 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/181226069' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T22:22:22.430 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.430 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.430 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.430 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: Updating smithi190:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:22.431 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:22:22.431 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:22.431 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: Updating smithi190:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:22:22.431 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/181226069' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T22:22:22.431 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:22:22.432 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:22.432 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.432 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:22.432 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:22.432 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:22.432 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:22.432 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:22.433 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:22.433 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi132:172.21.15.132=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:22.433 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi132:172.21.15.132=b;smithi190:172.21.15.190=c;count:3 2023-07-21T22:22:22.433 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: Deploying daemon mon.c on smithi190 2023-07-21T22:22:22.433 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:22.433 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: from='client.? 172.21.15.190:0/652758046' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:22:22.433 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-21T22:22:22.433 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-21T22:22:22.434 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: expand_channel_meta expand map: {default=false} 2023-07-21T22:22:22.434 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: expand_channel_meta from 'false' to 'false' 2023-07-21T22:22:22.434 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: expand_channel_meta expanded map: {default=false} 2023-07-21T22:22:22.434 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: expand_channel_meta expand map: {default=info} 2023-07-21T22:22:22.434 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: expand_channel_meta from 'info' to 'info' 2023-07-21T22:22:22.434 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: expand_channel_meta expanded map: {default=info} 2023-07-21T22:22:22.434 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: expand_channel_meta expand map: {default=daemon} 2023-07-21T22:22:22.434 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-21T22:22:22.435 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: expand_channel_meta expanded map: {default=daemon} 2023-07-21T22:22:22.435 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: expand_channel_meta expand map: {default=debug} 2023-07-21T22:22:22.435 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: expand_channel_meta from 'debug' to 'debug' 2023-07-21T22:22:22.435 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:22 smithi190 ceph-mon[98334]: expand_channel_meta expanded map: {default=debug} 2023-07-21T22:22:22.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:22 smithi107 ceph-mon[96543]: from='client.? 172.21.15.190:0/652758046' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:22:23.436 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T22:22:23.437 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T22:22:24.002 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:22:27.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:22:27.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:27.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: mon.a calling monitor election 2023-07-21T22:22:27.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:27.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:27.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:27.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: mon.c calling monitor election 2023-07-21T22:22:27.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:27.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:27.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:27.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:27.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:22:27.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: monmap e2: 2 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-07-21T22:22:27.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: fsmap 2023-07-21T22:22:27.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:22:27.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: mgrmap e13: x(active, since 41s) 2023-07-21T22:22:27.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: overall HEALTH_OK 2023-07-21T22:22:27.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:27.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:27.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:22:27.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:27.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: mon.a calling monitor election 2023-07-21T22:22:27.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:27.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:27.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:27.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: mon.c calling monitor election 2023-07-21T22:22:27.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:27.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:27.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:27.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:27.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:22:27.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: monmap e2: 2 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-07-21T22:22:27.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: fsmap 2023-07-21T22:22:27.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:22:27.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: mgrmap e13: x(active, since 41s) 2023-07-21T22:22:27.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: overall HEALTH_OK 2023-07-21T22:22:27.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:27.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:28.205 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.159+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3902815486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc540035c0 msgr2=0x7fcc54005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:28.205 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.159+0000 7fcc6aa77700 1 --2- 172.21.15.190:0/3902815486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc540035c0 0x7fcc54005a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcc640ffb40 tx=0x7fcc5800fe60 comp rx=0 tx=0).stop 2023-07-21T22:22:28.206 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.159+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3902815486 shutdown_connections 2023-07-21T22:22:28.206 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.159+0000 7fcc6aa77700 1 --2- 172.21.15.190:0/3902815486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc540035c0 0x7fcc54005a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:28.206 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.159+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3902815486 >> 172.21.15.190:0/3902815486 conn(0x7fcc640743d0 msgr2=0x7fcc640747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:28.206 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.159+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3902815486 shutdown_connections 2023-07-21T22:22:28.207 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.159+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3902815486 wait complete. 2023-07-21T22:22:28.210 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.160+0000 7fcc6aa77700 1 Processor -- start 2023-07-21T22:22:28.210 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.160+0000 7fcc6aa77700 1 -- start start 2023-07-21T22:22:28.210 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.160+0000 7fcc6aa77700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc540035c0 0x7fcc6419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:28.210 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.160+0000 7fcc6aa77700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc6419aef0 0x7fcc64194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:28.211 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.160+0000 7fcc6aa77700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6419b420 con 0x7fcc540035c0 2023-07-21T22:22:28.211 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.160+0000 7fcc6aa77700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6419b590 con 0x7fcc6419aef0 2023-07-21T22:22:28.211 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.160+0000 7fcc63fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc6419aef0 0x7fcc64194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:28.211 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.160+0000 7fcc63fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc6419aef0 0x7fcc64194a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:47896/0 (socket says 172.21.15.190:47896) 2023-07-21T22:22:28.212 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.160+0000 7fcc63fff700 1 -- 172.21.15.190:0/3124882166 learned_addr learned my addr 172.21.15.190:0/3124882166 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-07-21T22:22:28.212 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.160+0000 7fcc68813700 1 --2- 172.21.15.190:0/3124882166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc540035c0 0x7fcc6419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:28.212 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.160+0000 7fcc63fff700 1 -- 172.21.15.190:0/3124882166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc6419aef0 msgr2=0x7fcc64194a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-21T22:22:28.213 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.161+0000 7fcc63fff700 1 -- 172.21.15.190:0/3124882166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc6419aef0 msgr2=0x7fcc64194a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-21T22:22:28.213 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.161+0000 7fcc63fff700 1 --2- 172.21.15.190:0/3124882166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc6419aef0 0x7fcc64194a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-21T22:22:28.213 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.161+0000 7fcc63fff700 1 --2- 172.21.15.190:0/3124882166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc6419aef0 0x7fcc64194a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:22:28.213 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.161+0000 7fcc68813700 1 -- 172.21.15.190:0/3124882166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc6419aef0 msgr2=0x7fcc64194a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:22:28.214 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.161+0000 7fcc68813700 1 --2- 172.21.15.190:0/3124882166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc6419aef0 0x7fcc64194a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:28.214 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.161+0000 7fcc68813700 1 -- 172.21.15.190:0/3124882166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5800f8b0 con 0x7fcc540035c0 2023-07-21T22:22:28.214 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.161+0000 7fcc68813700 1 --2- 172.21.15.190:0/3124882166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc540035c0 0x7fcc6419a9b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc5801b000 tx=0x7fcc58013c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:28.214 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.161+0000 7fcc61ffb700 1 -- 172.21.15.190:0/3124882166 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fcc58023070 con 0x7fcc540035c0 2023-07-21T22:22:28.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.161+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3124882166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc64195030 con 0x7fcc540035c0 2023-07-21T22:22:28.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.161+0000 7fcc61ffb700 1 -- 172.21.15.190:0/3124882166 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc58004bd0 con 0x7fcc540035c0 2023-07-21T22:22:28.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.161+0000 7fcc61ffb700 1 -- 172.21.15.190:0/3124882166 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fcc58003d60 con 0x7fcc540035c0 2023-07-21T22:22:28.215 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.162+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3124882166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc64195600 con 0x7fcc540035c0 2023-07-21T22:22:28.216 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.163+0000 7fcc61ffb700 1 -- 172.21.15.190:0/3124882166 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fcc58004d40 con 0x7fcc540035c0 2023-07-21T22:22:28.216 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.163+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3124882166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc64061b40 con 0x7fcc540035c0 2023-07-21T22:22:28.216 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.163+0000 7fcc61ffb700 1 --2- 172.21.15.190:0/3124882166 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc480385f0 0x7fcc4803aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:28.217 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.163+0000 7fcc61ffb700 1 -- 172.21.15.190:0/3124882166 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcc5804d5e0 con 0x7fcc540035c0 2023-07-21T22:22:28.217 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.163+0000 7fcc63fff700 1 --2- 172.21.15.190:0/3124882166 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc480385f0 0x7fcc4803aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:28.217 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.164+0000 7fcc63fff700 1 --2- 172.21.15.190:0/3124882166 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc480385f0 0x7fcc4803aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009180 tx=0x7fcc50008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:28.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.168+0000 7fcc61ffb700 1 -- 172.21.15.190:0/3124882166 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcc58030580 con 0x7fcc540035c0 2023-07-21T22:22:28.483 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.437+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3124882166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcc6419f7c0 con 0x7fcc540035c0 2023-07-21T22:22:28.484 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.438+0000 7fcc61ffb700 1 -- 172.21.15.190:0/3124882166 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7fcc580303a0 con 0x7fcc540035c0 2023-07-21T22:22:28.484 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:22:28.484 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":2,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","modified":"2023-07-21T22:22:22.309719Z","created":"2023-07-21T22:20:54.562258Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-21T22:22:28.487 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.440+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3124882166 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc480385f0 msgr2=0x7fcc4803aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:28.487 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.440+0000 7fcc6aa77700 1 --2- 172.21.15.190:0/3124882166 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc480385f0 0x7fcc4803aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009180 tx=0x7fcc50008040 comp rx=0 tx=0).stop 2023-07-21T22:22:28.488 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.440+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3124882166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc540035c0 msgr2=0x7fcc6419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:28.488 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.440+0000 7fcc6aa77700 1 --2- 172.21.15.190:0/3124882166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc540035c0 0x7fcc6419a9b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcc5801b000 tx=0x7fcc58013c90 comp rx=0 tx=0).stop 2023-07-21T22:22:28.488 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.440+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3124882166 shutdown_connections 2023-07-21T22:22:28.488 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.440+0000 7fcc6aa77700 1 --2- 172.21.15.190:0/3124882166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc540035c0 0x7fcc6419a9b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:28.489 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.440+0000 7fcc6aa77700 1 --2- 172.21.15.190:0/3124882166 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc480385f0 0x7fcc4803aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:28.489 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.440+0000 7fcc6aa77700 1 --2- 172.21.15.190:0/3124882166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc6419aef0 0x7fcc64194a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:28.490 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.440+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3124882166 >> 172.21.15.190:0/3124882166 conn(0x7fcc640743d0 msgr2=0x7fcc64067c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:28.490 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.440+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3124882166 shutdown_connections 2023-07-21T22:22:28.490 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:28.440+0000 7fcc6aa77700 1 -- 172.21.15.190:0/3124882166 wait complete. 2023-07-21T22:22:28.490 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 2 2023-07-21T22:22:28.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:28 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:28.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:28 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:28.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:28 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:28.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:28 smithi190 ceph-mon[98334]: Deploying daemon mon.b on smithi132 2023-07-21T22:22:28.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:28 smithi190 ceph-mon[98334]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:28.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:28 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:28.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:28 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:28.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:28 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:28.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:28 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:28.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:28 smithi107 ceph-mon[96543]: Deploying daemon mon.b on smithi132 2023-07-21T22:22:28.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:28 smithi107 ceph-mon[96543]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:28.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:28 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:29.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:29 smithi107 ceph-mon[96543]: from='client.? 172.21.15.190:0/3124882166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:22:29.687 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:22:29 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:22:29.310+0000 7f932f150700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-21T22:22:29.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:29 smithi190 ceph-mon[98334]: from='client.? 172.21.15.190:0/3124882166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:22:29.954 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T22:22:29.955 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T22:22:30.180 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:29 smithi132 systemd[1]: Starting Ceph mon.b for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:22:30.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 podman[97823]: 2023-07-21T22:22:30.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T22:22:30.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev), process ceph-mon, pid 7 2023-07-21T22:22:30.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: pidfile_write: ignore empty --pid-file 2023-07-21T22:22:30.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: load: jerasure load: lrc 2023-07-21T22:22:30.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: RocksDB version: 7.9.2 2023-07-21T22:22:30.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Git sha 0 2023-07-21T22:22:30.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Compile date 2023-07-21 19:07:44 2023-07-21T22:22:30.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: DB SUMMARY 2023-07-21T22:22:30.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: DB Session ID: 1U6JE03K9VEGTV8HQ3JD 2023-07-21T22:22:30.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: CURRENT file: CURRENT 2023-07-21T22:22:30.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T22:22:30.435 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-21T22:22:30.435 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-21T22:22:30.435 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-21T22:22:30.435 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.error_if_exists: 0 2023-07-21T22:22:30.436 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.create_if_missing: 0 2023-07-21T22:22:30.436 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.paranoid_checks: 1 2023-07-21T22:22:30.436 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T22:22:30.436 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T22:22:30.436 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T22:22:30.437 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.env: 0x55d91993e720 2023-07-21T22:22:30.437 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T22:22:30.437 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.info_log: 0x55d91b453260 2023-07-21T22:22:30.437 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T22:22:30.437 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.statistics: (nil) 2023-07-21T22:22:30.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.use_fsync: 0 2023-07-21T22:22:30.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_log_file_size: 0 2023-07-21T22:22:30.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T22:22:30.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T22:22:30.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T22:22:30.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T22:22:30.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.allow_fallocate: 1 2023-07-21T22:22:30.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T22:22:30.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T22:22:30.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.use_direct_reads: 0 2023-07-21T22:22:30.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T22:22:30.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T22:22:30.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.db_log_dir: 2023-07-21T22:22:30.439 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.wal_dir: 2023-07-21T22:22:30.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T22:22:30.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T22:22:30.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T22:22:30.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T22:22:30.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T22:22:30.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T22:22:30.440 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T22:22:30.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T22:22:30.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.write_buffer_manager: 0x55d91b3743c0 2023-07-21T22:22:30.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T22:22:30.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T22:22:30.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T22:22:30.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T22:22:30.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T22:22:30.441 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T22:22:30.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T22:22:30.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T22:22:30.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.unordered_write: 0 2023-07-21T22:22:30.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T22:22:30.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T22:22:30.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T22:22:30.442 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T22:22:30.443 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.row_cache: None 2023-07-21T22:22:30.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.wal_filter: None 2023-07-21T22:22:30.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T22:22:30.444 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T22:22:30.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.two_write_queues: 0 2023-07-21T22:22:30.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T22:22:30.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.wal_compression: 0 2023-07-21T22:22:30.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.atomic_flush: 0 2023-07-21T22:22:30.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T22:22:30.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T22:22:30.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T22:22:30.445 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.log_readahead_size: 0 2023-07-21T22:22:30.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T22:22:30.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T22:22:30.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T22:22:30.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T22:22:30.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T22:22:30.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T22:22:30.446 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T22:22:30.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_background_jobs: 2 2023-07-21T22:22:30.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_background_compactions: -1 2023-07-21T22:22:30.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_subcompactions: 1 2023-07-21T22:22:30.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T22:22:30.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T22:22:30.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T22:22:30.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T22:22:30.447 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T22:22:30.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T22:22:30.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T22:22:30.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T22:22:30.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_open_files: -1 2023-07-21T22:22:30.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T22:22:30.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T22:22:30.448 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T22:22:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T22:22:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_background_flushes: -1 2023-07-21T22:22:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Compression algorithms supported: 2023-07-21T22:22:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T22:22:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: kZSTD supported: 0 2023-07-21T22:22:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: kXpressCompression supported: 0 2023-07-21T22:22:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T22:22:30.449 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: kZlibCompression supported: 1 2023-07-21T22:22:30.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: kSnappyCompression supported: 1 2023-07-21T22:22:30.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: kLZ4Compression supported: 1 2023-07-21T22:22:30.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: kBZip2Compression supported: 0 2023-07-21T22:22:30.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T22:22:30.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T22:22:30.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-21T22:22:30.450 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T22:22:30.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T22:22:30.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.merge_operator: 2023-07-21T22:22:30.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compaction_filter: None 2023-07-21T22:22:30.451 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T22:22:30.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T22:22:30.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T22:22:30.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T22:22:30.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d91b453360) 2023-07-21T22:22:30.452 INFO:journalctl@ceph.mon.b.smithi132.stdout: cache_index_and_filter_blocks: 1 2023-07-21T22:22:30.452 INFO:journalctl@ceph.mon.b.smithi132.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T22:22:30.452 INFO:journalctl@ceph.mon.b.smithi132.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T22:22:30.453 INFO:journalctl@ceph.mon.b.smithi132.stdout: pin_top_level_index_and_filter: 1 2023-07-21T22:22:30.453 INFO:journalctl@ceph.mon.b.smithi132.stdout: index_type: 0 2023-07-21T22:22:30.453 INFO:journalctl@ceph.mon.b.smithi132.stdout: data_block_index_type: 0 2023-07-21T22:22:30.453 INFO:journalctl@ceph.mon.b.smithi132.stdout: index_shortening: 1 2023-07-21T22:22:30.453 INFO:journalctl@ceph.mon.b.smithi132.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T22:22:30.453 INFO:journalctl@ceph.mon.b.smithi132.stdout: checksum: 4 2023-07-21T22:22:30.453 INFO:journalctl@ceph.mon.b.smithi132.stdout: no_block_cache: 0 2023-07-21T22:22:30.453 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_cache: 0x55d91b4131f0 2023-07-21T22:22:30.454 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_cache_name: BinnedLRUCache 2023-07-21T22:22:30.454 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_cache_options: 2023-07-21T22:22:30.454 INFO:journalctl@ceph.mon.b.smithi132.stdout: capacity : 536870912 2023-07-21T22:22:30.454 INFO:journalctl@ceph.mon.b.smithi132.stdout: num_shard_bits : 4 2023-07-21T22:22:30.454 INFO:journalctl@ceph.mon.b.smithi132.stdout: strict_capacity_limit : 0 2023-07-21T22:22:30.454 INFO:journalctl@ceph.mon.b.smithi132.stdout: high_pri_pool_ratio: 0.000 2023-07-21T22:22:30.454 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_cache_compressed: (nil) 2023-07-21T22:22:30.454 INFO:journalctl@ceph.mon.b.smithi132.stdout: persistent_cache: (nil) 2023-07-21T22:22:30.455 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_size: 4096 2023-07-21T22:22:30.455 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_size_deviation: 10 2023-07-21T22:22:30.455 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_restart_interval: 16 2023-07-21T22:22:30.455 INFO:journalctl@ceph.mon.b.smithi132.stdout: index_block_restart_interval: 1 2023-07-21T22:22:30.455 INFO:journalctl@ceph.mon.b.smithi132.stdout: metadata_block_size: 4096 2023-07-21T22:22:30.455 INFO:journalctl@ceph.mon.b.smithi132.stdout: partition_filters: 0 2023-07-21T22:22:30.455 INFO:journalctl@ceph.mon.b.smithi132.stdout: use_delta_encoding: 1 2023-07-21T22:22:30.455 INFO:journalctl@ceph.mon.b.smithi132.stdout: filter_policy: bloomfilter 2023-07-21T22:22:30.456 INFO:journalctl@ceph.mon.b.smithi132.stdout: whole_key_filtering: 1 2023-07-21T22:22:30.456 INFO:journalctl@ceph.mon.b.smithi132.stdout: verify_compression: 0 2023-07-21T22:22:30.456 INFO:journalctl@ceph.mon.b.smithi132.stdout: read_amp_bytes_per_bit: 0 2023-07-21T22:22:30.456 INFO:journalctl@ceph.mon.b.smithi132.stdout: format_version: 5 2023-07-21T22:22:30.456 INFO:journalctl@ceph.mon.b.smithi132.stdout: enable_index_compression: 1 2023-07-21T22:22:30.456 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_align: 0 2023-07-21T22:22:30.456 INFO:journalctl@ceph.mon.b.smithi132.stdout: max_auto_readahead_size: 262144 2023-07-21T22:22:30.457 INFO:journalctl@ceph.mon.b.smithi132.stdout: prepopulate_block_cache: 0 2023-07-21T22:22:30.457 INFO:journalctl@ceph.mon.b.smithi132.stdout: initial_auto_readahead_size: 8192 2023-07-21T22:22:30.457 INFO:journalctl@ceph.mon.b.smithi132.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T22:22:30.457 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T22:22:30.457 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T22:22:30.457 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compression: NoCompression 2023-07-21T22:22:30.457 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T22:22:30.457 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T22:22:30.458 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T22:22:30.458 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.num_levels: 7 2023-07-21T22:22:30.458 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T22:22:30.458 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T22:22:30.458 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T22:22:30.458 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T22:22:30.458 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T22:22:30.458 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T22:22:30.459 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T22:22:30.459 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:22:30.459 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T22:22:30.459 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T22:22:30.459 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:22:30.459 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:22:30.459 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T22:22:30.460 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 bash[97823]: a2ce252f5187869ee92b27d9f0ad52f41a054da212be607488b4ef8a5e0aef45 2023-07-21T22:22:30.460 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T22:22:30.460 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T22:22:30.460 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T22:22:30.460 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:22:30.460 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:22:30.460 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T22:22:30.461 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T22:22:30.461 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:22:30.461 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T22:22:30.461 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T22:22:30.461 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T22:22:30.461 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T22:22:30.461 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T22:22:30.461 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T22:22:30.462 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T22:22:30.462 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T22:22:30.462 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T22:22:30.462 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T22:22:30.462 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T22:22:30.462 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T22:22:30.462 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T22:22:30.462 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T22:22:30.463 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T22:22:30.463 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T22:22:30.463 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T22:22:30.463 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T22:22:30.463 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T22:22:30.463 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T22:22:30.463 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T22:22:30.463 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T22:22:30.464 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T22:22:30.464 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T22:22:30.465 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T22:22:30.465 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T22:22:30.465 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T22:22:30.465 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T22:22:30.465 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T22:22:30.465 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T22:22:30.466 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T22:22:30.466 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T22:22:30.466 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.table_properties_collectors: 2023-07-21T22:22:30.466 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.inplace_update_support: 0 2023-07-21T22:22:30.466 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T22:22:30.466 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T22:22:30.466 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T22:22:30.467 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T22:22:30.467 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.bloom_locality: 0 2023-07-21T22:22:30.467 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.max_successive_merges: 0 2023-07-21T22:22:30.467 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T22:22:30.467 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T22:22:30.467 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T22:22:30.467 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T22:22:30.467 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.ttl: 2592000 2023-07-21T22:22:30.468 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T22:22:30.468 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T22:22:30.468 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T22:22:30.468 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.enable_blob_files: false 2023-07-21T22:22:30.468 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.min_blob_size: 0 2023-07-21T22:22:30.468 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T22:22:30.468 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T22:22:30.468 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T22:22:30.469 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T22:22:30.469 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T22:22:30.469 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T22:22:30.469 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T22:22:30.469 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T22:22:30.469 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-21T22:22:30.469 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-21T22:22:30.470 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c7adfa13-e269-40d4-8246-2b30009055de 2023-07-21T22:22:30.470 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689978150365021, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-21T22:22:30.470 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-21T22:22:30.470 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689978150365420, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689978150, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c7adfa13-e269-40d4-8246-2b30009055de", "db_session_id": "1U6JE03K9VEGTV8HQ3JD", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-21T22:22:30.470 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689978150365496, "job": 1, "event": "recovery_finished"} 2023-07-21T22:22:30.470 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-21T22:22:30.470 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T22:22:30.470 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d91b4ac000 2023-07-21T22:22:30.471 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: DB pointer 0x55d91b498000 2023-07-21T22:22:30.471 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T22:22:30.471 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T22:22:30.471 INFO:journalctl@ceph.mon.b.smithi132.stdout: ** DB Stats ** 2023-07-21T22:22:30.471 INFO:journalctl@ceph.mon.b.smithi132.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:22:30.471 INFO:journalctl@ceph.mon.b.smithi132.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T22:22:30.471 INFO:journalctl@ceph.mon.b.smithi132.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:22:30.471 INFO:journalctl@ceph.mon.b.smithi132.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:22:30.472 INFO:journalctl@ceph.mon.b.smithi132.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T22:22:30.472 INFO:journalctl@ceph.mon.b.smithi132.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:22:30.472 INFO:journalctl@ceph.mon.b.smithi132.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:22:30.472 INFO:journalctl@ceph.mon.b.smithi132.stdout: 2023-07-21T22:22:30.472 INFO:journalctl@ceph.mon.b.smithi132.stdout: ** Compaction Stats [default] ** 2023-07-21T22:22:30.472 INFO:journalctl@ceph.mon.b.smithi132.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:22:30.472 INFO:journalctl@ceph.mon.b.smithi132.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T22:22:30.473 INFO:journalctl@ceph.mon.b.smithi132.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-21T22:22:30.473 INFO:journalctl@ceph.mon.b.smithi132.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-21T22:22:30.473 INFO:journalctl@ceph.mon.b.smithi132.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-21T22:22:30.473 INFO:journalctl@ceph.mon.b.smithi132.stdout: 2023-07-21T22:22:30.473 INFO:journalctl@ceph.mon.b.smithi132.stdout: ** Compaction Stats [default] ** 2023-07-21T22:22:30.473 INFO:journalctl@ceph.mon.b.smithi132.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:22:30.473 INFO:journalctl@ceph.mon.b.smithi132.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T22:22:30.473 INFO:journalctl@ceph.mon.b.smithi132.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-21T22:22:30.474 INFO:journalctl@ceph.mon.b.smithi132.stdout: 2023-07-21T22:22:30.474 INFO:journalctl@ceph.mon.b.smithi132.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T22:22:30.474 INFO:journalctl@ceph.mon.b.smithi132.stdout: 2023-07-21T22:22:30.474 INFO:journalctl@ceph.mon.b.smithi132.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:22:30.474 INFO:journalctl@ceph.mon.b.smithi132.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T22:22:30.474 INFO:journalctl@ceph.mon.b.smithi132.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T22:22:30.474 INFO:journalctl@ceph.mon.b.smithi132.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T22:22:30.475 INFO:journalctl@ceph.mon.b.smithi132.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T22:22:30.475 INFO:journalctl@ceph.mon.b.smithi132.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T22:22:30.475 INFO:journalctl@ceph.mon.b.smithi132.stdout: Cumulative compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:22:30.475 INFO:journalctl@ceph.mon.b.smithi132.stdout: Interval compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:22:30.475 INFO:journalctl@ceph.mon.b.smithi132.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T22:22:30.475 INFO:journalctl@ceph.mon.b.smithi132.stdout: Block cache BinnedLRUCache@0x55d91b4131f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-07-21T22:22:30.475 INFO:journalctl@ceph.mon.b.smithi132.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-21T22:22:30.475 INFO:journalctl@ceph.mon.b.smithi132.stdout: 2023-07-21T22:22:30.476 INFO:journalctl@ceph.mon.b.smithi132.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T22:22:30.476 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-21T22:22:30.476 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: using public_addr v2:172.21.15.132:0/0 -> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2023-07-21T22:22:30.476 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: starting mon.b rank -1 at public addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] at bind addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:22:30.476 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(???) e0 preinit fsid af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:22:30.476 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing).mds e1 new map 2023-07-21T22:22:30.476 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-21T22:22:30.476 INFO:journalctl@ceph.mon.b.smithi132.stdout: e1 2023-07-21T22:22:30.477 INFO:journalctl@ceph.mon.b.smithi132.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T22:22:30.477 INFO:journalctl@ceph.mon.b.smithi132.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:22:30.477 INFO:journalctl@ceph.mon.b.smithi132.stdout: legacy client fscid: -1 2023-07-21T22:22:30.477 INFO:journalctl@ceph.mon.b.smithi132.stdout: 2023-07-21T22:22:30.477 INFO:journalctl@ceph.mon.b.smithi132.stdout: No filesystems configured 2023-07-21T22:22:30.477 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-21T22:22:30.477 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-21T22:22:30.478 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-21T22:22:30.478 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-21T22:22:30.478 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-21T22:22:30.478 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-21T22:22:30.478 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T22:22:30.478 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:22:30.478 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:22:30.478 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:22:30.479 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi190", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:30.479 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: Updating smithi132:/etc/ceph/ceph.conf 2023-07-21T22:22:30.479 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: Updating smithi132:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:30.479 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:22:30.479 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: Deploying cephadm binary to smithi190 2023-07-21T22:22:30.479 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: Updating smithi132:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:22:30.479 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.479 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.480 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.480 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:30.480 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.480 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:30.480 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: Added host smithi190 2023-07-21T22:22:30.480 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.480 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.480 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:30.481 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.481 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:22:30.481 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:30.481 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.481 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.481 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.481 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.482 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi190", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:22:30.482 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:30.482 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:30.482 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: Updating smithi190:/etc/ceph/ceph.conf 2023-07-21T22:22:30.482 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/181226069' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T22:22:30.482 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.482 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.482 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.483 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: Updating smithi190:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:30.483 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: Updating smithi190:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:22:30.483 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:30.483 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: Updating smithi190:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:22:30.483 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/181226069' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T22:22:30.483 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:22:30.483 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:30.484 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.484 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:30.484 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:30.484 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:30.518 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:22:30.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:30.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:30.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi132:172.21.15.132=b;smithi190:172.21.15.190=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:30.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi132:172.21.15.132=b;smithi190:172.21.15.190=c;count:3 2023-07-21T22:22:30.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: Deploying daemon mon.c on smithi190 2023-07-21T22:22:30.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:30.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='client.? 172.21.15.190:0/652758046' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:22:30.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:22:30.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:30.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.a calling monitor election 2023-07-21T22:22:30.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:30.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:30.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:30.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.c calling monitor election 2023-07-21T22:22:30.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:30.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:30.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:30.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:30.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:22:30.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: monmap e2: 2 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-07-21T22:22:30.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: fsmap 2023-07-21T22:22:30.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:22:30.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mgrmap e13: x(active, since 41s) 2023-07-21T22:22:30.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: overall HEALTH_OK 2023-07-21T22:22:30.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:30.935 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:30.935 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:30.935 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: Deploying daemon mon.b on smithi132 2023-07-21T22:22:30.935 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:30.935 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:30.935 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: from='client.? 172.21.15.190:0/3124882166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:22:30.935 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-21T22:22:30.935 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: expand_channel_meta expand map: {default=false} 2023-07-21T22:22:30.936 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: expand_channel_meta from 'false' to 'false' 2023-07-21T22:22:30.936 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: expand_channel_meta expanded map: {default=false} 2023-07-21T22:22:30.936 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: expand_channel_meta expand map: {default=info} 2023-07-21T22:22:30.936 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: expand_channel_meta from 'info' to 'info' 2023-07-21T22:22:30.936 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: expand_channel_meta expanded map: {default=info} 2023-07-21T22:22:30.936 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: expand_channel_meta expand map: {default=daemon} 2023-07-21T22:22:30.936 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-21T22:22:30.937 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: expand_channel_meta expanded map: {default=daemon} 2023-07-21T22:22:30.937 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: expand_channel_meta expand map: {default=debug} 2023-07-21T22:22:30.937 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: expand_channel_meta from 'debug' to 'debug' 2023-07-21T22:22:30.937 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: expand_channel_meta expanded map: {default=debug} 2023-07-21T22:22:30.937 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 ceph-mon[97863]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-21T22:22:30.937 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:30 smithi132 systemd[1]: Started Ceph mon.b for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:22:35.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: mon.c calling monitor election 2023-07-21T22:22:35.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:22:35.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: mon.a calling monitor election 2023-07-21T22:22:35.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:35.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:35.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:35.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:35.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:35.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:35.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:35.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:35.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:35.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:22:35.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-07-21T22:22:35.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: fsmap 2023-07-21T22:22:35.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:22:35.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: mgrmap e13: x(active, since 49s) 2023-07-21T22:22:35.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-21T22:22:35.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-21T22:22:35.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-21T22:22:35.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: mon.b (rank 2) addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] is down (out of quorum) 2023-07-21T22:22:35.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:35.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:35.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:35.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:35 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:35.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: mon.c calling monitor election 2023-07-21T22:22:35.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:22:35.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: mon.a calling monitor election 2023-07-21T22:22:35.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:35.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:35.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:35.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:35.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:35.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:35.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:35.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:35.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:35.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:22:35.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-07-21T22:22:35.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: fsmap 2023-07-21T22:22:35.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:22:35.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: mgrmap e13: x(active, since 49s) 2023-07-21T22:22:35.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-21T22:22:35.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-21T22:22:35.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-21T22:22:35.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: mon.b (rank 2) addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] is down (out of quorum) 2023-07-21T22:22:35.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:35.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:35.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:35.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:35 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:36.015 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.970+0000 7f2877a60700 1 -- 172.21.15.190:0/1503972361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f285c0035c0 msgr2=0x7f285c005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:36.015 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.970+0000 7f2877a60700 1 --2- 172.21.15.190:0/1503972361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f285c0035c0 0x7f285c005a40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f28700ff660 tx=0x7f286c00fe10 comp rx=0 tx=0).stop 2023-07-21T22:22:36.015 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.970+0000 7f2877a60700 1 -- 172.21.15.190:0/1503972361 shutdown_connections 2023-07-21T22:22:36.016 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.970+0000 7f2877a60700 1 --2- 172.21.15.190:0/1503972361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f285c0035c0 0x7f285c005a40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:36.016 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.970+0000 7f2877a60700 1 -- 172.21.15.190:0/1503972361 >> 172.21.15.190:0/1503972361 conn(0x7f28700fb210 msgr2=0x7f28700fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:36.016 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.970+0000 7f2877a60700 1 -- 172.21.15.190:0/1503972361 shutdown_connections 2023-07-21T22:22:36.016 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.970+0000 7f2877a60700 1 -- 172.21.15.190:0/1503972361 wait complete. 2023-07-21T22:22:36.018 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2877a60700 1 Processor -- start 2023-07-21T22:22:36.019 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2877a60700 1 -- start start 2023-07-21T22:22:36.019 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2877a60700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0035c0 0x7f2870072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:36.019 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2877a60700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2870072d50 0x7f28700731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:36.019 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2877a60700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28701a0ed0 0x7f28701a32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:36.019 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2877a60700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28701a3900 con 0x7f2870072d50 2023-07-21T22:22:36.019 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2877a60700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28701a3a70 con 0x7f285c0035c0 2023-07-21T22:22:36.019 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2877a60700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28701a3c30 con 0x7f28701a0ed0 2023-07-21T22:22:36.020 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2875ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28701a0ed0 0x7f28701a32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:36.020 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2875ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28701a0ed0 0x7f28701a32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:47924/0 (socket says 172.21.15.190:47924) 2023-07-21T22:22:36.020 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2875ffd700 1 -- 172.21.15.190:0/4020514369 learned_addr learned my addr 172.21.15.190:0/4020514369 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-07-21T22:22:36.020 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f28757fc700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0035c0 0x7f2870072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:36.020 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2875ffd700 1 -- 172.21.15.190:0/4020514369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28701a0ed0 msgr2=0x7f28701a32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-07-21T22:22:36.020 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2875ffd700 1 -- 172.21.15.190:0/4020514369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28701a0ed0 msgr2=0x7f28701a32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-21T22:22:36.021 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2875ffd700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28701a0ed0 0x7f28701a32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-21T22:22:36.021 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2875ffd700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28701a0ed0 0x7f28701a32b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:22:36.021 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f2874ffb700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2870072d50 0x7f28700731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:36.021 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f28757fc700 1 -- 172.21.15.190:0/4020514369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0035c0 msgr2=0x7f2870072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-07-21T22:22:36.021 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f28757fc700 1 -- 172.21.15.190:0/4020514369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0035c0 msgr2=0x7f2870072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-21T22:22:36.021 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f28757fc700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0035c0 0x7f2870072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-21T22:22:36.021 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.971+0000 7f28757fc700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0035c0 0x7f2870072810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:22:36.022 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.972+0000 7f2874ffb700 1 -- 172.21.15.190:0/4020514369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28701a0ed0 msgr2=0x7f28701a32b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:22:36.022 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.972+0000 7f2874ffb700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28701a0ed0 0x7f28701a32b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:36.022 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.972+0000 7f2874ffb700 1 -- 172.21.15.190:0/4020514369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0035c0 msgr2=0x7f2870072810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:22:36.022 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.972+0000 7f2874ffb700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0035c0 0x7f2870072810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:36.022 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.972+0000 7f2874ffb700 1 -- 172.21.15.190:0/4020514369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c00f860 con 0x7f2870072d50 2023-07-21T22:22:36.022 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.972+0000 7f2874ffb700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2870072d50 0x7f28700731c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f286000b550 tx=0x7f286000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:36.022 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.972+0000 7f28667fc700 1 -- 172.21.15.190:0/4020514369 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860004230 con 0x7f2870072d50 2023-07-21T22:22:36.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.972+0000 7f28667fc700 1 -- 172.21.15.190:0/4020514369 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2860004390 con 0x7f2870072d50 2023-07-21T22:22:36.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.972+0000 7f2877a60700 1 -- 172.21.15.190:0/4020514369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28701a3f10 con 0x7f2870072d50 2023-07-21T22:22:36.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.972+0000 7f28667fc700 1 -- 172.21.15.190:0/4020514369 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860011af0 con 0x7f2870072d50 2023-07-21T22:22:36.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.972+0000 7f2877a60700 1 -- 172.21.15.190:0/4020514369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28701a4460 con 0x7f2870072d50 2023-07-21T22:22:36.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.973+0000 7f28667fc700 1 -- 172.21.15.190:0/4020514369 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f2860011c50 con 0x7f2870072d50 2023-07-21T22:22:36.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.973+0000 7f2877a60700 1 -- 172.21.15.190:0/4020514369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2870061b40 con 0x7f2870072d50 2023-07-21T22:22:36.023 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.974+0000 7f28667fc700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2854038c30 0x7f285403b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:36.024 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.974+0000 7f28667fc700 1 -- 172.21.15.190:0/4020514369 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f286004acc0 con 0x7f2870072d50 2023-07-21T22:22:36.024 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.974+0000 7f28757fc700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2854038c30 0x7f285403b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:36.024 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.974+0000 7f28757fc700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2854038c30 0x7f285403b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f286c020040 tx=0x7f286c00f3e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:36.025 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:35.979+0000 7f28667fc700 1 -- 172.21.15.190:0/4020514369 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f286001ae40 con 0x7f2870072d50 2023-07-21T22:22:36.308 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.263+0000 7f2877a60700 1 -- 172.21.15.190:0/4020514369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2870073d40 con 0x7f2870072d50 2023-07-21T22:22:36.312 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.263+0000 7f28667fc700 1 -- 172.21.15.190:0/4020514369 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1278 (secure 0 0 0) 0x7f286000f2e0 con 0x7f2870072d50 2023-07-21T22:22:36.313 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-07-21T22:22:36.313 INFO:teuthology.orchestra.run.smithi190.stdout:{"epoch":3,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","modified":"2023-07-21T22:22:30.395552Z","created":"2023-07-21T22:20:54.562258Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:3300","nonce":0},{"type":"v1","addr":"172.21.15.190:6789","nonce":0}]},"addr":"172.21.15.190:6789/0","public_addr":"172.21.15.190:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:3300","nonce":0},{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-21T22:22:36.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.266+0000 7f2877a60700 1 -- 172.21.15.190:0/4020514369 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2854038c30 msgr2=0x7f285403b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:36.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.266+0000 7f2877a60700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2854038c30 0x7f285403b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f286c020040 tx=0x7f286c00f3e0 comp rx=0 tx=0).stop 2023-07-21T22:22:36.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.266+0000 7f2877a60700 1 -- 172.21.15.190:0/4020514369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2870072d50 msgr2=0x7f28700731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:36.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.266+0000 7f2877a60700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2870072d50 0x7f28700731c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f286000b550 tx=0x7f286000b910 comp rx=0 tx=0).stop 2023-07-21T22:22:36.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.266+0000 7f2877a60700 1 -- 172.21.15.190:0/4020514369 shutdown_connections 2023-07-21T22:22:36.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.266+0000 7f2877a60700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2854038c30 0x7f285403b0e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:36.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.266+0000 7f2877a60700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f28701a0ed0 0x7f28701a32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:36.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.266+0000 7f2877a60700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0035c0 0x7f2870072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:36.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.266+0000 7f2877a60700 1 --2- 172.21.15.190:0/4020514369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2870072d50 0x7f28700731c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:36.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.266+0000 7f2877a60700 1 -- 172.21.15.190:0/4020514369 >> 172.21.15.190:0/4020514369 conn(0x7f28700fb210 msgr2=0x7f2870101270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:36.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.266+0000 7f2877a60700 1 -- 172.21.15.190:0/4020514369 shutdown_connections 2023-07-21T22:22:36.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:36.266+0000 7f2877a60700 1 -- 172.21.15.190:0/4020514369 wait complete. 2023-07-21T22:22:36.317 INFO:teuthology.orchestra.run.smithi190.stderr:dumped monmap epoch 3 2023-07-21T22:22:36.639 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:36.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:36 smithi190 ceph-mon[98334]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:36.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:36.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:36 smithi190 ceph-mon[98334]: from='client.? 172.21.15.190:0/4020514369' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:22:36.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:36.681 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-21T22:22:36.681 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph config generate-minimal-conf 2023-07-21T22:22:36.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:36.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:36 smithi107 ceph-mon[96543]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:36.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:36.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:36 smithi107 ceph-mon[96543]: from='client.? 172.21.15.190:0/4020514369' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:22:36.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:37.247 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:22:37.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: mon.c calling monitor election 2023-07-21T22:22:37.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:22:37.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: mon.a calling monitor election 2023-07-21T22:22:37.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:37.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:22:37.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:37.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:37.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:37.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:37.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:37.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:37.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:37.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:22:37.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-07-21T22:22:37.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: fsmap 2023-07-21T22:22:37.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:22:37.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: mgrmap e13: x(active, since 49s) 2023-07-21T22:22:37.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-21T22:22:37.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-21T22:22:37.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-21T22:22:37.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: mon.b (rank 2) addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] is down (out of quorum) 2023-07-21T22:22:37.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:37.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:37.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:37.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:37.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:37.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:37.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:37.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='client.? 172.21.15.190:0/4020514369' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:22:37.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:38.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.449+0000 7f6a2239a700 1 -- 172.21.15.107:0/427315953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0fe450 msgr2=0x7f6a1c0fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:38.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.449+0000 7f6a2239a700 1 --2- 172.21.15.107:0/427315953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0fe450 0x7f6a1c0fe820 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009910 tx=0x7f6a0c009c20 comp rx=0 tx=0).stop 2023-07-21T22:22:38.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.449+0000 7f6a2239a700 1 -- 172.21.15.107:0/427315953 shutdown_connections 2023-07-21T22:22:38.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.449+0000 7f6a2239a700 1 --2- 172.21.15.107:0/427315953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0fe450 0x7f6a1c0fe820 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:38.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.449+0000 7f6a2239a700 1 -- 172.21.15.107:0/427315953 >> 172.21.15.107:0/427315953 conn(0x7f6a1c074f40 msgr2=0x7f6a1c075340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:38.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.449+0000 7f6a2239a700 1 -- 172.21.15.107:0/427315953 shutdown_connections 2023-07-21T22:22:38.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.449+0000 7f6a2239a700 1 -- 172.21.15.107:0/427315953 wait complete. 2023-07-21T22:22:38.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a2239a700 1 Processor -- start 2023-07-21T22:22:38.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a2239a700 1 -- start start 2023-07-21T22:22:38.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a2239a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0fe450 0x7f6a1c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:38.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a2239a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a1c19af90 0x7f6a1c194a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:38.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a2239a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a1c194fd0 0x7f6a1c195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:38.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a2239a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c19b530 con 0x7f6a1c0fe450 2023-07-21T22:22:38.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a2239a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c1a1800 con 0x7f6a1c19af90 2023-07-21T22:22:38.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a2239a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c1a1940 con 0x7f6a1c194fd0 2023-07-21T22:22:38.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a1bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0fe450 0x7f6a1c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:38.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a1bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0fe450 0x7f6a1c19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45524/0 (socket says 172.21.15.107:45524) 2023-07-21T22:22:38.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a1bfff700 1 -- 172.21.15.107:0/3414579224 learned_addr learned my addr 172.21.15.107:0/3414579224 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:22:38.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a1b7fe700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a1c19af90 0x7f6a1c194a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:38.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a20937700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a1c194fd0 0x7f6a1c195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:38.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.450+0000 7f6a1bfff700 1 -- 172.21.15.107:0/3414579224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a1c194fd0 msgr2=0x7f6a1c195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:38.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.451+0000 7f6a1bfff700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a1c194fd0 0x7f6a1c195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:38.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.451+0000 7f6a1bfff700 1 -- 172.21.15.107:0/3414579224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a1c19af90 msgr2=0x7f6a1c194a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:38.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.451+0000 7f6a1bfff700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a1c19af90 0x7f6a1c194a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:38.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.451+0000 7f6a1bfff700 1 -- 172.21.15.107:0/3414579224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c0095f0 con 0x7f6a1c0fe450 2023-07-21T22:22:38.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.451+0000 7f6a1bfff700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0fe450 0x7f6a1c19aa50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c004cb0 tx=0x7f6a0c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:38.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.451+0000 7f6a197fa700 1 -- 172.21.15.107:0/3414579224 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c01d070 con 0x7f6a1c0fe450 2023-07-21T22:22:38.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.451+0000 7f6a197fa700 1 -- 172.21.15.107:0/3414579224 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a0c00be20 con 0x7f6a1c0fe450 2023-07-21T22:22:38.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.451+0000 7f6a197fa700 1 -- 172.21.15.107:0/3414579224 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c022e50 con 0x7f6a1c0fe450 2023-07-21T22:22:38.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.451+0000 7f6a2239a700 1 -- 172.21.15.107:0/3414579224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1c1a1bc0 con 0x7f6a1c0fe450 2023-07-21T22:22:38.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.451+0000 7f6a2239a700 1 -- 172.21.15.107:0/3414579224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1c1a2060 con 0x7f6a1c0fe450 2023-07-21T22:22:38.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.452+0000 7f6a197fa700 1 -- 172.21.15.107:0/3414579224 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f6a0c00f460 con 0x7f6a1c0fe450 2023-07-21T22:22:38.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.452+0000 7f6a197fa700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6a04038c30 0x7f6a0403b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:38.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.452+0000 7f6a197fa700 1 -- 172.21.15.107:0/3414579224 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6a0c04c8a0 con 0x7f6a1c0fe450 2023-07-21T22:22:38.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.452+0000 7f6a1b7fe700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6a04038c30 0x7f6a0403b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:38.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.452+0000 7f6a2239a700 1 -- 172.21.15.107:0/3414579224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1c18ed80 con 0x7f6a1c0fe450 2023-07-21T22:22:38.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.452+0000 7f6a1b7fe700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6a04038c30 0x7f6a0403b0e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009ad0 tx=0x7f6a10009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:38.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.457+0000 7f6a197fa700 1 -- 172.21.15.107:0/3414579224 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6a0c027020 con 0x7f6a1c0fe450 2023-07-21T22:22:38.520 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: mon.b calling monitor election 2023-07-21T22:22:38.520 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: Updating smithi107:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:38.520 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: Updating smithi132:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:38.520 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: Updating smithi190:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:38.520 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: mon.b calling monitor election 2023-07-21T22:22:38.521 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: mon.c calling monitor election 2023-07-21T22:22:38.521 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: mon.a calling monitor election 2023-07-21T22:22:38.521 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T22:22:38.521 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-07-21T22:22:38.521 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: fsmap 2023-07-21T22:22:38.521 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:22:38.521 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: mgrmap e13: x(active, since 51s) 2023-07-21T22:22:38.522 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-21T22:22:38.522 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: Cluster is now healthy 2023-07-21T22:22:38.522 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: overall HEALTH_OK 2023-07-21T22:22:38.522 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.522 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.522 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.522 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.522 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.523 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.523 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.523 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:38.523 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T22:22:38.523 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:38.523 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:22:38.523 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:38.523 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:38 smithi107 ceph-mon[96543]: Reconfiguring daemon mon.a on smithi107 2023-07-21T22:22:38.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: mon.b calling monitor election 2023-07-21T22:22:38.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: Updating smithi107:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:38.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: Updating smithi132:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:38.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: Updating smithi190:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:38.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: mon.b calling monitor election 2023-07-21T22:22:38.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: mon.c calling monitor election 2023-07-21T22:22:38.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: mon.a calling monitor election 2023-07-21T22:22:38.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T22:22:38.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-07-21T22:22:38.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: fsmap 2023-07-21T22:22:38.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:22:38.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: mgrmap e13: x(active, since 51s) 2023-07-21T22:22:38.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-21T22:22:38.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: Cluster is now healthy 2023-07-21T22:22:38.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: overall HEALTH_OK 2023-07-21T22:22:38.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:38.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T22:22:38.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:38.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:22:38.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:38.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:38 smithi190 ceph-mon[98334]: Reconfiguring daemon mon.a on smithi107 2023-07-21T22:22:38.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: mon.b calling monitor election 2023-07-21T22:22:38.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: Updating smithi107:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:38.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: Updating smithi132:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:38.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: Updating smithi190:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:22:38.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: mon.b calling monitor election 2023-07-21T22:22:38.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: mon.c calling monitor election 2023-07-21T22:22:38.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: mon.a calling monitor election 2023-07-21T22:22:38.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T22:22:38.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0]} removed_ranks: {} 2023-07-21T22:22:38.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: fsmap 2023-07-21T22:22:38.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:22:38.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: mgrmap e13: x(active, since 51s) 2023-07-21T22:22:38.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-21T22:22:38.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: Cluster is now healthy 2023-07-21T22:22:38.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: overall HEALTH_OK 2023-07-21T22:22:38.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:38.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:38.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T22:22:38.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:38.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:22:38.685 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:38.685 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:38 smithi132 ceph-mon[97863]: Reconfiguring daemon mon.a on smithi107 2023-07-21T22:22:38.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.657+0000 7f6a2239a700 1 -- 172.21.15.107:0/3414579224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6a1c195f70 con 0x7f6a1c0fe450 2023-07-21T22:22:38.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.658+0000 7f6a197fa700 1 -- 172.21.15.107:0/3414579224 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f6a0c034070 con 0x7f6a1c0fe450 2023-07-21T22:22:38.702 INFO:teuthology.orchestra.run.smithi107.stdout:# minimal ceph.conf for af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:22:38.702 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2023-07-21T22:22:38.703 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:22:38.703 INFO:teuthology.orchestra.run.smithi107.stdout: mon_host = [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] 2023-07-21T22:22:38.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.659+0000 7f6a2239a700 1 -- 172.21.15.107:0/3414579224 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6a04038c30 msgr2=0x7f6a0403b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:38.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.659+0000 7f6a2239a700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6a04038c30 0x7f6a0403b0e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009ad0 tx=0x7f6a10009210 comp rx=0 tx=0).stop 2023-07-21T22:22:38.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.659+0000 7f6a2239a700 1 -- 172.21.15.107:0/3414579224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0fe450 msgr2=0x7f6a1c19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:38.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.659+0000 7f6a2239a700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0fe450 0x7f6a1c19aa50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c004cb0 tx=0x7f6a0c004ce0 comp rx=0 tx=0).stop 2023-07-21T22:22:38.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.660+0000 7f6a2239a700 1 -- 172.21.15.107:0/3414579224 shutdown_connections 2023-07-21T22:22:38.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.660+0000 7f6a2239a700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6a04038c30 0x7f6a0403b0e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:38.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.660+0000 7f6a2239a700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a1c194fd0 0x7f6a1c195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:38.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.660+0000 7f6a2239a700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0fe450 0x7f6a1c19aa50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:38.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.660+0000 7f6a2239a700 1 --2- 172.21.15.107:0/3414579224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a1c19af90 0x7f6a1c194a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:38.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.660+0000 7f6a2239a700 1 -- 172.21.15.107:0/3414579224 >> 172.21.15.107:0/3414579224 conn(0x7f6a1c074f40 msgr2=0x7f6a1c101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:38.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.660+0000 7f6a2239a700 1 -- 172.21.15.107:0/3414579224 shutdown_connections 2023-07-21T22:22:38.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:38.660+0000 7f6a2239a700 1 -- 172.21.15.107:0/3414579224 wait complete. 2023-07-21T22:22:39.395 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-21T22:22:39.396 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:22:39.396 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T22:22:39.425 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:22:39.425 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:22:39.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:39.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:39 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3414579224' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:39.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:39.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:39.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:39.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:22:39.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:39.465 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-07-21T22:22:39.465 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T22:22:39.498 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-07-21T22:22:39.498 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:22:39.564 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-07-21T22:22:39.564 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T22:22:39.595 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:39.596 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:39 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3414579224' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:39.596 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:39.596 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:39.596 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:39.596 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:22:39.596 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:39.600 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-07-21T22:22:39.600 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:22:39.672 INFO:tasks.cephadm:Adding mgr.x on smithi107 2023-07-21T22:22:39.672 INFO:tasks.cephadm:Adding mgr.y on smithi132 2023-07-21T22:22:39.673 INFO:tasks.cephadm:Adding mgr.z on smithi190 2023-07-21T22:22:39.674 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch apply mgr '3;smithi107=x;smithi132=y;smithi190=z' 2023-07-21T22:22:39.680 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:22:39.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:39 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3414579224' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:39.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:39.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:39.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:39.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:22:39.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:40.260 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:22:40.637 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:40 smithi190 ceph-mon[98334]: Reconfiguring mon.b (monmap changed)... 2023-07-21T22:22:40.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:40 smithi190 ceph-mon[98334]: Reconfiguring daemon mon.b on smithi132 2023-07-21T22:22:40.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:40 smithi190 ceph-mon[98334]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:40.638 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:40.639 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:40.680 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:40 smithi132 ceph-mon[97863]: Reconfiguring mon.b (monmap changed)... 2023-07-21T22:22:40.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:40 smithi132 ceph-mon[97863]: Reconfiguring daemon mon.b on smithi132 2023-07-21T22:22:40.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:40 smithi132 ceph-mon[97863]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:40.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:40.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:40.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:40 smithi107 ceph-mon[96543]: Reconfiguring mon.b (monmap changed)... 2023-07-21T22:22:40.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:40 smithi107 ceph-mon[96543]: Reconfiguring daemon mon.b on smithi132 2023-07-21T22:22:40.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:40 smithi107 ceph-mon[96543]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:40.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:40.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:41.639 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:41 smithi190 ceph-mon[98334]: Reconfiguring mon.c (monmap changed)... 2023-07-21T22:22:41.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:41 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:41.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:41 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:22:41.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:41 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:41.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:41 smithi190 ceph-mon[98334]: Reconfiguring daemon mon.c on smithi190 2023-07-21T22:22:41.680 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:41 smithi132 ceph-mon[97863]: Reconfiguring mon.c (monmap changed)... 2023-07-21T22:22:41.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:41 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:41.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:41 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:22:41.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:41 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:41.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:41 smithi132 ceph-mon[97863]: Reconfiguring daemon mon.c on smithi190 2023-07-21T22:22:41.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:41 smithi107 ceph-mon[96543]: Reconfiguring mon.c (monmap changed)... 2023-07-21T22:22:41.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:41 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:22:41.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:41 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:22:41.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:41 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:41.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:41 smithi107 ceph-mon[96543]: Reconfiguring daemon mon.c on smithi190 2023-07-21T22:22:41.690 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.476+0000 7efd034d7700 1 -- 172.21.15.190:0/2302787356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcfc0fe600 msgr2=0x7efcfc0fea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:41.690 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.476+0000 7efd034d7700 1 --2- 172.21.15.190:0/2302787356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcfc0fe600 0x7efcfc0fea10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efcec009910 tx=0x7efcec009c20 comp rx=0 tx=0).stop 2023-07-21T22:22:41.690 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd034d7700 1 -- 172.21.15.190:0/2302787356 shutdown_connections 2023-07-21T22:22:41.690 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd034d7700 1 --2- 172.21.15.190:0/2302787356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcfc0fe600 0x7efcfc0fea10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:41.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd034d7700 1 -- 172.21.15.190:0/2302787356 >> 172.21.15.190:0/2302787356 conn(0x7efcfc0f9bb0 msgr2=0x7efcfc0fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:41.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd034d7700 1 -- 172.21.15.190:0/2302787356 shutdown_connections 2023-07-21T22:22:41.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd034d7700 1 -- 172.21.15.190:0/2302787356 wait complete. 2023-07-21T22:22:41.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd034d7700 1 Processor -- start 2023-07-21T22:22:41.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd034d7700 1 -- start start 2023-07-21T22:22:41.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd034d7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcfc0fe600 0x7efcfc068780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:41.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd034d7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcfc197760 0x7efcfc199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:41.691 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd034d7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcfc19a080 0x7efcfc19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:41.692 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd034d7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19cc80 con 0x7efcfc197760 2023-07-21T22:22:41.692 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd034d7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19cdf0 con 0x7efcfc19a080 2023-07-21T22:22:41.692 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd034d7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19cfb0 con 0x7efcfc0fe600 2023-07-21T22:22:41.692 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.477+0000 7efd01273700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcfc0fe600 0x7efcfc068780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:41.692 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efd01273700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcfc0fe600 0x7efcfc068780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:47930/0 (socket says 172.21.15.190:47930) 2023-07-21T22:22:41.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efd01273700 1 -- 172.21.15.190:0/3266269942 learned_addr learned my addr 172.21.15.190:0/3266269942 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-07-21T22:22:41.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efd00a72700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcfc197760 0x7efcfc199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:41.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efd01a74700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcfc19a080 0x7efcfc19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:41.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efd01273700 1 -- 172.21.15.190:0/3266269942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcfc19a080 msgr2=0x7efcfc19c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:41.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efd01273700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcfc19a080 0x7efcfc19c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:41.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efd01273700 1 -- 172.21.15.190:0/3266269942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcfc197760 msgr2=0x7efcfc199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:41.693 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efd01273700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcfc197760 0x7efcfc199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:41.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efd01273700 1 -- 172.21.15.190:0/3266269942 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec0095f0 con 0x7efcfc0fe600 2023-07-21T22:22:41.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efd01273700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcfc0fe600 0x7efcfc068780 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efcec000c00 tx=0x7efcec004cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:41.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efcf27fc700 1 -- 172.21.15.190:0/3266269942 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec01d070 con 0x7efcfc0fe600 2023-07-21T22:22:41.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efd034d7700 1 -- 172.21.15.190:0/3266269942 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc19d230 con 0x7efcfc0fe600 2023-07-21T22:22:41.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efcf27fc700 1 -- 172.21.15.190:0/3266269942 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcec00be20 con 0x7efcfc0fe600 2023-07-21T22:22:41.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efcf27fc700 1 -- 172.21.15.190:0/3266269942 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec00f460 con 0x7efcfc0fe600 2023-07-21T22:22:41.694 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.478+0000 7efd034d7700 1 -- 172.21.15.190:0/3266269942 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc19d750 con 0x7efcfc0fe600 2023-07-21T22:22:41.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.479+0000 7efcf27fc700 1 -- 172.21.15.190:0/3266269942 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7efcec00f5c0 con 0x7efcfc0fe600 2023-07-21T22:22:41.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.479+0000 7efcf27fc700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efce8038c30 0x7efce803b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:41.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.479+0000 7efcf27fc700 1 -- 172.21.15.190:0/3266269942 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efcec04c900 con 0x7efcfc0fe600 2023-07-21T22:22:41.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.479+0000 7efd034d7700 1 -- 172.21.15.190:0/3266269942 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc18d180 con 0x7efcfc0fe600 2023-07-21T22:22:41.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.479+0000 7efd00a72700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efce8038c30 0x7efce803b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:41.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.480+0000 7efd00a72700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efce8038c30 0x7efce803b0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009ad0 tx=0x7efcf8009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:41.695 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.484+0000 7efcf27fc700 1 -- 172.21.15.190:0/3266269942 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7efcec027030 con 0x7efcfc0fe600 2023-07-21T22:22:41.730 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.685+0000 7efd034d7700 1 -- 172.21.15.190:0/3266269942 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi132=y;smithi190=z", "target": ["mon-mgr", ""]}) v1 -- 0x7efcfc0609e0 con 0x7efce8038c30 2023-07-21T22:22:41.737 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.692+0000 7efcf27fc700 1 -- 172.21.15.190:0/3266269942 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efcfc0609e0 con 0x7efce8038c30 2023-07-21T22:22:41.737 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mgr update... 2023-07-21T22:22:41.739 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.694+0000 7efd034d7700 1 -- 172.21.15.190:0/3266269942 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efce8038c30 msgr2=0x7efce803b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:41.739 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.694+0000 7efd034d7700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efce8038c30 0x7efce803b0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009ad0 tx=0x7efcf8009210 comp rx=0 tx=0).stop 2023-07-21T22:22:41.739 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.694+0000 7efd034d7700 1 -- 172.21.15.190:0/3266269942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcfc0fe600 msgr2=0x7efcfc068780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:41.740 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.694+0000 7efd034d7700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcfc0fe600 0x7efcfc068780 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efcec000c00 tx=0x7efcec004cb0 comp rx=0 tx=0).stop 2023-07-21T22:22:41.740 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.694+0000 7efd034d7700 1 -- 172.21.15.190:0/3266269942 shutdown_connections 2023-07-21T22:22:41.740 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.694+0000 7efd034d7700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcfc19a080 0x7efcfc19c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:41.741 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.694+0000 7efd034d7700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efce8038c30 0x7efce803b0e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:41.741 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.694+0000 7efd034d7700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcfc0fe600 0x7efcfc068780 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:41.741 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.694+0000 7efd034d7700 1 --2- 172.21.15.190:0/3266269942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcfc197760 0x7efcfc199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:41.742 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.694+0000 7efd034d7700 1 -- 172.21.15.190:0/3266269942 >> 172.21.15.190:0/3266269942 conn(0x7efcfc0f9bb0 msgr2=0x7efcfc0fa860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:41.742 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.694+0000 7efd034d7700 1 -- 172.21.15.190:0/3266269942 shutdown_connections 2023-07-21T22:22:41.742 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:22:41.694+0000 7efd034d7700 1 -- 172.21.15.190:0/3266269942 wait complete. 2023-07-21T22:22:42.155 DEBUG:teuthology.orchestra.run.smithi132:mgr.y> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.y.service 2023-07-21T22:22:42.159 DEBUG:teuthology.orchestra.run.smithi190:mgr.z> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.z.service 2023-07-21T22:22:42.162 INFO:tasks.cephadm:Deploying OSDs... 2023-07-21T22:22:42.163 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:22:42.163 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T22:22:42.184 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T22:22:42.184 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2023-07-21T22:22:42.191 INFO:journalctl@ceph.mgr.z.smithi190.stdout:-- Logs begin at Fri 2023-07-21 22:03:29 UTC. -- 2023-07-21T22:22:42.192 INFO:journalctl@ceph.mgr.y.smithi132.stdout:-- Logs begin at Fri 2023-07-21 22:03:19 UTC. -- 2023-07-21T22:22:42.244 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T22:22:42.245 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:22:42.245 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 6h/6d Inode: 378278 Links: 1 2023-07-21T22:22:42.245 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:22:42.245 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:22:42.245 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2023-07-21 22:21:55.668738422 +0000 2023-07-21T22:22:42.245 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2023-07-21 22:15:13.804851809 +0000 2023-07-21T22:22:42.246 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2023-07-21 22:15:13.804851809 +0000 2023-07-21T22:22:42.246 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2023-07-21T22:22:42.246 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T22:22:42.315 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2023-07-21T22:22:42.316 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2023-07-21T22:22:42.316 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.00027039 s, 1.9 MB/s 2023-07-21T22:22:42.317 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T22:22:42.377 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2023-07-21T22:22:42.437 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T22:22:42.437 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:22:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 6h/6d Inode: 371677 Links: 1 2023-07-21T22:22:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:22:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:22:42.437 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2023-07-21 22:21:55.668738422 +0000 2023-07-21T22:22:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2023-07-21 22:15:14.315843288 +0000 2023-07-21T22:22:42.438 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2023-07-21 22:15:14.315843288 +0000 2023-07-21T22:22:42.438 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2023-07-21T22:22:42.438 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T22:22:42.505 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2023-07-21T22:22:42.505 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2023-07-21T22:22:42.506 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000253314 s, 2.0 MB/s 2023-07-21T22:22:42.506 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T22:22:42.565 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2023-07-21T22:22:42.623 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T22:22:42.623 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:22:42.623 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 6h/6d Inode: 378436 Links: 1 2023-07-21T22:22:42.623 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:22:42.623 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:22:42.624 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2023-07-21 22:21:55.668738422 +0000 2023-07-21T22:22:42.624 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2023-07-21 22:15:14.847834418 +0000 2023-07-21T22:22:42.624 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2023-07-21 22:15:14.847834418 +0000 2023-07-21T22:22:42.624 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2023-07-21T22:22:42.624 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T22:22:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2023-07-21T22:22:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2023-07-21T22:22:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000230848 s, 2.2 MB/s 2023-07-21T22:22:42.690 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T22:22:42.749 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2023-07-21T22:22:42.800 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi132=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:42.800 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: Saving service mgr spec with placement smithi107=x;smithi132=y;smithi190=z;count:3 2023-07-21T22:22:42.800 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:42.801 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:42.801 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:42.801 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:42.801 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:42.802 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:42.802 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:42.802 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:42.802 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:22:42.802 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:22:42.803 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:22:42.803 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:42.803 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:42 smithi190 ceph-mon[98334]: Deploying daemon mgr.z on smithi190 2023-07-21T22:22:42.806 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T22:22:42.807 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:22:42.807 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 6h/6d Inode: 384149 Links: 1 2023-07-21T22:22:42.807 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:22:42.807 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:22:42.807 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2023-07-21 22:21:55.668738422 +0000 2023-07-21T22:22:42.808 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2023-07-21 22:15:15.349826048 +0000 2023-07-21T22:22:42.808 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2023-07-21 22:15:15.349826048 +0000 2023-07-21T22:22:42.808 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2023-07-21T22:22:42.808 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T22:22:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2023-07-21T22:22:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2023-07-21T22:22:42.872 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000218382 s, 2.3 MB/s 2023-07-21T22:22:42.873 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T22:22:42.931 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-07-21T22:22:42.931 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T22:22:42.952 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T22:22:42.952 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2023-07-21T22:22:43.011 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T22:22:43.012 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:22:43.012 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 377398 Links: 1 2023-07-21T22:22:43.012 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:22:43.012 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:22:43.012 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-07-21 22:22:29.792771936 +0000 2023-07-21T22:22:43.012 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-07-21 22:15:28.600879405 +0000 2023-07-21T22:22:43.013 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-07-21 22:15:28.600879405 +0000 2023-07-21T22:22:43.013 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-07-21T22:22:43.013 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T22:22:43.078 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi132=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:43.078 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: Saving service mgr spec with placement smithi107=x;smithi132=y;smithi190=z;count:3 2023-07-21T22:22:43.079 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:43.079 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:43.079 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:43.079 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:43.079 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:43.079 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:43.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:43.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:43.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:22:43.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:22:43.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:22:43.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:43.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:42 smithi132 ceph-mon[97863]: Deploying daemon mgr.z on smithi190 2023-07-21T22:22:43.081 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-07-21T22:22:43.082 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-07-21T22:22:43.082 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000248193 s, 2.1 MB/s 2023-07-21T22:22:43.082 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T22:22:43.141 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2023-07-21T22:22:43.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi132=y;smithi190=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:43.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: Saving service mgr spec with placement smithi107=x;smithi132=y;smithi190=z;count:3 2023-07-21T22:22:43.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:43.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:43.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:43.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:43.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:43.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:43.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:43.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:43.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:22:43.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:22:43.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:22:43.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:43.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:42 smithi107 ceph-mon[96543]: Deploying daemon mgr.z on smithi190 2023-07-21T22:22:43.199 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T22:22:43.199 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:22:43.200 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 377446 Links: 1 2023-07-21T22:22:43.200 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:22:43.200 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:22:43.200 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-07-21 22:22:29.792771936 +0000 2023-07-21T22:22:43.200 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-07-21 22:15:29.121866245 +0000 2023-07-21T22:22:43.200 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-07-21 22:15:29.121866245 +0000 2023-07-21T22:22:43.200 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-07-21T22:22:43.201 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T22:22:43.266 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-07-21T22:22:43.267 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-07-21T22:22:43.267 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000230458 s, 2.2 MB/s 2023-07-21T22:22:43.268 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T22:22:43.328 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2023-07-21T22:22:43.385 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T22:22:43.386 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:22:43.386 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 379697 Links: 1 2023-07-21T22:22:43.386 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:22:43.386 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:22:43.386 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-07-21 22:22:29.792771936 +0000 2023-07-21T22:22:43.386 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-07-21 22:15:29.627853465 +0000 2023-07-21T22:22:43.386 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-07-21 22:15:29.627853465 +0000 2023-07-21T22:22:43.387 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-07-21T22:22:43.387 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T22:22:43.389 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:43 smithi190 systemd[1]: Starting Ceph mgr.z for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:22:43.452 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-07-21T22:22:43.452 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-07-21T22:22:43.452 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.00024312 s, 2.1 MB/s 2023-07-21T22:22:43.453 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T22:22:43.512 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2023-07-21T22:22:43.569 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T22:22:43.569 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:22:43.569 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 384104 Links: 1 2023-07-21T22:22:43.570 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:22:43.570 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:22:43.570 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-07-21 22:22:29.793771911 +0000 2023-07-21T22:22:43.570 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-07-21 22:15:30.138840558 +0000 2023-07-21T22:22:43.570 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-07-21 22:15:30.138840558 +0000 2023-07-21T22:22:43.570 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-07-21T22:22:43.571 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T22:22:43.634 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-07-21T22:22:43.634 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-07-21T22:22:43.635 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000231165 s, 2.2 MB/s 2023-07-21T22:22:43.635 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T22:22:43.693 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-07-21T22:22:43.694 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T22:22:43.711 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T22:22:43.712 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2023-07-21T22:22:43.769 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T22:22:43.769 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:22:43.770 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 6h/6d Inode: 370507 Links: 1 2023-07-21T22:22:43.770 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:22:43.770 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:22:43.770 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-07-21 22:22:43.261247131 +0000 2023-07-21T22:22:43.770 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-07-21 22:15:49.815578116 +0000 2023-07-21T22:22:43.770 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-07-21 22:15:49.815578116 +0000 2023-07-21T22:22:43.770 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-07-21T22:22:43.771 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T22:22:43.839 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-07-21T22:22:43.840 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-07-21T22:22:43.840 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.00024218 s, 2.1 MB/s 2023-07-21T22:22:43.841 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T22:22:43.889 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:43 smithi190 podman[100173]: 2023-07-21T22:22:43.898 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2023-07-21T22:22:43.956 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T22:22:43.956 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:22:43.956 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 6h/6d Inode: 370574 Links: 1 2023-07-21T22:22:43.957 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:22:43.957 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:22:43.957 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-07-21 22:22:43.261247131 +0000 2023-07-21T22:22:43.957 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-07-21 22:15:50.332563898 +0000 2023-07-21T22:22:43.957 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-07-21 22:15:50.332563898 +0000 2023-07-21T22:22:43.957 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-07-21T22:22:43.958 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T22:22:44.026 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-07-21T22:22:44.026 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-07-21T22:22:44.026 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000229502 s, 2.2 MB/s 2023-07-21T22:22:44.027 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T22:22:44.086 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2023-07-21T22:22:44.143 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T22:22:44.143 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:22:44.143 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 6h/6d Inode: 382030 Links: 1 2023-07-21T22:22:44.144 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:22:44.144 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:22:44.144 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-07-21 22:22:43.261247131 +0000 2023-07-21T22:22:44.144 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-07-21 22:15:50.851549625 +0000 2023-07-21T22:22:44.144 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-07-21 22:15:50.851549625 +0000 2023-07-21T22:22:44.144 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-07-21T22:22:44.145 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T22:22:44.206 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:43 smithi190 bash[100173]: ebf3befdbd6c825c7b0d261f81f538fd0ace54c1ff2894f5311dc8d1c327a1e3 2023-07-21T22:22:44.206 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:43 smithi190 ceph-mgr[100273]: -- 172.21.15.190:0/2173052493 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f7bc4a11e0 con 0x55f7bc477000 2023-07-21T22:22:44.207 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:43 smithi190 systemd[1]: Started Ceph mgr.z for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:22:44.207 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:43 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:43.996+0000 7f747bc28200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:22:44.207 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:44.080+0000 7f747bc28200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:22:44.208 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-07-21T22:22:44.208 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-07-21T22:22:44.209 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000218832 s, 2.3 MB/s 2023-07-21T22:22:44.209 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T22:22:44.267 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2023-07-21T22:22:44.323 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T22:22:44.323 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:22:44.323 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 6h/6d Inode: 378648 Links: 1 2023-07-21T22:22:44.323 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:22:44.323 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:22:44.324 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-07-21 22:22:43.261247131 +0000 2023-07-21T22:22:44.324 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-07-21 22:15:51.343536094 +0000 2023-07-21T22:22:44.324 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-07-21 22:15:51.343536094 +0000 2023-07-21T22:22:44.324 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: - 2023-07-21T22:22:44.324 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T22:22:44.388 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-07-21T22:22:44.388 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-07-21T22:22:44.388 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000204522 s, 2.5 MB/s 2023-07-21T22:22:44.389 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T22:22:44.446 INFO:tasks.cephadm:Deploying osd.0 on smithi107 with /dev/vg_nvme/lv_4... 2023-07-21T22:22:44.447 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T22:22:44.487 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:44.172+0000 7f747bc28200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:22:44.488 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:44.267+0000 7f747bc28200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:22:44.793 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:44.442+0000 7f747bc28200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:22:45.008 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:22:45.102 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:44.748+0000 7f747bc28200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:22:45.102 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:44.835+0000 7f747bc28200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:22:45.102 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:44.963+0000 7f747bc28200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:22:45.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-mon[98334]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:45.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:45.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:45.103 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:45.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:22:45.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:22:45.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:22:45.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:45.104 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:44 smithi190 ceph-mon[98334]: Deploying daemon mgr.y on smithi132 2023-07-21T22:22:45.263 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:44 smithi132 ceph-mon[97863]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:45.263 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:44 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:45.263 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:44 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:45.263 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:44 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:45.263 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:44 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:22:45.263 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:44 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:22:45.264 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:44 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:22:45.264 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:44 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:45.264 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:44 smithi132 ceph-mon[97863]: Deploying daemon mgr.y on smithi132 2023-07-21T22:22:45.353 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:44 smithi107 ceph-mon[96543]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:45.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:44 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:45.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:44 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:45.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:44 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:45.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:44 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:22:45.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:44 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:22:45.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:44 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:22:45.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:44 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:45.355 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:44 smithi107 ceph-mon[96543]: Deploying daemon mgr.y on smithi132 2023-07-21T22:22:45.389 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:45 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:45.057+0000 7f747bc28200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:22:45.681 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:45 smithi132 systemd[1]: Starting Ceph mgr.y for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:22:45.889 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:45 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:45.474+0000 7f747bc28200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:22:46.073 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:45 smithi132 podman[99297]: 2023-07-21T22:22:46.073 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:45 smithi132 bash[99297]: 186e0c403623778878d6ab9ab8237a1d0a40ece89971375d95d3bc7d9638e72e 2023-07-21T22:22:46.074 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:45 smithi132 ceph-mgr[99338]: -- 172.21.15.132:0/938745 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e1f832f1e0 con 0x55e1f8304800 2023-07-21T22:22:46.389 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:46 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:46.042+0000 7f747bc28200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:22:46.390 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:46 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:46.214+0000 7f747bc28200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:22:46.419 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:46 smithi132 systemd[1]: Started Ceph mgr.y for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:22:46.419 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:46.103+0000 7fe6f2436200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:22:46.419 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:46.183+0000 7fe6f2436200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:22:46.419 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:46.275+0000 7fe6f2436200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:22:46.419 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:46.373+0000 7fe6f2436200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:22:46.471 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T22:22:46.471 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T22:22:46.471 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2023-07-21T22:22:46.471 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2023-07-21T22:22:46.471 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0136755 s, 767 MB/s 2023-07-21T22:22:46.472 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2023-07-21T22:22:46.680 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:46.548+0000 7fe6f2436200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:22:46.864 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2023-07-21T22:22:47.180 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:47 smithi132 ceph-mon[97863]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:47.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:47.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:47.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:47.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:47.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:47.182 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:46.874+0000 7fe6f2436200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:22:47.182 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:46.967+0000 7fe6f2436200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:22:47.182 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:47 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:47.104+0000 7fe6f2436200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:22:47.343 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:47 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:47.021+0000 7f747bc28200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:22:47.343 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:47 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:47.115+0000 7f747bc28200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:22:47.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:47 smithi190 ceph-mon[98334]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:47.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:47.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:47.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:47.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:47.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:47.413 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:22:47.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:47 smithi107 ceph-mon[96543]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:47.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:47.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:47.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:47.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:47.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:47.636 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:47 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:47.298+0000 7f747bc28200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:22:47.680 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:47 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:47.212+0000 7fe6f2436200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:22:47.889 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:47 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:47.591+0000 7f747bc28200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:22:47.890 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:47 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:47.822+0000 7f747bc28200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:22:48.180 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:47 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:47.664+0000 7fe6f2436200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:22:48.277 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:47 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:47.973+0000 7f747bc28200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:22:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.584+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/1733460119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28100610 msgr2=0x7fdc28106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.584+0000 7fdc2d0bb700 1 --2- 172.21.15.107:0/1733460119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28100610 0x7fdc28106ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009990 tx=0x7fdc20009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.584+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/1733460119 shutdown_connections 2023-07-21T22:22:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.584+0000 7fdc2d0bb700 1 --2- 172.21.15.107:0/1733460119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28100610 0x7fdc28106ed0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.584+0000 7fdc2d0bb700 1 --2- 172.21.15.107:0/1733460119 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc280fea80 0x7fdc280fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.584+0000 7fdc2d0bb700 1 --2- 172.21.15.107:0/1733460119 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc280ffc80 0x7fdc281000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:48.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.584+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/1733460119 >> 172.21.15.107:0/1733460119 conn(0x7fdc280fa0b0 msgr2=0x7fdc280fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:48.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.584+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/1733460119 shutdown_connections 2023-07-21T22:22:48.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.584+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/1733460119 wait complete. 2023-07-21T22:22:48.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.585+0000 7fdc2d0bb700 1 Processor -- start 2023-07-21T22:22:48.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.585+0000 7fdc2d0bb700 1 -- start start 2023-07-21T22:22:48.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.585+0000 7fdc2d0bb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc280fea80 0x7fdc28068b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:48.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.585+0000 7fdc2d0bb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc280ffc80 0x7fdc2819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:48.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.585+0000 7fdc2d0bb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc28100610 0x7fdc2819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:48.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.585+0000 7fdc2d0bb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2819d330 con 0x7fdc280fea80 2023-07-21T22:22:48.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.585+0000 7fdc2d0bb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2819d4a0 con 0x7fdc280ffc80 2023-07-21T22:22:48.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.585+0000 7fdc2d0bb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2819d660 con 0x7fdc28100610 2023-07-21T22:22:48.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.585+0000 7fdc26d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc280fea80 0x7fdc28068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:48.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.585+0000 7fdc26d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc280fea80 0x7fdc28068b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45536/0 (socket says 172.21.15.107:45536) 2023-07-21T22:22:48.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.585+0000 7fdc26d9d700 1 -- 172.21.15.107:0/2519390066 learned_addr learned my addr 172.21.15.107:0/2519390066 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:22:48.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.585+0000 7fdc2759e700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc28100610 0x7fdc2819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:48.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.585+0000 7fdc2659c700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc280ffc80 0x7fdc2819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:48.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.586+0000 7fdc26d9d700 1 -- 172.21.15.107:0/2519390066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc28100610 msgr2=0x7fdc2819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:48.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.586+0000 7fdc26d9d700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc28100610 0x7fdc2819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:48.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.586+0000 7fdc26d9d700 1 -- 172.21.15.107:0/2519390066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc280ffc80 msgr2=0x7fdc2819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:48.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.586+0000 7fdc26d9d700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc280ffc80 0x7fdc2819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:48.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.586+0000 7fdc26d9d700 1 -- 172.21.15.107:0/2519390066 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc200095f0 con 0x7fdc280fea80 2023-07-21T22:22:48.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.586+0000 7fdc26d9d700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc280fea80 0x7fdc28068b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdc18007ee0 tx=0x7fdc1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:48.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.586+0000 7fdc1ffff700 1 -- 172.21.15.107:0/2519390066 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800dd20 con 0x7fdc280fea80 2023-07-21T22:22:48.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.586+0000 7fdc1ffff700 1 -- 172.21.15.107:0/2519390066 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc1800f040 con 0x7fdc280fea80 2023-07-21T22:22:48.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.586+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/2519390066 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2819d940 con 0x7fdc280fea80 2023-07-21T22:22:48.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.586+0000 7fdc1ffff700 1 -- 172.21.15.107:0/2519390066 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180145e0 con 0x7fdc280fea80 2023-07-21T22:22:48.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.586+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/2519390066 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2819de40 con 0x7fdc280fea80 2023-07-21T22:22:48.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.587+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/2519390066 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc28106020 con 0x7fdc280fea80 2023-07-21T22:22:48.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.587+0000 7fdc1ffff700 1 -- 172.21.15.107:0/2519390066 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fdc18015400 con 0x7fdc280fea80 2023-07-21T22:22:48.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.588+0000 7fdc1ffff700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdc140389e0 0x7fdc1403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:48.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.588+0000 7fdc1ffff700 1 -- 172.21.15.107:0/2519390066 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdc1804b290 con 0x7fdc280fea80 2023-07-21T22:22:48.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.588+0000 7fdc2659c700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdc140389e0 0x7fdc1403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:48.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.588+0000 7fdc2659c700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdc140389e0 0x7fdc1403ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc280690b0 tx=0x7fdc1000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:48.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.592+0000 7fdc1ffff700 1 -- 172.21.15.107:0/2519390066 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdc18012db0 con 0x7fdc280fea80 2023-07-21T22:22:48.639 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:48.232+0000 7f747bc28200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:22:48.680 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:48.243+0000 7fe6f2436200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:22:48.680 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:48.413+0000 7fe6f2436200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:22:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:48.800+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/2519390066 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc28060950 con 0x7fdc140389e0 2023-07-21T22:22:48.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:48 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:48.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:48 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:48.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:48 smithi107 ceph-mon[96543]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:48.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:48 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:48.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:48 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:48.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:48 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:48.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:48 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:48.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:48 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:48.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:48 smithi107 ceph-mon[96543]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T22:22:48.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:48 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:22:48.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:48 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:22:48.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:48 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:48.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:48 smithi107 ceph-mon[96543]: Reconfiguring daemon mgr.x on smithi107 2023-07-21T22:22:48.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:48.980 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:48.981 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-mon[98334]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:48.981 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:48.981 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:48.981 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:48.981 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:48.981 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:48.981 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-mon[98334]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T22:22:48.982 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:22:48.982 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:22:48.982 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:48.982 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-mon[98334]: Reconfiguring daemon mgr.x on smithi107 2023-07-21T22:22:49.180 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:49.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:49.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-mon[97863]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:49.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:49.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:49.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:49.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:49.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:49.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-mon[97863]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T22:22:49.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:22:49.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:22:49.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:49.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:48 smithi132 ceph-mon[97863]: Reconfiguring daemon mgr.x on smithi107 2023-07-21T22:22:49.300 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:48 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:48.935+0000 7f747bc28200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:22:49.300 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:49 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:49.065+0000 7f747bc28200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:22:49.474 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:49 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:49.170+0000 7fe6f2436200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:22:49.475 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:49 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:49.259+0000 7fe6f2436200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:22:49.626 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:49 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:49.255+0000 7f747bc28200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:22:49.627 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:49 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:49.368+0000 7f747bc28200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:22:49.751 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:49 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:49.429+0000 7fe6f2436200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:22:49.751 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:49 smithi132 ceph-mon[97863]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:49.751 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:49 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:22:49.752 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:49 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:22:49.752 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:49 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:49.752 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:49 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:49.752 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:49 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:49.752 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:49 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:49.889 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:49 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:49.581+0000 7f747bc28200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:22:49.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:49 smithi190 ceph-mon[98334]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:49.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:49 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:22:49.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:49 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:22:49.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:49 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:49.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:49 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:49.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:49 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:49.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:49 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:50.027 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:49 smithi107 ceph-mon[96543]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:22:50.027 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:49 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:22:50.027 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:49 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:22:50.027 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:49 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:50.027 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:49 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:50.028 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:49 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:50.028 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:49 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:50.125 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:49 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:49.705+0000 7fe6f2436200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:22:50.126 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:49 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:49.935+0000 7fe6f2436200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:22:50.383 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:50 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:50.080+0000 7fe6f2436200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:22:50.680 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:50 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:50.338+0000 7fe6f2436200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:22:50.839 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:50.793+0000 7fdc1ffff700 1 -- 172.21.15.107:0/2519390066 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 14) v1 ==== 90600+0+0 (secure 0 0 0) 0x7fdc18010d10 con 0x7fdc280fea80 2023-07-21T22:22:50.890 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:22:50 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z[100190]: 2023-07-21T22:22:50.434+0000 7f747bc28200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:22:50.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:50 smithi190 ceph-mon[98334]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:50.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:50.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:50.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:50.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:50 smithi190 ceph-mon[98334]: Standby manager daemon z started 2023-07-21T22:22:50.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:50 smithi190 ceph-mon[98334]: from='mgr.? 172.21.15.190:0/3126153434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:22:50.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:50 smithi190 ceph-mon[98334]: from='mgr.? 172.21.15.190:0/3126153434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:22:50.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:50 smithi190 ceph-mon[98334]: from='mgr.? 172.21.15.190:0/3126153434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:22:50.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:50 smithi190 ceph-mon[98334]: from='mgr.? 172.21.15.190:0/3126153434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:22:50.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:50 smithi107 ceph-mon[96543]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:50.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:50.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:50.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:50.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:50 smithi107 ceph-mon[96543]: Standby manager daemon z started 2023-07-21T22:22:50.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:50 smithi107 ceph-mon[96543]: from='mgr.? 172.21.15.190:0/3126153434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:22:50.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:50 smithi107 ceph-mon[96543]: from='mgr.? 172.21.15.190:0/3126153434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:22:50.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:50 smithi107 ceph-mon[96543]: from='mgr.? 172.21.15.190:0/3126153434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:22:50.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:50 smithi107 ceph-mon[96543]: from='mgr.? 172.21.15.190:0/3126153434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:22:51.080 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:50 smithi132 ceph-mon[97863]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:51.081 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:51.081 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:51.081 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:51.081 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:50 smithi132 ceph-mon[97863]: Standby manager daemon z started 2023-07-21T22:22:51.081 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:50 smithi132 ceph-mon[97863]: from='mgr.? 172.21.15.190:0/3126153434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:22:51.082 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:50 smithi132 ceph-mon[97863]: from='mgr.? 172.21.15.190:0/3126153434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:22:51.082 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:50 smithi132 ceph-mon[97863]: from='mgr.? 172.21.15.190:0/3126153434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:22:51.082 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:50 smithi132 ceph-mon[97863]: from='mgr.? 172.21.15.190:0/3126153434' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:22:51.397 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:51 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:51.035+0000 7fe6f2436200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:22:51.397 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:51 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:51.164+0000 7fe6f2436200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:22:51.680 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:51 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:51.352+0000 7fe6f2436200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:22:51.681 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:51 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:51.466+0000 7fe6f2436200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:22:52.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:51 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:52.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:51 smithi190 ceph-mon[98334]: mgrmap e14: x(active, since 65s), standbys: z 2023-07-21T22:22:52.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:51 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:22:52.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:51 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3069926714' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc595a3a-edf0-47ac-b533-25c80518c634"}]: dispatch 2023-07-21T22:22:52.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:51 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3069926714' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc595a3a-edf0-47ac-b533-25c80518c634"}]': finished 2023-07-21T22:22:52.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:51 smithi190 ceph-mon[98334]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T22:22:52.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:51 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:22:52.180 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:51 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:51.680+0000 7fe6f2436200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:22:52.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:51 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:52.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:51 smithi132 ceph-mon[97863]: mgrmap e14: x(active, since 65s), standbys: z 2023-07-21T22:22:52.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:51 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:22:52.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:51 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3069926714' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc595a3a-edf0-47ac-b533-25c80518c634"}]: dispatch 2023-07-21T22:22:52.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:51 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3069926714' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc595a3a-edf0-47ac-b533-25c80518c634"}]': finished 2023-07-21T22:22:52.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:51 smithi132 ceph-mon[97863]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T22:22:52.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:51 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:22:52.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:51 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:22:52.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:51 smithi107 ceph-mon[96543]: mgrmap e14: x(active, since 65s), standbys: z 2023-07-21T22:22:52.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:51 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:22:52.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:51 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3069926714' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc595a3a-edf0-47ac-b533-25c80518c634"}]: dispatch 2023-07-21T22:22:52.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:51 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3069926714' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc595a3a-edf0-47ac-b533-25c80518c634"}]': finished 2023-07-21T22:22:52.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:51 smithi107 ceph-mon[96543]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T22:22:52.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:51 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:22:52.930 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:22:52 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y[99313]: 2023-07-21T22:22:52.530+0000 7fe6f2436200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:22:52.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:52 smithi132 ceph-mon[97863]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:52.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:52 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2729080196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:22:52.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:52 smithi132 ceph-mon[97863]: Standby manager daemon y started 2023-07-21T22:22:52.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:52 smithi132 ceph-mon[97863]: from='mgr.? 172.21.15.132:0/4166456230' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:22:52.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:52 smithi132 ceph-mon[97863]: from='mgr.? 172.21.15.132:0/4166456230' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:22:52.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:52 smithi132 ceph-mon[97863]: from='mgr.? 172.21.15.132:0/4166456230' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:22:52.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:52 smithi132 ceph-mon[97863]: from='mgr.? 172.21.15.132:0/4166456230' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:22:53.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:52 smithi190 ceph-mon[98334]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:53.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:52 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2729080196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:22:53.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:52 smithi190 ceph-mon[98334]: Standby manager daemon y started 2023-07-21T22:22:53.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:52 smithi190 ceph-mon[98334]: from='mgr.? 172.21.15.132:0/4166456230' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:22:53.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:52 smithi190 ceph-mon[98334]: from='mgr.? 172.21.15.132:0/4166456230' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:22:53.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:52 smithi190 ceph-mon[98334]: from='mgr.? 172.21.15.132:0/4166456230' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:22:53.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:52 smithi190 ceph-mon[98334]: from='mgr.? 172.21.15.132:0/4166456230' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:22:53.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:52 smithi107 ceph-mon[96543]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:53.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:52 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2729080196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:22:53.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:52 smithi107 ceph-mon[96543]: Standby manager daemon y started 2023-07-21T22:22:53.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:52 smithi107 ceph-mon[96543]: from='mgr.? 172.21.15.132:0/4166456230' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:22:53.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:52 smithi107 ceph-mon[96543]: from='mgr.? 172.21.15.132:0/4166456230' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:22:53.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:52 smithi107 ceph-mon[96543]: from='mgr.? 172.21.15.132:0/4166456230' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:22:53.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:52 smithi107 ceph-mon[96543]: from='mgr.? 172.21.15.132:0/4166456230' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:22:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:22:53.265+0000 7fdc1ffff700 1 -- 172.21.15.107:0/2519390066 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7fdc180156b0 con 0x7fdc280fea80 2023-07-21T22:22:54.639 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:54 smithi190 ceph-mon[98334]: mgrmap e15: x(active, since 67s), standbys: z, y 2023-07-21T22:22:54.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:54 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:22:54.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:54 smithi190 ceph-mon[98334]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:54.680 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:54 smithi132 ceph-mon[97863]: mgrmap e15: x(active, since 67s), standbys: z, y 2023-07-21T22:22:54.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:54 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:22:54.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:54 smithi132 ceph-mon[97863]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:54.686 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:54 smithi107 ceph-mon[96543]: mgrmap e15: x(active, since 67s), standbys: z, y 2023-07-21T22:22:54.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:54 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:22:54.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:54 smithi107 ceph-mon[96543]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:57.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:56 smithi190 ceph-mon[98334]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:57.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:56 smithi132 ceph-mon[97863]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:57.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:56 smithi107 ceph-mon[96543]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:58.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:57 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T22:22:58.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:57 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:58.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:57 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T22:22:58.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:57 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:58.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:57 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T22:22:58.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:57 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:59.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:58 smithi107 ceph-mon[96543]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:59.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:22:58 smithi107 ceph-mon[96543]: Deploying daemon osd.0 on smithi107 2023-07-21T22:22:59.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:58 smithi190 ceph-mon[98334]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:59.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:22:58 smithi190 ceph-mon[98334]: Deploying daemon osd.0 on smithi107 2023-07-21T22:22:59.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:58 smithi132 ceph-mon[97863]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:22:59.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:22:58 smithi132 ceph-mon[97863]: Deploying daemon osd.0 on smithi107 2023-07-21T22:23:01.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:00 smithi107 ceph-mon[96543]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:01.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:00 smithi190 ceph-mon[98334]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:01.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:00 smithi132 ceph-mon[97863]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:03.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:02 smithi107 ceph-mon[96543]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:03.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:02 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:03.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:02 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:03.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:02 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:03.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:02 smithi190 ceph-mon[98334]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:03.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:02 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:03.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:02 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:03.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:02 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:03.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:02 smithi132 ceph-mon[97863]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:03.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:02 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:03.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:02 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:03.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:02 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:04.949 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 0 on host 'smithi107' 2023-07-21T22:23:04.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:04.903+0000 7fdc1ffff700 1 -- 172.21.15.107:0/2519390066 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdc28060950 con 0x7fdc140389e0 2023-07-21T22:23:04.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:04.906+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/2519390066 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdc140389e0 msgr2=0x7fdc1403ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:04.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:04.906+0000 7fdc2d0bb700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdc140389e0 0x7fdc1403ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc280690b0 tx=0x7fdc1000b410 comp rx=0 tx=0).stop 2023-07-21T22:23:04.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:04.906+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/2519390066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc280fea80 msgr2=0x7fdc28068b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:04.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:04.906+0000 7fdc2d0bb700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc280fea80 0x7fdc28068b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdc18007ee0 tx=0x7fdc1800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:04.906+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/2519390066 shutdown_connections 2023-07-21T22:23:04.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:04.906+0000 7fdc2d0bb700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdc140389e0 0x7fdc1403ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:04.906+0000 7fdc2d0bb700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc28100610 0x7fdc2819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:04.906+0000 7fdc2d0bb700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc280fea80 0x7fdc28068b30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:04.906+0000 7fdc2d0bb700 1 --2- 172.21.15.107:0/2519390066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc280ffc80 0x7fdc2819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:04.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:04.906+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/2519390066 >> 172.21.15.107:0/2519390066 conn(0x7fdc280fa0b0 msgr2=0x7fdc28103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:04.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:04.906+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/2519390066 shutdown_connections 2023-07-21T22:23:04.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:04.906+0000 7fdc2d0bb700 1 -- 172.21.15.107:0/2519390066 wait complete. 2023-07-21T22:23:05.357 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:04 smithi107 ceph-mon[96543]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:05.358 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:04 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.358 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:04 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.358 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:04 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:05.358 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:04 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:05.358 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:04 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.358 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:04 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:05.358 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:04 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.359 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:04 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:04 smithi190 ceph-mon[98334]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:05.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:04 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:04 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:04 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:05.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:04 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:05.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:04 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:04 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:05.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:04 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:04 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.401 DEBUG:teuthology.orchestra.run.smithi107:osd.0> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.0.service 2023-07-21T22:23:05.403 INFO:tasks.cephadm:Deploying osd.1 on smithi132 with /dev/vg_nvme/lv_4... 2023-07-21T22:23:05.403 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T22:23:05.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:04 smithi132 ceph-mon[97863]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:05.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:04 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:04 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:04 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:05.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:04 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:05.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:04 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:04 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:05.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:04 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:04 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:05.434 INFO:journalctl@ceph.osd.0.smithi107.stdout:-- Logs begin at Fri 2023-07-21 22:03:40 UTC. -- 2023-07-21T22:23:05.963 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:23:06.671 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:23:06 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-0[109479]: 2023-07-21T22:23:06.219+0000 7f1d6898d640 -1 osd.0 0 log_to_monitors true 2023-07-21T22:23:07.033 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:06 smithi132 ceph-mon[97863]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:07.034 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:06 smithi132 ceph-mon[97863]: from='osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T22:23:07.365 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:06 smithi107 ceph-mon[96543]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:07.365 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:06 smithi107 ceph-mon[96543]: from='osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T22:23:07.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:06 smithi190 ceph-mon[98334]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:07.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:06 smithi190 ceph-mon[98334]: from='osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T22:23:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T22:23:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T22:23:07.471 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2023-07-21T22:23:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2023-07-21T22:23:07.471 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111306 s, 942 MB/s 2023-07-21T22:23:07.471 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2023-07-21T22:23:07.917 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi132:vg_nvme/lv_4 2023-07-21T22:23:08.180 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:07 smithi132 ceph-mon[97863]: from='osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T22:23:08.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:07 smithi132 ceph-mon[97863]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T22:23:08.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:07 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:08.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:07 smithi132 ceph-mon[97863]: from='osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2023-07-21T22:23:08.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:07 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:08.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:07 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:08.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:07 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:23:08.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:07 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:08.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:07 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:08.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:07 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:08.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:07 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:08.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:07 smithi190 ceph-mon[98334]: from='osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T22:23:08.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:07 smithi190 ceph-mon[98334]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T22:23:08.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:07 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:08.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:07 smithi190 ceph-mon[98334]: from='osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2023-07-21T22:23:08.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:07 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:08.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:07 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:08.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:07 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:23:08.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:07 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:08.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:07 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:08.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:07 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:08.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:07 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:08.437 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:23:07 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-0[109479]: 2023-07-21T22:23:07.965+0000 7f1d5d804700 -1 osd.0 0 waiting for initial osdmap 2023-07-21T22:23:08.437 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:23:07 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-0[109479]: 2023-07-21T22:23:07.971+0000 7f1d59dfa700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:23:08.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:07 smithi107 ceph-mon[96543]: from='osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T22:23:08.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:07 smithi107 ceph-mon[96543]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T22:23:08.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:07 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:08.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:07 smithi107 ceph-mon[96543]: from='osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2023-07-21T22:23:08.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:07 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:08.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:07 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:08.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:07 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:23:08.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:07 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:08.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:07 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:08.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:07 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:08.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:07 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:08.508 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:23:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:08 smithi190 ceph-mon[98334]: Detected new or changed devices on smithi107 2023-07-21T22:23:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:08 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi107 to 17168M 2023-07-21T22:23:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:08 smithi190 ceph-mon[98334]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:08 smithi190 ceph-mon[98334]: from='osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2023-07-21T22:23:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:08 smithi190 ceph-mon[98334]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T22:23:09.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:08 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:09.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:08 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:09.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:08 smithi132 ceph-mon[97863]: Detected new or changed devices on smithi107 2023-07-21T22:23:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:08 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi107 to 17168M 2023-07-21T22:23:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:08 smithi132 ceph-mon[97863]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:08 smithi132 ceph-mon[97863]: from='osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2023-07-21T22:23:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:08 smithi132 ceph-mon[97863]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T22:23:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:08 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:08 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:09.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:08 smithi107 ceph-mon[96543]: Detected new or changed devices on smithi107 2023-07-21T22:23:09.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:08 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi107 to 17168M 2023-07-21T22:23:09.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:08 smithi107 ceph-mon[96543]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:09.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:08 smithi107 ceph-mon[96543]: from='osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2023-07-21T22:23:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:08 smithi107 ceph-mon[96543]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T22:23:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:09.739 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.693+0000 7efcce219700 1 -- 172.21.15.132:0/4007686839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcc8100610 msgr2=0x7efcc8106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:09.739 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.693+0000 7efcce219700 1 --2- 172.21.15.132:0/4007686839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcc8100610 0x7efcc8106ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efcc000b0a0 tx=0x7efcc000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:23:09.739 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.693+0000 7efcce219700 1 -- 172.21.15.132:0/4007686839 shutdown_connections 2023-07-21T22:23:09.739 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.693+0000 7efcce219700 1 --2- 172.21.15.132:0/4007686839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcc8100610 0x7efcc8106ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:09.739 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.693+0000 7efcce219700 1 --2- 172.21.15.132:0/4007686839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcc80fea80 0x7efcc80fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:09.740 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.693+0000 7efcce219700 1 --2- 172.21.15.132:0/4007686839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcc80ffc80 0x7efcc81000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:09.740 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.693+0000 7efcce219700 1 -- 172.21.15.132:0/4007686839 >> 172.21.15.132:0/4007686839 conn(0x7efcc80fa0b0 msgr2=0x7efcc80fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:09.740 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.693+0000 7efcce219700 1 -- 172.21.15.132:0/4007686839 shutdown_connections 2023-07-21T22:23:09.740 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.694+0000 7efcce219700 1 -- 172.21.15.132:0/4007686839 wait complete. 2023-07-21T22:23:09.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.694+0000 7efcce219700 1 Processor -- start 2023-07-21T22:23:09.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.694+0000 7efcce219700 1 -- start start 2023-07-21T22:23:09.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.694+0000 7efcce219700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcc80fea80 0x7efcc8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:09.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.694+0000 7efcce219700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcc80ffc80 0x7efcc819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:09.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.694+0000 7efcce219700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcc8100610 0x7efcc819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:09.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.694+0000 7efcce219700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc819d480 con 0x7efcc8100610 2023-07-21T22:23:09.742 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.694+0000 7efcce219700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc819d5f0 con 0x7efcc80ffc80 2023-07-21T22:23:09.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.694+0000 7efcce219700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc819d7b0 con 0x7efcc80fea80 2023-07-21T22:23:09.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.694+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcc80ffc80 0x7efcc819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:09.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.694+0000 7efcc6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcc80ffc80 0x7efcc819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50460/0 (socket says 172.21.15.132:50460) 2023-07-21T22:23:09.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcc6ffd700 1 -- 172.21.15.132:0/4236068889 learned_addr learned my addr 172.21.15.132:0/4236068889 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-07-21T22:23:09.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcc77fe700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcc80fea80 0x7efcc8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:09.743 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcc7fff700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcc8100610 0x7efcc819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:09.744 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcc6ffd700 1 -- 172.21.15.132:0/4236068889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcc80fea80 msgr2=0x7efcc8068b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:09.744 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcc6ffd700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcc80fea80 0x7efcc8068b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:09.744 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcc6ffd700 1 -- 172.21.15.132:0/4236068889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcc8100610 msgr2=0x7efcc819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:09.744 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcc6ffd700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcc8100610 0x7efcc819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:09.744 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcc6ffd700 1 -- 172.21.15.132:0/4236068889 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc0009d00 con 0x7efcc80ffc80 2023-07-21T22:23:09.744 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcc77fe700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcc80fea80 0x7efcc8068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:23:09.744 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcc6ffd700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcc80ffc80 0x7efcc819a690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00e9a0 tx=0x7efcbc00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:09.744 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcc4ff9700 1 -- 172.21.15.132:0/4236068889 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc009790 con 0x7efcc80ffc80 2023-07-21T22:23:09.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcc4ff9700 1 -- 172.21.15.132:0/4236068889 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcbc00cbd0 con 0x7efcc80ffc80 2023-07-21T22:23:09.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcce219700 1 -- 172.21.15.132:0/4236068889 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc819da90 con 0x7efcc80ffc80 2023-07-21T22:23:09.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcc4ff9700 1 -- 172.21.15.132:0/4236068889 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc018680 con 0x7efcc80ffc80 2023-07-21T22:23:09.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.695+0000 7efcce219700 1 -- 172.21.15.132:0/4236068889 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc819dfe0 con 0x7efcc80ffc80 2023-07-21T22:23:09.745 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.696+0000 7efcce219700 1 -- 172.21.15.132:0/4236068889 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc8105fe0 con 0x7efcc80ffc80 2023-07-21T22:23:09.746 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.697+0000 7efcc4ff9700 1 -- 172.21.15.132:0/4236068889 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7efcbc00cd40 con 0x7efcc80ffc80 2023-07-21T22:23:09.746 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.698+0000 7efcc4ff9700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efcb00a1140 0x7efcb00a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:09.746 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.698+0000 7efcc4ff9700 1 -- 172.21.15.132:0/4236068889 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7efcbc014070 con 0x7efcc80ffc80 2023-07-21T22:23:09.746 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.698+0000 7efcc77fe700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efcb00a1140 0x7efcb00a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:09.746 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.698+0000 7efcc77fe700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efcb00a1140 0x7efcb00a35f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efcb8005950 tx=0x7efcb80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:09.747 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.702+0000 7efcc4ff9700 1 -- 172.21.15.132:0/4236068889 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7efcbc051770 con 0x7efcc80ffc80 2023-07-21T22:23:09.940 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:09.895+0000 7efcce219700 1 -- 172.21.15.132:0/4236068889 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7efcc8060950 con 0x7efcb00a1140 2023-07-21T22:23:10.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:09 smithi190 ceph-mon[98334]: purged_snaps scrub starts 2023-07-21T22:23:10.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:09 smithi190 ceph-mon[98334]: purged_snaps scrub ok 2023-07-21T22:23:10.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:09 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:10.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:09 smithi190 ceph-mon[98334]: osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012] boot 2023-07-21T22:23:10.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:09 smithi190 ceph-mon[98334]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T22:23:10.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:09 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:10.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:09 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:23:10.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:09 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:23:10.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:09 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:10.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:09 smithi132 ceph-mon[97863]: purged_snaps scrub starts 2023-07-21T22:23:10.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:09 smithi132 ceph-mon[97863]: purged_snaps scrub ok 2023-07-21T22:23:10.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:09 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:10.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:09 smithi132 ceph-mon[97863]: osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012] boot 2023-07-21T22:23:10.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:09 smithi132 ceph-mon[97863]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T22:23:10.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:09 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:10.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:09 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:23:10.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:09 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:23:10.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:09 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:10.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:09 smithi107 ceph-mon[96543]: purged_snaps scrub starts 2023-07-21T22:23:10.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:09 smithi107 ceph-mon[96543]: purged_snaps scrub ok 2023-07-21T22:23:10.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:09 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:10.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:09 smithi107 ceph-mon[96543]: osd.0 [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012] boot 2023-07-21T22:23:10.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:09 smithi107 ceph-mon[96543]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T22:23:10.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:09 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:23:10.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:09 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:23:10.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:09 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:23:10.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:09 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:11.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:10 smithi190 ceph-mon[98334]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:11.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:10 smithi190 ceph-mon[98334]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:23:11.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:10 smithi190 ceph-mon[98334]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T22:23:11.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:10 smithi132 ceph-mon[97863]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:11.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:10 smithi132 ceph-mon[97863]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:23:11.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:10 smithi132 ceph-mon[97863]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T22:23:11.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:10 smithi107 ceph-mon[96543]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:23:11.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:10 smithi107 ceph-mon[96543]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:23:11.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:10 smithi107 ceph-mon[96543]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T22:23:13.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:13 smithi190 ceph-mon[98334]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:13 smithi190 ceph-mon[98334]: from='client.? 172.21.15.132:0/1277320878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d5fd94a-6288-42ed-9f8a-413b2e687e26"}]: dispatch 2023-07-21T22:23:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:13 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d5fd94a-6288-42ed-9f8a-413b2e687e26"}]: dispatch 2023-07-21T22:23:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:13 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d5fd94a-6288-42ed-9f8a-413b2e687e26"}]': finished 2023-07-21T22:23:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:13 smithi190 ceph-mon[98334]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T22:23:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:13 smithi190 ceph-mon[98334]: from='client.? 172.21.15.132:0/1843185863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:23:13.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:13 smithi132 ceph-mon[97863]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:13.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:13 smithi132 ceph-mon[97863]: from='client.? 172.21.15.132:0/1277320878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d5fd94a-6288-42ed-9f8a-413b2e687e26"}]: dispatch 2023-07-21T22:23:13.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:13 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d5fd94a-6288-42ed-9f8a-413b2e687e26"}]: dispatch 2023-07-21T22:23:13.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:13 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d5fd94a-6288-42ed-9f8a-413b2e687e26"}]': finished 2023-07-21T22:23:13.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:13 smithi132 ceph-mon[97863]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T22:23:13.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:13.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:13 smithi132 ceph-mon[97863]: from='client.? 172.21.15.132:0/1843185863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:23:13.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:13 smithi107 ceph-mon[96543]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:13.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:13 smithi107 ceph-mon[96543]: from='client.? 172.21.15.132:0/1277320878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d5fd94a-6288-42ed-9f8a-413b2e687e26"}]: dispatch 2023-07-21T22:23:13.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:13 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d5fd94a-6288-42ed-9f8a-413b2e687e26"}]: dispatch 2023-07-21T22:23:13.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:13 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d5fd94a-6288-42ed-9f8a-413b2e687e26"}]': finished 2023-07-21T22:23:13.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:13 smithi107 ceph-mon[96543]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T22:23:13.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:13.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:13 smithi107 ceph-mon[96543]: from='client.? 172.21.15.132:0/1843185863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:23:15.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:15 smithi190 ceph-mon[98334]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:15.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:15 smithi132 ceph-mon[97863]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:15.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:15 smithi107 ceph-mon[96543]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:17.345 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:17 smithi132 ceph-mon[97863]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:17.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:17 smithi190 ceph-mon[98334]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:17.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:17 smithi107 ceph-mon[96543]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:19.180 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:19 smithi132 ceph-mon[97863]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:19.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:19 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T22:23:19.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:19 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:19.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:19 smithi190 ceph-mon[98334]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:19.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:19 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T22:23:19.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:19 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:19.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:19 smithi107 ceph-mon[96543]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:19.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T22:23:19.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:20.361 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:20 smithi132 ceph-mon[97863]: Deploying daemon osd.1 on smithi132 2023-07-21T22:23:20.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:20 smithi190 ceph-mon[98334]: Deploying daemon osd.1 on smithi132 2023-07-21T22:23:20.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:20 smithi107 ceph-mon[96543]: Deploying daemon osd.1 on smithi132 2023-07-21T22:23:21.383 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:21 smithi132 ceph-mon[97863]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:21.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:21 smithi190 ceph-mon[98334]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:21.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:21 smithi107 ceph-mon[96543]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:23.344 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:23 smithi132 ceph-mon[97863]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:23.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:23 smithi190 ceph-mon[98334]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:23.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:23 smithi107 ceph-mon[96543]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:24.621 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:24 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:24.622 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:24 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:24.622 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:24 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:24.622 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:24 smithi132 ceph-mon[97863]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:24.639 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:24 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:24.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:24 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:24.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:24 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:24.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:24 smithi190 ceph-mon[98334]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:24.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:24 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:24.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:24 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:24.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:24 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:24.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:24 smithi107 ceph-mon[96543]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:26.198 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 1 on host 'smithi132' 2023-07-21T22:23:26.198 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:26.152+0000 7efcc4ff9700 1 -- 172.21.15.132:0/4236068889 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efcc8060950 con 0x7efcb00a1140 2023-07-21T22:23:26.200 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:26.155+0000 7efcce219700 1 -- 172.21.15.132:0/4236068889 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efcb00a1140 msgr2=0x7efcb00a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:26.200 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:26.155+0000 7efcce219700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efcb00a1140 0x7efcb00a35f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efcb8005950 tx=0x7efcb80058e0 comp rx=0 tx=0).stop 2023-07-21T22:23:26.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:26.155+0000 7efcce219700 1 -- 172.21.15.132:0/4236068889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcc80ffc80 msgr2=0x7efcc819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:26.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:26.155+0000 7efcce219700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcc80ffc80 0x7efcc819a690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00e9a0 tx=0x7efcbc00ed60 comp rx=0 tx=0).stop 2023-07-21T22:23:26.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:26.155+0000 7efcce219700 1 -- 172.21.15.132:0/4236068889 shutdown_connections 2023-07-21T22:23:26.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:26.155+0000 7efcce219700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcc8100610 0x7efcc819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:26.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:26.155+0000 7efcce219700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efcb00a1140 0x7efcb00a35f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:26.201 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:26.155+0000 7efcce219700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcc80fea80 0x7efcc8068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:26.202 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:26.155+0000 7efcce219700 1 --2- 172.21.15.132:0/4236068889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efcc80ffc80 0x7efcc819a690 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:26.202 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:26.155+0000 7efcce219700 1 -- 172.21.15.132:0/4236068889 >> 172.21.15.132:0/4236068889 conn(0x7efcc80fa0b0 msgr2=0x7efcc8103120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:26.202 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:26.155+0000 7efcce219700 1 -- 172.21.15.132:0/4236068889 shutdown_connections 2023-07-21T22:23:26.202 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:23:26.155+0000 7efcce219700 1 -- 172.21.15.132:0/4236068889 wait complete. 2023-07-21T22:23:26.603 DEBUG:teuthology.orchestra.run.smithi132:osd.1> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.1.service 2023-07-21T22:23:26.605 INFO:tasks.cephadm:Deploying osd.2 on smithi190 with /dev/vg_nvme/lv_4... 2023-07-21T22:23:26.605 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T22:23:26.643 INFO:journalctl@ceph.osd.1.smithi132.stdout:-- Logs begin at Fri 2023-07-21 22:03:19 UTC. -- 2023-07-21T22:23:26.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:26 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:26 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:26 smithi190 ceph-mon[98334]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:26.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:26 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:26.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:26 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:26.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:26 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:26 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:26 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:26 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:26.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:26 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:26 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:26 smithi132 ceph-mon[97863]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:26.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:26 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:26.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:26 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:26.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:26 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:26 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:26 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:26 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:26.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:26 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:26 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:26 smithi107 ceph-mon[96543]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:26.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:26 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:26.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:26 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:26.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:26 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:26 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:26 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:26.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:26 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:27.166 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:23:27.617 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:27 smithi132 ceph-mon[97863]: from='osd.1 [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:23:27.618 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:27 smithi132 ceph-mon[97863]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:23:27.618 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:23:27 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-1[102848]: 2023-07-21T22:23:27.297+0000 7f5808413640 -1 osd.1 0 log_to_monitors true 2023-07-21T22:23:27.626 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:27 smithi190 ceph-mon[98334]: from='osd.1 [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:23:27.627 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:27 smithi190 ceph-mon[98334]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:23:27.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:27 smithi107 ceph-mon[96543]: from='osd.1 [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:23:27.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:27 smithi107 ceph-mon[96543]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:23:28.661 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T22:23:28.661 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T22:23:28.661 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2023-07-21T22:23:28.661 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2023-07-21T22:23:28.662 INFO:teuthology.orchestra.run.smithi190.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110522 s, 949 MB/s 2023-07-21T22:23:28.662 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 2023-07-21T22:23:28.662 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2023-07-21T22:23:28.663 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:23:28 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-1[102848]: 2023-07-21T22:23:28.445+0000 7f57fd28a700 -1 osd.1 0 waiting for initial osdmap 2023-07-21T22:23:28.664 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:23:28 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-1[102848]: 2023-07-21T22:23:28.448+0000 7f57f787c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:23:28.664 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:28 smithi132 ceph-mon[97863]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T22:23:28.664 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:28 smithi132 ceph-mon[97863]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T22:23:28.664 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:28 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:28.664 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:28 smithi132 ceph-mon[97863]: from='osd.1 [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:23:28.664 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:28 smithi132 ceph-mon[97863]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:23:28.665 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:28 smithi132 ceph-mon[97863]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:28.792 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:28 smithi190 ceph-mon[98334]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T22:23:28.793 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:28 smithi190 ceph-mon[98334]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T22:23:28.793 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:28 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:28.793 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:28 smithi190 ceph-mon[98334]: from='osd.1 [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:23:28.793 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:28 smithi190 ceph-mon[98334]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:23:28.793 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:28 smithi190 ceph-mon[98334]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:28.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:28 smithi107 ceph-mon[96543]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T22:23:28.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:28 smithi107 ceph-mon[96543]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T22:23:28.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:28 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:28.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:28 smithi107 ceph-mon[96543]: from='osd.1 [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:23:28.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:28 smithi107 ceph-mon[96543]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:23:28.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:28 smithi107 ceph-mon[96543]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:29.021 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi190:vg_nvme/lv_4 2023-07-21T22:23:29.568 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:23:29.773 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:29 smithi190 ceph-mon[98334]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-07-21T22:23:29.773 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:29 smithi190 ceph-mon[98334]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T22:23:29.773 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:29 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:29.774 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:29 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:29.774 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:29 smithi190 ceph-mon[98334]: Detected new or changed devices on smithi132 2023-07-21T22:23:29.774 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:29 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:29.774 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:29 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:29.774 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:29 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:23:29.775 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:29 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi132 to 17168M 2023-07-21T22:23:29.775 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:29 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:29.775 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:29 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:29.775 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:29 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:29.775 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:29 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:29.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:29 smithi132 ceph-mon[97863]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-07-21T22:23:29.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:29 smithi132 ceph-mon[97863]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T22:23:29.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:29 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:29.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:29 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:29.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:29 smithi132 ceph-mon[97863]: Detected new or changed devices on smithi132 2023-07-21T22:23:29.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:29 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:29.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:29 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:29.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:29 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:23:29.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:29 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi132 to 17168M 2023-07-21T22:23:29.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:29 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:29.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:29 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:29.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:29 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:29.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:29 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:29.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:29 smithi107 ceph-mon[96543]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-07-21T22:23:29.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:29 smithi107 ceph-mon[96543]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T22:23:29.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:29 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:29.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:29 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:29.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:29 smithi107 ceph-mon[96543]: Detected new or changed devices on smithi132 2023-07-21T22:23:29.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:29 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:29.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:29 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:29.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:29 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:23:29.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:29 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi132 to 17168M 2023-07-21T22:23:29.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:29 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:29.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:29 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:29.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:29 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:29.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:29 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:30.738 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.691+0000 7f755a2e4700 1 -- 172.21.15.190:0/1667383483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75540fece0 msgr2=0x7f75540ff150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.738 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.691+0000 7f755a2e4700 1 --2- 172.21.15.190:0/1667383483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75540fece0 0x7f75540ff150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7544009990 tx=0x7f7544009ca0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.738 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.691+0000 7f755a2e4700 1 -- 172.21.15.190:0/1667383483 shutdown_connections 2023-07-21T22:23:30.738 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.691+0000 7f755a2e4700 1 --2- 172.21.15.190:0/1667383483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75540ff690 0x7f7554106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.738 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.691+0000 7f755a2e4700 1 --2- 172.21.15.190:0/1667383483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7554100110 0x7f7554100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.739 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.691+0000 7f755a2e4700 1 --2- 172.21.15.190:0/1667383483 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75540fece0 0x7f75540ff150 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.739 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.691+0000 7f755a2e4700 1 -- 172.21.15.190:0/1667383483 >> 172.21.15.190:0/1667383483 conn(0x7f75540f9860 msgr2=0x7f75540fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.739 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.691+0000 7f755a2e4700 1 -- 172.21.15.190:0/1667383483 shutdown_connections 2023-07-21T22:23:30.739 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.691+0000 7f755a2e4700 1 -- 172.21.15.190:0/1667383483 wait complete. 2023-07-21T22:23:30.739 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.692+0000 7f755a2e4700 1 Processor -- start 2023-07-21T22:23:30.739 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.692+0000 7f755a2e4700 1 -- start start 2023-07-21T22:23:30.741 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.692+0000 7f755a2e4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75540fece0 0x7f7554194880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.742 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.692+0000 7f755a2e4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75540ff690 0x7f7554194dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.742 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.692+0000 7f755a2e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7554100110 0x7f755419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.742 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.692+0000 7f755a2e4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755419c410 con 0x7f75540fece0 2023-07-21T22:23:30.742 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.692+0000 7f755a2e4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755419c550 con 0x7f75540ff690 2023-07-21T22:23:30.742 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.692+0000 7f755a2e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755419c6e0 con 0x7f7554100110 2023-07-21T22:23:30.742 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.692+0000 7f7558881700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7554100110 0x7f755419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.742 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.692+0000 7f7558881700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7554100110 0x7f755419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:47966/0 (socket says 172.21.15.190:47966) 2023-07-21T22:23:30.743 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f7558881700 1 -- 172.21.15.190:0/3351817980 learned_addr learned my addr 172.21.15.190:0/3351817980 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-07-21T22:23:30.743 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f7553fff700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75540fece0 0x7f7554194880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.743 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f75537fe700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75540ff690 0x7f7554194dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.743 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f7558881700 1 -- 172.21.15.190:0/3351817980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75540ff690 msgr2=0x7f7554194dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.743 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f7558881700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75540ff690 0x7f7554194dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.743 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f7558881700 1 -- 172.21.15.190:0/3351817980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75540fece0 msgr2=0x7f7554194880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.743 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f7558881700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75540fece0 0x7f7554194880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.744 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f7558881700 1 -- 172.21.15.190:0/3351817980 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75440095f0 con 0x7f7554100110 2023-07-21T22:23:30.744 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f75537fe700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75540ff690 0x7f7554194dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:23:30.744 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f7558881700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7554100110 0x7f755419be40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f754c00b9a0 tx=0x7f754c00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.744 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f75517fa700 1 -- 172.21.15.190:0/3351817980 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c00c9b0 con 0x7f7554100110 2023-07-21T22:23:30.744 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f75517fa700 1 -- 172.21.15.190:0/3351817980 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f754c014440 con 0x7f7554100110 2023-07-21T22:23:30.744 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f755a2e4700 1 -- 172.21.15.190:0/3351817980 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755419c9c0 con 0x7f7554100110 2023-07-21T22:23:30.744 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f75517fa700 1 -- 172.21.15.190:0/3351817980 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c0075d0 con 0x7f7554100110 2023-07-21T22:23:30.744 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.693+0000 7f755a2e4700 1 -- 172.21.15.190:0/3351817980 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755419cef0 con 0x7f7554100110 2023-07-21T22:23:30.745 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.694+0000 7f755a2e4700 1 -- 172.21.15.190:0/3351817980 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75541060b0 con 0x7f7554100110 2023-07-21T22:23:30.745 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.695+0000 7f75517fa700 1 -- 172.21.15.190:0/3351817980 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7f754c0145b0 con 0x7f7554100110 2023-07-21T22:23:30.745 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.696+0000 7f75517fa700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f753c0a10c0 0x7f753c0a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.746 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.696+0000 7f75517fa700 1 -- 172.21.15.190:0/3351817980 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f754c006e40 con 0x7f7554100110 2023-07-21T22:23:30.746 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.696+0000 7f7553fff700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f753c0a10c0 0x7f753c0a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.746 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.697+0000 7f7553fff700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f753c0a10c0 0x7f753c0a3570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7548006fd0 tx=0x7f7548008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.746 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.700+0000 7f75517fa700 1 -- 172.21.15.190:0/3351817980 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f754c095c30 con 0x7f7554100110 2023-07-21T22:23:30.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:30 smithi190 ceph-mon[98334]: purged_snaps scrub starts 2023-07-21T22:23:30.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:30 smithi190 ceph-mon[98334]: purged_snaps scrub ok 2023-07-21T22:23:30.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:30 smithi190 ceph-mon[98334]: osd.1 [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199] boot 2023-07-21T22:23:30.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:30 smithi190 ceph-mon[98334]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T22:23:30.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:30.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:30 smithi190 ceph-mon[98334]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:30.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:30 smithi132 ceph-mon[97863]: purged_snaps scrub starts 2023-07-21T22:23:30.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:30 smithi132 ceph-mon[97863]: purged_snaps scrub ok 2023-07-21T22:23:30.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:30 smithi132 ceph-mon[97863]: osd.1 [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199] boot 2023-07-21T22:23:30.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:30 smithi132 ceph-mon[97863]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T22:23:30.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:30.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:30 smithi132 ceph-mon[97863]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:30.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:30 smithi107 ceph-mon[96543]: purged_snaps scrub starts 2023-07-21T22:23:30.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:30 smithi107 ceph-mon[96543]: purged_snaps scrub ok 2023-07-21T22:23:30.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:30 smithi107 ceph-mon[96543]: osd.1 [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199] boot 2023-07-21T22:23:30.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:30 smithi107 ceph-mon[96543]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T22:23:30.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:23:30.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:30 smithi107 ceph-mon[96543]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:23:30.938 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:30.892+0000 7f755a2e4700 1 -- 172.21.15.190:0/3351817980 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7554060950 con 0x7f753c0a10c0 2023-07-21T22:23:31.768 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:31 smithi190 ceph-mon[98334]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T22:23:31.768 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:31 smithi190 ceph-mon[98334]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:23:31.768 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:31 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:23:31.769 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:31 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:23:31.769 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:31 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:31.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:31 smithi132 ceph-mon[97863]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T22:23:31.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:31 smithi132 ceph-mon[97863]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:23:31.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:31 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:23:31.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:31 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:23:31.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:31 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:31.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:31 smithi107 ceph-mon[96543]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T22:23:31.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:31 smithi107 ceph-mon[96543]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:23:31.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:31 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:23:31.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:31 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:23:31.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:31 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:32.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:32 smithi190 ceph-mon[98334]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:32.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:32 smithi132 ceph-mon[97863]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:32.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:32 smithi107 ceph-mon[96543]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:33.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:33 smithi190 ceph-mon[98334]: from='client.? 172.21.15.190:0/1765695709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5ce38db-d9b7-4f49-9945-fe7e1aacf27d"}]: dispatch 2023-07-21T22:23:33.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:33 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5ce38db-d9b7-4f49-9945-fe7e1aacf27d"}]: dispatch 2023-07-21T22:23:33.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:33 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e5ce38db-d9b7-4f49-9945-fe7e1aacf27d"}]': finished 2023-07-21T22:23:33.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:33 smithi190 ceph-mon[98334]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T22:23:33.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:33 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:33.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:33 smithi132 ceph-mon[97863]: from='client.? 172.21.15.190:0/1765695709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5ce38db-d9b7-4f49-9945-fe7e1aacf27d"}]: dispatch 2023-07-21T22:23:33.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:33 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5ce38db-d9b7-4f49-9945-fe7e1aacf27d"}]: dispatch 2023-07-21T22:23:33.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:33 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e5ce38db-d9b7-4f49-9945-fe7e1aacf27d"}]': finished 2023-07-21T22:23:33.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:33 smithi132 ceph-mon[97863]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T22:23:33.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:33 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:33.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:33 smithi107 ceph-mon[96543]: from='client.? 172.21.15.190:0/1765695709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5ce38db-d9b7-4f49-9945-fe7e1aacf27d"}]: dispatch 2023-07-21T22:23:33.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:33 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5ce38db-d9b7-4f49-9945-fe7e1aacf27d"}]: dispatch 2023-07-21T22:23:33.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:33 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e5ce38db-d9b7-4f49-9945-fe7e1aacf27d"}]': finished 2023-07-21T22:23:33.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:33 smithi107 ceph-mon[96543]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T22:23:33.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:33 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:34.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:34 smithi190 ceph-mon[98334]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:34.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:34 smithi190 ceph-mon[98334]: from='client.? 172.21.15.190:0/1204630178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:23:34.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:34 smithi132 ceph-mon[97863]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:34.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:34 smithi132 ceph-mon[97863]: from='client.? 172.21.15.190:0/1204630178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:23:34.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:34 smithi107 ceph-mon[96543]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:34.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:34 smithi107 ceph-mon[96543]: from='client.? 172.21.15.190:0/1204630178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:23:37.316 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:36 smithi190 ceph-mon[98334]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:37.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:36 smithi132 ceph-mon[97863]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:37.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:36 smithi107 ceph-mon[96543]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:39.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:38 smithi190 ceph-mon[98334]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:39.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:38 smithi132 ceph-mon[97863]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:39.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:38 smithi107 ceph-mon[96543]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:40.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T22:23:40.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:40.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T22:23:40.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:40.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T22:23:40.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:41.259 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:40 smithi190 ceph-mon[98334]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:41.259 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:40 smithi190 ceph-mon[98334]: Deploying daemon osd.2 on smithi190 2023-07-21T22:23:41.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:40 smithi132 ceph-mon[97863]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:41.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:40 smithi132 ceph-mon[97863]: Deploying daemon osd.2 on smithi190 2023-07-21T22:23:41.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:40 smithi107 ceph-mon[96543]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:41.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:40 smithi107 ceph-mon[96543]: Deploying daemon osd.2 on smithi190 2023-07-21T22:23:43.307 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:42 smithi190 ceph-mon[98334]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:43.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:42 smithi132 ceph-mon[97863]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:43.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:42 smithi107 ceph-mon[96543]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:45.290 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:45 smithi190 ceph-mon[98334]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:45.290 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:45 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:45.290 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:45 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:45.291 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:45 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:45.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:45 smithi132 ceph-mon[97863]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:45.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:45 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:45.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:45 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:45.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:45 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:45.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:45 smithi107 ceph-mon[96543]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:45.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:45 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:45.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:45 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:45.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:45 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:46.948 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:46.903+0000 7f75517fa700 1 -- 172.21.15.190:0/3351817980 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7554060950 con 0x7f753c0a10c0 2023-07-21T22:23:46.949 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 2 on host 'smithi190' 2023-07-21T22:23:46.950 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:46.905+0000 7f755a2e4700 1 -- 172.21.15.190:0/3351817980 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f753c0a10c0 msgr2=0x7f753c0a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:46.950 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:46.905+0000 7f755a2e4700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f753c0a10c0 0x7f753c0a3570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7548006fd0 tx=0x7f7548008040 comp rx=0 tx=0).stop 2023-07-21T22:23:46.951 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:46.905+0000 7f755a2e4700 1 -- 172.21.15.190:0/3351817980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7554100110 msgr2=0x7f755419be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:46.951 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:46.905+0000 7f755a2e4700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7554100110 0x7f755419be40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f754c00b9a0 tx=0x7f754c00bd60 comp rx=0 tx=0).stop 2023-07-21T22:23:46.951 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:46.905+0000 7f755a2e4700 1 -- 172.21.15.190:0/3351817980 shutdown_connections 2023-07-21T22:23:46.951 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:46.905+0000 7f755a2e4700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f753c0a10c0 0x7f753c0a3570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:46.951 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:46.905+0000 7f755a2e4700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7554100110 0x7f755419be40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:46.951 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:46.905+0000 7f755a2e4700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75540fece0 0x7f7554194880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:46.951 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:46.905+0000 7f755a2e4700 1 --2- 172.21.15.190:0/3351817980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75540ff690 0x7f7554194dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:46.952 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:46.905+0000 7f755a2e4700 1 -- 172.21.15.190:0/3351817980 >> 172.21.15.190:0/3351817980 conn(0x7f75540f9860 msgr2=0x7f75541031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:46.952 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:46.906+0000 7f755a2e4700 1 -- 172.21.15.190:0/3351817980 shutdown_connections 2023-07-21T22:23:46.952 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:23:46.906+0000 7f755a2e4700 1 -- 172.21.15.190:0/3351817980 wait complete. 2023-07-21T22:23:47.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:46 smithi190 ceph-mon[98334]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:47.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:47.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:47.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:47.351 DEBUG:teuthology.orchestra.run.smithi190:osd.2> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.2.service 2023-07-21T22:23:47.353 INFO:tasks.cephadm:Deploying osd.3 on smithi107 with /dev/vg_nvme/lv_3... 2023-07-21T22:23:47.353 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T22:23:47.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:46 smithi107 ceph-mon[96543]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:47.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:47.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:47.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:47.392 INFO:journalctl@ceph.osd.2.smithi190.stdout:-- Logs begin at Fri 2023-07-21 22:03:29 UTC. -- 2023-07-21T22:23:47.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:46 smithi132 ceph-mon[97863]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:47.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:47.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:47.922 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:23:48.390 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:23:48 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-2[104149]: 2023-07-21T22:23:48.060+0000 7f1f8a5dd640 -1 osd.2 0 log_to_monitors true 2023-07-21T22:23:49.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:48 smithi107 ceph-mon[96543]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:49.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:48 smithi107 ceph-mon[96543]: from='osd.2 [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:23:49.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:48 smithi107 ceph-mon[96543]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:23:49.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:48 smithi190 ceph-mon[98334]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:49.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:48 smithi190 ceph-mon[98334]: from='osd.2 [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:23:49.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:48 smithi190 ceph-mon[98334]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:23:49.420 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T22:23:49.420 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T22:23:49.420 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2023-07-21T22:23:49.420 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2023-07-21T22:23:49.421 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111957 s, 937 MB/s 2023-07-21T22:23:49.421 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 2023-07-21T22:23:49.421 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2023-07-21T22:23:49.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:48 smithi132 ceph-mon[97863]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:49.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:48 smithi132 ceph-mon[97863]: from='osd.2 [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:23:49.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:48 smithi132 ceph-mon[97863]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:23:49.864 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2023-07-21T22:23:50.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-mon[98334]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T22:23:50.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-mon[98334]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T22:23:50.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:50.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-mon[98334]: from='osd.2 [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:23:50.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-mon[98334]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:23:50.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:50.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:50.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:23:50.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:50.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:50.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:50.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:50.392 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-2[104149]: 2023-07-21T22:23:50.008+0000 7f1f7f454700 -1 osd.2 0 waiting for initial osdmap 2023-07-21T22:23:50.392 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:23:50 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-2[104149]: 2023-07-21T22:23:50.013+0000 7f1f77a42700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:23:50.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:50 smithi132 ceph-mon[97863]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T22:23:50.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:50 smithi132 ceph-mon[97863]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T22:23:50.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:50.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:50 smithi132 ceph-mon[97863]: from='osd.2 [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:23:50.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:50 smithi132 ceph-mon[97863]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:23:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:23:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:50.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:50 smithi107 ceph-mon[96543]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T22:23:50.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:50 smithi107 ceph-mon[96543]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T22:23:50.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:50.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:50 smithi107 ceph-mon[96543]: from='osd.2 [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:23:50.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:50 smithi107 ceph-mon[96543]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:23:50.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:50.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:50.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:23:50.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:50.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:50.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:50.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:50.457 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:23:51.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:51 smithi190 ceph-mon[98334]: Detected new or changed devices on smithi190 2023-07-21T22:23:51.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:51 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi190 to 17168M 2023-07-21T22:23:51.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:51 smithi190 ceph-mon[98334]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:51.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:51 smithi190 ceph-mon[98334]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-07-21T22:23:51.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:51 smithi190 ceph-mon[98334]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T22:23:51.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:51 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:51.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:51 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:51.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:51 smithi132 ceph-mon[97863]: Detected new or changed devices on smithi190 2023-07-21T22:23:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:51 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi190 to 17168M 2023-07-21T22:23:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:51 smithi132 ceph-mon[97863]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:51 smithi132 ceph-mon[97863]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-07-21T22:23:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:51 smithi132 ceph-mon[97863]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T22:23:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:51 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:51 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:51.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:51 smithi107 ceph-mon[96543]: Detected new or changed devices on smithi190 2023-07-21T22:23:51.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:51 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi190 to 17168M 2023-07-21T22:23:51.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:51 smithi107 ceph-mon[96543]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:51.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:51 smithi107 ceph-mon[96543]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-07-21T22:23:51.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:51 smithi107 ceph-mon[96543]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T22:23:51.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:51 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:51.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:51 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:51.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.607+0000 7f35ed942700 1 -- 172.21.15.107:0/1185607254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35e80ffc10 msgr2=0x7f35e8100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:51.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.607+0000 7f35ed942700 1 --2- 172.21.15.107:0/1185607254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35e80ffc10 0x7f35e8100020 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009990 tx=0x7f35d8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:23:51.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.607+0000 7f35ed942700 1 -- 172.21.15.107:0/1185607254 shutdown_connections 2023-07-21T22:23:51.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.607+0000 7f35ed942700 1 --2- 172.21.15.107:0/1185607254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35e81017a0 0x7f35e8108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:51.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.607+0000 7f35ed942700 1 --2- 172.21.15.107:0/1185607254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35e80ffc10 0x7f35e8100020 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:51.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.607+0000 7f35ed942700 1 --2- 172.21.15.107:0/1185607254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e8100e10 0x7f35e8101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:51.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.607+0000 7f35ed942700 1 -- 172.21.15.107:0/1185607254 >> 172.21.15.107:0/1185607254 conn(0x7f35e80fb220 msgr2=0x7f35e80fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:51.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.607+0000 7f35ed942700 1 -- 172.21.15.107:0/1185607254 shutdown_connections 2023-07-21T22:23:51.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.608+0000 7f35ed942700 1 -- 172.21.15.107:0/1185607254 wait complete. 2023-07-21T22:23:51.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.608+0000 7f35ed942700 1 Processor -- start 2023-07-21T22:23:51.659 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.608+0000 7f35ed942700 1 -- start start 2023-07-21T22:23:51.659 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.608+0000 7f35ed942700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e80ffc10 0x7f35e8195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:51.660 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.608+0000 7f35ed942700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35e8100e10 0x7f35e8195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:51.660 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.608+0000 7f35ed942700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35e81017a0 0x7f35e819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:51.660 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.608+0000 7f35ed942700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e819d490 con 0x7f35e8100e10 2023-07-21T22:23:51.660 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.608+0000 7f35ed942700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e819d5d0 con 0x7f35e81017a0 2023-07-21T22:23:51.661 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.608+0000 7f35ed942700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e819d760 con 0x7f35e80ffc10 2023-07-21T22:23:51.661 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35e67fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35e8100e10 0x7f35e8195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:51.661 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35e67fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35e8100e10 0x7f35e8195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45608/0 (socket says 172.21.15.107:45608) 2023-07-21T22:23:51.661 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35e67fc700 1 -- 172.21.15.107:0/3434653949 learned_addr learned my addr 172.21.15.107:0/3434653949 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:23:51.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35e77fe700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35e81017a0 0x7f35e819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:51.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35e67fc700 1 -- 172.21.15.107:0/3434653949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e80ffc10 msgr2=0x7f35e8195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:51.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35e67fc700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e80ffc10 0x7f35e8195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:51.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35e67fc700 1 -- 172.21.15.107:0/3434653949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35e81017a0 msgr2=0x7f35e819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:51.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35e67fc700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35e81017a0 0x7f35e819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:51.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35e67fc700 1 -- 172.21.15.107:0/3434653949 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d80095f0 con 0x7f35e8100e10 2023-07-21T22:23:51.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35e77fe700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35e81017a0 0x7f35e819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:23:51.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35e67fc700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35e8100e10 0x7f35e8195e40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00b560 tx=0x7f35dc00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:51.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35ec940700 1 -- 172.21.15.107:0/3434653949 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc011830 con 0x7f35e8100e10 2023-07-21T22:23:51.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35ec940700 1 -- 172.21.15.107:0/3434653949 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35dc011e50 con 0x7f35e8100e10 2023-07-21T22:23:51.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35ed942700 1 -- 172.21.15.107:0/3434653949 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e819da40 con 0x7f35e8100e10 2023-07-21T22:23:51.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35ec940700 1 -- 172.21.15.107:0/3434653949 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00f2e0 con 0x7f35e8100e10 2023-07-21T22:23:51.665 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.609+0000 7f35ed942700 1 -- 172.21.15.107:0/3434653949 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e819df40 con 0x7f35e8100e10 2023-07-21T22:23:51.665 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.610+0000 7f35ed942700 1 -- 172.21.15.107:0/3434653949 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e810b450 con 0x7f35e8100e10 2023-07-21T22:23:51.665 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.611+0000 7f35ec940700 1 -- 172.21.15.107:0/3434653949 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7f35dc011990 con 0x7f35e8100e10 2023-07-21T22:23:51.665 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.612+0000 7f35ec940700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f35d00a1140 0x7f35d00a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:51.666 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.612+0000 7f35e6ffd700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f35d00a1140 0x7f35d00a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:51.666 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.612+0000 7f35ec940700 1 -- 172.21.15.107:0/3434653949 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f35dc0cb3d0 con 0x7f35e8100e10 2023-07-21T22:23:51.666 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.612+0000 7f35e6ffd700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f35d00a1140 0x7f35d00a35f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f35d80095c0 tx=0x7f35d8009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:51.667 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.615+0000 7f35ec940700 1 -- 172.21.15.107:0/3434653949 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f35dc0960b0 con 0x7f35e8100e10 2023-07-21T22:23:51.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:51.810+0000 7f35ed942700 1 -- 172.21.15.107:0/3434653949 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f35e8060950 con 0x7f35d00a1140 2023-07-21T22:23:52.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:52 smithi190 ceph-mon[98334]: purged_snaps scrub starts 2023-07-21T22:23:52.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:52 smithi190 ceph-mon[98334]: purged_snaps scrub ok 2023-07-21T22:23:52.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:52 smithi190 ceph-mon[98334]: osd.2 [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995] boot 2023-07-21T22:23:52.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:52 smithi190 ceph-mon[98334]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T22:23:52.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:52 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:52.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:52 smithi132 ceph-mon[97863]: purged_snaps scrub starts 2023-07-21T22:23:52.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:52 smithi132 ceph-mon[97863]: purged_snaps scrub ok 2023-07-21T22:23:52.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:52 smithi132 ceph-mon[97863]: osd.2 [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995] boot 2023-07-21T22:23:52.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:52 smithi132 ceph-mon[97863]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T22:23:52.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:52 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:52.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:52 smithi107 ceph-mon[96543]: purged_snaps scrub starts 2023-07-21T22:23:52.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:52 smithi107 ceph-mon[96543]: purged_snaps scrub ok 2023-07-21T22:23:52.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:52 smithi107 ceph-mon[96543]: osd.2 [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995] boot 2023-07-21T22:23:52.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:52 smithi107 ceph-mon[96543]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T22:23:52.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:52 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:23:53.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:53 smithi107 ceph-mon[96543]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:53.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:23:53.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:53 smithi107 ceph-mon[96543]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:23:53.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:23:53.173 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:23:53.173 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:53.173 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T22:23:53.173 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:53 smithi107 ceph-mon[96543]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T22:23:53.173 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:23:53.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:53 smithi190 ceph-mon[98334]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:53.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:23:53.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:53 smithi190 ceph-mon[98334]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:23:53.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:23:53.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:23:53.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:53.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T22:23:53.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:53 smithi190 ceph-mon[98334]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T22:23:53.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:23:53.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:53 smithi132 ceph-mon[97863]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:23:53.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:23:53.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:53 smithi132 ceph-mon[97863]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:23:53.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:23:53.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:23:53.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:53.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T22:23:53.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:53 smithi132 ceph-mon[97863]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T22:23:53.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:23:53.682 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:23:53 smithi107 sudo[112481]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:23:53.682 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:23:53 smithi107 sudo[112481]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:23:53.682 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:23:53 smithi107 sudo[112481]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:23:53.683 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:23:53 smithi107 sudo[112481]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:23:53.937 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:23:53 smithi107 sudo[112520]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:23:53.937 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:23:53 smithi107 sudo[112520]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:23:53.937 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:23:53 smithi107 sudo[112520]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:23:53.937 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:23:53 smithi107 sudo[112520]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:23:54.226 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:54 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:23:54.226 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:54 smithi132 ceph-mon[97863]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T22:23:54.226 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:54 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:54.226 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:23:53 smithi132 sudo[105186]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:23:54.227 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:23:53 smithi132 sudo[105186]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:23:54.227 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:23:53 smithi132 sudo[105186]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:23:54.227 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:23:53 smithi132 sudo[105186]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:23:54.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:54 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:23:54.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:54 smithi190 ceph-mon[98334]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T22:23:54.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:54 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:54.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:54 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:23:54.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:54 smithi107 ceph-mon[96543]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T22:23:54.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:54 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:54.680 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:23:54 smithi132 sudo[105189]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:23:54.681 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:23:54 smithi132 sudo[105189]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:23:54.681 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:23:54 smithi132 sudo[105189]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:23:54.681 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:23:54 smithi132 sudo[105189]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:23:54.745 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:23:54 smithi190 sudo[106402]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:23:54.745 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:23:54 smithi190 sudo[106402]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:23:54.745 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:23:54 smithi190 sudo[106402]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:23:54.745 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:23:54 smithi190 sudo[106402]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:23:55.139 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:23:54 smithi190 sudo[106405]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:23:55.140 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:23:54 smithi190 sudo[106405]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:23:55.140 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:23:54 smithi190 sudo[106405]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:23:55.140 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:23:54 smithi190 sudo[106405]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:23:55.275 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:54 smithi107 sudo[112607]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:23:55.275 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:54 smithi107 sudo[112607]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:23:55.275 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:54 smithi107 sudo[112607]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:23:55.275 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:54 smithi107 sudo[112607]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:23:55.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:55 smithi107 sudo[112635]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:23:55.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:55 smithi107 sudo[112635]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:23:55.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:55 smithi107 sudo[112635]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:23:55.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:55 smithi107 sudo[112635]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:23:55.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:55 smithi107 ceph-mon[96543]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:23:55.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:55 smithi107 ceph-mon[96543]: osdmap e21: 3 total, 3 up, 3 in 2023-07-21T22:23:55.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:55.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:55 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3861053183' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaefb60a-452f-4eb7-a984-5b459aa49d4e"}]: dispatch 2023-07-21T22:23:55.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:55 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3861053183' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eaefb60a-452f-4eb7-a984-5b459aa49d4e"}]': finished 2023-07-21T22:23:55.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:55 smithi107 ceph-mon[96543]: osdmap e22: 4 total, 3 up, 4 in 2023-07-21T22:23:55.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:23:55.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:55.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:55 smithi132 ceph-mon[97863]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:23:55.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:55 smithi132 ceph-mon[97863]: osdmap e21: 3 total, 3 up, 3 in 2023-07-21T22:23:55.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:55 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:55.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:55 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3861053183' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaefb60a-452f-4eb7-a984-5b459aa49d4e"}]: dispatch 2023-07-21T22:23:55.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:55 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3861053183' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eaefb60a-452f-4eb7-a984-5b459aa49d4e"}]': finished 2023-07-21T22:23:55.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:55 smithi132 ceph-mon[97863]: osdmap e22: 4 total, 3 up, 4 in 2023-07-21T22:23:55.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:55 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:23:55.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:55 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:55.958 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 sudo[106408]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:23:55.959 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 sudo[106408]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:23:55.959 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 sudo[106408]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:23:55.959 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 sudo[106408]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:23:55.974 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:23:55.928+0000 7f35ec940700 1 -- 172.21.15.107:0/3434653949 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f35dc096480 con 0x7f35e8100e10 2023-07-21T22:23:56.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 sudo[106411]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:23:56.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 sudo[106411]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:23:56.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 sudo[106411]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:23:56.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 sudo[106411]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:23:56.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 ceph-mon[98334]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:23:56.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 ceph-mon[98334]: osdmap e21: 3 total, 3 up, 3 in 2023-07-21T22:23:56.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:56.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3861053183' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaefb60a-452f-4eb7-a984-5b459aa49d4e"}]: dispatch 2023-07-21T22:23:56.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3861053183' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eaefb60a-452f-4eb7-a984-5b459aa49d4e"}]': finished 2023-07-21T22:23:56.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 ceph-mon[98334]: osdmap e22: 4 total, 3 up, 4 in 2023-07-21T22:23:56.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:23:56.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:55 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:23:56.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 sudo[105192]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:23:56.452 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 sudo[105192]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:23:56.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 sudo[105192]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:23:56.453 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 sudo[105192]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:23:56.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:23:56.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2624264696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:23:56.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:23:56.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:23:56.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:23:56.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:23:56.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:23:56.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:23:56.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:23:56.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:23:56.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:23:56.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:23:56.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:23:56.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:23:56.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:23:56.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:23:56.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T22:23:56.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:56 smithi190 ceph-mon[98334]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:23:56.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 sudo[105195]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:23:56.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 sudo[105195]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:23:56.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 sudo[105195]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:23:56.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 sudo[105195]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:23:56.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:23:56.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2624264696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:23:56.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:23:56.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:23:56.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:23:56.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:23:56.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:23:56.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:23:56.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:23:56.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:23:56.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:23:56.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:23:56.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:23:56.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:23:56.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:23:56.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:23:56.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T22:23:56.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:56 smithi132 ceph-mon[97863]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:23:56.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:23:56.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2624264696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:23:56.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:23:56.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:23:56.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:23:56.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:23:56.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:23:56.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:23:56.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:23:56.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:23:56.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:23:56.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:23:56.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:23:56.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:23:56.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:23:56.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:23:56.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T22:23:56.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:56 smithi107 ceph-mon[96543]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:23:59.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:23:58 smithi190 ceph-mon[98334]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:23:59.180 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:23:58 smithi132 ceph-mon[97863]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:23:59.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:23:58 smithi107 ceph-mon[96543]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:24:01.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:00 smithi190 ceph-mon[98334]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:24:01.162 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:00 smithi107 ceph-mon[96543]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:24:01.180 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:00 smithi132 ceph-mon[97863]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:24:02.070 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:01 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T22:24:02.070 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:01 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:02.070 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:01 smithi107 ceph-mon[96543]: Deploying daemon osd.3 on smithi107 2023-07-21T22:24:02.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:01 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T22:24:02.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:01 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:02.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:01 smithi190 ceph-mon[98334]: Deploying daemon osd.3 on smithi107 2023-07-21T22:24:02.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:01 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T22:24:02.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:01 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:02.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:01 smithi132 ceph-mon[97863]: Deploying daemon osd.3 on smithi107 2023-07-21T22:24:03.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:02 smithi107 ceph-mon[96543]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-07-21T22:24:03.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:02 smithi190 ceph-mon[98334]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-07-21T22:24:03.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:02 smithi132 ceph-mon[97863]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-07-21T22:24:05.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:04 smithi107 ceph-mon[96543]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-07-21T22:24:05.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:04 smithi190 ceph-mon[98334]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-07-21T22:24:05.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:04 smithi132 ceph-mon[97863]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-07-21T22:24:06.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:06 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:06.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:06 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:06.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:06 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:06.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:06 smithi107 ceph-mon[96543]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-21T22:24:06.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:06 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:06.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:06 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:06.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:06 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:06.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:06 smithi190 ceph-mon[98334]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-21T22:24:06.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:06 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:06.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:06 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:06.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:06 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:06.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:06 smithi132 ceph-mon[97863]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-21T22:24:08.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:08.279+0000 7f35ec940700 1 -- 172.21.15.107:0/3434653949 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f35e8060950 con 0x7f35d00a1140 2023-07-21T22:24:08.325 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 3 on host 'smithi107' 2023-07-21T22:24:08.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:08.281+0000 7f35ed942700 1 -- 172.21.15.107:0/3434653949 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f35d00a1140 msgr2=0x7f35d00a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:08.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:08.281+0000 7f35ed942700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f35d00a1140 0x7f35d00a35f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f35d80095c0 tx=0x7f35d8009530 comp rx=0 tx=0).stop 2023-07-21T22:24:08.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:08.281+0000 7f35ed942700 1 -- 172.21.15.107:0/3434653949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35e8100e10 msgr2=0x7f35e8195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:08.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:08.281+0000 7f35ed942700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35e8100e10 0x7f35e8195e40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00b560 tx=0x7f35dc00b920 comp rx=0 tx=0).stop 2023-07-21T22:24:08.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:08.281+0000 7f35ed942700 1 -- 172.21.15.107:0/3434653949 shutdown_connections 2023-07-21T22:24:08.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:08.281+0000 7f35ed942700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35e81017a0 0x7f35e819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:08.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:08.281+0000 7f35ed942700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f35d00a1140 0x7f35d00a35f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:08.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:08.281+0000 7f35ed942700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35e80ffc10 0x7f35e8195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:08.281+0000 7f35ed942700 1 --2- 172.21.15.107:0/3434653949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35e8100e10 0x7f35e8195e40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:08.281+0000 7f35ed942700 1 -- 172.21.15.107:0/3434653949 >> 172.21.15.107:0/3434653949 conn(0x7f35e80fb220 msgr2=0x7f35e80fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:08.281+0000 7f35ed942700 1 -- 172.21.15.107:0/3434653949 shutdown_connections 2023-07-21T22:24:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:08.281+0000 7f35ed942700 1 -- 172.21.15.107:0/3434653949 wait complete. 2023-07-21T22:24:08.739 DEBUG:teuthology.orchestra.run.smithi107:osd.3> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.3.service 2023-07-21T22:24:08.741 INFO:tasks.cephadm:Deploying osd.4 on smithi132 with /dev/vg_nvme/lv_3... 2023-07-21T22:24:08.742 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T22:24:08.773 INFO:journalctl@ceph.osd.3.smithi107.stdout:-- Logs begin at Fri 2023-07-21 22:03:40 UTC. -- 2023-07-21T22:24:09.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:08 smithi190 ceph-mon[98334]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-21T22:24:09.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:08 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:09.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:08 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:09.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:08 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:09.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:08 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:09.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:08 smithi132 ceph-mon[97863]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-21T22:24:09.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:08 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:09.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:08 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:09.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:08 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:09.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:08 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:08 smithi107 ceph-mon[96543]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-21T22:24:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:09.310 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:24:09.610 INFO:journalctl@ceph.osd.3.smithi107.stdout:Jul 21 22:24:09 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-3[114835]: 2023-07-21T22:24:09.454+0000 7f92c7465640 -1 osd.3 0 log_to_monitors true 2023-07-21T22:24:09.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:09 smithi107 ceph-mon[96543]: from='osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T22:24:10.064 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:09 smithi132 ceph-mon[97863]: from='osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T22:24:10.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:09 smithi190 ceph-mon[98334]: from='osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T22:24:10.788 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T22:24:10.788 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T22:24:10.788 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2023-07-21T22:24:10.789 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2023-07-21T22:24:10.789 INFO:teuthology.orchestra.run.smithi132.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0149398 s, 702 MB/s 2023-07-21T22:24:10.789 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2023-07-21T22:24:10.937 INFO:journalctl@ceph.osd.3.smithi107.stdout:Jul 21 22:24:10 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-3[114835]: 2023-07-21T22:24:10.758+0000 7f92bdadf700 -1 osd.3 0 waiting for initial osdmap 2023-07-21T22:24:10.937 INFO:journalctl@ceph.osd.3.smithi107.stdout:Jul 21 22:24:10 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-3[114835]: 2023-07-21T22:24:10.764+0000 7f92b58cc700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:24:10.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:10 smithi107 ceph-mon[96543]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-21T22:24:10.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:10 smithi107 ceph-mon[96543]: from='osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T22:24:10.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:10 smithi107 ceph-mon[96543]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T22:24:10.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:10 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:24:10.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:10 smithi107 ceph-mon[96543]: from='osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2023-07-21T22:24:11.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:10 smithi190 ceph-mon[98334]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-21T22:24:11.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:10 smithi190 ceph-mon[98334]: from='osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T22:24:11.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:10 smithi190 ceph-mon[98334]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T22:24:11.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:10 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:24:11.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:10 smithi190 ceph-mon[98334]: from='osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2023-07-21T22:24:11.180 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:10 smithi132 ceph-mon[97863]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-21T22:24:11.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:10 smithi132 ceph-mon[97863]: from='osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T22:24:11.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:10 smithi132 ceph-mon[97863]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T22:24:11.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:10 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:24:11.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:10 smithi132 ceph-mon[97863]: from='osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2023-07-21T22:24:11.251 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi132:vg_nvme/lv_3 2023-07-21T22:24:11.804 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:24:12.055 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2023-07-21T22:24:12.055 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: osdmap e24: 4 total, 3 up, 4 in 2023-07-21T22:24:12.055 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:24:12.055 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:24:12.056 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004] boot 2023-07-21T22:24:12.056 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: osdmap e25: 4 total, 4 up, 4 in 2023-07-21T22:24:12.056 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:24:12.056 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: Detected new or changed devices on smithi107 2023-07-21T22:24:12.056 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.056 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.056 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:12.057 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:12.057 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi107 to 8584M 2023-07-21T22:24:12.057 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.057 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:12.057 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:12.057 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.057 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:12.057 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:12.058 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:12.058 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2023-07-21T22:24:12.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: osdmap e24: 4 total, 3 up, 4 in 2023-07-21T22:24:12.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:24:12.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:24:12.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004] boot 2023-07-21T22:24:12.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: osdmap e25: 4 total, 4 up, 4 in 2023-07-21T22:24:12.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:24:12.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: Detected new or changed devices on smithi107 2023-07-21T22:24:12.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:12.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:12.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi107 to 8584M 2023-07-21T22:24:12.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:12.143 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:12.143 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.143 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:12.143 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:12.143 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:12.143 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2023-07-21T22:24:12.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: osdmap e24: 4 total, 3 up, 4 in 2023-07-21T22:24:12.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:24:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:24:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: osd.3 [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004] boot 2023-07-21T22:24:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: osdmap e25: 4 total, 4 up, 4 in 2023-07-21T22:24:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:24:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: Detected new or changed devices on smithi107 2023-07-21T22:24:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:12.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:12.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi107 to 8584M 2023-07-21T22:24:12.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:12.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:12.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:12.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:12.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:12.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:12.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:13.106 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.059+0000 7fe2262bb700 1 -- 172.21.15.132:0/2748085811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe220100e30 msgr2=0x7fe220101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:13.107 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.059+0000 7fe2262bb700 1 --2- 172.21.15.132:0/2748085811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe220100e30 0x7fe220101280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe214009990 tx=0x7fe214009ca0 comp rx=0 tx=0).stop 2023-07-21T22:24:13.107 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.059+0000 7fe2262bb700 1 -- 172.21.15.132:0/2748085811 shutdown_connections 2023-07-21T22:24:13.107 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.059+0000 7fe2262bb700 1 --2- 172.21.15.132:0/2748085811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2201017c0 0x7fe220108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:13.107 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.059+0000 7fe2262bb700 1 --2- 172.21.15.132:0/2748085811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2200ffc30 0x7fe220100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:13.107 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.059+0000 7fe2262bb700 1 --2- 172.21.15.132:0/2748085811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe220100e30 0x7fe220101280 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:13.107 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.059+0000 7fe2262bb700 1 -- 172.21.15.132:0/2748085811 >> 172.21.15.132:0/2748085811 conn(0x7fe2200fb2a0 msgr2=0x7fe2200fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:13.108 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.060+0000 7fe2262bb700 1 -- 172.21.15.132:0/2748085811 shutdown_connections 2023-07-21T22:24:13.108 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.060+0000 7fe2262bb700 1 -- 172.21.15.132:0/2748085811 wait complete. 2023-07-21T22:24:13.108 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.060+0000 7fe2262bb700 1 Processor -- start 2023-07-21T22:24:13.108 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.060+0000 7fe2262bb700 1 -- start start 2023-07-21T22:24:13.108 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.060+0000 7fe2262bb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2200ffc30 0x7fe2201958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:13.108 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.060+0000 7fe2262bb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220100e30 0x7fe220195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:13.108 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.060+0000 7fe2262bb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2201017c0 0x7fe22019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:13.109 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.060+0000 7fe2262bb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22019d460 con 0x7fe2200ffc30 2023-07-21T22:24:13.109 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.060+0000 7fe2262bb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22019d5a0 con 0x7fe2201017c0 2023-07-21T22:24:13.109 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.060+0000 7fe2262bb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22019d730 con 0x7fe220100e30 2023-07-21T22:24:13.109 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe224858700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2201017c0 0x7fe22019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:13.109 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe224858700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2201017c0 0x7fe22019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50542/0 (socket says 172.21.15.132:50542) 2023-07-21T22:24:13.109 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe224858700 1 -- 172.21.15.132:0/2376453195 learned_addr learned my addr 172.21.15.132:0/2376453195 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-07-21T22:24:13.109 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe21ffff700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2200ffc30 0x7fe2201958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:13.110 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe21f7fe700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220100e30 0x7fe220195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:13.110 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe224858700 1 -- 172.21.15.132:0/2376453195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220100e30 msgr2=0x7fe220195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:13.110 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe224858700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220100e30 0x7fe220195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:13.110 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe224858700 1 -- 172.21.15.132:0/2376453195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2200ffc30 msgr2=0x7fe2201958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:13.110 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe224858700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2200ffc30 0x7fe2201958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:13.110 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe224858700 1 -- 172.21.15.132:0/2376453195 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2140095f0 con 0x7fe2201017c0 2023-07-21T22:24:13.110 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe224858700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2201017c0 0x7fe22019ce90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe21800ada0 tx=0x7fe21800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:13.111 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe21d7fa700 1 -- 172.21.15.132:0/2376453195 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800f070 con 0x7fe2201017c0 2023-07-21T22:24:13.111 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe21d7fa700 1 -- 172.21.15.132:0/2376453195 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe218007b30 con 0x7fe2201017c0 2023-07-21T22:24:13.111 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe21d7fa700 1 -- 172.21.15.132:0/2376453195 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218005280 con 0x7fe2201017c0 2023-07-21T22:24:13.111 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe2262bb700 1 -- 172.21.15.132:0/2376453195 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22019da10 con 0x7fe2201017c0 2023-07-21T22:24:13.111 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.061+0000 7fe2262bb700 1 -- 172.21.15.132:0/2376453195 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22019dec0 con 0x7fe2201017c0 2023-07-21T22:24:13.112 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.062+0000 7fe2262bb700 1 -- 172.21.15.132:0/2376453195 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22010b3f0 con 0x7fe2201017c0 2023-07-21T22:24:13.112 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.063+0000 7fe21d7fa700 1 -- 172.21.15.132:0/2376453195 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe21800c8d0 con 0x7fe2201017c0 2023-07-21T22:24:13.112 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.064+0000 7fe21d7fa700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe2080a1150 0x7fe2080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:13.112 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.064+0000 7fe21d7fa700 1 -- 172.21.15.132:0/2376453195 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fe2180cac90 con 0x7fe2201017c0 2023-07-21T22:24:13.112 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.064+0000 7fe21ffff700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe2080a1150 0x7fe2080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:13.113 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.065+0000 7fe21ffff700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe2080a1150 0x7fe2080a3600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe220100c90 tx=0x7fe21000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:13.114 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.068+0000 7fe21d7fa700 1 -- 172.21.15.132:0/2376453195 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe218095670 con 0x7fe2201017c0 2023-07-21T22:24:13.180 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:12 smithi132 ceph-mon[97863]: purged_snaps scrub starts 2023-07-21T22:24:13.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:12 smithi132 ceph-mon[97863]: purged_snaps scrub ok 2023-07-21T22:24:13.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:12 smithi132 ceph-mon[97863]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:24:13.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:12 smithi132 ceph-mon[97863]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T22:24:13.309 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:13.264+0000 7fe2262bb700 1 -- 172.21.15.132:0/2376453195 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe220060950 con 0x7fe2080a1150 2023-07-21T22:24:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:12 smithi190 ceph-mon[98334]: purged_snaps scrub starts 2023-07-21T22:24:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:12 smithi190 ceph-mon[98334]: purged_snaps scrub ok 2023-07-21T22:24:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:12 smithi190 ceph-mon[98334]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:24:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:12 smithi190 ceph-mon[98334]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T22:24:13.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:12 smithi107 ceph-mon[96543]: purged_snaps scrub starts 2023-07-21T22:24:13.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:12 smithi107 ceph-mon[96543]: purged_snaps scrub ok 2023-07-21T22:24:13.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:12 smithi107 ceph-mon[96543]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:24:13.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:12 smithi107 ceph-mon[96543]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T22:24:14.352 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:13 smithi132 ceph-mon[97863]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T22:24:14.352 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:24:14.353 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:24:14.353 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:14.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:13 smithi190 ceph-mon[98334]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T22:24:14.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:24:14.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:24:14.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:14.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:13 smithi107 ceph-mon[96543]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T22:24:14.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:24:14.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:24:14.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:15.169 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:14 smithi132 ceph-mon[97863]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:24:15.170 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:14 smithi132 ceph-mon[97863]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:15.170 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:14 smithi132 ceph-mon[97863]: osdmap e28: 4 total, 4 up, 4 in 2023-07-21T22:24:15.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:14 smithi190 ceph-mon[98334]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:24:15.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:14 smithi190 ceph-mon[98334]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:15.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:14 smithi190 ceph-mon[98334]: osdmap e28: 4 total, 4 up, 4 in 2023-07-21T22:24:15.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:14 smithi107 ceph-mon[96543]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:24:15.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:14 smithi107 ceph-mon[96543]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:15.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:14 smithi107 ceph-mon[96543]: osdmap e28: 4 total, 4 up, 4 in 2023-07-21T22:24:16.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:15 smithi190 ceph-mon[98334]: osdmap e29: 4 total, 4 up, 4 in 2023-07-21T22:24:16.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:15 smithi190 ceph-mon[98334]: from='client.? 172.21.15.132:0/2837197401' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f114d7a-5079-4c48-a9fa-c477d9447aa7"}]: dispatch 2023-07-21T22:24:16.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:15 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f114d7a-5079-4c48-a9fa-c477d9447aa7"}]: dispatch 2023-07-21T22:24:16.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:15 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f114d7a-5079-4c48-a9fa-c477d9447aa7"}]': finished 2023-07-21T22:24:16.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:15 smithi190 ceph-mon[98334]: osdmap e30: 5 total, 4 up, 5 in 2023-07-21T22:24:16.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:16.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:15 smithi132 ceph-mon[97863]: osdmap e29: 4 total, 4 up, 4 in 2023-07-21T22:24:16.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:15 smithi132 ceph-mon[97863]: from='client.? 172.21.15.132:0/2837197401' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f114d7a-5079-4c48-a9fa-c477d9447aa7"}]: dispatch 2023-07-21T22:24:16.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:15 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f114d7a-5079-4c48-a9fa-c477d9447aa7"}]: dispatch 2023-07-21T22:24:16.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:15 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f114d7a-5079-4c48-a9fa-c477d9447aa7"}]': finished 2023-07-21T22:24:16.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:15 smithi132 ceph-mon[97863]: osdmap e30: 5 total, 4 up, 5 in 2023-07-21T22:24:16.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:16.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:15 smithi107 ceph-mon[96543]: osdmap e29: 4 total, 4 up, 4 in 2023-07-21T22:24:16.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:15 smithi107 ceph-mon[96543]: from='client.? 172.21.15.132:0/2837197401' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f114d7a-5079-4c48-a9fa-c477d9447aa7"}]: dispatch 2023-07-21T22:24:16.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:15 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f114d7a-5079-4c48-a9fa-c477d9447aa7"}]: dispatch 2023-07-21T22:24:16.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:15 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f114d7a-5079-4c48-a9fa-c477d9447aa7"}]': finished 2023-07-21T22:24:16.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:15 smithi107 ceph-mon[96543]: osdmap e30: 5 total, 4 up, 5 in 2023-07-21T22:24:16.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:17.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:16 smithi190 ceph-mon[98334]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:24:17.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:16 smithi190 ceph-mon[98334]: from='client.? 172.21.15.132:0/158913146' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:24:17.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:16 smithi132 ceph-mon[97863]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:24:17.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:16 smithi132 ceph-mon[97863]: from='client.? 172.21.15.132:0/158913146' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:24:17.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:16 smithi107 ceph-mon[96543]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:24:17.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:16 smithi107 ceph-mon[96543]: from='client.? 172.21.15.132:0/158913146' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:24:19.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:18 smithi190 ceph-mon[98334]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 121 KiB/s, 0 objects/s recovering 2023-07-21T22:24:19.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:18 smithi132 ceph-mon[97863]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 121 KiB/s, 0 objects/s recovering 2023-07-21T22:24:19.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:18 smithi107 ceph-mon[96543]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 121 KiB/s, 0 objects/s recovering 2023-07-21T22:24:21.337 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:20 smithi132 ceph-mon[97863]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:24:21.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:20 smithi190 ceph-mon[98334]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:24:21.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:20 smithi107 ceph-mon[96543]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:24:23.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:23 smithi190 ceph-mon[98334]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-07-21T22:24:23.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:23 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T22:24:23.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:23 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:23.395 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:23 smithi132 ceph-mon[97863]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-07-21T22:24:23.396 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:23 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T22:24:23.396 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:23 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:23.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:23 smithi107 ceph-mon[96543]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-07-21T22:24:23.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:23 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T22:24:23.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:23 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:24.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:24 smithi190 ceph-mon[98334]: Deploying daemon osd.4 on smithi132 2023-07-21T22:24:24.398 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:24 smithi132 ceph-mon[97863]: Deploying daemon osd.4 on smithi132 2023-07-21T22:24:24.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:24 smithi107 ceph-mon[96543]: Deploying daemon osd.4 on smithi132 2023-07-21T22:24:25.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:25 smithi190 ceph-mon[98334]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-07-21T22:24:25.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:25 smithi132 ceph-mon[97863]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-07-21T22:24:25.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:25 smithi107 ceph-mon[96543]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-07-21T22:24:27.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:27 smithi190 ceph-mon[98334]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:27.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:27.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:27.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:27.418 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:27 smithi132 ceph-mon[97863]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:27.418 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:27 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:27.418 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:27 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:27.418 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:27 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:27.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:27 smithi107 ceph-mon[96543]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:27.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:27.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:27.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:29.326 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:29 smithi132 ceph-mon[97863]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:29.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:29 smithi190 ceph-mon[98334]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:29.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:29 smithi107 ceph-mon[96543]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:29.672 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 4 on host 'smithi132' 2023-07-21T22:24:29.672 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:29.626+0000 7fe21d7fa700 1 -- 172.21.15.132:0/2376453195 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe220060950 con 0x7fe2080a1150 2023-07-21T22:24:29.674 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:29.628+0000 7fe2262bb700 1 -- 172.21.15.132:0/2376453195 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe2080a1150 msgr2=0x7fe2080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:29.674 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:29.628+0000 7fe2262bb700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe2080a1150 0x7fe2080a3600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe220100c90 tx=0x7fe21000b410 comp rx=0 tx=0).stop 2023-07-21T22:24:29.674 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:29.628+0000 7fe2262bb700 1 -- 172.21.15.132:0/2376453195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2201017c0 msgr2=0x7fe22019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:29.674 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:29.628+0000 7fe2262bb700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2201017c0 0x7fe22019ce90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe21800ada0 tx=0x7fe21800c330 comp rx=0 tx=0).stop 2023-07-21T22:24:29.674 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:29.628+0000 7fe2262bb700 1 -- 172.21.15.132:0/2376453195 shutdown_connections 2023-07-21T22:24:29.674 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:29.628+0000 7fe2262bb700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2201017c0 0x7fe22019ce90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:29.675 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:29.628+0000 7fe2262bb700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2200ffc30 0x7fe2201958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:29.675 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:29.628+0000 7fe2262bb700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe2080a1150 0x7fe2080a3600 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:29.675 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:29.628+0000 7fe2262bb700 1 --2- 172.21.15.132:0/2376453195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe220100e30 0x7fe220195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:29.675 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:29.628+0000 7fe2262bb700 1 -- 172.21.15.132:0/2376453195 >> 172.21.15.132:0/2376453195 conn(0x7fe2200fb2a0 msgr2=0x7fe2200fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:29.675 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:29.628+0000 7fe2262bb700 1 -- 172.21.15.132:0/2376453195 shutdown_connections 2023-07-21T22:24:29.675 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:24:29.629+0000 7fe2262bb700 1 -- 172.21.15.132:0/2376453195 wait complete. 2023-07-21T22:24:30.096 DEBUG:teuthology.orchestra.run.smithi132:osd.4> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.4.service 2023-07-21T22:24:30.098 INFO:tasks.cephadm:Deploying osd.5 on smithi190 with /dev/vg_nvme/lv_3... 2023-07-21T22:24:30.098 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T22:24:30.126 INFO:journalctl@ceph.osd.4.smithi132.stdout:-- Logs begin at Fri 2023-07-21 22:03:19 UTC. -- 2023-07-21T22:24:30.389 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:30.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:30.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:30.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:30.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:30.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:30.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:30.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:30.661 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:24:30.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:30.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:30.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:30.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:31.227 INFO:journalctl@ceph.osd.4.smithi132.stdout:Jul 21 22:24:30 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-4[108301]: 2023-07-21T22:24:30.917+0000 7f575b321640 -1 osd.4 0 log_to_monitors true 2023-07-21T22:24:31.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:31 smithi190 ceph-mon[98334]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:31.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:31 smithi190 ceph-mon[98334]: from='osd.4 [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:24:31.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:31 smithi190 ceph-mon[98334]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:24:31.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:31 smithi132 ceph-mon[97863]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:31.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:31 smithi132 ceph-mon[97863]: from='osd.4 [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:24:31.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:31 smithi132 ceph-mon[97863]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:24:31.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:31 smithi107 ceph-mon[96543]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:31.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:31 smithi107 ceph-mon[96543]: from='osd.4 [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:24:31.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:31 smithi107 ceph-mon[96543]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:24:32.133 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T22:24:32.133 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T22:24:32.133 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2023-07-21T22:24:32.133 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2023-07-21T22:24:32.133 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011035 s, 950 MB/s 2023-07-21T22:24:32.134 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2023-07-21T22:24:32.503 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi190:vg_nvme/lv_3 2023-07-21T22:24:32.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T22:24:32.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: osdmap e31: 5 total, 4 up, 5 in 2023-07-21T22:24:32.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:32.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='osd.4 [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:24:32.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:24:32.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:32.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:32.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:32.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:32.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:32.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:32.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:32.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:32.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.931 INFO:journalctl@ceph.osd.4.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-4[108301]: 2023-07-21T22:24:32.629+0000 7f5750198700 -1 osd.4 0 waiting for initial osdmap 2023-07-21T22:24:32.931 INFO:journalctl@ceph.osd.4.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-4[108301]: 2023-07-21T22:24:32.635+0000 7f574af8b700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:24:32.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T22:24:32.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: osdmap e31: 5 total, 4 up, 5 in 2023-07-21T22:24:32.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:32.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='osd.4 [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:24:32.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:24:32.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:32.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:32.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:32.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:32.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:32.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:32.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:32.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:32.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T22:24:32.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: osdmap e31: 5 total, 4 up, 5 in 2023-07-21T22:24:32.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:32.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='osd.4 [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:24:32.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:24:32.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:32.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:32.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:32.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:32.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:32.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:32.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:32.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:32.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:32.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:33.053 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:24:33.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:33 smithi190 ceph-mon[98334]: Detected new or changed devices on smithi132 2023-07-21T22:24:33.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:33 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi132 to 8584M 2023-07-21T22:24:33.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:33 smithi190 ceph-mon[98334]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-07-21T22:24:33.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:33 smithi190 ceph-mon[98334]: osdmap e32: 5 total, 4 up, 5 in 2023-07-21T22:24:33.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:33 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:33.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:33 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:33.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:33 smithi132 ceph-mon[97863]: Detected new or changed devices on smithi132 2023-07-21T22:24:33.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:33 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi132 to 8584M 2023-07-21T22:24:33.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:33 smithi132 ceph-mon[97863]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-07-21T22:24:33.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:33 smithi132 ceph-mon[97863]: osdmap e32: 5 total, 4 up, 5 in 2023-07-21T22:24:33.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:33 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:33.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:33 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:33.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:33 smithi107 ceph-mon[96543]: Detected new or changed devices on smithi132 2023-07-21T22:24:33.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:33 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi132 to 8584M 2023-07-21T22:24:33.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:33 smithi107 ceph-mon[96543]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-07-21T22:24:33.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:33 smithi107 ceph-mon[96543]: osdmap e32: 5 total, 4 up, 5 in 2023-07-21T22:24:33.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:33 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:33.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:33 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:34.148 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.103+0000 7f8205269700 1 -- 172.21.15.190:0/28397948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82000fde70 msgr2=0x7f82000fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:34.148 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.103+0000 7f8205269700 1 --2- 172.21.15.190:0/28397948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82000fde70 0x7f82000fe280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009990 tx=0x7f81f4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:24:34.149 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.103+0000 7f8205269700 1 -- 172.21.15.190:0/28397948 shutdown_connections 2023-07-21T22:24:34.149 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.103+0000 7f8205269700 1 --2- 172.21.15.190:0/28397948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8200101880 0x7f8200108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:34.149 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.103+0000 7f8205269700 1 --2- 172.21.15.190:0/28397948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82000fde70 0x7f82000fe280 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:34.149 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.103+0000 7f8205269700 1 --2- 172.21.15.190:0/28397948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8200100ef0 0x7f8200101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:34.149 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.103+0000 7f8205269700 1 -- 172.21.15.190:0/28397948 >> 172.21.15.190:0/28397948 conn(0x7f82000f9860 msgr2=0x7f82000fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:34.149 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.103+0000 7f8205269700 1 -- 172.21.15.190:0/28397948 shutdown_connections 2023-07-21T22:24:34.149 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.103+0000 7f8205269700 1 -- 172.21.15.190:0/28397948 wait complete. 2023-07-21T22:24:34.151 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f8205269700 1 Processor -- start 2023-07-21T22:24:34.151 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f8205269700 1 -- start start 2023-07-21T22:24:34.151 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f8205269700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82000fde70 0x7f8200197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:34.151 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f8205269700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8200100ef0 0x7f8200198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:34.151 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f8205269700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8200101880 0x7f820019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:34.152 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f8205269700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820019f660 con 0x7f8200101880 2023-07-21T22:24:34.152 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f8205269700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820019f7a0 con 0x7f82000fde70 2023-07-21T22:24:34.152 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f8205269700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820019f930 con 0x7f8200100ef0 2023-07-21T22:24:34.152 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f81fe7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8200100ef0 0x7f8200198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:34.152 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f81fe7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8200100ef0 0x7f8200198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:48044/0 (socket says 172.21.15.190:48044) 2023-07-21T22:24:34.152 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f81fe7fc700 1 -- 172.21.15.190:0/149772586 learned_addr learned my addr 172.21.15.190:0/149772586 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-07-21T22:24:34.153 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f81feffd700 1 --2- 172.21.15.190:0/149772586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82000fde70 0x7f8200197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:34.153 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f81fe7fc700 1 -- 172.21.15.190:0/149772586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82000fde70 msgr2=0x7f8200197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:34.153 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f81fe7fc700 1 --2- 172.21.15.190:0/149772586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82000fde70 0x7f8200197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:34.153 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f81fe7fc700 1 -- 172.21.15.190:0/149772586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8200101880 msgr2=0x7f820019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:34.153 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f81fe7fc700 1 --2- 172.21.15.190:0/149772586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8200101880 0x7f820019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:34.153 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.104+0000 7f81fe7fc700 1 -- 172.21.15.190:0/149772586 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f40095f0 con 0x7f8200100ef0 2023-07-21T22:24:34.153 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.105+0000 7f81fe7fc700 1 --2- 172.21.15.190:0/149772586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8200100ef0 0x7f8200198010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f81f000d980 tx=0x7f81f000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:34.154 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.105+0000 7f81e7fff700 1 -- 172.21.15.190:0/149772586 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f0009b40 con 0x7f8200100ef0 2023-07-21T22:24:34.154 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.105+0000 7f8205269700 1 -- 172.21.15.190:0/149772586 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820019fc10 con 0x7f8200100ef0 2023-07-21T22:24:34.154 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.105+0000 7f81e7fff700 1 -- 172.21.15.190:0/149772586 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81f000b6e0 con 0x7f8200100ef0 2023-07-21T22:24:34.154 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.105+0000 7f81e7fff700 1 -- 172.21.15.190:0/149772586 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f000f620 con 0x7f8200100ef0 2023-07-21T22:24:34.154 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.105+0000 7f8205269700 1 -- 172.21.15.190:0/149772586 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82001a0110 con 0x7f8200100ef0 2023-07-21T22:24:34.154 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.106+0000 7f8205269700 1 -- 172.21.15.190:0/149772586 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820010b360 con 0x7f8200100ef0 2023-07-21T22:24:34.155 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.107+0000 7f81e7fff700 1 -- 172.21.15.190:0/149772586 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f81f0010460 con 0x7f8200100ef0 2023-07-21T22:24:34.155 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.107+0000 7f81e7fff700 1 --2- 172.21.15.190:0/149772586 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81e80a1150 0x7f81e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:34.155 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.107+0000 7f81e7fff700 1 -- 172.21.15.190:0/149772586 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f81f00cbb20 con 0x7f8200100ef0 2023-07-21T22:24:34.155 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.108+0000 7f81feffd700 1 --2- 172.21.15.190:0/149772586 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81e80a1150 0x7f81e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:34.156 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.108+0000 7f81feffd700 1 --2- 172.21.15.190:0/149772586 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81e80a1150 0x7f81e80a3600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000c00 tx=0x7f81f4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:34.156 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.111+0000 7f81e7fff700 1 -- 172.21.15.190:0/149772586 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f81f00962e0 con 0x7f8200100ef0 2023-07-21T22:24:34.348 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:34.303+0000 7f8205269700 1 -- 172.21.15.190:0/149772586 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8200107850 con 0x7f81e80a1150 2023-07-21T22:24:34.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:34 smithi107 ceph-mon[96543]: purged_snaps scrub starts 2023-07-21T22:24:34.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:34 smithi107 ceph-mon[96543]: purged_snaps scrub ok 2023-07-21T22:24:34.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:34 smithi107 ceph-mon[96543]: osd.4 [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216] boot 2023-07-21T22:24:34.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:34 smithi107 ceph-mon[96543]: osdmap e33: 5 total, 5 up, 5 in 2023-07-21T22:24:34.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:34.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:34 smithi107 ceph-mon[96543]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:34.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:24:34.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:24:34.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:35.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:34 smithi190 ceph-mon[98334]: purged_snaps scrub starts 2023-07-21T22:24:35.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:34 smithi190 ceph-mon[98334]: purged_snaps scrub ok 2023-07-21T22:24:35.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:34 smithi190 ceph-mon[98334]: osd.4 [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216] boot 2023-07-21T22:24:35.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:34 smithi190 ceph-mon[98334]: osdmap e33: 5 total, 5 up, 5 in 2023-07-21T22:24:35.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:35.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:34 smithi190 ceph-mon[98334]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:35.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:24:35.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:24:35.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:35.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:34 smithi132 ceph-mon[97863]: purged_snaps scrub starts 2023-07-21T22:24:35.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:34 smithi132 ceph-mon[97863]: purged_snaps scrub ok 2023-07-21T22:24:35.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:34 smithi132 ceph-mon[97863]: osd.4 [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216] boot 2023-07-21T22:24:35.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:34 smithi132 ceph-mon[97863]: osdmap e33: 5 total, 5 up, 5 in 2023-07-21T22:24:35.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:24:35.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:34 smithi132 ceph-mon[97863]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:24:35.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:24:35.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:24:35.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:35.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:35 smithi107 ceph-mon[96543]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:24:35.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:35 smithi107 ceph-mon[96543]: osdmap e34: 5 total, 5 up, 5 in 2023-07-21T22:24:35.938 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:35 smithi190 ceph-mon[98334]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:24:35.938 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:35 smithi190 ceph-mon[98334]: osdmap e34: 5 total, 5 up, 5 in 2023-07-21T22:24:36.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:35 smithi132 ceph-mon[97863]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:24:36.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:35 smithi132 ceph-mon[97863]: osdmap e34: 5 total, 5 up, 5 in 2023-07-21T22:24:37.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:36 smithi190 ceph-mon[98334]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:37.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:36 smithi190 ceph-mon[98334]: from='client.? 172.21.15.190:0/2135034320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93ba2f1d-2b22-4199-9cae-05fef817b637"}]: dispatch 2023-07-21T22:24:37.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:36 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93ba2f1d-2b22-4199-9cae-05fef817b637"}]: dispatch 2023-07-21T22:24:37.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:36 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93ba2f1d-2b22-4199-9cae-05fef817b637"}]': finished 2023-07-21T22:24:37.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:36 smithi190 ceph-mon[98334]: osdmap e35: 6 total, 5 up, 6 in 2023-07-21T22:24:37.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:36 smithi132 ceph-mon[97863]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:36 smithi132 ceph-mon[97863]: from='client.? 172.21.15.190:0/2135034320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93ba2f1d-2b22-4199-9cae-05fef817b637"}]: dispatch 2023-07-21T22:24:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:36 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93ba2f1d-2b22-4199-9cae-05fef817b637"}]: dispatch 2023-07-21T22:24:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:36 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93ba2f1d-2b22-4199-9cae-05fef817b637"}]': finished 2023-07-21T22:24:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:36 smithi132 ceph-mon[97863]: osdmap e35: 6 total, 5 up, 6 in 2023-07-21T22:24:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:37.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:36 smithi107 ceph-mon[96543]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:37.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:36 smithi107 ceph-mon[96543]: from='client.? 172.21.15.190:0/2135034320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93ba2f1d-2b22-4199-9cae-05fef817b637"}]: dispatch 2023-07-21T22:24:37.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:36 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93ba2f1d-2b22-4199-9cae-05fef817b637"}]: dispatch 2023-07-21T22:24:37.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:36 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93ba2f1d-2b22-4199-9cae-05fef817b637"}]': finished 2023-07-21T22:24:37.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:36 smithi107 ceph-mon[96543]: osdmap e35: 6 total, 5 up, 6 in 2023-07-21T22:24:37.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:38.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:37 smithi190 ceph-mon[98334]: from='client.? 172.21.15.190:0/3492914868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:24:38.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:37 smithi132 ceph-mon[97863]: from='client.? 172.21.15.190:0/3492914868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:24:38.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:37 smithi107 ceph-mon[96543]: from='client.? 172.21.15.190:0/3492914868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:24:39.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:38 smithi190 ceph-mon[98334]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:39.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:38 smithi132 ceph-mon[97863]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:39.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:38 smithi107 ceph-mon[96543]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:41.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:40 smithi190 ceph-mon[98334]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:41.430 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:40 smithi132 ceph-mon[97863]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:41.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:40 smithi107 ceph-mon[96543]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:43.348 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:42 smithi190 ceph-mon[98334]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:43.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:42 smithi132 ceph-mon[97863]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:43.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:42 smithi107 ceph-mon[96543]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:44.172 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:43 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T22:24:44.172 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:43 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:44.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:43 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T22:24:44.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:43 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:44.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:43 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T22:24:44.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:43 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:45.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:44 smithi190 ceph-mon[98334]: Deploying daemon osd.5 on smithi190 2023-07-21T22:24:45.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:44 smithi190 ceph-mon[98334]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:45.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:44 smithi132 ceph-mon[97863]: Deploying daemon osd.5 on smithi190 2023-07-21T22:24:45.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:44 smithi132 ceph-mon[97863]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:45.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:44 smithi107 ceph-mon[96543]: Deploying daemon osd.5 on smithi190 2023-07-21T22:24:45.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:44 smithi107 ceph-mon[96543]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:47.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:46 smithi190 ceph-mon[98334]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:47.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:46 smithi132 ceph-mon[97863]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:47.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:46 smithi107 ceph-mon[96543]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:48.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:48 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:48.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:48 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:48.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:48 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:48.847 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:48 smithi190 ceph-mon[98334]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:49.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:48 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:49.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:48 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:49.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:48 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:49.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:48 smithi132 ceph-mon[97863]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:49.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:48 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:49.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:48 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:49.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:48 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:49.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:48 smithi107 ceph-mon[96543]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:50.356 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:50.310+0000 7f81e7fff700 1 -- 172.21.15.190:0/149772586 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8200107850 con 0x7f81e80a1150 2023-07-21T22:24:50.356 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 5 on host 'smithi190' 2023-07-21T22:24:50.358 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:50.312+0000 7f8205269700 1 -- 172.21.15.190:0/149772586 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81e80a1150 msgr2=0x7f81e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:50.358 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:50.312+0000 7f8205269700 1 --2- 172.21.15.190:0/149772586 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81e80a1150 0x7f81e80a3600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000c00 tx=0x7f81f4005fb0 comp rx=0 tx=0).stop 2023-07-21T22:24:50.358 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:50.312+0000 7f8205269700 1 -- 172.21.15.190:0/149772586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8200100ef0 msgr2=0x7f8200198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:50.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:50.312+0000 7f8205269700 1 --2- 172.21.15.190:0/149772586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8200100ef0 0x7f8200198010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f81f000d980 tx=0x7f81f000dc90 comp rx=0 tx=0).stop 2023-07-21T22:24:50.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:50.312+0000 7f8205269700 1 -- 172.21.15.190:0/149772586 shutdown_connections 2023-07-21T22:24:50.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:50.312+0000 7f8205269700 1 --2- 172.21.15.190:0/149772586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8200101880 0x7f820019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:50.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:50.312+0000 7f8205269700 1 --2- 172.21.15.190:0/149772586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82000fde70 0x7f8200197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:50.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:50.312+0000 7f8205269700 1 --2- 172.21.15.190:0/149772586 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81e80a1150 0x7f81e80a3600 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:50.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:50.312+0000 7f8205269700 1 --2- 172.21.15.190:0/149772586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8200100ef0 0x7f8200198010 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:50.359 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:50.312+0000 7f8205269700 1 -- 172.21.15.190:0/149772586 >> 172.21.15.190:0/149772586 conn(0x7f82000f9860 msgr2=0x7f82000fbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:50.360 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:50.312+0000 7f8205269700 1 -- 172.21.15.190:0/149772586 shutdown_connections 2023-07-21T22:24:50.360 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:24:50.312+0000 7f8205269700 1 -- 172.21.15.190:0/149772586 wait complete. 2023-07-21T22:24:50.742 DEBUG:teuthology.orchestra.run.smithi190:osd.5> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.5.service 2023-07-21T22:24:50.743 INFO:tasks.cephadm:Deploying osd.6 on smithi107 with /dev/vg_nvme/lv_2... 2023-07-21T22:24:50.744 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T22:24:50.786 INFO:journalctl@ceph.osd.5.smithi190.stdout:-- Logs begin at Fri 2023-07-21 22:03:29 UTC. -- 2023-07-21T22:24:51.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:50 smithi190 ceph-mon[98334]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:51.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:51.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:51.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:51.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:50 smithi107 ceph-mon[96543]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:51.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:51.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:51.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:51.323 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:24:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:50 smithi132 ceph-mon[97863]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:51.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:51.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:51.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:51.890 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:24:51 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:24:51.641+0000 7f568fd71640 -1 osd.5 0 log_to_monitors true 2023-07-21T22:24:52.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:51 smithi107 ceph-mon[96543]: from='osd.5 [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:24:52.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:51 smithi107 ceph-mon[96543]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:24:52.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:51 smithi190 ceph-mon[98334]: from='osd.5 [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:24:52.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:51 smithi190 ceph-mon[98334]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:24:52.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:51 smithi132 ceph-mon[97863]: from='osd.5 [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:24:52.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:51 smithi132 ceph-mon[97863]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:24:52.823 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T22:24:52.823 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T22:24:52.823 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2023-07-21T22:24:52.823 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2023-07-21T22:24:52.824 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125615 s, 835 MB/s 2023-07-21T22:24:52.824 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 2023-07-21T22:24:52.824 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2023-07-21T22:24:53.177 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:52 smithi107 ceph-mon[96543]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:53.178 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:52 smithi107 ceph-mon[96543]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T22:24:53.178 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:52 smithi107 ceph-mon[96543]: osdmap e36: 6 total, 5 up, 6 in 2023-07-21T22:24:53.178 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:52 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:53.178 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:52 smithi107 ceph-mon[96543]: from='osd.5 [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:24:53.178 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:52 smithi107 ceph-mon[96543]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:24:53.179 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2023-07-21T22:24:53.390 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:24:52 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:24:52.986+0000 7f5684be8700 -1 osd.5 0 waiting for initial osdmap 2023-07-21T22:24:53.390 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:24:52 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:24:52.993+0000 7f567e9d9700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:24:53.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:52 smithi190 ceph-mon[98334]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:53.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:52 smithi190 ceph-mon[98334]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T22:24:53.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:52 smithi190 ceph-mon[98334]: osdmap e36: 6 total, 5 up, 6 in 2023-07-21T22:24:53.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:52 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:53.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:52 smithi190 ceph-mon[98334]: from='osd.5 [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:24:53.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:52 smithi190 ceph-mon[98334]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:24:53.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:52 smithi132 ceph-mon[97863]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:53.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:52 smithi132 ceph-mon[97863]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T22:24:53.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:52 smithi132 ceph-mon[97863]: osdmap e36: 6 total, 5 up, 6 in 2023-07-21T22:24:53.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:52 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:53.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:52 smithi132 ceph-mon[97863]: from='osd.5 [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:24:53.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:52 smithi132 ceph-mon[97863]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:24:53.734 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:24:54.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-07-21T22:24:54.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: osdmap e37: 6 total, 5 up, 6 in 2023-07-21T22:24:54.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:54.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:54.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: Detected new or changed devices on smithi190 2023-07-21T22:24:54.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:54.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:54.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:54.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:54.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi190 to 8584M 2023-07-21T22:24:54.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:54.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:54.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:54.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:54.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-07-21T22:24:54.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: osdmap e37: 6 total, 5 up, 6 in 2023-07-21T22:24:54.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:54.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:54.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: Detected new or changed devices on smithi190 2023-07-21T22:24:54.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:54.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:54.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:54.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:54.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi190 to 8584M 2023-07-21T22:24:54.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:54.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:54.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:54.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:54.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-07-21T22:24:54.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: osdmap e37: 6 total, 5 up, 6 in 2023-07-21T22:24:54.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:54.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:54.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: Detected new or changed devices on smithi190 2023-07-21T22:24:54.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:54.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:54.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:54.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:24:54.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi190 to 8584M 2023-07-21T22:24:54.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:54.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:54.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:54.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:24:54.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.916+0000 7f2048931700 1 -- 172.21.15.107:0/2958796439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20440ffda0 msgr2=0x7f2044106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:54.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.916+0000 7f2048931700 1 --2- 172.21.15.107:0/2958796439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20440ffda0 0x7f2044106660 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f203c00b0a0 tx=0x7f203c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:24:54.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.916+0000 7f2048931700 1 -- 172.21.15.107:0/2958796439 shutdown_connections 2023-07-21T22:24:54.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.916+0000 7f2048931700 1 --2- 172.21.15.107:0/2958796439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20440ffda0 0x7f2044106660 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:54.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.916+0000 7f2048931700 1 --2- 172.21.15.107:0/2958796439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20440fe210 0x7f20440fe620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:54.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.917+0000 7f2048931700 1 --2- 172.21.15.107:0/2958796439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20440ff410 0x7f20440ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:54.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.917+0000 7f2048931700 1 -- 172.21.15.107:0/2958796439 >> 172.21.15.107:0/2958796439 conn(0x7f20440f9860 msgr2=0x7f20440fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:54.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.917+0000 7f2048931700 1 -- 172.21.15.107:0/2958796439 shutdown_connections 2023-07-21T22:24:54.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.917+0000 7f2048931700 1 -- 172.21.15.107:0/2958796439 wait complete. 2023-07-21T22:24:54.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.917+0000 7f2048931700 1 Processor -- start 2023-07-21T22:24:54.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.917+0000 7f2048931700 1 -- start start 2023-07-21T22:24:54.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2048931700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20440fe210 0x7f20441958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:54.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2048931700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20440ff410 0x7f2044195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:54.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2048931700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20440ffda0 0x7f204419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:54.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2048931700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204419d470 con 0x7f20440ff410 2023-07-21T22:24:54.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2048931700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204419d5b0 con 0x7f20440ffda0 2023-07-21T22:24:54.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2048931700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204419d740 con 0x7f20440fe210 2023-07-21T22:24:54.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2041d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20440ff410 0x7f2044195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:54.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2041d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20440ff410 0x7f2044195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45734/0 (socket says 172.21.15.107:45734) 2023-07-21T22:24:54.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2041d9b700 1 -- 172.21.15.107:0/571299678 learned_addr learned my addr 172.21.15.107:0/571299678 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:24:54.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f204259c700 1 --2- 172.21.15.107:0/571299678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20440fe210 0x7f20441958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:54.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2041d9b700 1 -- 172.21.15.107:0/571299678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20440fe210 msgr2=0x7f20441958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:54.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2041d9b700 1 --2- 172.21.15.107:0/571299678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20440fe210 0x7f20441958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:54.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2041d9b700 1 -- 172.21.15.107:0/571299678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20440ffda0 msgr2=0x7f204419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:54.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2041d9b700 1 --2- 172.21.15.107:0/571299678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20440ffda0 0x7f204419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:54.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2041d9b700 1 -- 172.21.15.107:0/571299678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c009d00 con 0x7f20440ff410 2023-07-21T22:24:54.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2041d9b700 1 --2- 172.21.15.107:0/571299678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20440ff410 0x7f2044195e20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f203400c9e0 tx=0x7f203400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:54.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f20337fe700 1 -- 172.21.15.107:0/571299678 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034007ac0 con 0x7f20440ff410 2023-07-21T22:24:54.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f20337fe700 1 -- 172.21.15.107:0/571299678 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2034007c20 con 0x7f20440ff410 2023-07-21T22:24:54.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2048931700 1 -- 172.21.15.107:0/571299678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204419da20 con 0x7f20440ff410 2023-07-21T22:24:54.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f20337fe700 1 -- 172.21.15.107:0/571299678 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034018660 con 0x7f20440ff410 2023-07-21T22:24:54.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.918+0000 7f2048931700 1 -- 172.21.15.107:0/571299678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204419df70 con 0x7f20440ff410 2023-07-21T22:24:54.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.919+0000 7f2048931700 1 -- 172.21.15.107:0/571299678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204410a2f0 con 0x7f20440ff410 2023-07-21T22:24:54.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.920+0000 7f20337fe700 1 -- 172.21.15.107:0/571299678 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f203400f450 con 0x7f20440ff410 2023-07-21T22:24:54.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.921+0000 7f20337fe700 1 --2- 172.21.15.107:0/571299678 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f202c0a11d0 0x7f202c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:54.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.921+0000 7f204259c700 1 --2- 172.21.15.107:0/571299678 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f202c0a11d0 0x7f202c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:54.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.921+0000 7f20337fe700 1 -- 172.21.15.107:0/571299678 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4566+0+0 (secure 0 0 0) 0x7f20340cb9c0 con 0x7f20440ff410 2023-07-21T22:24:54.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.921+0000 7f204259c700 1 --2- 172.21.15.107:0/571299678 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f202c0a11d0 0x7f202c0a3680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2038005fd0 tx=0x7f2038005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:54.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:54.925+0000 7f20337fe700 1 -- 172.21.15.107:0/571299678 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2034095fb0 con 0x7f20440ff410 2023-07-21T22:24:55.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:24:55.122+0000 7f2048931700 1 -- 172.21.15.107:0/571299678 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2044107660 con 0x7f202c0a11d0 2023-07-21T22:24:55.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:54 smithi190 ceph-mon[98334]: purged_snaps scrub starts 2023-07-21T22:24:55.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:54 smithi190 ceph-mon[98334]: purged_snaps scrub ok 2023-07-21T22:24:55.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:54 smithi190 ceph-mon[98334]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:55.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:54 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:55.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:54 smithi190 ceph-mon[98334]: osd.5 [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533] boot 2023-07-21T22:24:55.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:54 smithi190 ceph-mon[98334]: osdmap e38: 6 total, 6 up, 6 in 2023-07-21T22:24:55.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:54 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:55.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:54 smithi132 ceph-mon[97863]: purged_snaps scrub starts 2023-07-21T22:24:55.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:54 smithi132 ceph-mon[97863]: purged_snaps scrub ok 2023-07-21T22:24:55.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:54 smithi132 ceph-mon[97863]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:55.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:54 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:55.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:54 smithi132 ceph-mon[97863]: osd.5 [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533] boot 2023-07-21T22:24:55.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:54 smithi132 ceph-mon[97863]: osdmap e38: 6 total, 6 up, 6 in 2023-07-21T22:24:55.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:54 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:55.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:54 smithi107 ceph-mon[96543]: purged_snaps scrub starts 2023-07-21T22:24:55.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:54 smithi107 ceph-mon[96543]: purged_snaps scrub ok 2023-07-21T22:24:55.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:54 smithi107 ceph-mon[96543]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:24:55.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:54 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:55.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:54 smithi107 ceph-mon[96543]: osd.5 [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533] boot 2023-07-21T22:24:55.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:54 smithi107 ceph-mon[96543]: osdmap e38: 6 total, 6 up, 6 in 2023-07-21T22:24:55.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:54 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:24:56.299 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:55 smithi107 ceph-mon[96543]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T22:24:56.300 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:55 smithi107 ceph-mon[96543]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:24:56.300 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:24:56.300 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:24:56.300 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:56.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:55 smithi190 ceph-mon[98334]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T22:24:56.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:55 smithi190 ceph-mon[98334]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:24:56.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:55 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:24:56.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:55 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:24:56.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:55 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:56.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:55 smithi132 ceph-mon[97863]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T22:24:56.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:55 smithi132 ceph-mon[97863]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:24:56.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:55 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:24:56.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:55 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:24:56.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:55 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:57.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:56 smithi190 ceph-mon[98334]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:24:57.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:56 smithi190 ceph-mon[98334]: osdmap e40: 6 total, 6 up, 6 in 2023-07-21T22:24:57.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:56 smithi132 ceph-mon[97863]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:24:57.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:56 smithi132 ceph-mon[97863]: osdmap e40: 6 total, 6 up, 6 in 2023-07-21T22:24:57.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:56 smithi107 ceph-mon[96543]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:24:57.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:56 smithi107 ceph-mon[96543]: osdmap e40: 6 total, 6 up, 6 in 2023-07-21T22:24:58.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:58 smithi190 ceph-mon[98334]: osdmap e41: 6 total, 6 up, 6 in 2023-07-21T22:24:58.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:58 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/4124451289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59ce5822-cb79-41f8-bc1e-de34c37311d9"}]: dispatch 2023-07-21T22:24:58.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:58 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/4124451289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59ce5822-cb79-41f8-bc1e-de34c37311d9"}]': finished 2023-07-21T22:24:58.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:58 smithi190 ceph-mon[98334]: osdmap e42: 7 total, 6 up, 7 in 2023-07-21T22:24:58.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:58 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:24:58.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:58 smithi132 ceph-mon[97863]: osdmap e41: 6 total, 6 up, 6 in 2023-07-21T22:24:58.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:58 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/4124451289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59ce5822-cb79-41f8-bc1e-de34c37311d9"}]: dispatch 2023-07-21T22:24:58.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:58 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/4124451289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59ce5822-cb79-41f8-bc1e-de34c37311d9"}]': finished 2023-07-21T22:24:58.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:58 smithi132 ceph-mon[97863]: osdmap e42: 7 total, 6 up, 7 in 2023-07-21T22:24:58.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:58 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:24:58.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:58 smithi107 ceph-mon[96543]: osdmap e41: 6 total, 6 up, 6 in 2023-07-21T22:24:58.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:58 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/4124451289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59ce5822-cb79-41f8-bc1e-de34c37311d9"}]: dispatch 2023-07-21T22:24:58.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:58 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/4124451289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59ce5822-cb79-41f8-bc1e-de34c37311d9"}]': finished 2023-07-21T22:24:58.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:58 smithi107 ceph-mon[96543]: osdmap e42: 7 total, 6 up, 7 in 2023-07-21T22:24:58.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:58 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:24:59.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:59 smithi190 ceph-mon[98334]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:24:59.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:24:59 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/727675902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:24:59.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:59 smithi132 ceph-mon[97863]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:24:59.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:24:59 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/727675902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:24:59.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:59 smithi107 ceph-mon[96543]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:24:59.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:24:59 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/727675902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:25:01.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:01 smithi190 ceph-mon[98334]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:25:01.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:01 smithi132 ceph-mon[97863]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:25:01.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:01 smithi107 ceph-mon[96543]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:25:03.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:03 smithi107 ceph-mon[96543]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:25:03.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:03 smithi190 ceph-mon[98334]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:25:03.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:03 smithi132 ceph-mon[97863]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:25:05.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:05 smithi190 ceph-mon[98334]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-07-21T22:25:05.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:05 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T22:25:05.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:05 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:05.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:05 smithi190 ceph-mon[98334]: Deploying daemon osd.6 on smithi107 2023-07-21T22:25:05.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:05 smithi132 ceph-mon[97863]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-07-21T22:25:05.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:05 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T22:25:05.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:05 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:05.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:05 smithi132 ceph-mon[97863]: Deploying daemon osd.6 on smithi107 2023-07-21T22:25:05.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:05 smithi107 ceph-mon[96543]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-07-21T22:25:05.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:05 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T22:25:05.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:05 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:05.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:05 smithi107 ceph-mon[96543]: Deploying daemon osd.6 on smithi107 2023-07-21T22:25:07.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:07 smithi190 ceph-mon[98334]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-07-21T22:25:07.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:07 smithi132 ceph-mon[97863]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-07-21T22:25:07.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:07 smithi107 ceph-mon[96543]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-07-21T22:25:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:09 smithi107 ceph-mon[96543]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-07-21T22:25:09.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:09 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:09.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:09 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:09.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:09 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:09 smithi190 ceph-mon[98334]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-07-21T22:25:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:09 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:09 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:09 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:09 smithi132 ceph-mon[97863]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-07-21T22:25:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:09 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:09 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:09 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:11.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:11.073+0000 7f20337fe700 1 -- 172.21.15.107:0/571299678 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2044107660 con 0x7f202c0a11d0 2023-07-21T22:25:11.119 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 6 on host 'smithi107' 2023-07-21T22:25:11.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:11.075+0000 7f2048931700 1 -- 172.21.15.107:0/571299678 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f202c0a11d0 msgr2=0x7f202c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:11.075+0000 7f2048931700 1 --2- 172.21.15.107:0/571299678 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f202c0a11d0 0x7f202c0a3680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2038005fd0 tx=0x7f2038005dd0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:11.075+0000 7f2048931700 1 -- 172.21.15.107:0/571299678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20440ff410 msgr2=0x7f2044195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:11.075+0000 7f2048931700 1 --2- 172.21.15.107:0/571299678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20440ff410 0x7f2044195e20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f203400c9e0 tx=0x7f203400cda0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:11.075+0000 7f2048931700 1 -- 172.21.15.107:0/571299678 shutdown_connections 2023-07-21T22:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:11.075+0000 7f2048931700 1 --2- 172.21.15.107:0/571299678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20440ffda0 0x7f204419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:11.075+0000 7f2048931700 1 --2- 172.21.15.107:0/571299678 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f202c0a11d0 0x7f202c0a3680 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:11.075+0000 7f2048931700 1 --2- 172.21.15.107:0/571299678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20440fe210 0x7f20441958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:11.075+0000 7f2048931700 1 --2- 172.21.15.107:0/571299678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20440ff410 0x7f2044195e20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:11.075+0000 7f2048931700 1 -- 172.21.15.107:0/571299678 >> 172.21.15.107:0/571299678 conn(0x7f20440f9860 msgr2=0x7f2044106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:11.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:11.075+0000 7f2048931700 1 -- 172.21.15.107:0/571299678 shutdown_connections 2023-07-21T22:25:11.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:11.075+0000 7f2048931700 1 -- 172.21.15.107:0/571299678 wait complete. 2023-07-21T22:25:11.272 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:11 smithi107 ceph-mon[96543]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:25:11.272 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:11.272 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:11.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:11 smithi190 ceph-mon[98334]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:25:11.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:11.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:11.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:11 smithi132 ceph-mon[97863]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:25:11.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:11.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:11.527 DEBUG:teuthology.orchestra.run.smithi107:osd.6> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.6.service 2023-07-21T22:25:11.529 INFO:tasks.cephadm:Deploying osd.7 on smithi132 with /dev/vg_nvme/lv_2... 2023-07-21T22:25:11.529 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T22:25:11.558 INFO:journalctl@ceph.osd.6.smithi107.stdout:-- Logs begin at Fri 2023-07-21 22:03:40 UTC. -- 2023-07-21T22:25:12.107 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:25:12.347 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:12 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:12.347 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:12 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:12.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:12 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:12.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:12 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:12.398 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:12 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:12.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:12 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:12.687 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:25:12 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:25:12.351+0000 7f30199cc640 -1 osd.6 0 log_to_monitors true 2023-07-21T22:25:13.148 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:13 smithi132 ceph-mon[97863]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:25:13.148 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:13 smithi132 ceph-mon[97863]: from='osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T22:25:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:13 smithi190 ceph-mon[98334]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:25:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:13 smithi190 ceph-mon[98334]: from='osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T22:25:13.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:13 smithi107 ceph-mon[96543]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:25:13.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:13 smithi107 ceph-mon[96543]: from='osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T22:25:13.582 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T22:25:13.582 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T22:25:13.582 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2023-07-21T22:25:13.582 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2023-07-21T22:25:13.582 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124688 s, 841 MB/s 2023-07-21T22:25:13.583 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2023-07-21T22:25:14.063 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi132:vg_nvme/lv_2 2023-07-21T22:25:14.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T22:25:14.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: osdmap e43: 7 total, 6 up, 7 in 2023-07-21T22:25:14.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:25:14.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2023-07-21T22:25:14.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:14.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:14.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:14.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:14.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:14.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:14.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:14.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:14.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T22:25:14.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: osdmap e43: 7 total, 6 up, 7 in 2023-07-21T22:25:14.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:25:14.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2023-07-21T22:25:14.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:14.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:14.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:14.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:14.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:14.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:14.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:14.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:14.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.437 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:25:14.060+0000 7f300e843700 -1 osd.6 0 waiting for initial osdmap 2023-07-21T22:25:14.438 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:25:14.072+0000 7f3008634700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:25:14.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T22:25:14.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: osdmap e43: 7 total, 6 up, 7 in 2023-07-21T22:25:14.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:25:14.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2023-07-21T22:25:14.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:14.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:14.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:14.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:14.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:14.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:14.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:14.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:14.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:14.620 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:25:15.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:15 smithi190 ceph-mon[98334]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:25:15.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:15 smithi190 ceph-mon[98334]: Detected new or changed devices on smithi107 2023-07-21T22:25:15.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:15 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi107 to 5722M 2023-07-21T22:25:15.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:15 smithi190 ceph-mon[98334]: from='osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2023-07-21T22:25:15.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:15 smithi190 ceph-mon[98334]: osdmap e44: 7 total, 6 up, 7 in 2023-07-21T22:25:15.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:25:15.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:25:15.414 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:15 smithi132 ceph-mon[97863]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:25:15.414 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:15 smithi132 ceph-mon[97863]: Detected new or changed devices on smithi107 2023-07-21T22:25:15.415 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:15 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi107 to 5722M 2023-07-21T22:25:15.415 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:15 smithi132 ceph-mon[97863]: from='osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2023-07-21T22:25:15.415 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:15 smithi132 ceph-mon[97863]: osdmap e44: 7 total, 6 up, 7 in 2023-07-21T22:25:15.415 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:25:15.415 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:25:15.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:15 smithi107 ceph-mon[96543]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:25:15.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:15 smithi107 ceph-mon[96543]: Detected new or changed devices on smithi107 2023-07-21T22:25:15.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:15 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi107 to 5722M 2023-07-21T22:25:15.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:15 smithi107 ceph-mon[96543]: from='osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2023-07-21T22:25:15.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:15 smithi107 ceph-mon[96543]: osdmap e44: 7 total, 6 up, 7 in 2023-07-21T22:25:15.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:25:15.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:25:15.775 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.730+0000 7f643b0af700 1 -- 172.21.15.132:0/2790417508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64340ffc80 msgr2=0x7f64341000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:15.776 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.730+0000 7f643b0af700 1 --2- 172.21.15.132:0/2790417508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64340ffc80 0x7f64341000d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6420009990 tx=0x7f6420009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.776 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.730+0000 7f643b0af700 1 -- 172.21.15.132:0/2790417508 shutdown_connections 2023-07-21T22:25:15.776 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.730+0000 7f643b0af700 1 --2- 172.21.15.132:0/2790417508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6434100610 0x7f6434106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.776 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.730+0000 7f643b0af700 1 --2- 172.21.15.132:0/2790417508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64340fea80 0x7f64340fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.776 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.730+0000 7f643b0af700 1 --2- 172.21.15.132:0/2790417508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64340ffc80 0x7f64341000d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.776 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.730+0000 7f643b0af700 1 -- 172.21.15.132:0/2790417508 >> 172.21.15.132:0/2790417508 conn(0x7f64340fa0b0 msgr2=0x7f64340fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:15.777 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.730+0000 7f643b0af700 1 -- 172.21.15.132:0/2790417508 shutdown_connections 2023-07-21T22:25:15.777 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.730+0000 7f643b0af700 1 -- 172.21.15.132:0/2790417508 wait complete. 2023-07-21T22:25:15.778 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.730+0000 7f643b0af700 1 Processor -- start 2023-07-21T22:25:15.778 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.730+0000 7f643b0af700 1 -- start start 2023-07-21T22:25:15.779 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f643b0af700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64340fea80 0x7f6434102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.779 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f643b0af700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64340ffc80 0x7f6434101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.779 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f643b0af700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6434100610 0x7f64341015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.779 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f643b0af700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6434101d20 con 0x7f64340fea80 2023-07-21T22:25:15.779 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f643b0af700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6434101e90 con 0x7f64340ffc80 2023-07-21T22:25:15.779 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f643b0af700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643419f8f0 con 0x7f6434100610 2023-07-21T22:25:15.779 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f6433fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64340ffc80 0x7f6434101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.780 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f6433fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64340ffc80 0x7f6434101060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50652/0 (socket says 172.21.15.132:50652) 2023-07-21T22:25:15.780 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f6433fff700 1 -- 172.21.15.132:0/2413208641 learned_addr learned my addr 172.21.15.132:0/2413208641 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-07-21T22:25:15.780 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f6438e4b700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64340fea80 0x7f6434102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.780 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f643964c700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6434100610 0x7f64341015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.780 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f6433fff700 1 -- 172.21.15.132:0/2413208641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6434100610 msgr2=0x7f64341015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:15.780 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f6433fff700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6434100610 0x7f64341015a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.781 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f6433fff700 1 -- 172.21.15.132:0/2413208641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64340fea80 msgr2=0x7f6434102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:15.781 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f6433fff700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64340fea80 0x7f6434102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:15.781 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f6433fff700 1 -- 172.21.15.132:0/2413208641 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64200095f0 con 0x7f64340ffc80 2023-07-21T22:25:15.781 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f6433fff700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64340ffc80 0x7f6434101060 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f64200098e0 tx=0x7f6420004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:15.781 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f6431ffb700 1 -- 172.21.15.132:0/2413208641 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642001d070 con 0x7f64340ffc80 2023-07-21T22:25:15.781 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f6431ffb700 1 -- 172.21.15.132:0/2413208641 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6420021410 con 0x7f64340ffc80 2023-07-21T22:25:15.781 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f6431ffb700 1 -- 172.21.15.132:0/2413208641 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6420017650 con 0x7f64340ffc80 2023-07-21T22:25:15.781 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.731+0000 7f643b0af700 1 -- 172.21.15.132:0/2413208641 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643419fb70 con 0x7f64340ffc80 2023-07-21T22:25:15.782 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.732+0000 7f643b0af700 1 -- 172.21.15.132:0/2413208641 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643419ff30 con 0x7f64340ffc80 2023-07-21T22:25:15.782 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.732+0000 7f643b0af700 1 -- 172.21.15.132:0/2413208641 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643410a240 con 0x7f64340ffc80 2023-07-21T22:25:15.782 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.734+0000 7f6431ffb700 1 -- 172.21.15.132:0/2413208641 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6420021580 con 0x7f64340ffc80 2023-07-21T22:25:15.783 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.734+0000 7f6431ffb700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64240a11d0 0x7f64240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:15.783 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.735+0000 7f6431ffb700 1 -- 172.21.15.132:0/2413208641 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f64200cd560 con 0x7f64340ffc80 2023-07-21T22:25:15.783 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.735+0000 7f6438e4b700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64240a11d0 0x7f64240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:15.783 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.735+0000 7f6438e4b700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64240a11d0 0x7f64240a3680 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f64340ffae0 tx=0x7f6428009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:15.783 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.738+0000 7f6431ffb700 1 -- 172.21.15.132:0/2413208641 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f64200979b0 con 0x7f64340ffc80 2023-07-21T22:25:15.975 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:15.929+0000 7f643b0af700 1 -- 172.21.15.132:0/2413208641 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6434068fd0 con 0x7f64240a11d0 2023-07-21T22:25:16.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:16 smithi190 ceph-mon[98334]: purged_snaps scrub starts 2023-07-21T22:25:16.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:16 smithi190 ceph-mon[98334]: purged_snaps scrub ok 2023-07-21T22:25:16.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:16 smithi190 ceph-mon[98334]: osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368] boot 2023-07-21T22:25:16.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:16 smithi190 ceph-mon[98334]: osdmap e45: 7 total, 7 up, 7 in 2023-07-21T22:25:16.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:16 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:25:16.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:16 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:25:16.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:16 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:25:16.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:16 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:16.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:16 smithi132 ceph-mon[97863]: purged_snaps scrub starts 2023-07-21T22:25:16.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:16 smithi132 ceph-mon[97863]: purged_snaps scrub ok 2023-07-21T22:25:16.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:16 smithi132 ceph-mon[97863]: osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368] boot 2023-07-21T22:25:16.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:16 smithi132 ceph-mon[97863]: osdmap e45: 7 total, 7 up, 7 in 2023-07-21T22:25:16.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:16 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:25:16.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:16 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:25:16.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:16 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:25:16.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:16 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:16.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:16 smithi107 ceph-mon[96543]: purged_snaps scrub starts 2023-07-21T22:25:16.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:16 smithi107 ceph-mon[96543]: purged_snaps scrub ok 2023-07-21T22:25:16.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:16 smithi107 ceph-mon[96543]: osd.6 [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368] boot 2023-07-21T22:25:16.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:16 smithi107 ceph-mon[96543]: osdmap e45: 7 total, 7 up, 7 in 2023-07-21T22:25:16.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:25:16.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:25:16.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:25:16.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:16 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:17.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:17 smithi190 ceph-mon[98334]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:25:17.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:17 smithi190 ceph-mon[98334]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:25:17.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:17 smithi190 ceph-mon[98334]: osdmap e46: 7 total, 7 up, 7 in 2023-07-21T22:25:17.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:17 smithi132 ceph-mon[97863]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:25:17.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:17 smithi132 ceph-mon[97863]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:25:17.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:17 smithi132 ceph-mon[97863]: osdmap e46: 7 total, 7 up, 7 in 2023-07-21T22:25:17.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:17 smithi107 ceph-mon[96543]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:25:17.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:17 smithi107 ceph-mon[96543]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:25:17.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:17 smithi107 ceph-mon[96543]: osdmap e46: 7 total, 7 up, 7 in 2023-07-21T22:25:19.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:19 smithi190 ceph-mon[98334]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:19.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:19 smithi190 ceph-mon[98334]: from='client.? 172.21.15.132:0/3589698016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0013f445-b592-48e8-9620-ca2ff64a8e24"}]: dispatch 2023-07-21T22:25:19.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:19 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0013f445-b592-48e8-9620-ca2ff64a8e24"}]: dispatch 2023-07-21T22:25:19.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:19 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0013f445-b592-48e8-9620-ca2ff64a8e24"}]': finished 2023-07-21T22:25:19.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:19 smithi190 ceph-mon[98334]: osdmap e47: 8 total, 7 up, 8 in 2023-07-21T22:25:19.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:19 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:19.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:19 smithi190 ceph-mon[98334]: from='client.? 172.21.15.132:0/554996419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:25:19.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:19 smithi132 ceph-mon[97863]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:19.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:19 smithi132 ceph-mon[97863]: from='client.? 172.21.15.132:0/3589698016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0013f445-b592-48e8-9620-ca2ff64a8e24"}]: dispatch 2023-07-21T22:25:19.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:19 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0013f445-b592-48e8-9620-ca2ff64a8e24"}]: dispatch 2023-07-21T22:25:19.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:19 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0013f445-b592-48e8-9620-ca2ff64a8e24"}]': finished 2023-07-21T22:25:19.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:19 smithi132 ceph-mon[97863]: osdmap e47: 8 total, 7 up, 8 in 2023-07-21T22:25:19.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:19 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:19.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:19 smithi132 ceph-mon[97863]: from='client.? 172.21.15.132:0/554996419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:25:19.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:19 smithi107 ceph-mon[96543]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:19.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:19 smithi107 ceph-mon[96543]: from='client.? 172.21.15.132:0/3589698016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0013f445-b592-48e8-9620-ca2ff64a8e24"}]: dispatch 2023-07-21T22:25:19.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:19 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0013f445-b592-48e8-9620-ca2ff64a8e24"}]: dispatch 2023-07-21T22:25:19.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:19 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0013f445-b592-48e8-9620-ca2ff64a8e24"}]': finished 2023-07-21T22:25:19.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:19 smithi107 ceph-mon[96543]: osdmap e47: 8 total, 7 up, 8 in 2023-07-21T22:25:19.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:19.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:19 smithi107 ceph-mon[96543]: from='client.? 172.21.15.132:0/554996419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:25:21.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:21 smithi132 ceph-mon[97863]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:21.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:21 smithi107 ceph-mon[96543]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:21.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:21 smithi190 ceph-mon[98334]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:23.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:23 smithi132 ceph-mon[97863]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:23.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:23 smithi107 ceph-mon[96543]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:23.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:23 smithi190 ceph-mon[98334]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:25.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:25 smithi132 ceph-mon[97863]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:25.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:25 smithi107 ceph-mon[96543]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:25.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:25 smithi190 ceph-mon[98334]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:26.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:26 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T22:25:26.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:26 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:26.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:26 smithi132 ceph-mon[97863]: Deploying daemon osd.7 on smithi132 2023-07-21T22:25:26.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:26 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T22:25:26.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:26 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:26.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:26 smithi107 ceph-mon[96543]: Deploying daemon osd.7 on smithi132 2023-07-21T22:25:26.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:26 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T22:25:26.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:26 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:26.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:26 smithi190 ceph-mon[98334]: Deploying daemon osd.7 on smithi132 2023-07-21T22:25:27.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:27 smithi132 ceph-mon[97863]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:27.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:27 smithi107 ceph-mon[96543]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:27.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:27 smithi190 ceph-mon[98334]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:29.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:29 smithi132 ceph-mon[97863]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:29.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:29 smithi107 ceph-mon[96543]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:29.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:29 smithi190 ceph-mon[98334]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:30.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:30.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:30.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:30.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:30 smithi132 ceph-mon[97863]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:30.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:30.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:30.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:30.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:30 smithi190 ceph-mon[98334]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:30.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:30.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:30.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:30.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:30 smithi107 ceph-mon[96543]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:32.584 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 7 on host 'smithi132' 2023-07-21T22:25:32.584 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:32.538+0000 7f6431ffb700 1 -- 172.21.15.132:0/2413208641 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6434068fd0 con 0x7f64240a11d0 2023-07-21T22:25:32.586 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:32.540+0000 7f643b0af700 1 -- 172.21.15.132:0/2413208641 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64240a11d0 msgr2=0x7f64240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:32.586 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:32.540+0000 7f643b0af700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64240a11d0 0x7f64240a3680 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f64340ffae0 tx=0x7f6428009260 comp rx=0 tx=0).stop 2023-07-21T22:25:32.587 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:32.541+0000 7f643b0af700 1 -- 172.21.15.132:0/2413208641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64340ffc80 msgr2=0x7f6434101060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:32.587 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:32.541+0000 7f643b0af700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64340ffc80 0x7f6434101060 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f64200098e0 tx=0x7f6420004710 comp rx=0 tx=0).stop 2023-07-21T22:25:32.587 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:32.541+0000 7f643b0af700 1 -- 172.21.15.132:0/2413208641 shutdown_connections 2023-07-21T22:25:32.587 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:32.541+0000 7f643b0af700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64240a11d0 0x7f64240a3680 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:32.587 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:32.541+0000 7f643b0af700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6434100610 0x7f64341015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:32.587 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:32.541+0000 7f643b0af700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64340fea80 0x7f6434102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:32.587 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:32.541+0000 7f643b0af700 1 --2- 172.21.15.132:0/2413208641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64340ffc80 0x7f6434101060 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:32.588 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:32.541+0000 7f643b0af700 1 -- 172.21.15.132:0/2413208641 >> 172.21.15.132:0/2413208641 conn(0x7f64340fa0b0 msgr2=0x7f64340688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:32.588 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:32.541+0000 7f643b0af700 1 -- 172.21.15.132:0/2413208641 shutdown_connections 2023-07-21T22:25:32.588 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:25:32.541+0000 7f643b0af700 1 -- 172.21.15.132:0/2413208641 wait complete. 2023-07-21T22:25:32.999 DEBUG:teuthology.orchestra.run.smithi132:osd.7> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.7.service 2023-07-21T22:25:33.001 INFO:tasks.cephadm:Deploying osd.8 on smithi190 with /dev/vg_nvme/lv_2... 2023-07-21T22:25:33.001 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T22:25:33.030 INFO:journalctl@ceph.osd.7.smithi132.stdout:-- Logs begin at Fri 2023-07-21 22:03:19 UTC. -- 2023-07-21T22:25:33.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:33 smithi190 ceph-mon[98334]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:33.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:33 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:33.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:33 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:33.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:33 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:33.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:33 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:33.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:33 smithi132 ceph-mon[97863]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:33.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:33 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:33.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:33 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:33.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:33 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:33.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:33 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:33.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:33 smithi107 ceph-mon[96543]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:33.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:33 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:33.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:33 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:33.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:33 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:33.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:33 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:33.565 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:25:33.743 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:25:33 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:25:33.443+0000 7fd38bb90640 -1 osd.7 0 log_to_monitors true 2023-07-21T22:25:34.359 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:34 smithi132 ceph-mon[97863]: from='osd.7 [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:25:34.359 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:34 smithi132 ceph-mon[97863]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:25:34.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:34 smithi190 ceph-mon[98334]: from='osd.7 [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:25:34.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:34 smithi190 ceph-mon[98334]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:25:34.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:34 smithi107 ceph-mon[96543]: from='osd.7 [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:25:34.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:34 smithi107 ceph-mon[96543]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:25:35.045 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T22:25:35.045 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T22:25:35.046 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2023-07-21T22:25:35.046 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2023-07-21T22:25:35.046 INFO:teuthology.orchestra.run.smithi190.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111473 s, 941 MB/s 2023-07-21T22:25:35.046 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2023-07-21T22:25:35.181 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:25:35 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:25:35.045+0000 7fd380a07700 -1 osd.7 0 waiting for initial osdmap 2023-07-21T22:25:35.181 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:25:35 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:25:35.055+0000 7fd37bffb700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:25:35.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:35 smithi132 ceph-mon[97863]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:35.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:35 smithi132 ceph-mon[97863]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T22:25:35.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:35 smithi132 ceph-mon[97863]: osdmap e48: 8 total, 7 up, 8 in 2023-07-21T22:25:35.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:35 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:35.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:35 smithi132 ceph-mon[97863]: from='osd.7 [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:25:35.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:35 smithi132 ceph-mon[97863]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:25:35.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:35 smithi190 ceph-mon[98334]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:35.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:35 smithi190 ceph-mon[98334]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T22:25:35.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:35 smithi190 ceph-mon[98334]: osdmap e48: 8 total, 7 up, 8 in 2023-07-21T22:25:35.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:35 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:35.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:35 smithi190 ceph-mon[98334]: from='osd.7 [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:25:35.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:35 smithi190 ceph-mon[98334]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:25:35.410 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi190:vg_nvme/lv_2 2023-07-21T22:25:35.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:35 smithi107 ceph-mon[96543]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:35.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:35 smithi107 ceph-mon[96543]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T22:25:35.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:35 smithi107 ceph-mon[96543]: osdmap e48: 8 total, 7 up, 8 in 2023-07-21T22:25:35.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:35 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:35.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:35 smithi107 ceph-mon[96543]: from='osd.7 [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:25:35.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:35 smithi107 ceph-mon[96543]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:25:35.963 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:25:36.341 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-07-21T22:25:36.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: osdmap e49: 8 total, 7 up, 8 in 2023-07-21T22:25:36.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:36.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:36.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: Detected new or changed devices on smithi132 2023-07-21T22:25:36.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.342 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:36.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:36.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:36.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi132 to 5722M 2023-07-21T22:25:36.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:36.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:36.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.343 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:36.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:36.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:36.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: osd.7 [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834] boot 2023-07-21T22:25:36.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: osdmap e50: 8 total, 8 up, 8 in 2023-07-21T22:25:36.344 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:36 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:36.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-07-21T22:25:36.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: osdmap e49: 8 total, 7 up, 8 in 2023-07-21T22:25:36.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:36.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:36.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: Detected new or changed devices on smithi132 2023-07-21T22:25:36.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:36.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:36.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:36.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi132 to 5722M 2023-07-21T22:25:36.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:36.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:36.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:36.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:36.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:36.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: osd.7 [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834] boot 2023-07-21T22:25:36.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: osdmap e50: 8 total, 8 up, 8 in 2023-07-21T22:25:36.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:36 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:36.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-07-21T22:25:36.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: osdmap e49: 8 total, 7 up, 8 in 2023-07-21T22:25:36.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:36.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:36.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: Detected new or changed devices on smithi132 2023-07-21T22:25:36.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:36.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:36.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:36.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi132 to 5722M 2023-07-21T22:25:36.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:36.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:36.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:36.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:36.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:36.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:36.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: osd.7 [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834] boot 2023-07-21T22:25:36.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: osdmap e50: 8 total, 8 up, 8 in 2023-07-21T22:25:36.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:36 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:25:37.129 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.083+0000 7f6a15e2c700 1 -- 172.21.15.190:0/2299619167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a10100e30 msgr2=0x7f6a10101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:37.130 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.083+0000 7f6a15e2c700 1 --2- 172.21.15.190:0/2299619167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a10100e30 0x7f6a10101280 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009990 tx=0x7f6a00009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:37.130 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.083+0000 7f6a15e2c700 1 -- 172.21.15.190:0/2299619167 shutdown_connections 2023-07-21T22:25:37.130 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.083+0000 7f6a15e2c700 1 --2- 172.21.15.190:0/2299619167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a101017c0 0x7f6a10108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:37.130 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.083+0000 7f6a15e2c700 1 --2- 172.21.15.190:0/2299619167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a100ffc30 0x7f6a10100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:37.130 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.083+0000 7f6a15e2c700 1 --2- 172.21.15.190:0/2299619167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a10100e30 0x7f6a10101280 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:37.130 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.083+0000 7f6a15e2c700 1 -- 172.21.15.190:0/2299619167 >> 172.21.15.190:0/2299619167 conn(0x7f6a100fb2a0 msgr2=0x7f6a100fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:37.131 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.083+0000 7f6a15e2c700 1 -- 172.21.15.190:0/2299619167 shutdown_connections 2023-07-21T22:25:37.131 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.083+0000 7f6a15e2c700 1 -- 172.21.15.190:0/2299619167 wait complete. 2023-07-21T22:25:37.131 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.084+0000 7f6a15e2c700 1 Processor -- start 2023-07-21T22:25:37.131 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.084+0000 7f6a15e2c700 1 -- start start 2023-07-21T22:25:37.131 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.084+0000 7f6a15e2c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a100ffc30 0x7f6a100742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:37.131 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.084+0000 7f6a15e2c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10100e30 0x7f6a10072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:37.131 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.084+0000 7f6a15e2c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a101017c0 0x7f6a10072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:37.131 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.084+0000 7f6a15e2c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a100735c0 con 0x7f6a10100e30 2023-07-21T22:25:37.132 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.084+0000 7f6a15e2c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a10073730 con 0x7f6a100ffc30 2023-07-21T22:25:37.132 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.084+0000 7f6a15e2c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1019f7d0 con 0x7f6a101017c0 2023-07-21T22:25:37.132 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.084+0000 7f6a0ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a101017c0 0x7f6a10072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:37.132 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.084+0000 7f6a0ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a101017c0 0x7f6a10072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:48154/0 (socket says 172.21.15.190:48154) 2023-07-21T22:25:37.132 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.084+0000 7f6a0ffff700 1 -- 172.21.15.190:0/3234240071 learned_addr learned my addr 172.21.15.190:0/3234240071 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-07-21T22:25:37.132 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.084+0000 7f6a0effd700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10100e30 0x7f6a10072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:37.134 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.085+0000 7f6a0f7fe700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a100ffc30 0x7f6a100742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:37.134 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.085+0000 7f6a0ffff700 1 -- 172.21.15.190:0/3234240071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a100ffc30 msgr2=0x7f6a100742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:37.134 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.085+0000 7f6a0ffff700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a100ffc30 0x7f6a100742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:37.134 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.085+0000 7f6a0ffff700 1 -- 172.21.15.190:0/3234240071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10100e30 msgr2=0x7f6a10072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:37.134 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.085+0000 7f6a0ffff700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10100e30 0x7f6a10072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:37.134 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.085+0000 7f6a0ffff700 1 -- 172.21.15.190:0/3234240071 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a000095f0 con 0x7f6a101017c0 2023-07-21T22:25:37.135 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.085+0000 7f6a0effd700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10100e30 0x7f6a10072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:37.135 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.085+0000 7f6a0ffff700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a101017c0 0x7f6a10072e40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6a08007d70 tx=0x7f6a0800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:37.135 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.085+0000 7f6a0cff9700 1 -- 172.21.15.190:0/3234240071 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0800ad40 con 0x7f6a101017c0 2023-07-21T22:25:37.135 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.085+0000 7f6a0cff9700 1 -- 172.21.15.190:0/3234240071 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a080128e0 con 0x7f6a101017c0 2023-07-21T22:25:37.135 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.085+0000 7f6a0cff9700 1 -- 172.21.15.190:0/3234240071 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a08011780 con 0x7f6a101017c0 2023-07-21T22:25:37.135 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.085+0000 7f6a15e2c700 1 -- 172.21.15.190:0/3234240071 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1019fab0 con 0x7f6a101017c0 2023-07-21T22:25:37.135 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.085+0000 7f6a15e2c700 1 -- 172.21.15.190:0/3234240071 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1019ff60 con 0x7f6a101017c0 2023-07-21T22:25:37.136 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.086+0000 7f6a15e2c700 1 -- 172.21.15.190:0/3234240071 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1010b3f0 con 0x7f6a101017c0 2023-07-21T22:25:37.136 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.087+0000 7f6a0cff9700 1 -- 172.21.15.190:0/3234240071 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6a08012a50 con 0x7f6a101017c0 2023-07-21T22:25:37.136 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.088+0000 7f6a0cff9700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f69fc0a1150 0x7f69fc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:37.136 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.088+0000 7f6a0cff9700 1 -- 172.21.15.190:0/3234240071 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f6a080cc520 con 0x7f6a101017c0 2023-07-21T22:25:37.136 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.088+0000 7f6a0f7fe700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f69fc0a1150 0x7f69fc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:37.136 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.089+0000 7f6a0f7fe700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f69fc0a1150 0x7f69fc0a3600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a10100c90 tx=0x7f69f8005d70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:37.137 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.091+0000 7f6a0cff9700 1 -- 172.21.15.190:0/3234240071 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6a080967b0 con 0x7f6a101017c0 2023-07-21T22:25:37.326 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:37.281+0000 7f6a15e2c700 1 -- 172.21.15.190:0/3234240071 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a10060950 con 0x7f69fc0a1150 2023-07-21T22:25:37.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:37 smithi190 ceph-mon[98334]: purged_snaps scrub starts 2023-07-21T22:25:37.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:37 smithi190 ceph-mon[98334]: purged_snaps scrub ok 2023-07-21T22:25:37.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:37 smithi190 ceph-mon[98334]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:37.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:37 smithi190 ceph-mon[98334]: osdmap e51: 8 total, 8 up, 8 in 2023-07-21T22:25:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:37 smithi132 ceph-mon[97863]: purged_snaps scrub starts 2023-07-21T22:25:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:37 smithi132 ceph-mon[97863]: purged_snaps scrub ok 2023-07-21T22:25:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:37 smithi132 ceph-mon[97863]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:37 smithi132 ceph-mon[97863]: osdmap e51: 8 total, 8 up, 8 in 2023-07-21T22:25:37.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:37 smithi107 ceph-mon[96543]: purged_snaps scrub starts 2023-07-21T22:25:37.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:37 smithi107 ceph-mon[96543]: purged_snaps scrub ok 2023-07-21T22:25:37.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:37 smithi107 ceph-mon[96543]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:25:37.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:37 smithi107 ceph-mon[96543]: osdmap e51: 8 total, 8 up, 8 in 2023-07-21T22:25:38.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:38 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:25:38.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:38 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:25:38.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:38 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:38.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:38 smithi190 ceph-mon[98334]: osdmap e52: 8 total, 8 up, 8 in 2023-07-21T22:25:38.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:38 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:25:38.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:38 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:25:38.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:38 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:38.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:38 smithi132 ceph-mon[97863]: osdmap e52: 8 total, 8 up, 8 in 2023-07-21T22:25:38.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:38 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:25:38.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:38 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:25:38.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:38 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:38.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:38 smithi107 ceph-mon[96543]: osdmap e52: 8 total, 8 up, 8 in 2023-07-21T22:25:39.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:39 smithi190 ceph-mon[98334]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:25:39.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:39 smithi190 ceph-mon[98334]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:39.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:39 smithi190 ceph-mon[98334]: osdmap e53: 8 total, 8 up, 8 in 2023-07-21T22:25:39.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:39 smithi132 ceph-mon[97863]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:25:39.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:39 smithi132 ceph-mon[97863]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:39.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:39 smithi132 ceph-mon[97863]: osdmap e53: 8 total, 8 up, 8 in 2023-07-21T22:25:39.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:39 smithi107 ceph-mon[96543]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:25:39.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:39 smithi107 ceph-mon[96543]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:39.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:39 smithi107 ceph-mon[96543]: osdmap e53: 8 total, 8 up, 8 in 2023-07-21T22:25:40.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:40 smithi190 ceph-mon[98334]: from='client.? 172.21.15.190:0/1951811650' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0beb03d-307c-4f41-8e17-6e0418063c75"}]: dispatch 2023-07-21T22:25:40.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:40 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0beb03d-307c-4f41-8e17-6e0418063c75"}]: dispatch 2023-07-21T22:25:40.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:40 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0beb03d-307c-4f41-8e17-6e0418063c75"}]': finished 2023-07-21T22:25:40.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:40 smithi190 ceph-mon[98334]: osdmap e54: 9 total, 8 up, 9 in 2023-07-21T22:25:40.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:40.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:40 smithi132 ceph-mon[97863]: from='client.? 172.21.15.190:0/1951811650' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0beb03d-307c-4f41-8e17-6e0418063c75"}]: dispatch 2023-07-21T22:25:40.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:40 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0beb03d-307c-4f41-8e17-6e0418063c75"}]: dispatch 2023-07-21T22:25:40.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:40 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0beb03d-307c-4f41-8e17-6e0418063c75"}]': finished 2023-07-21T22:25:40.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:40 smithi132 ceph-mon[97863]: osdmap e54: 9 total, 8 up, 9 in 2023-07-21T22:25:40.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:40.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:40 smithi107 ceph-mon[96543]: from='client.? 172.21.15.190:0/1951811650' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0beb03d-307c-4f41-8e17-6e0418063c75"}]: dispatch 2023-07-21T22:25:40.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:40 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0beb03d-307c-4f41-8e17-6e0418063c75"}]: dispatch 2023-07-21T22:25:40.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:40 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0beb03d-307c-4f41-8e17-6e0418063c75"}]': finished 2023-07-21T22:25:40.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:40 smithi107 ceph-mon[96543]: osdmap e54: 9 total, 8 up, 9 in 2023-07-21T22:25:40.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:41.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:41 smithi190 ceph-mon[98334]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:41.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:41 smithi190 ceph-mon[98334]: from='client.? 172.21.15.190:0/2565126362' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:25:41.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:41 smithi132 ceph-mon[97863]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:41.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:41 smithi132 ceph-mon[97863]: from='client.? 172.21.15.190:0/2565126362' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:25:41.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:41 smithi107 ceph-mon[96543]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:41.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:41 smithi107 ceph-mon[96543]: from='client.? 172.21.15.190:0/2565126362' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:25:43.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:43 smithi190 ceph-mon[98334]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:43.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:43 smithi132 ceph-mon[97863]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:43.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:43 smithi107 ceph-mon[96543]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:45.372 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:45 smithi190 ceph-mon[98334]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:45.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:45 smithi132 ceph-mon[97863]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:45.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:45 smithi107 ceph-mon[96543]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:47.338 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:47 smithi190 ceph-mon[98334]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:47.339 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T22:25:47.339 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:47.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:47 smithi132 ceph-mon[97863]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:47.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T22:25:47.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:47.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:47 smithi107 ceph-mon[96543]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:47.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T22:25:47.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:48.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:48 smithi190 ceph-mon[98334]: Deploying daemon osd.8 on smithi190 2023-07-21T22:25:48.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:48 smithi132 ceph-mon[97863]: Deploying daemon osd.8 on smithi190 2023-07-21T22:25:48.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:48 smithi107 ceph-mon[96543]: Deploying daemon osd.8 on smithi190 2023-07-21T22:25:49.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:49 smithi190 ceph-mon[98334]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:49.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:49 smithi132 ceph-mon[97863]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:49.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:49 smithi107 ceph-mon[96543]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:51.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:51 smithi190 ceph-mon[98334]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:51.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:51 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:51.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:51 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:51.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:51 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:51 smithi132 ceph-mon[97863]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:51 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:51 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:51 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:51.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:51 smithi107 ceph-mon[96543]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:51.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:51 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:51.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:51 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:51.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:51 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:53.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:53 smithi190 ceph-mon[98334]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:53.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:53.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:53.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:53.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:53.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:53.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:53 smithi132 ceph-mon[97863]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:53.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:53.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:53.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:53.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:53.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:53.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:53 smithi107 ceph-mon[96543]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:53.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:53.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:53.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:53.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:53.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:53.838 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:53.792+0000 7f6a0cff9700 1 -- 172.21.15.190:0/3234240071 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6a10060950 con 0x7f69fc0a1150 2023-07-21T22:25:53.839 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 8 on host 'smithi190' 2023-07-21T22:25:53.841 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:53.794+0000 7f6a15e2c700 1 -- 172.21.15.190:0/3234240071 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f69fc0a1150 msgr2=0x7f69fc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:53.841 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:53.794+0000 7f6a15e2c700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f69fc0a1150 0x7f69fc0a3600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a10100c90 tx=0x7f69f8005d70 comp rx=0 tx=0).stop 2023-07-21T22:25:53.841 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:53.794+0000 7f6a15e2c700 1 -- 172.21.15.190:0/3234240071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a101017c0 msgr2=0x7f6a10072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:53.841 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:53.794+0000 7f6a15e2c700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a101017c0 0x7f6a10072e40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6a08007d70 tx=0x7f6a0800a340 comp rx=0 tx=0).stop 2023-07-21T22:25:53.841 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:53.794+0000 7f6a15e2c700 1 -- 172.21.15.190:0/3234240071 shutdown_connections 2023-07-21T22:25:53.841 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:53.794+0000 7f6a15e2c700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f69fc0a1150 0x7f69fc0a3600 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:53.842 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:53.794+0000 7f6a15e2c700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a101017c0 0x7f6a10072e40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:53.842 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:53.794+0000 7f6a15e2c700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a100ffc30 0x7f6a100742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:53.842 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:53.794+0000 7f6a15e2c700 1 --2- 172.21.15.190:0/3234240071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a10100e30 0x7f6a10072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:53.842 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:53.794+0000 7f6a15e2c700 1 -- 172.21.15.190:0/3234240071 >> 172.21.15.190:0/3234240071 conn(0x7f6a100fb2a0 msgr2=0x7f6a100fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:53.842 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:53.795+0000 7f6a15e2c700 1 -- 172.21.15.190:0/3234240071 shutdown_connections 2023-07-21T22:25:53.842 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:25:53.795+0000 7f6a15e2c700 1 -- 172.21.15.190:0/3234240071 wait complete. 2023-07-21T22:25:54.255 DEBUG:teuthology.orchestra.run.smithi190:osd.8> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.8.service 2023-07-21T22:25:54.258 INFO:tasks.cephadm:Deploying osd.9 on smithi107 with /dev/vg_nvme/lv_1... 2023-07-21T22:25:54.258 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T22:25:54.288 INFO:journalctl@ceph.osd.8.smithi190.stdout:-- Logs begin at Fri 2023-07-21 22:03:29 UTC. -- 2023-07-21T22:25:54.824 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:25:55.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:54 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:55.076 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:54 smithi107 ceph-mon[96543]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:55.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:54 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:55.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:54 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:55.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:54 smithi107 ceph-mon[96543]: from='osd.8 [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:25:55.077 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:54 smithi107 ceph-mon[96543]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:25:55.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:54 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:55.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:54 smithi190 ceph-mon[98334]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:55.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:54 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:55.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:54 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:55.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:54 smithi190 ceph-mon[98334]: from='osd.8 [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:25:55.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:54 smithi190 ceph-mon[98334]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:25:55.142 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:25:54 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:25:54.747+0000 7fb28cef6640 -1 osd.8 0 log_to_monitors true 2023-07-21T22:25:55.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:54 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:55.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:54 smithi132 ceph-mon[97863]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:55.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:54 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:55.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:54 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:55.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:54 smithi132 ceph-mon[97863]: from='osd.8 [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:25:55.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:54 smithi132 ceph-mon[97863]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:25:55.846 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:55 smithi107 ceph-mon[96543]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T22:25:55.846 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:55 smithi107 ceph-mon[96543]: osdmap e55: 9 total, 8 up, 9 in 2023-07-21T22:25:55.846 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:55.846 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:55 smithi107 ceph-mon[96543]: from='osd.8 [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:25:55.846 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:55 smithi107 ceph-mon[96543]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:25:55.890 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:25:55 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:25:55.775+0000 7fb281d6d700 -1 osd.8 0 waiting for initial osdmap 2023-07-21T22:25:55.890 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:25:55 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:25:55.786+0000 7fb27a35b700 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:25:55.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:55 smithi190 ceph-mon[98334]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T22:25:55.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:55 smithi190 ceph-mon[98334]: osdmap e55: 9 total, 8 up, 9 in 2023-07-21T22:25:55.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:55 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:55.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:55 smithi190 ceph-mon[98334]: from='osd.8 [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:25:55.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:55 smithi190 ceph-mon[98334]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:25:56.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:55 smithi132 ceph-mon[97863]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T22:25:56.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:55 smithi132 ceph-mon[97863]: osdmap e55: 9 total, 8 up, 9 in 2023-07-21T22:25:56.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:55 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:56.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:55 smithi132 ceph-mon[97863]: from='osd.8 [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:25:56.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:55 smithi132 ceph-mon[97863]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:25:56.315 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T22:25:56.316 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T22:25:56.316 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2023-07-21T22:25:56.316 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2023-07-21T22:25:56.316 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111312 s, 942 MB/s 2023-07-21T22:25:56.316 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2023-07-21T22:25:56.780 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2023-07-21T22:25:57.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:57.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-07-21T22:25:57.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: osdmap e56: 9 total, 8 up, 9 in 2023-07-21T22:25:57.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:57.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:57.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: osd.8 [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159] boot 2023-07-21T22:25:57.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: osdmap e57: 9 total, 9 up, 9 in 2023-07-21T22:25:57.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:57.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:57.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:57.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:57.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:57.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:57.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:57.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:57.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:57.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:57.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:57.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-07-21T22:25:57.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: osdmap e56: 9 total, 8 up, 9 in 2023-07-21T22:25:57.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:57.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:57.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: osd.8 [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159] boot 2023-07-21T22:25:57.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: osdmap e57: 9 total, 9 up, 9 in 2023-07-21T22:25:57.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:57.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:57.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:57.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:57.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:57.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:57.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:57.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:57.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:57.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:57.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:25:57.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-07-21T22:25:57.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: osdmap e56: 9 total, 8 up, 9 in 2023-07-21T22:25:57.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:57.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:57.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: osd.8 [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159] boot 2023-07-21T22:25:57.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: osdmap e57: 9 total, 9 up, 9 in 2023-07-21T22:25:57.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:25:57.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:57.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:57.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:57.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:57.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:25:57.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:57.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:57.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:57.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:25:57.334 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:25:58.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:57 smithi107 ceph-mon[96543]: purged_snaps scrub starts 2023-07-21T22:25:58.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:57 smithi107 ceph-mon[96543]: purged_snaps scrub ok 2023-07-21T22:25:58.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:57 smithi107 ceph-mon[96543]: Detected new or changed devices on smithi190 2023-07-21T22:25:58.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:57 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi190 to 5722M 2023-07-21T22:25:58.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:57 smithi107 ceph-mon[96543]: osdmap e58: 9 total, 9 up, 9 in 2023-07-21T22:25:58.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:57 smithi190 ceph-mon[98334]: purged_snaps scrub starts 2023-07-21T22:25:58.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:57 smithi190 ceph-mon[98334]: purged_snaps scrub ok 2023-07-21T22:25:58.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:57 smithi190 ceph-mon[98334]: Detected new or changed devices on smithi190 2023-07-21T22:25:58.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:57 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi190 to 5722M 2023-07-21T22:25:58.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:57 smithi190 ceph-mon[98334]: osdmap e58: 9 total, 9 up, 9 in 2023-07-21T22:25:58.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:57 smithi132 ceph-mon[97863]: purged_snaps scrub starts 2023-07-21T22:25:58.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:57 smithi132 ceph-mon[97863]: purged_snaps scrub ok 2023-07-21T22:25:58.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:57 smithi132 ceph-mon[97863]: Detected new or changed devices on smithi190 2023-07-21T22:25:58.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:57 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi190 to 5722M 2023-07-21T22:25:58.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:57 smithi132 ceph-mon[97863]: osdmap e58: 9 total, 9 up, 9 in 2023-07-21T22:25:58.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.415+0000 7fa6827ae700 1 -- 172.21.15.107:0/2223609944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa67c0ffc80 msgr2=0x7fa67c1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:58.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.415+0000 7fa6827ae700 1 --2- 172.21.15.107:0/2223609944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa67c0ffc80 0x7fa67c1000d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa670009990 tx=0x7fa670009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:58.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.415+0000 7fa6827ae700 1 -- 172.21.15.107:0/2223609944 shutdown_connections 2023-07-21T22:25:58.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.415+0000 7fa6827ae700 1 --2- 172.21.15.107:0/2223609944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa67c100610 0x7fa67c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:58.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.415+0000 7fa6827ae700 1 --2- 172.21.15.107:0/2223609944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa67c0fea80 0x7fa67c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:58.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.415+0000 7fa6827ae700 1 --2- 172.21.15.107:0/2223609944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa67c0ffc80 0x7fa67c1000d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:58.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.415+0000 7fa6827ae700 1 -- 172.21.15.107:0/2223609944 >> 172.21.15.107:0/2223609944 conn(0x7fa67c0fa0b0 msgr2=0x7fa67c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:58.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.415+0000 7fa6827ae700 1 -- 172.21.15.107:0/2223609944 shutdown_connections 2023-07-21T22:25:58.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.415+0000 7fa6827ae700 1 -- 172.21.15.107:0/2223609944 wait complete. 2023-07-21T22:25:58.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.416+0000 7fa6827ae700 1 Processor -- start 2023-07-21T22:25:58.464 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.416+0000 7fa6827ae700 1 -- start start 2023-07-21T22:25:58.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.416+0000 7fa6827ae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa67c0fea80 0x7fa67c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:58.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.416+0000 7fa6827ae700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa67c0ffc80 0x7fa67c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:58.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.416+0000 7fa6827ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa67c100610 0x7fa67c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:58.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.416+0000 7fa6827ae700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c19d400 con 0x7fa67c0fea80 2023-07-21T22:25:58.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.416+0000 7fa6827ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c19d570 con 0x7fa67c100610 2023-07-21T22:25:58.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.416+0000 7fa6827ae700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c19d730 con 0x7fa67c0ffc80 2023-07-21T22:25:58.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.416+0000 7fa67bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa67c0fea80 0x7fa67c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:58.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.416+0000 7fa67bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa67c0fea80 0x7fa67c0693e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45878/0 (socket says 172.21.15.107:45878) 2023-07-21T22:25:58.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.416+0000 7fa67bfff700 1 -- 172.21.15.107:0/1323164687 learned_addr learned my addr 172.21.15.107:0/1323164687 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:25:58.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa67b7fe700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa67c0ffc80 0x7fa67c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:58.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa680d4b700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa67c100610 0x7fa67c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:58.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa67bfff700 1 -- 172.21.15.107:0/1323164687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa67c0ffc80 msgr2=0x7fa67c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:58.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa67bfff700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa67c0ffc80 0x7fa67c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:58.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa67bfff700 1 -- 172.21.15.107:0/1323164687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa67c100610 msgr2=0x7fa67c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:58.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa67bfff700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa67c100610 0x7fa67c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:58.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa67bfff700 1 -- 172.21.15.107:0/1323164687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700095f0 con 0x7fa67c0fea80 2023-07-21T22:25:58.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa680d4b700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa67c100610 0x7fa67c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:25:58.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa67bfff700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa67c0fea80 0x7fa67c0693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00c970 tx=0x7fa66c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:58.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa6797fa700 1 -- 172.21.15.107:0/1323164687 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c004220 con 0x7fa67c0fea80 2023-07-21T22:25:58.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa6797fa700 1 -- 172.21.15.107:0/1323164687 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa66c007d60 con 0x7fa67c0fea80 2023-07-21T22:25:58.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa6827ae700 1 -- 172.21.15.107:0/1323164687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67c19da10 con 0x7fa67c0fea80 2023-07-21T22:25:58.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa6797fa700 1 -- 172.21.15.107:0/1323164687 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c003d00 con 0x7fa67c0fea80 2023-07-21T22:25:58.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.417+0000 7fa6827ae700 1 -- 172.21.15.107:0/1323164687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67c19df10 con 0x7fa67c0fea80 2023-07-21T22:25:58.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.418+0000 7fa6827ae700 1 -- 172.21.15.107:0/1323164687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67c106070 con 0x7fa67c0fea80 2023-07-21T22:25:58.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.419+0000 7fa6797fa700 1 -- 172.21.15.107:0/1323164687 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa66c004380 con 0x7fa67c0fea80 2023-07-21T22:25:58.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.419+0000 7fa6797fa700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa6640a1150 0x7fa6640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:58.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.420+0000 7fa67b7fe700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa6640a1150 0x7fa6640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:58.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.420+0000 7fa6797fa700 1 -- 172.21.15.107:0/1323164687 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fa66c0cbf70 con 0x7fa67c0fea80 2023-07-21T22:25:58.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.420+0000 7fa67b7fe700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa6640a1150 0x7fa6640a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa670006010 tx=0x7fa67000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:58.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.423+0000 7fa6797fa700 1 -- 172.21.15.107:0/1323164687 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa66c096050 con 0x7fa67c0fea80 2023-07-21T22:25:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:25:58.618+0000 7fa6827ae700 1 -- 172.21.15.107:0/1323164687 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa67c060950 con 0x7fa6640a1150 2023-07-21T22:25:59.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:58 smithi190 ceph-mon[98334]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:25:59.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:58 smithi190 ceph-mon[98334]: osdmap e59: 9 total, 9 up, 9 in 2023-07-21T22:25:59.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:58 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:25:59.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:58 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:25:59.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:25:58 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:59.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:58 smithi132 ceph-mon[97863]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:25:59.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:58 smithi132 ceph-mon[97863]: osdmap e59: 9 total, 9 up, 9 in 2023-07-21T22:25:59.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:58 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:25:59.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:58 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:25:59.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:25:58 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:59.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:58 smithi107 ceph-mon[96543]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:25:59.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:58 smithi107 ceph-mon[96543]: osdmap e59: 9 total, 9 up, 9 in 2023-07-21T22:25:59.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:58 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:25:59.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:58 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:25:59.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:25:58 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:00.315 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:00 smithi107 ceph-mon[96543]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:26:00.315 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:00 smithi107 ceph-mon[96543]: osdmap e60: 9 total, 9 up, 9 in 2023-07-21T22:26:00.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:00 smithi190 ceph-mon[98334]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:26:00.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:00 smithi190 ceph-mon[98334]: osdmap e60: 9 total, 9 up, 9 in 2023-07-21T22:26:00.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:00 smithi132 ceph-mon[97863]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:26:00.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:00 smithi132 ceph-mon[97863]: osdmap e60: 9 total, 9 up, 9 in 2023-07-21T22:26:01.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:01 smithi190 ceph-mon[98334]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:01.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:01 smithi190 ceph-mon[98334]: osdmap e61: 9 total, 9 up, 9 in 2023-07-21T22:26:01.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:01 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1824804615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6203384-ccdb-419c-aa4a-0f64c3bdcba0"}]: dispatch 2023-07-21T22:26:01.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:01 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1824804615' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6203384-ccdb-419c-aa4a-0f64c3bdcba0"}]': finished 2023-07-21T22:26:01.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:01 smithi190 ceph-mon[98334]: osdmap e62: 10 total, 9 up, 10 in 2023-07-21T22:26:01.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:01 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:01.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:01 smithi132 ceph-mon[97863]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:01.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:01 smithi132 ceph-mon[97863]: osdmap e61: 9 total, 9 up, 9 in 2023-07-21T22:26:01.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:01 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1824804615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6203384-ccdb-419c-aa4a-0f64c3bdcba0"}]: dispatch 2023-07-21T22:26:01.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:01 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1824804615' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6203384-ccdb-419c-aa4a-0f64c3bdcba0"}]': finished 2023-07-21T22:26:01.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:01 smithi132 ceph-mon[97863]: osdmap e62: 10 total, 9 up, 10 in 2023-07-21T22:26:01.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:01 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:01.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:01 smithi107 ceph-mon[96543]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:01.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:01 smithi107 ceph-mon[96543]: osdmap e61: 9 total, 9 up, 9 in 2023-07-21T22:26:01.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:01 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1824804615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6203384-ccdb-419c-aa4a-0f64c3bdcba0"}]: dispatch 2023-07-21T22:26:01.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:01 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1824804615' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6203384-ccdb-419c-aa4a-0f64c3bdcba0"}]': finished 2023-07-21T22:26:01.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:01 smithi107 ceph-mon[96543]: osdmap e62: 10 total, 9 up, 10 in 2023-07-21T22:26:01.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:01 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:02.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:02 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2197297199' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:26:02.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:02 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2197297199' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:26:02.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:02 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2197297199' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:26:03.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:03 smithi190 ceph-mon[98334]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:26:03.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:03 smithi132 ceph-mon[97863]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:26:03.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:03 smithi107 ceph-mon[96543]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:26:05.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:05 smithi107 ceph-mon[96543]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-07-21T22:26:05.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:05 smithi190 ceph-mon[98334]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-07-21T22:26:05.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:05 smithi132 ceph-mon[97863]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2023-07-21T22:26:07.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:07 smithi190 ceph-mon[98334]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-07-21T22:26:07.427 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:07 smithi107 ceph-mon[96543]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-07-21T22:26:07.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:07 smithi132 ceph-mon[97863]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2023-07-21T22:26:08.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:08 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T22:26:08.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:08 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:08.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:08 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T22:26:08.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:08 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:08.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T22:26:08.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:09.332 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:09 smithi107 ceph-mon[96543]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T22:26:09.332 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:09 smithi107 ceph-mon[96543]: Deploying daemon osd.9 on smithi107 2023-07-21T22:26:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:09 smithi190 ceph-mon[98334]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T22:26:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:09 smithi190 ceph-mon[98334]: Deploying daemon osd.9 on smithi107 2023-07-21T22:26:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:09 smithi132 ceph-mon[97863]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T22:26:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:09 smithi132 ceph-mon[97863]: Deploying daemon osd.9 on smithi107 2023-07-21T22:26:11.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:11 smithi190 ceph-mon[98334]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-07-21T22:26:11.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:11 smithi132 ceph-mon[97863]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-07-21T22:26:11.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:11 smithi107 ceph-mon[96543]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-07-21T22:26:13.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:13 smithi107 ceph-mon[96543]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-07-21T22:26:13.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:13.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:13.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:13.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:13 smithi190 ceph-mon[98334]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-07-21T22:26:13.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:13.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:13.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:13.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:13 smithi132 ceph-mon[97863]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-07-21T22:26:13.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:13.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:13.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:14.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:14.918+0000 7fa6797fa700 1 -- 172.21.15.107:0/1323164687 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa67c060950 con 0x7fa6640a1150 2023-07-21T22:26:14.965 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 9 on host 'smithi107' 2023-07-21T22:26:14.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:14.921+0000 7fa6827ae700 1 -- 172.21.15.107:0/1323164687 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa6640a1150 msgr2=0x7fa6640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:14.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:14.921+0000 7fa6827ae700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa6640a1150 0x7fa6640a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa670006010 tx=0x7fa67000b540 comp rx=0 tx=0).stop 2023-07-21T22:26:14.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:14.921+0000 7fa6827ae700 1 -- 172.21.15.107:0/1323164687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa67c0fea80 msgr2=0x7fa67c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:14.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:14.921+0000 7fa6827ae700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa67c0fea80 0x7fa67c0693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00c970 tx=0x7fa66c00cd30 comp rx=0 tx=0).stop 2023-07-21T22:26:14.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:14.921+0000 7fa6827ae700 1 -- 172.21.15.107:0/1323164687 shutdown_connections 2023-07-21T22:26:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:14.921+0000 7fa6827ae700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa67c100610 0x7fa67c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:14.921+0000 7fa6827ae700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa67c0fea80 0x7fa67c0693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:14.921+0000 7fa6827ae700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa6640a1150 0x7fa6640a3600 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:14.921+0000 7fa6827ae700 1 --2- 172.21.15.107:0/1323164687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa67c0ffc80 0x7fa67c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:14.921+0000 7fa6827ae700 1 -- 172.21.15.107:0/1323164687 >> 172.21.15.107:0/1323164687 conn(0x7fa67c0fa0b0 msgr2=0x7fa67c1031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:14.921+0000 7fa6827ae700 1 -- 172.21.15.107:0/1323164687 shutdown_connections 2023-07-21T22:26:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:14.921+0000 7fa6827ae700 1 -- 172.21.15.107:0/1323164687 wait complete. 2023-07-21T22:26:15.324 DEBUG:teuthology.orchestra.run.smithi107:osd.9> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.9.service 2023-07-21T22:26:15.326 INFO:tasks.cephadm:Deploying osd.10 on smithi132 with /dev/vg_nvme/lv_1... 2023-07-21T22:26:15.326 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T22:26:15.359 INFO:journalctl@ceph.osd.9.smithi107.stdout:-- Logs begin at Fri 2023-07-21 22:03:40 UTC. -- 2023-07-21T22:26:15.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:15 smithi190 ceph-mon[98334]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:15.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:15.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:15.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:15.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:15.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:15 smithi132 ceph-mon[97863]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:15.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:15.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:15.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:15.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:15.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:15 smithi107 ceph-mon[96543]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:15.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:15.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:15.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:15.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:15.890 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:26:16.291 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:26:16 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:26:16.083+0000 7f39739bf640 -1 osd.9 0 log_to_monitors true 2023-07-21T22:26:17.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:16 smithi132 ceph-mon[97863]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:17.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:16 smithi132 ceph-mon[97863]: from='osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T22:26:17.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:16 smithi107 ceph-mon[96543]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:17.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:16 smithi107 ceph-mon[96543]: from='osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T22:26:17.371 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T22:26:17.371 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T22:26:17.371 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2023-07-21T22:26:17.371 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2023-07-21T22:26:17.372 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110756 s, 947 MB/s 2023-07-21T22:26:17.372 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2023-07-21T22:26:17.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:16 smithi190 ceph-mon[98334]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:17.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:16 smithi190 ceph-mon[98334]: from='osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T22:26:17.747 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi132:vg_nvme/lv_1 2023-07-21T22:26:18.300 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:26:18.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T22:26:18.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: osdmap e63: 10 total, 9 up, 10 in 2023-07-21T22:26:18.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:18.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2023-07-21T22:26:18.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:18.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:18.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:18.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:18.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:18.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:18.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:18.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:18.393 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:18.393 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:18 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T22:26:18.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: osdmap e63: 10 total, 9 up, 10 in 2023-07-21T22:26:18.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:18.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2023-07-21T22:26:18.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:18.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:18.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:18.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:18.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:18.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:18.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:18.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:18.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:18.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T22:26:18.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: osdmap e63: 10 total, 9 up, 10 in 2023-07-21T22:26:18.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:18.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2023-07-21T22:26:18.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:18.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:18.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:18.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:18.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:18.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:18.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:18.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:18.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:18.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:18.440 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:26:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:26:17.998+0000 7f3968836700 -1 osd.9 0 waiting for initial osdmap 2023-07-21T22:26:18.441 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:26:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:26:18.012+0000 7f3961625700 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:26:19.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:19 smithi132 ceph-mon[97863]: Detected new or changed devices on smithi107 2023-07-21T22:26:19.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:19 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi107 to 4292M 2023-07-21T22:26:19.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:19 smithi132 ceph-mon[97863]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:19.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:19 smithi132 ceph-mon[97863]: from='osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2023-07-21T22:26:19.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:19 smithi132 ceph-mon[97863]: osdmap e64: 10 total, 9 up, 10 in 2023-07-21T22:26:19.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:19 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:19.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:19 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:19.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:19 smithi190 ceph-mon[98334]: Detected new or changed devices on smithi107 2023-07-21T22:26:19.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:19 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi107 to 4292M 2023-07-21T22:26:19.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:19 smithi190 ceph-mon[98334]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:19.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:19 smithi190 ceph-mon[98334]: from='osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2023-07-21T22:26:19.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:19 smithi190 ceph-mon[98334]: osdmap e64: 10 total, 9 up, 10 in 2023-07-21T22:26:19.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:19 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:19.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:19 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:19.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:19 smithi107 ceph-mon[96543]: Detected new or changed devices on smithi107 2023-07-21T22:26:19.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:19 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi107 to 4292M 2023-07-21T22:26:19.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:19 smithi107 ceph-mon[96543]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:19.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:19 smithi107 ceph-mon[96543]: from='osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2023-07-21T22:26:19.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:19 smithi107 ceph-mon[96543]: osdmap e64: 10 total, 9 up, 10 in 2023-07-21T22:26:19.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:19.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:19.537 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.489+0000 7f9d01b92700 1 -- 172.21.15.132:0/830549977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fecc0 msgr2=0x7f9cfc0ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.537 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.489+0000 7f9d01b92700 1 --2- 172.21.15.132:0/830549977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fecc0 0x7f9cfc0ff130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009990 tx=0x7f9cec009ca0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.537 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.489+0000 7f9d01b92700 1 -- 172.21.15.132:0/830549977 shutdown_connections 2023-07-21T22:26:19.537 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.489+0000 7f9d01b92700 1 --2- 172.21.15.132:0/830549977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cfc0ff670 0x7f9cfc106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.538 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.489+0000 7f9d01b92700 1 --2- 172.21.15.132:0/830549977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc100060 0x7f9cfc100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.538 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.489+0000 7f9d01b92700 1 --2- 172.21.15.132:0/830549977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fecc0 0x7f9cfc0ff130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.538 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.489+0000 7f9d01b92700 1 -- 172.21.15.132:0/830549977 >> 172.21.15.132:0/830549977 conn(0x7f9cfc0f9860 msgr2=0x7f9cfc0fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:19.538 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.489+0000 7f9d01b92700 1 -- 172.21.15.132:0/830549977 shutdown_connections 2023-07-21T22:26:19.538 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.489+0000 7f9d01b92700 1 -- 172.21.15.132:0/830549977 wait complete. 2023-07-21T22:26:19.538 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.490+0000 7f9d01b92700 1 Processor -- start 2023-07-21T22:26:19.538 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.490+0000 7f9d01b92700 1 -- start start 2023-07-21T22:26:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.490+0000 7f9d01b92700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fecc0 0x7f9cfc0fe710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.490+0000 7f9d01b92700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0ff670 0x7f9cfc19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.490+0000 7f9d01b92700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cfc100060 0x7f9cfc19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.490+0000 7f9d01b92700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc19f690 con 0x7f9cfc0ff670 2023-07-21T22:26:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.490+0000 7f9d01b92700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc19f800 con 0x7f9cfc0fecc0 2023-07-21T22:26:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.490+0000 7f9d01b92700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc19f9c0 con 0x7f9cfc100060 2023-07-21T22:26:19.539 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.490+0000 7f9cfb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fecc0 0x7f9cfc0fe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.540 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.490+0000 7f9cfb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fecc0 0x7f9cfc0fe710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:50784/0 (socket says 172.21.15.132:50784) 2023-07-21T22:26:19.540 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.490+0000 7f9cfb7fe700 1 -- 172.21.15.132:0/316309842 learned_addr learned my addr 172.21.15.132:0/316309842 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-07-21T22:26:19.540 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.490+0000 7f9cfaffd700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0ff670 0x7f9cfc19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.540 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.490+0000 7f9cfbfff700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cfc100060 0x7f9cfc19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.540 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.491+0000 7f9cfb7fe700 1 -- 172.21.15.132:0/316309842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cfc100060 msgr2=0x7f9cfc19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.540 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.491+0000 7f9cfb7fe700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cfc100060 0x7f9cfc19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.541 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.491+0000 7f9cfb7fe700 1 -- 172.21.15.132:0/316309842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0ff670 msgr2=0x7f9cfc19ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.541 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.491+0000 7f9cfb7fe700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0ff670 0x7f9cfc19ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.541 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.491+0000 7f9cfb7fe700 1 -- 172.21.15.132:0/316309842 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec0095f0 con 0x7f9cfc0fecc0 2023-07-21T22:26:19.541 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.491+0000 7f9cfb7fe700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fecc0 0x7f9cfc0fe710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400d980 tx=0x7f9ce400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.541 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.491+0000 7f9cf8ff9700 1 -- 172.21.15.132:0/316309842 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4004020 con 0x7f9cfc0fecc0 2023-07-21T22:26:19.541 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.491+0000 7f9cf8ff9700 1 -- 172.21.15.132:0/316309842 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ce400f690 con 0x7f9cfc0fecc0 2023-07-21T22:26:19.541 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.491+0000 7f9cf8ff9700 1 -- 172.21.15.132:0/316309842 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4003a60 con 0x7f9cfc0fecc0 2023-07-21T22:26:19.542 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.491+0000 7f9d01b92700 1 -- 172.21.15.132:0/316309842 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cfc19fca0 con 0x7f9cfc0fecc0 2023-07-21T22:26:19.542 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.491+0000 7f9d01b92700 1 -- 172.21.15.132:0/316309842 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cfc1a0170 con 0x7f9cfc0fecc0 2023-07-21T22:26:19.543 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.492+0000 7f9d01b92700 1 -- 172.21.15.132:0/316309842 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cfc106070 con 0x7f9cfc0fecc0 2023-07-21T22:26:19.543 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.493+0000 7f9cf8ff9700 1 -- 172.21.15.132:0/316309842 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9ce400fb00 con 0x7f9cfc0fecc0 2023-07-21T22:26:19.543 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.494+0000 7f9cf8ff9700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9ce80a11d0 0x7f9ce80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.543 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.494+0000 7f9cf8ff9700 1 -- 172.21.15.132:0/316309842 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f9ce40cbfc0 con 0x7f9cfc0fecc0 2023-07-21T22:26:19.543 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.494+0000 7f9cfaffd700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9ce80a11d0 0x7f9ce80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.543 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.495+0000 7f9cfaffd700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9ce80a11d0 0x7f9ce80a3680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9cec003790 tx=0x7f9cec003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.544 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.498+0000 7f9cf8ff9700 1 -- 172.21.15.132:0/316309842 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9ce4095f90 con 0x7f9cfc0fecc0 2023-07-21T22:26:19.738 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:19.692+0000 7f9d01b92700 1 -- 172.21.15.132:0/316309842 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cfc060950 con 0x7f9ce80a11d0 2023-07-21T22:26:20.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:20 smithi190 ceph-mon[98334]: purged_snaps scrub starts 2023-07-21T22:26:20.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:20 smithi190 ceph-mon[98334]: purged_snaps scrub ok 2023-07-21T22:26:20.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:20 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:20.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:20 smithi190 ceph-mon[98334]: osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252] boot 2023-07-21T22:26:20.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:20 smithi190 ceph-mon[98334]: osdmap e65: 10 total, 10 up, 10 in 2023-07-21T22:26:20.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:20 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:20.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:20 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:26:20.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:20 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:26:20.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:20 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:20.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:20 smithi132 ceph-mon[97863]: purged_snaps scrub starts 2023-07-21T22:26:20.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:20 smithi132 ceph-mon[97863]: purged_snaps scrub ok 2023-07-21T22:26:20.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:20 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:20.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:20 smithi132 ceph-mon[97863]: osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252] boot 2023-07-21T22:26:20.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:20 smithi132 ceph-mon[97863]: osdmap e65: 10 total, 10 up, 10 in 2023-07-21T22:26:20.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:20 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:20.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:20 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:26:20.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:20 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:26:20.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:20 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:20.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:20 smithi107 ceph-mon[96543]: purged_snaps scrub starts 2023-07-21T22:26:20.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:20 smithi107 ceph-mon[96543]: purged_snaps scrub ok 2023-07-21T22:26:20.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:20 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:20.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:20 smithi107 ceph-mon[96543]: osd.9 [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252] boot 2023-07-21T22:26:20.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:20 smithi107 ceph-mon[96543]: osdmap e65: 10 total, 10 up, 10 in 2023-07-21T22:26:20.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:20 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:26:20.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:20 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:26:20.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:20 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:26:20.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:20 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:21.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:21 smithi132 ceph-mon[97863]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:26:21.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:21 smithi132 ceph-mon[97863]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:21.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:21 smithi132 ceph-mon[97863]: osdmap e66: 10 total, 10 up, 10 in 2023-07-21T22:26:21.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:21 smithi190 ceph-mon[98334]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:26:21.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:21 smithi190 ceph-mon[98334]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:21.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:21 smithi190 ceph-mon[98334]: osdmap e66: 10 total, 10 up, 10 in 2023-07-21T22:26:21.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:21 smithi107 ceph-mon[96543]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:26:21.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:21 smithi107 ceph-mon[96543]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:26:21.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:21 smithi107 ceph-mon[96543]: osdmap e66: 10 total, 10 up, 10 in 2023-07-21T22:26:23.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:23 smithi190 ceph-mon[98334]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:23.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:23 smithi190 ceph-mon[98334]: from='client.? 172.21.15.132:0/259582130' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc031611-c261-490b-ab22-31fd52691970"}]: dispatch 2023-07-21T22:26:23.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:23 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc031611-c261-490b-ab22-31fd52691970"}]: dispatch 2023-07-21T22:26:23.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:23 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc031611-c261-490b-ab22-31fd52691970"}]': finished 2023-07-21T22:26:23.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:23 smithi190 ceph-mon[98334]: osdmap e67: 11 total, 10 up, 11 in 2023-07-21T22:26:23.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:23 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:23.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:23 smithi190 ceph-mon[98334]: from='client.? 172.21.15.132:0/1644689227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:26:23.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:23 smithi132 ceph-mon[97863]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:23.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:23 smithi132 ceph-mon[97863]: from='client.? 172.21.15.132:0/259582130' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc031611-c261-490b-ab22-31fd52691970"}]: dispatch 2023-07-21T22:26:23.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:23 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc031611-c261-490b-ab22-31fd52691970"}]: dispatch 2023-07-21T22:26:23.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:23 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc031611-c261-490b-ab22-31fd52691970"}]': finished 2023-07-21T22:26:23.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:23 smithi132 ceph-mon[97863]: osdmap e67: 11 total, 10 up, 11 in 2023-07-21T22:26:23.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:23 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:23.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:23 smithi132 ceph-mon[97863]: from='client.? 172.21.15.132:0/1644689227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:26:23.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:23 smithi107 ceph-mon[96543]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:23.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:23 smithi107 ceph-mon[96543]: from='client.? 172.21.15.132:0/259582130' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc031611-c261-490b-ab22-31fd52691970"}]: dispatch 2023-07-21T22:26:23.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:23 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc031611-c261-490b-ab22-31fd52691970"}]: dispatch 2023-07-21T22:26:23.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:23 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc031611-c261-490b-ab22-31fd52691970"}]': finished 2023-07-21T22:26:23.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:23 smithi107 ceph-mon[96543]: osdmap e67: 11 total, 10 up, 11 in 2023-07-21T22:26:23.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:23 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:23.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:23 smithi107 ceph-mon[96543]: from='client.? 172.21.15.132:0/1644689227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:26:25.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:25 smithi190 ceph-mon[98334]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:25.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:25 smithi132 ceph-mon[97863]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:25.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:25 smithi107 ceph-mon[96543]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:27.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:27 smithi190 ceph-mon[98334]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:27.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:27 smithi132 ceph-mon[97863]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:27.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:27 smithi107 ceph-mon[96543]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:29.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:29 smithi190 ceph-mon[98334]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:29.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:29 smithi132 ceph-mon[97863]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:29.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:29 smithi107 ceph-mon[96543]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:30.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T22:26:30.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:30.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:30 smithi190 ceph-mon[98334]: Deploying daemon osd.10 on smithi132 2023-07-21T22:26:30.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T22:26:30.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:30.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:30 smithi132 ceph-mon[97863]: Deploying daemon osd.10 on smithi132 2023-07-21T22:26:30.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T22:26:30.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:30.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:30 smithi107 ceph-mon[96543]: Deploying daemon osd.10 on smithi132 2023-07-21T22:26:31.333 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:31 smithi132 ceph-mon[97863]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:31.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:31 smithi190 ceph-mon[98334]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:31.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:31 smithi107 ceph-mon[96543]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:33.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:33 smithi190 ceph-mon[98334]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:33.403 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:33 smithi132 ceph-mon[97863]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:33.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:33 smithi107 ceph-mon[96543]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:34.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:34.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:34.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:34.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:34 smithi132 ceph-mon[97863]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:34.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:34.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:34.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:34.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:34 smithi190 ceph-mon[98334]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:34.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:34.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:34.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:34.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:34 smithi107 ceph-mon[96543]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:36.349 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 10 on host 'smithi132' 2023-07-21T22:26:36.349 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:36.303+0000 7f9cf8ff9700 1 -- 172.21.15.132:0/316309842 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f9cfc060950 con 0x7f9ce80a11d0 2023-07-21T22:26:36.351 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:36.305+0000 7f9d01b92700 1 -- 172.21.15.132:0/316309842 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9ce80a11d0 msgr2=0x7f9ce80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:36.351 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:36.305+0000 7f9d01b92700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9ce80a11d0 0x7f9ce80a3680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9cec003790 tx=0x7f9cec003680 comp rx=0 tx=0).stop 2023-07-21T22:26:36.352 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:36.305+0000 7f9d01b92700 1 -- 172.21.15.132:0/316309842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fecc0 msgr2=0x7f9cfc0fe710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:36.352 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:36.305+0000 7f9d01b92700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fecc0 0x7f9cfc0fe710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400d980 tx=0x7f9ce400dc90 comp rx=0 tx=0).stop 2023-07-21T22:26:36.352 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:36.306+0000 7f9d01b92700 1 -- 172.21.15.132:0/316309842 shutdown_connections 2023-07-21T22:26:36.352 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:36.306+0000 7f9d01b92700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9ce80a11d0 0x7f9ce80a3680 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:36.352 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:36.306+0000 7f9d01b92700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cfc100060 0x7f9cfc19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:36.352 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:36.306+0000 7f9d01b92700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fecc0 0x7f9cfc0fe710 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:36.353 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:36.306+0000 7f9d01b92700 1 --2- 172.21.15.132:0/316309842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0ff670 0x7f9cfc19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:36.353 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:36.306+0000 7f9d01b92700 1 -- 172.21.15.132:0/316309842 >> 172.21.15.132:0/316309842 conn(0x7f9cfc0f9860 msgr2=0x7f9cfc1031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:36.353 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:36.306+0000 7f9d01b92700 1 -- 172.21.15.132:0/316309842 shutdown_connections 2023-07-21T22:26:36.353 INFO:teuthology.orchestra.run.smithi132.stderr:2023-07-21T22:26:36.306+0000 7f9d01b92700 1 -- 172.21.15.132:0/316309842 wait complete. 2023-07-21T22:26:36.795 DEBUG:teuthology.orchestra.run.smithi132:osd.10> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.10.service 2023-07-21T22:26:36.797 INFO:tasks.cephadm:Deploying osd.11 on smithi190 with /dev/vg_nvme/lv_1... 2023-07-21T22:26:36.798 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T22:26:36.830 INFO:journalctl@ceph.osd.10.smithi132.stdout:-- Logs begin at Fri 2023-07-21 22:03:19 UTC. -- 2023-07-21T22:26:37.362 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:26:37.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:37 smithi107 ceph-mon[96543]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:37.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:37 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:37.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:37 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:37.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:37 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:37.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:37 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:37.466 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:37 smithi132 ceph-mon[97863]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:37.467 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:37.467 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:37.467 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:37.467 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:37.562 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:37 smithi190 ceph-mon[98334]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:37.562 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:37 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:37.562 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:37 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:37.562 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:37 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:37.563 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:37 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:37.775 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:26:37 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:26:37.552+0000 7f3e296c5640 -1 osd.10 0 log_to_monitors true 2023-07-21T22:26:38.374 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:38 smithi190 ceph-mon[98334]: from='osd.10 [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:26:38.375 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:38 smithi190 ceph-mon[98334]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:26:38.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:38 smithi107 ceph-mon[96543]: from='osd.10 [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:26:38.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:38 smithi107 ceph-mon[96543]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:26:38.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:38 smithi132 ceph-mon[97863]: from='osd.10 [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:26:38.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:38 smithi132 ceph-mon[97863]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:26:38.829 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T22:26:38.829 INFO:teuthology.orchestra.run.smithi190.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T22:26:38.830 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10+0 records in 2023-07-21T22:26:38.830 INFO:teuthology.orchestra.run.smithi190.stderr:10+0 records out 2023-07-21T22:26:38.830 INFO:teuthology.orchestra.run.smithi190.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131698 s, 796 MB/s 2023-07-21T22:26:38.830 INFO:teuthology.orchestra.run.smithi190.stderr:--> Zapping successful for: 2023-07-21T22:26:39.219 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi190:vg_nvme/lv_1 2023-07-21T22:26:39.431 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:26:39 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:26:39.150+0000 7f3e1e53c700 -1 osd.10 0 waiting for initial osdmap 2023-07-21T22:26:39.431 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:26:39 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:26:39.163+0000 7f3e16b2a700 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:26:39.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:39 smithi132 ceph-mon[97863]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:39.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:39 smithi132 ceph-mon[97863]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T22:26:39.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:39 smithi132 ceph-mon[97863]: osdmap e68: 11 total, 10 up, 11 in 2023-07-21T22:26:39.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:39.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:39 smithi132 ceph-mon[97863]: from='osd.10 [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:26:39.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:39 smithi132 ceph-mon[97863]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:26:39.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:39 smithi190 ceph-mon[98334]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:39.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:39 smithi190 ceph-mon[98334]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T22:26:39.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:39 smithi190 ceph-mon[98334]: osdmap e68: 11 total, 10 up, 11 in 2023-07-21T22:26:39.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:39.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:39 smithi190 ceph-mon[98334]: from='osd.10 [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:26:39.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:39 smithi190 ceph-mon[98334]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:26:39.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:39 smithi107 ceph-mon[96543]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:39.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:39 smithi107 ceph-mon[96543]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T22:26:39.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:39 smithi107 ceph-mon[96543]: osdmap e68: 11 total, 10 up, 11 in 2023-07-21T22:26:39.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:39.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:39 smithi107 ceph-mon[96543]: from='osd.10 [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:26:39.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:39 smithi107 ceph-mon[96543]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-07-21T22:26:39.772 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:26:40.255 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-07-21T22:26:40.255 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: osdmap e69: 11 total, 10 up, 11 in 2023-07-21T22:26:40.255 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:40.256 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:40.256 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: Detected new or changed devices on smithi132 2023-07-21T22:26:40.256 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.256 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.256 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:40.256 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:40.256 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:40.257 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:40.257 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi132 to 4292M 2023-07-21T22:26:40.257 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.257 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:40.257 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:40.257 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.257 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:40.258 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:40.258 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:40.258 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-07-21T22:26:40.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: osdmap e69: 11 total, 10 up, 11 in 2023-07-21T22:26:40.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:40.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:40.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: Detected new or changed devices on smithi132 2023-07-21T22:26:40.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:40.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:40.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:40.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:40.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi132 to 4292M 2023-07-21T22:26:40.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:40.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:40.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:40.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:40.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:40.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-07-21T22:26:40.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: osdmap e69: 11 total, 10 up, 11 in 2023-07-21T22:26:40.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:40.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:40.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: Detected new or changed devices on smithi132 2023-07-21T22:26:40.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:40.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:40.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:40.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:26:40.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi132 to 4292M 2023-07-21T22:26:40.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:40.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:40.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:40.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:40.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:40.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:40.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.913+0000 7f47118fc700 1 -- 172.21.15.190:0/4103741531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470c0fdea0 msgr2=0x7f470c0fe2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:40.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.913+0000 7f47118fc700 1 --2- 172.21.15.190:0/4103741531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470c0fdea0 0x7f470c0fe2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009990 tx=0x7f46fc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:26:40.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.913+0000 7f47118fc700 1 -- 172.21.15.190:0/4103741531 shutdown_connections 2023-07-21T22:26:40.960 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.913+0000 7f47118fc700 1 --2- 172.21.15.190:0/4103741531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f470c101810 0x7f470c1080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:40.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.913+0000 7f47118fc700 1 --2- 172.21.15.190:0/4103741531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470c0fdea0 0x7f470c0fe2b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:40.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.913+0000 7f47118fc700 1 --2- 172.21.15.190:0/4103741531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f470c100e80 0x7f470c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:40.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.913+0000 7f47118fc700 1 -- 172.21.15.190:0/4103741531 >> 172.21.15.190:0/4103741531 conn(0x7f470c0f9890 msgr2=0x7f470c0fbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:40.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.913+0000 7f47118fc700 1 -- 172.21.15.190:0/4103741531 shutdown_connections 2023-07-21T22:26:40.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.913+0000 7f47118fc700 1 -- 172.21.15.190:0/4103741531 wait complete. 2023-07-21T22:26:40.961 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f47118fc700 1 Processor -- start 2023-07-21T22:26:40.963 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f47118fc700 1 -- start start 2023-07-21T22:26:40.963 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f47118fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f470c0fdea0 0x7f470c195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:40.963 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f47118fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470c100e80 0x7f470c195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:40.963 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f47118fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f470c101810 0x7f470c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:40.963 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f47118fc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c19d420 con 0x7f470c0fdea0 2023-07-21T22:26:40.963 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f47118fc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c19d560 con 0x7f470c101810 2023-07-21T22:26:40.964 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f47118fc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c19d6f0 con 0x7f470c100e80 2023-07-21T22:26:40.964 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f470a7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470c100e80 0x7f470c195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:40.964 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f470a7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470c100e80 0x7f470c195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:48304/0 (socket says 172.21.15.190:48304) 2023-07-21T22:26:40.964 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f470a7fc700 1 -- 172.21.15.190:0/960261674 learned_addr learned my addr 172.21.15.190:0/960261674 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-07-21T22:26:40.964 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f470b7fe700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f470c101810 0x7f470c19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:40.964 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f470a7fc700 1 -- 172.21.15.190:0/960261674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f470c101810 msgr2=0x7f470c19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:40.965 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f470a7fc700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f470c101810 0x7f470c19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:40.965 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f470a7fc700 1 -- 172.21.15.190:0/960261674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f470c0fdea0 msgr2=0x7f470c195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:40.965 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f470affd700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f470c0fdea0 0x7f470c195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:40.965 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f470a7fc700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f470c0fdea0 0x7f470c195890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:40.965 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.914+0000 7f470a7fc700 1 -- 172.21.15.190:0/960261674 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc0095f0 con 0x7f470c100e80 2023-07-21T22:26:40.965 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.915+0000 7f470a7fc700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470c100e80 0x7f470c195dd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f470000d980 tx=0x7f470000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:40.965 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.915+0000 7f47108fa700 1 -- 172.21.15.190:0/960261674 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4700009b40 con 0x7f470c100e80 2023-07-21T22:26:40.965 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.915+0000 7f47118fc700 1 -- 172.21.15.190:0/960261674 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c19d9d0 con 0x7f470c100e80 2023-07-21T22:26:40.966 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.915+0000 7f47108fa700 1 -- 172.21.15.190:0/960261674 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f470000b6e0 con 0x7f470c100e80 2023-07-21T22:26:40.966 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.915+0000 7f47108fa700 1 -- 172.21.15.190:0/960261674 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470000f620 con 0x7f470c100e80 2023-07-21T22:26:40.966 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.915+0000 7f47118fc700 1 -- 172.21.15.190:0/960261674 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c19df20 con 0x7f470c100e80 2023-07-21T22:26:40.966 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.916+0000 7f47118fc700 1 -- 172.21.15.190:0/960261674 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470c107200 con 0x7f470c100e80 2023-07-21T22:26:40.966 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.917+0000 7f47108fa700 1 -- 172.21.15.190:0/960261674 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4700010460 con 0x7f470c100e80 2023-07-21T22:26:40.967 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.917+0000 7f47108fa700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f46f40a1150 0x7f46f40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:40.967 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.917+0000 7f47108fa700 1 -- 172.21.15.190:0/960261674 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f47000cc550 con 0x7f470c100e80 2023-07-21T22:26:40.967 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.917+0000 7f470affd700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f46f40a1150 0x7f46f40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:40.968 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.918+0000 7f470affd700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f46f40a1150 0x7f46f40a3600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0095c0 tx=0x7f46fc009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:40.968 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:40.921+0000 7f47108fa700 1 -- 172.21.15.190:0/960261674 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f47000962d0 con 0x7f470c100e80 2023-07-21T22:26:41.162 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:41.116+0000 7f47118fc700 1 -- 172.21.15.190:0/960261674 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f470c060950 con 0x7f46f40a1150 2023-07-21T22:26:41.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:41 smithi190 ceph-mon[98334]: purged_snaps scrub starts 2023-07-21T22:26:41.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:41 smithi190 ceph-mon[98334]: purged_snaps scrub ok 2023-07-21T22:26:41.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:41 smithi190 ceph-mon[98334]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:41.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:41 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:41.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:41 smithi190 ceph-mon[98334]: osd.10 [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367] boot 2023-07-21T22:26:41.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:41 smithi190 ceph-mon[98334]: osdmap e70: 11 total, 11 up, 11 in 2023-07-21T22:26:41.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:41 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:41.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:41 smithi190 ceph-mon[98334]: osdmap e71: 11 total, 11 up, 11 in 2023-07-21T22:26:41.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:41 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:26:41.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:41 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:26:41.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:41 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:41.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:41 smithi132 ceph-mon[97863]: purged_snaps scrub starts 2023-07-21T22:26:41.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:41 smithi132 ceph-mon[97863]: purged_snaps scrub ok 2023-07-21T22:26:41.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:41 smithi132 ceph-mon[97863]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:41.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:41 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:41.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:41 smithi132 ceph-mon[97863]: osd.10 [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367] boot 2023-07-21T22:26:41.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:41 smithi132 ceph-mon[97863]: osdmap e70: 11 total, 11 up, 11 in 2023-07-21T22:26:41.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:41 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:41.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:41 smithi132 ceph-mon[97863]: osdmap e71: 11 total, 11 up, 11 in 2023-07-21T22:26:41.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:41 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:26:41.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:41 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:26:41.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:41 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:41.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:41 smithi107 ceph-mon[96543]: purged_snaps scrub starts 2023-07-21T22:26:41.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:41 smithi107 ceph-mon[96543]: purged_snaps scrub ok 2023-07-21T22:26:41.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:41 smithi107 ceph-mon[96543]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:26:41.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:41 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:41.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:41 smithi107 ceph-mon[96543]: osd.10 [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367] boot 2023-07-21T22:26:41.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:41 smithi107 ceph-mon[96543]: osdmap e70: 11 total, 11 up, 11 in 2023-07-21T22:26:41.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:41 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:26:41.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:41 smithi107 ceph-mon[96543]: osdmap e71: 11 total, 11 up, 11 in 2023-07-21T22:26:41.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:41 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:26:41.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:41 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:26:41.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:41 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:42.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:42 smithi190 ceph-mon[98334]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:26:42.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:42 smithi190 ceph-mon[98334]: osdmap e72: 11 total, 11 up, 11 in 2023-07-21T22:26:42.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:42 smithi132 ceph-mon[97863]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:26:42.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:42 smithi132 ceph-mon[97863]: osdmap e72: 11 total, 11 up, 11 in 2023-07-21T22:26:42.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:42 smithi107 ceph-mon[96543]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi190:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:26:42.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:42 smithi107 ceph-mon[96543]: osdmap e72: 11 total, 11 up, 11 in 2023-07-21T22:26:43.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:43 smithi190 ceph-mon[98334]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:26:43.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:43 smithi132 ceph-mon[97863]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:26:43.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:43 smithi107 ceph-mon[96543]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:26:44.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:44 smithi190 ceph-mon[98334]: osdmap e73: 11 total, 11 up, 11 in 2023-07-21T22:26:44.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:44 smithi190 ceph-mon[98334]: from='client.? 172.21.15.190:0/2193622390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9d95a8c-e6d0-49a6-9c50-2adee825168b"}]: dispatch 2023-07-21T22:26:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:44 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9d95a8c-e6d0-49a6-9c50-2adee825168b"}]: dispatch 2023-07-21T22:26:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:44 smithi190 ceph-mon[98334]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9d95a8c-e6d0-49a6-9c50-2adee825168b"}]': finished 2023-07-21T22:26:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:44 smithi190 ceph-mon[98334]: osdmap e74: 12 total, 11 up, 12 in 2023-07-21T22:26:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:44 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:26:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:44 smithi190 ceph-mon[98334]: from='client.? 172.21.15.190:0/1525531988' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:26:44.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:44 smithi132 ceph-mon[97863]: osdmap e73: 11 total, 11 up, 11 in 2023-07-21T22:26:44.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:44 smithi132 ceph-mon[97863]: from='client.? 172.21.15.190:0/2193622390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9d95a8c-e6d0-49a6-9c50-2adee825168b"}]: dispatch 2023-07-21T22:26:44.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:44 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9d95a8c-e6d0-49a6-9c50-2adee825168b"}]: dispatch 2023-07-21T22:26:44.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:44 smithi132 ceph-mon[97863]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9d95a8c-e6d0-49a6-9c50-2adee825168b"}]': finished 2023-07-21T22:26:44.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:44 smithi132 ceph-mon[97863]: osdmap e74: 12 total, 11 up, 12 in 2023-07-21T22:26:44.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:44 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:26:44.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:44 smithi132 ceph-mon[97863]: from='client.? 172.21.15.190:0/1525531988' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:26:44.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:44 smithi107 ceph-mon[96543]: osdmap e73: 11 total, 11 up, 11 in 2023-07-21T22:26:44.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:44 smithi107 ceph-mon[96543]: from='client.? 172.21.15.190:0/2193622390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9d95a8c-e6d0-49a6-9c50-2adee825168b"}]: dispatch 2023-07-21T22:26:44.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:44 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9d95a8c-e6d0-49a6-9c50-2adee825168b"}]: dispatch 2023-07-21T22:26:44.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:44 smithi107 ceph-mon[96543]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9d95a8c-e6d0-49a6-9c50-2adee825168b"}]': finished 2023-07-21T22:26:44.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:44 smithi107 ceph-mon[96543]: osdmap e74: 12 total, 11 up, 12 in 2023-07-21T22:26:44.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:44 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:26:44.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:44 smithi107 ceph-mon[96543]: from='client.? 172.21.15.190:0/1525531988' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:26:45.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:45 smithi190 ceph-mon[98334]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:26:45.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:45 smithi132 ceph-mon[97863]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:26:45.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:45 smithi107 ceph-mon[96543]: pgmap v213: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:26:47.562 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:47 smithi190 ceph-mon[98334]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:26:47.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:47 smithi132 ceph-mon[97863]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:26:47.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:47 smithi107 ceph-mon[96543]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:26:49.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:49 smithi190 ceph-mon[98334]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:26:49.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:49 smithi132 ceph-mon[97863]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:26:49.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:49 smithi107 ceph-mon[96543]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:26:50.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T22:26:50.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:50.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T22:26:50.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:50.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T22:26:50.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:51.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:51 smithi190 ceph-mon[98334]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-07-21T22:26:51.531 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:51 smithi190 ceph-mon[98334]: Deploying daemon osd.11 on smithi190 2023-07-21T22:26:51.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:51 smithi132 ceph-mon[97863]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-07-21T22:26:51.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:51 smithi132 ceph-mon[97863]: Deploying daemon osd.11 on smithi190 2023-07-21T22:26:51.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:51 smithi107 ceph-mon[96543]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-07-21T22:26:51.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:51 smithi107 ceph-mon[96543]: Deploying daemon osd.11 on smithi190 2023-07-21T22:26:53.523 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:53 smithi190 ceph-mon[98334]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-07-21T22:26:53.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:53 smithi132 ceph-mon[97863]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-07-21T22:26:53.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:53 smithi107 ceph-mon[96543]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-07-21T22:26:55.602 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:55 smithi190 ceph-mon[98334]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-07-21T22:26:55.603 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:55 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:55.603 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:55 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:55.603 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:55 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:55.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:55 smithi132 ceph-mon[97863]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-07-21T22:26:55.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:55 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:55.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:55 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:55.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:55 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:55.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:55 smithi107 ceph-mon[96543]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-07-21T22:26:55.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:55.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:55.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:55 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:57.170 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:57.124+0000 7f47108fa700 1 -- 172.21.15.190:0/960261674 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f470c060950 con 0x7f46f40a1150 2023-07-21T22:26:57.170 INFO:teuthology.orchestra.run.smithi190.stdout:Created osd(s) 11 on host 'smithi190' 2023-07-21T22:26:57.172 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:57.126+0000 7f47118fc700 1 -- 172.21.15.190:0/960261674 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f46f40a1150 msgr2=0x7f46f40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:57.172 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:57.126+0000 7f47118fc700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f46f40a1150 0x7f46f40a3600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0095c0 tx=0x7f46fc009530 comp rx=0 tx=0).stop 2023-07-21T22:26:57.172 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:57.126+0000 7f47118fc700 1 -- 172.21.15.190:0/960261674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470c100e80 msgr2=0x7f470c195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:57.172 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:57.126+0000 7f47118fc700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470c100e80 0x7f470c195dd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f470000d980 tx=0x7f470000dc90 comp rx=0 tx=0).stop 2023-07-21T22:26:57.173 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:57.127+0000 7f47118fc700 1 -- 172.21.15.190:0/960261674 shutdown_connections 2023-07-21T22:26:57.173 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:57.127+0000 7f47118fc700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f470c101810 0x7f470c19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:57.173 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:57.127+0000 7f47118fc700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f470c0fdea0 0x7f470c195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:57.173 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:57.127+0000 7f47118fc700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f46f40a1150 0x7f46f40a3600 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:57.173 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:57.127+0000 7f47118fc700 1 --2- 172.21.15.190:0/960261674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f470c100e80 0x7f470c195dd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:57.173 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:57.127+0000 7f47118fc700 1 -- 172.21.15.190:0/960261674 >> 172.21.15.190:0/960261674 conn(0x7f470c0f9890 msgr2=0x7f470c104340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:57.173 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:57.127+0000 7f47118fc700 1 -- 172.21.15.190:0/960261674 shutdown_connections 2023-07-21T22:26:57.174 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:26:57.127+0000 7f47118fc700 1 -- 172.21.15.190:0/960261674 wait complete. 2023-07-21T22:26:57.327 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:57 smithi190 ceph-mon[98334]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:26:57.327 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:57 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.327 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:57 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:57 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:57.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:57 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:57.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:57 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:57 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:57 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.328 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:57 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:57.553 DEBUG:teuthology.orchestra.run.smithi190:osd.11> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.11.service 2023-07-21T22:26:57.554 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-21T22:26:57.555 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-21T22:26:57.587 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:57 smithi107 ceph-mon[96543]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:26:57.587 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:57 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.587 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:57 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.587 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:57 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:57.587 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:57 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:57.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:57 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:57 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:57 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:57 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:57.588 INFO:journalctl@ceph.osd.11.smithi190.stdout:-- Logs begin at Fri 2023-07-21 22:03:29 UTC. -- 2023-07-21T22:26:57.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:57 smithi132 ceph-mon[97863]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:26:57.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:57 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:57 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:57 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:57.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:57 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:57.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:57 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:57 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:57 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:26:57.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:57 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:58.123 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:26:58.640 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:26:58 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:26:58.315+0000 7f577c069640 -1 osd.11 0 log_to_monitors true 2023-07-21T22:26:59.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.272+0000 7f1c51671700 1 -- 172.21.15.107:0/3268604654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c100610 msgr2=0x7f1c4c106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:59.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.272+0000 7f1c51671700 1 --2- 172.21.15.107:0/3268604654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c100610 0x7f1c4c106ed0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400b0a0 tx=0x7f1c4400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:26:59.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.272+0000 7f1c51671700 1 -- 172.21.15.107:0/3268604654 shutdown_connections 2023-07-21T22:26:59.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.272+0000 7f1c51671700 1 --2- 172.21.15.107:0/3268604654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c100610 0x7f1c4c106ed0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:59.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.272+0000 7f1c51671700 1 --2- 172.21.15.107:0/3268604654 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c4c0fea80 0x7f1c4c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.272+0000 7f1c51671700 1 --2- 172.21.15.107:0/3268604654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c4c0ffc80 0x7f1c4c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.272+0000 7f1c51671700 1 -- 172.21.15.107:0/3268604654 >> 172.21.15.107:0/3268604654 conn(0x7f1c4c0fa0b0 msgr2=0x7f1c4c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.272+0000 7f1c51671700 1 -- 172.21.15.107:0/3268604654 shutdown_connections 2023-07-21T22:26:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.272+0000 7f1c51671700 1 -- 172.21.15.107:0/3268604654 wait complete. 2023-07-21T22:26:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.272+0000 7f1c51671700 1 Processor -- start 2023-07-21T22:26:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.272+0000 7f1c51671700 1 -- start start 2023-07-21T22:26:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c51671700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c4c0fea80 0x7f1c4c1948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c51671700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c0ffc80 0x7f1c4c194de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c51671700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c4c100610 0x7f1c4c19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c51671700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c19c430 con 0x7f1c4c0ffc80 2023-07-21T22:26:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c51671700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c19c570 con 0x7f1c4c0fea80 2023-07-21T22:26:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c51671700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c19c700 con 0x7f1c4c100610 2023-07-21T22:26:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c4a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c0ffc80 0x7f1c4c194de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c4a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c0ffc80 0x7f1c4c194de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46038/0 (socket says 172.21.15.107:46038) 2023-07-21T22:26:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c4a7fc700 1 -- 172.21.15.107:0/3107492890 learned_addr learned my addr 172.21.15.107:0/3107492890 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:26:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c4b7fe700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c4c100610 0x7f1c4c19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c4affd700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c4c0fea80 0x7f1c4c1948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c4a7fc700 1 -- 172.21.15.107:0/3107492890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c4c100610 msgr2=0x7f1c4c19be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c4a7fc700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c4c100610 0x7f1c4c19be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c4a7fc700 1 -- 172.21.15.107:0/3107492890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c4c0fea80 msgr2=0x7f1c4c1948a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c4a7fc700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c4c0fea80 0x7f1c4c1948a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c4a7fc700 1 -- 172.21.15.107:0/3107492890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c44009d00 con 0x7f1c4c0ffc80 2023-07-21T22:26:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c4a7fc700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c0ffc80 0x7f1c4c194de0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0099a0 tx=0x7f1c3c009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c33fff700 1 -- 172.21.15.107:0/3107492890 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c012070 con 0x7f1c4c0ffc80 2023-07-21T22:26:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c51671700 1 -- 172.21.15.107:0/3107492890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c0688c0 con 0x7f1c4c0ffc80 2023-07-21T22:26:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c33fff700 1 -- 172.21.15.107:0/3107492890 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c3c00d8f0 con 0x7f1c4c0ffc80 2023-07-21T22:26:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c33fff700 1 -- 172.21.15.107:0/3107492890 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c00c9c0 con 0x7f1c4c0ffc80 2023-07-21T22:26:59.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.273+0000 7f1c51671700 1 -- 172.21.15.107:0/3107492890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c068dc0 con 0x7f1c4c0ffc80 2023-07-21T22:26:59.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.274+0000 7f1c51671700 1 -- 172.21.15.107:0/3107492890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4c10a270 con 0x7f1c4c0ffc80 2023-07-21T22:26:59.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.275+0000 7f1c33fff700 1 -- 172.21.15.107:0/3107492890 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1c3c007c00 con 0x7f1c4c0ffc80 2023-07-21T22:26:59.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.276+0000 7f1c33fff700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1c340a1150 0x7f1c340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:59.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.276+0000 7f1c4affd700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1c340a1150 0x7f1c340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:59.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.276+0000 7f1c33fff700 1 -- 172.21.15.107:0/3107492890 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f1c3c0cd2d0 con 0x7f1c4c0ffc80 2023-07-21T22:26:59.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.277+0000 7f1c4affd700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1c340a1150 0x7f1c340a3600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005950 tx=0x7f1c400058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:59.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.280+0000 7f1c33fff700 1 -- 172.21.15.107:0/3107492890 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1c3c050770 con 0x7f1c4c0ffc80 2023-07-21T22:26:59.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:59 smithi107 ceph-mon[96543]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:26:59.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:59 smithi107 ceph-mon[96543]: from='osd.11 [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:26:59.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:26:59 smithi107 ceph-mon[96543]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:26:59.513 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.467+0000 7f1c51671700 1 -- 172.21.15.107:0/3107492890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f1c4c10a460 con 0x7f1c4c0ffc80 2023-07-21T22:26:59.514 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.467+0000 7f1c33fff700 1 -- 172.21.15.107:0/3107492890 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f1c3c09aa80 con 0x7f1c4c0ffc80 2023-07-21T22:26:59.515 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:26:59.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.470+0000 7f1c51671700 1 -- 172.21.15.107:0/3107492890 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1c340a1150 msgr2=0x7f1c340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:59.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.470+0000 7f1c51671700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1c340a1150 0x7f1c340a3600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005950 tx=0x7f1c400058e0 comp rx=0 tx=0).stop 2023-07-21T22:26:59.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.470+0000 7f1c51671700 1 -- 172.21.15.107:0/3107492890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c0ffc80 msgr2=0x7f1c4c194de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:59.517 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.470+0000 7f1c51671700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c0ffc80 0x7f1c4c194de0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0099a0 tx=0x7f1c3c009cb0 comp rx=0 tx=0).stop 2023-07-21T22:26:59.517 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.470+0000 7f1c51671700 1 -- 172.21.15.107:0/3107492890 shutdown_connections 2023-07-21T22:26:59.517 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.470+0000 7f1c51671700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1c340a1150 0x7f1c340a3600 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:59.517 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.470+0000 7f1c51671700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1c4c100610 0x7f1c4c19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:59.517 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.470+0000 7f1c51671700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c4c0fea80 0x7f1c4c1948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:59.517 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.470+0000 7f1c51671700 1 --2- 172.21.15.107:0/3107492890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c4c0ffc80 0x7f1c4c194de0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:59.517 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.470+0000 7f1c51671700 1 -- 172.21.15.107:0/3107492890 >> 172.21.15.107:0/3107492890 conn(0x7f1c4c0fa0b0 msgr2=0x7f1c4c103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:59.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.470+0000 7f1c51671700 1 -- 172.21.15.107:0/3107492890 shutdown_connections 2023-07-21T22:26:59.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:26:59.470+0000 7f1c51671700 1 -- 172.21.15.107:0/3107492890 wait complete. 2023-07-21T22:26:59.519 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:59 smithi190 ceph-mon[98334]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:26:59.519 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:59 smithi190 ceph-mon[98334]: from='osd.11 [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:26:59.519 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:26:59 smithi190 ceph-mon[98334]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:26:59.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:59 smithi132 ceph-mon[97863]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:26:59.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:59 smithi132 ceph-mon[97863]: from='osd.11 [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:26:59.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:26:59 smithi132 ceph-mon[97863]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:26:59.882 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1689978400,"num_in_osds":12,"osd_in_since":1689978403,"num_remapped_pgs":0} 2023-07-21T22:27:00.640 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:27:00.228+0000 7f57726e3700 -1 osd.11 0 waiting for initial osdmap 2023-07-21T22:27:00.640 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:27:00.243+0000 7f57694ce700 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:27:00.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T22:27:00.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: osdmap e75: 12 total, 11 up, 12 in 2023-07-21T22:27:00.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:00.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='osd.11 [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:27:00.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:27:00.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3107492890' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:27:00.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:00.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:00.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:27:00.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:27:00.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:27:00.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:27:00.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:00.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:00.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:00.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:00 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:00.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T22:27:00.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: osdmap e75: 12 total, 11 up, 12 in 2023-07-21T22:27:00.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:00.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='osd.11 [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:27:00.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:27:00.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3107492890' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:27:00.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:00.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:00.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:27:00.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:27:00.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:27:00.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:27:00.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:00.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:00.685 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:00.685 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:00 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:00.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T22:27:00.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: osdmap e75: 12 total, 11 up, 12 in 2023-07-21T22:27:00.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:00.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='osd.11 [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:27:00.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]: dispatch 2023-07-21T22:27:00.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3107492890' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:27:00.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:00.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:00.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:27:00.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:27:00.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:27:00.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:27:00.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:00.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:00.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:00.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:00 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:00.884 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-21T22:27:01.449 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:27:01.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:01 smithi190 ceph-mon[98334]: Detected new or changed devices on smithi190 2023-07-21T22:27:01.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:01 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi190 to 4292M 2023-07-21T22:27:01.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:01 smithi190 ceph-mon[98334]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:27:01.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:01 smithi190 ceph-mon[98334]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-07-21T22:27:01.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:01 smithi190 ceph-mon[98334]: osdmap e76: 12 total, 11 up, 12 in 2023-07-21T22:27:01.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:01 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:01.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:01 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:01.653 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:01 smithi107 ceph-mon[96543]: Detected new or changed devices on smithi190 2023-07-21T22:27:01.653 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:01 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi190 to 4292M 2023-07-21T22:27:01.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:01 smithi107 ceph-mon[96543]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:27:01.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:01 smithi107 ceph-mon[96543]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-07-21T22:27:01.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:01 smithi107 ceph-mon[96543]: osdmap e76: 12 total, 11 up, 12 in 2023-07-21T22:27:01.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:01 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:01.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:01 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:01.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:01 smithi132 ceph-mon[97863]: Detected new or changed devices on smithi190 2023-07-21T22:27:01.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:01 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi190 to 4292M 2023-07-21T22:27:01.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:01 smithi132 ceph-mon[97863]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:27:01.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:01 smithi132 ceph-mon[97863]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi190", "root=default"]}]': finished 2023-07-21T22:27:01.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:01 smithi132 ceph-mon[97863]: osdmap e76: 12 total, 11 up, 12 in 2023-07-21T22:27:01.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:01 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:01.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:01 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:02.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:02 smithi190 ceph-mon[98334]: purged_snaps scrub starts 2023-07-21T22:27:02.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:02 smithi190 ceph-mon[98334]: purged_snaps scrub ok 2023-07-21T22:27:02.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:02 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:02.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:02 smithi190 ceph-mon[98334]: osd.11 [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112] boot 2023-07-21T22:27:02.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:02 smithi190 ceph-mon[98334]: osdmap e77: 12 total, 12 up, 12 in 2023-07-21T22:27:02.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:02 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:02.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.599+0000 7fa15359e700 1 -- 172.21.15.107:0/1882816677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1540ff9f0 msgr2=0x7fa1540ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:02.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.599+0000 7fa15359e700 1 --2- 172.21.15.107:0/1882816677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1540ff9f0 0x7fa1540ffe00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa1480099a0 tx=0x7fa148009cb0 comp rx=0 tx=0).stop 2023-07-21T22:27:02.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.599+0000 7fa15359e700 1 -- 172.21.15.107:0/1882816677 shutdown_connections 2023-07-21T22:27:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.599+0000 7fa15359e700 1 --2- 172.21.15.107:0/1882816677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa154101580 0x7fa154107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.599+0000 7fa15359e700 1 --2- 172.21.15.107:0/1882816677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1540ff9f0 0x7fa1540ffe00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.599+0000 7fa15359e700 1 --2- 172.21.15.107:0/1882816677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa154100bf0 0x7fa154101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.599+0000 7fa15359e700 1 -- 172.21.15.107:0/1882816677 >> 172.21.15.107:0/1882816677 conn(0x7fa1540fb040 msgr2=0x7fa1540fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.599+0000 7fa15359e700 1 -- 172.21.15.107:0/1882816677 shutdown_connections 2023-07-21T22:27:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.599+0000 7fa15359e700 1 -- 172.21.15.107:0/1882816677 wait complete. 2023-07-21T22:27:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa15359e700 1 Processor -- start 2023-07-21T22:27:02.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa15359e700 1 -- start start 2023-07-21T22:27:02.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa15359e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1540ff9f0 0x7fa154195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:02.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa15359e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa154100bf0 0x7fa154195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:02.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa15359e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa154101580 0x7fa15419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:02.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa15359e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15419d410 con 0x7fa154100bf0 2023-07-21T22:27:02.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa15359e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15419d550 con 0x7fa154101580 2023-07-21T22:27:02.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa15359e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15419d6e0 con 0x7fa1540ff9f0 2023-07-21T22:27:02.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa151d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa154100bf0 0x7fa154195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa151d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa154100bf0 0x7fa154195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46064/0 (socket says 172.21.15.107:46064) 2023-07-21T22:27:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa151d9b700 1 -- 172.21.15.107:0/1126756703 learned_addr learned my addr 172.21.15.107:0/1126756703 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa15259c700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1540ff9f0 0x7fa154195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa152d9d700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa154101580 0x7fa15419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa151d9b700 1 -- 172.21.15.107:0/1126756703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1540ff9f0 msgr2=0x7fa154195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa151d9b700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1540ff9f0 0x7fa154195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa151d9b700 1 -- 172.21.15.107:0/1126756703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa154101580 msgr2=0x7fa15419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa151d9b700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa154101580 0x7fa15419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa151d9b700 1 -- 172.21.15.107:0/1126756703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa148009580 con 0x7fa154100bf0 2023-07-21T22:27:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.600+0000 7fa152d9d700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa154101580 0x7fa15419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:27:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.601+0000 7fa151d9b700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa154100bf0 0x7fa154195dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa14400b560 tx=0x7fa14400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.601+0000 7fa1437fe700 1 -- 172.21.15.107:0/1126756703 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144011830 con 0x7fa154100bf0 2023-07-21T22:27:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.601+0000 7fa1437fe700 1 -- 172.21.15.107:0/1126756703 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa144011e50 con 0x7fa154100bf0 2023-07-21T22:27:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.601+0000 7fa15359e700 1 -- 172.21.15.107:0/1126756703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15419d9c0 con 0x7fa154100bf0 2023-07-21T22:27:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.601+0000 7fa1437fe700 1 -- 172.21.15.107:0/1126756703 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400f2e0 con 0x7fa154100bf0 2023-07-21T22:27:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.601+0000 7fa15359e700 1 -- 172.21.15.107:0/1126756703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15419df40 con 0x7fa154100bf0 2023-07-21T22:27:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.602+0000 7fa15359e700 1 -- 172.21.15.107:0/1126756703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1541070a0 con 0x7fa154100bf0 2023-07-21T22:27:02.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.603+0000 7fa1437fe700 1 -- 172.21.15.107:0/1126756703 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa144010580 con 0x7fa154100bf0 2023-07-21T22:27:02.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.604+0000 7fa1437fe700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa13c0a12a0 0x7fa13c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:02.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.604+0000 7fa15259c700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa13c0a12a0 0x7fa13c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:02.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.604+0000 7fa1437fe700 1 -- 172.21.15.107:0/1126756703 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa1440cc2f0 con 0x7fa154100bf0 2023-07-21T22:27:02.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.604+0000 7fa15259c700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa13c0a12a0 0x7fa13c0a3750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa148009970 tx=0x7fa148005a50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:02.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.607+0000 7fa1437fe700 1 -- 172.21.15.107:0/1126756703 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa144095f60 con 0x7fa154100bf0 2023-07-21T22:27:02.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:02 smithi132 ceph-mon[97863]: purged_snaps scrub starts 2023-07-21T22:27:02.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:02 smithi132 ceph-mon[97863]: purged_snaps scrub ok 2023-07-21T22:27:02.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:02 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:02.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:02 smithi132 ceph-mon[97863]: osd.11 [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112] boot 2023-07-21T22:27:02.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:02 smithi132 ceph-mon[97863]: osdmap e77: 12 total, 12 up, 12 in 2023-07-21T22:27:02.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:02 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:02.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:02 smithi107 ceph-mon[96543]: purged_snaps scrub starts 2023-07-21T22:27:02.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:02 smithi107 ceph-mon[96543]: purged_snaps scrub ok 2023-07-21T22:27:02.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:02 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:02.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:02 smithi107 ceph-mon[96543]: osd.11 [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112] boot 2023-07-21T22:27:02.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:02 smithi107 ceph-mon[96543]: osdmap e77: 12 total, 12 up, 12 in 2023-07-21T22:27:02.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:02 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:27:02.840 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.794+0000 7fa15359e700 1 -- 172.21.15.107:0/1126756703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fa15402caa0 con 0x7fa154100bf0 2023-07-21T22:27:02.841 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.794+0000 7fa1437fe700 1 -- 172.21.15.107:0/1126756703 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7fa144015090 con 0x7fa154100bf0 2023-07-21T22:27:02.841 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:02.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.796+0000 7fa15359e700 1 -- 172.21.15.107:0/1126756703 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa13c0a12a0 msgr2=0x7fa13c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:02.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.796+0000 7fa15359e700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa13c0a12a0 0x7fa13c0a3750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa148009970 tx=0x7fa148005a50 comp rx=0 tx=0).stop 2023-07-21T22:27:02.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.796+0000 7fa15359e700 1 -- 172.21.15.107:0/1126756703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa154100bf0 msgr2=0x7fa154195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:02.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.796+0000 7fa15359e700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa154100bf0 0x7fa154195dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa14400b560 tx=0x7fa14400b870 comp rx=0 tx=0).stop 2023-07-21T22:27:02.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.796+0000 7fa15359e700 1 -- 172.21.15.107:0/1126756703 shutdown_connections 2023-07-21T22:27:02.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.796+0000 7fa15359e700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa154101580 0x7fa15419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:02.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.796+0000 7fa15359e700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa13c0a12a0 0x7fa13c0a3750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:02.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.797+0000 7fa15359e700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1540ff9f0 0x7fa154195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:02.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.797+0000 7fa15359e700 1 --2- 172.21.15.107:0/1126756703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa154100bf0 0x7fa154195dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:02.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.797+0000 7fa15359e700 1 -- 172.21.15.107:0/1126756703 >> 172.21.15.107:0/1126756703 conn(0x7fa1540fb040 msgr2=0x7fa1541041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:02.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.797+0000 7fa15359e700 1 -- 172.21.15.107:0/1126756703 shutdown_connections 2023-07-21T22:27:02.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:02.797+0000 7fa15359e700 1 -- 172.21.15.107:0/1126756703 wait complete. 2023-07-21T22:27:03.306 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1689978421,"num_in_osds":12,"osd_in_since":1689978403,"num_remapped_pgs":0} 2023-07-21T22:27:03.307 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T22:27:03.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:03 smithi190 ceph-mon[98334]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:27:03.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:03 smithi190 ceph-mon[98334]: osdmap e78: 12 total, 12 up, 12 in 2023-07-21T22:27:03.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:03 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1126756703' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:27:03.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:03 smithi132 ceph-mon[97863]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:27:03.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:03 smithi132 ceph-mon[97863]: osdmap e78: 12 total, 12 up, 12 in 2023-07-21T22:27:03.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:03 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1126756703' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:27:03.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:03 smithi107 ceph-mon[96543]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:27:03.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:03 smithi107 ceph-mon[96543]: osdmap e78: 12 total, 12 up, 12 in 2023-07-21T22:27:03.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:03 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1126756703' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:27:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:27:05.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.967+0000 7fe5134eb700 1 -- 172.21.15.107:0/807596201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe50c0ff4b0 msgr2=0x7fe50c0ff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:05.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.967+0000 7fe5134eb700 1 --2- 172.21.15.107:0/807596201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe50c0ff4b0 0x7fe50c0ff880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe5080099c0 tx=0x7fe508009cd0 comp rx=0 tx=0).stop 2023-07-21T22:27:05.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 -- 172.21.15.107:0/807596201 shutdown_connections 2023-07-21T22:27:05.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 --2- 172.21.15.107:0/807596201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe50c103e20 0x7fe50c10b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:05.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 --2- 172.21.15.107:0/807596201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe50c0ff4b0 0x7fe50c0ff880 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:05.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 --2- 172.21.15.107:0/807596201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe50c0ffe50 0x7fe50c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:05.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 -- 172.21.15.107:0/807596201 >> 172.21.15.107:0/807596201 conn(0x7fe50c075150 msgr2=0x7fe50c075550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:05.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 -- 172.21.15.107:0/807596201 shutdown_connections 2023-07-21T22:27:05.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 -- 172.21.15.107:0/807596201 wait complete. 2023-07-21T22:27:05.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 Processor -- start 2023-07-21T22:27:05.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 -- start start 2023-07-21T22:27:05.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe50c0ff4b0 0x7fe50c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:05.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe50c0ffe50 0x7fe50c1962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:05.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe50c103e20 0x7fe50c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:05.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c19ad70 con 0x7fe50c0ffe50 2023-07-21T22:27:05.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c19aee0 con 0x7fe50c103e20 2023-07-21T22:27:05.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.968+0000 7fe5134eb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c19b0a0 con 0x7fe50c0ff4b0 2023-07-21T22:27:05.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe510a86700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe50c0ffe50 0x7fe50c1962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:05.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe510a86700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe50c0ffe50 0x7fe50c1962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46076/0 (socket says 172.21.15.107:46076) 2023-07-21T22:27:05.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe510a86700 1 -- 172.21.15.107:0/2322751186 learned_addr learned my addr 172.21.15.107:0/2322751186 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:05.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe511a88700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe50c103e20 0x7fe50c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:05.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe511287700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe50c0ff4b0 0x7fe50c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:05.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe510a86700 1 -- 172.21.15.107:0/2322751186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe50c0ff4b0 msgr2=0x7fe50c195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:05.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe510a86700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe50c0ff4b0 0x7fe50c195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:05.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe510a86700 1 -- 172.21.15.107:0/2322751186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe50c103e20 msgr2=0x7fe50c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:05.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe510a86700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe50c103e20 0x7fe50c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:05.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe510a86700 1 -- 172.21.15.107:0/2322751186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080095f0 con 0x7fe50c0ffe50 2023-07-21T22:27:05.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe510a86700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe50c0ffe50 0x7fe50c1962f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00b500 tx=0x7fe4fc00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:05.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe5027fc700 1 -- 172.21.15.107:0/2322751186 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc011830 con 0x7fe50c0ffe50 2023-07-21T22:27:05.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe5027fc700 1 -- 172.21.15.107:0/2322751186 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4fc011e50 con 0x7fe50c0ffe50 2023-07-21T22:27:05.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe5134eb700 1 -- 172.21.15.107:0/2322751186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50c19b380 con 0x7fe50c0ffe50 2023-07-21T22:27:05.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe5027fc700 1 -- 172.21.15.107:0/2322751186 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc00f2e0 con 0x7fe50c0ffe50 2023-07-21T22:27:05.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.969+0000 7fe5134eb700 1 -- 172.21.15.107:0/2322751186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50c1a2be0 con 0x7fe50c0ffe50 2023-07-21T22:27:05.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.970+0000 7fe5134eb700 1 -- 172.21.15.107:0/2322751186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe50c04e210 con 0x7fe50c0ffe50 2023-07-21T22:27:05.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.971+0000 7fe5027fc700 1 -- 172.21.15.107:0/2322751186 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe4fc0105a0 con 0x7fe50c0ffe50 2023-07-21T22:27:05.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.972+0000 7fe5027fc700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe4f80a11d0 0x7fe4f80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:05.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.972+0000 7fe511287700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe4f80a11d0 0x7fe4f80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:05.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.972+0000 7fe5027fc700 1 -- 172.21.15.107:0/2322751186 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe4fc0cc470 con 0x7fe50c0ffe50 2023-07-21T22:27:05.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.972+0000 7fe511287700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe4f80a11d0 0x7fe4f80a3680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe508000c00 tx=0x7fe508005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:05.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:04.975+0000 7fe5027fc700 1 -- 172.21.15.107:0/2322751186 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe4fc096060 con 0x7fe50c0ffe50 2023-07-21T22:27:05.209 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.163+0000 7fe5134eb700 1 -- 172.21.15.107:0/2322751186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe50c1a2e30 con 0x7fe50c0ffe50 2023-07-21T22:27:05.210 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.164+0000 7fe5027fc700 1 -- 172.21.15.107:0/2322751186 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18504 (secure 0 0 0) 0x7fe4fc099680 con 0x7fe50c0ffe50 2023-07-21T22:27:05.210 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:05.210 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":78,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:27:02.233693+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:27:05.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.166+0000 7fe5134eb700 1 -- 172.21.15.107:0/2322751186 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe4f80a11d0 msgr2=0x7fe4f80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:05.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.166+0000 7fe5134eb700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe4f80a11d0 0x7fe4f80a3680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe508000c00 tx=0x7fe508005fb0 comp rx=0 tx=0).stop 2023-07-21T22:27:05.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.166+0000 7fe5134eb700 1 -- 172.21.15.107:0/2322751186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe50c0ffe50 msgr2=0x7fe50c1962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:05.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.166+0000 7fe5134eb700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe50c0ffe50 0x7fe50c1962f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00b500 tx=0x7fe4fc00b8c0 comp rx=0 tx=0).stop 2023-07-21T22:27:05.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.166+0000 7fe5134eb700 1 -- 172.21.15.107:0/2322751186 shutdown_connections 2023-07-21T22:27:05.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.166+0000 7fe5134eb700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe50c103e20 0x7fe50c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:05.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.166+0000 7fe5134eb700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe4f80a11d0 0x7fe4f80a3680 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:05.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.166+0000 7fe5134eb700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe50c0ff4b0 0x7fe50c195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:05.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.166+0000 7fe5134eb700 1 --2- 172.21.15.107:0/2322751186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe50c0ffe50 0x7fe50c1962f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:05.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.166+0000 7fe5134eb700 1 -- 172.21.15.107:0/2322751186 >> 172.21.15.107:0/2322751186 conn(0x7fe50c075150 msgr2=0x7fe50c0fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:05.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.166+0000 7fe5134eb700 1 -- 172.21.15.107:0/2322751186 shutdown_connections 2023-07-21T22:27:05.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:05.166+0000 7fe5134eb700 1 -- 172.21.15.107:0/2322751186 wait complete. 2023-07-21T22:27:05.499 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:05 smithi107 ceph-mon[96543]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:05.500 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:05 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2322751186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:05.513 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:23:52.031819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:27:05.514 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-21T22:27:05.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:05 smithi190 ceph-mon[98334]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:05.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:05 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2322751186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:05.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:05 smithi132 ceph-mon[97863]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:05.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:05 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2322751186' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:06.062 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:27:07.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.220+0000 7f9853a3b700 1 -- 172.21.15.107:0/1647996620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c0feaf0 msgr2=0x7f984c10caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:07.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.220+0000 7f9853a3b700 1 --2- 172.21.15.107:0/1647996620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c0feaf0 0x7f984c10caa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f984400b0a0 tx=0x7f984400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:07.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.220+0000 7f9853a3b700 1 -- 172.21.15.107:0/1647996620 shutdown_connections 2023-07-21T22:27:07.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.221+0000 7f9853a3b700 1 --2- 172.21.15.107:0/1647996620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c0feaf0 0x7f984c10caa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:07.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.221+0000 7f9853a3b700 1 --2- 172.21.15.107:0/1647996620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f984c105cd0 0x7f984c1060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:07.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.221+0000 7f9853a3b700 1 --2- 172.21.15.107:0/1647996620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f984c0fe140 0x7f984c0fe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:07.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.221+0000 7f9853a3b700 1 -- 172.21.15.107:0/1647996620 >> 172.21.15.107:0/1647996620 conn(0x7f984c075150 msgr2=0x7f984c075550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:07.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.221+0000 7f9853a3b700 1 -- 172.21.15.107:0/1647996620 shutdown_connections 2023-07-21T22:27:07.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.221+0000 7f9853a3b700 1 -- 172.21.15.107:0/1647996620 wait complete. 2023-07-21T22:27:07.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.221+0000 7f9853a3b700 1 Processor -- start 2023-07-21T22:27:07.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.221+0000 7f9853a3b700 1 -- start start 2023-07-21T22:27:07.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f9853a3b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c0fe140 0x7f984c195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:07.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f9853a3b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f984c0feaf0 0x7f984c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:07.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f9853a3b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f984c105cd0 0x7f984c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:07.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f9853a3b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c19ad30 con 0x7f984c0fe140 2023-07-21T22:27:07.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f9853a3b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c19aea0 con 0x7f984c0feaf0 2023-07-21T22:27:07.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f9853a3b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c19b060 con 0x7f984c105cd0 2023-07-21T22:27:07.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f98517d7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c0fe140 0x7f984c195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:07.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f98517d7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c0fe140 0x7f984c195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46086/0 (socket says 172.21.15.107:46086) 2023-07-21T22:27:07.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f98517d7700 1 -- 172.21.15.107:0/2240364688 learned_addr learned my addr 172.21.15.107:0/2240364688 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:07.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f9851fd8700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f984c105cd0 0x7f984c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:07.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f9850fd6700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f984c0feaf0 0x7f984c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:07.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f98517d7700 1 -- 172.21.15.107:0/2240364688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f984c105cd0 msgr2=0x7f984c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:07.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f98517d7700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f984c105cd0 0x7f984c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:07.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f98517d7700 1 -- 172.21.15.107:0/2240364688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f984c0feaf0 msgr2=0x7f984c196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:07.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f98517d7700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f984c0feaf0 0x7f984c196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:07.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f98517d7700 1 -- 172.21.15.107:0/2240364688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9844009d00 con 0x7f984c0fe140 2023-07-21T22:27:07.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f98517d7700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c0fe140 0x7f984c195dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f983c00d950 tx=0x7f983c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:07.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f98427fc700 1 -- 172.21.15.107:0/2240364688 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c009940 con 0x7f984c0fe140 2023-07-21T22:27:07.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f98427fc700 1 -- 172.21.15.107:0/2240364688 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f983c010460 con 0x7f984c0fe140 2023-07-21T22:27:07.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f9853a3b700 1 -- 172.21.15.107:0/2240364688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984c19b340 con 0x7f984c0fe140 2023-07-21T22:27:07.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f98427fc700 1 -- 172.21.15.107:0/2240364688 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c00f620 con 0x7f984c0fe140 2023-07-21T22:27:07.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.222+0000 7f9853a3b700 1 -- 172.21.15.107:0/2240364688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c1a2ba0 con 0x7f984c0fe140 2023-07-21T22:27:07.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.223+0000 7f9853a3b700 1 -- 172.21.15.107:0/2240364688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984c04e210 con 0x7f984c0fe140 2023-07-21T22:27:07.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.224+0000 7f98427fc700 1 -- 172.21.15.107:0/2240364688 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f983c0108d0 con 0x7f984c0fe140 2023-07-21T22:27:07.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.225+0000 7f98427fc700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f98380a11d0 0x7f98380a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:07.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.225+0000 7f9850fd6700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f98380a11d0 0x7f98380a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:07.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.225+0000 7f98427fc700 1 -- 172.21.15.107:0/2240364688 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f983c0cc300 con 0x7f984c0fe140 2023-07-21T22:27:07.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.225+0000 7f9850fd6700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f98380a11d0 0x7f98380a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9848005950 tx=0x7f98480058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:07.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.228+0000 7f98427fc700 1 -- 172.21.15.107:0/2240364688 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f983c095e50 con 0x7f984c0fe140 2023-07-21T22:27:07.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:07 smithi107 ceph-mon[96543]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:07.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.415+0000 7f9853a3b700 1 -- 172.21.15.107:0/2240364688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f984c1a2d30 con 0x7f984c0fe140 2023-07-21T22:27:07.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.416+0000 7f98427fc700 1 -- 172.21.15.107:0/2240364688 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f983c099470 con 0x7f984c0fe140 2023-07-21T22:27:07.462 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2023-07-21T22:27:07.464 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.418+0000 7f9853a3b700 1 -- 172.21.15.107:0/2240364688 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f98380a11d0 msgr2=0x7f98380a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:07.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.418+0000 7f9853a3b700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f98380a11d0 0x7f98380a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9848005950 tx=0x7f98480058e0 comp rx=0 tx=0).stop 2023-07-21T22:27:07.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.418+0000 7f9853a3b700 1 -- 172.21.15.107:0/2240364688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c0fe140 msgr2=0x7f984c195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:07.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.418+0000 7f9853a3b700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c0fe140 0x7f984c195dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f983c00d950 tx=0x7f983c00dc60 comp rx=0 tx=0).stop 2023-07-21T22:27:07.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.418+0000 7f9853a3b700 1 -- 172.21.15.107:0/2240364688 shutdown_connections 2023-07-21T22:27:07.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.418+0000 7f9853a3b700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f98380a11d0 0x7f98380a3680 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:07.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.418+0000 7f9853a3b700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f984c105cd0 0x7f984c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:07.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.418+0000 7f9853a3b700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f984c0fe140 0x7f984c195dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:07.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.418+0000 7f9853a3b700 1 --2- 172.21.15.107:0/2240364688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f984c0feaf0 0x7f984c196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:07.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.418+0000 7f9853a3b700 1 -- 172.21.15.107:0/2240364688 >> 172.21.15.107:0/2240364688 conn(0x7f984c075150 msgr2=0x7f984c0fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:07.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.418+0000 7f9853a3b700 1 -- 172.21.15.107:0/2240364688 shutdown_connections 2023-07-21T22:27:07.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:07.418+0000 7f9853a3b700 1 -- 172.21.15.107:0/2240364688 wait complete. 2023-07-21T22:27:07.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:07 smithi190 ceph-mon[98334]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:07.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:07 smithi132 ceph-mon[97863]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:07.814 INFO:tasks.cephadm:Adding mds.a on smithi107 2023-07-21T22:27:07.814 INFO:tasks.cephadm:Adding mds.d on smithi107 2023-07-21T22:27:07.814 INFO:tasks.cephadm:Adding mds.g on smithi107 2023-07-21T22:27:07.815 INFO:tasks.cephadm:Adding mds.j on smithi107 2023-07-21T22:27:07.815 INFO:tasks.cephadm:Adding mds.b on smithi132 2023-07-21T22:27:07.815 INFO:tasks.cephadm:Adding mds.e on smithi132 2023-07-21T22:27:07.815 INFO:tasks.cephadm:Adding mds.h on smithi132 2023-07-21T22:27:07.815 INFO:tasks.cephadm:Adding mds.k on smithi132 2023-07-21T22:27:07.816 INFO:tasks.cephadm:Adding mds.c on smithi190 2023-07-21T22:27:07.816 INFO:tasks.cephadm:Adding mds.f on smithi190 2023-07-21T22:27:07.816 INFO:tasks.cephadm:Adding mds.i on smithi190 2023-07-21T22:27:07.816 INFO:tasks.cephadm:Adding mds.l on smithi190 2023-07-21T22:27:07.817 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph orch apply mds all '12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l' 2023-07-21T22:27:08.386 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:27:08.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:08 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2240364688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:27:08.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:08 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2240364688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:27:08.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:08 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2240364688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:27:09.555 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.508+0000 7fac70b91700 1 -- 172.21.15.190:0/2348605212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac6c100610 msgr2=0x7fac6c106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:09.555 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.508+0000 7fac70b91700 1 --2- 172.21.15.190:0/2348605212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac6c100610 0x7fac6c106ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fac6400b0a0 tx=0x7fac6400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:09.556 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.508+0000 7fac70b91700 1 -- 172.21.15.190:0/2348605212 shutdown_connections 2023-07-21T22:27:09.556 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.508+0000 7fac70b91700 1 --2- 172.21.15.190:0/2348605212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac6c100610 0x7fac6c106ed0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:09.556 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.508+0000 7fac70b91700 1 --2- 172.21.15.190:0/2348605212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac6c0fea80 0x7fac6c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:09.556 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.508+0000 7fac70b91700 1 --2- 172.21.15.190:0/2348605212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac6c0ffc80 0x7fac6c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:09.556 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.508+0000 7fac70b91700 1 -- 172.21.15.190:0/2348605212 >> 172.21.15.190:0/2348605212 conn(0x7fac6c0fa0b0 msgr2=0x7fac6c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:09.556 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.508+0000 7fac70b91700 1 -- 172.21.15.190:0/2348605212 shutdown_connections 2023-07-21T22:27:09.556 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.508+0000 7fac70b91700 1 -- 172.21.15.190:0/2348605212 wait complete. 2023-07-21T22:27:09.558 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.509+0000 7fac70b91700 1 Processor -- start 2023-07-21T22:27:09.558 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.509+0000 7fac70b91700 1 -- start start 2023-07-21T22:27:09.558 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.509+0000 7fac70b91700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac6c0fea80 0x7fac6c1948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:09.558 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.509+0000 7fac70b91700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac6c0ffc80 0x7fac6c194de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:09.558 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.509+0000 7fac70b91700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac6c100610 0x7fac6c19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:09.559 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.509+0000 7fac70b91700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c19c430 con 0x7fac6c100610 2023-07-21T22:27:09.559 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.509+0000 7fac70b91700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c19c570 con 0x7fac6c0ffc80 2023-07-21T22:27:09.559 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.509+0000 7fac70b91700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c19c700 con 0x7fac6c0fea80 2023-07-21T22:27:09.559 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.509+0000 7fac6a59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac6c0fea80 0x7fac6c1948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:09.559 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac6a59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac6c0fea80 0x7fac6c1948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:48422/0 (socket says 172.21.15.190:48422) 2023-07-21T22:27:09.559 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac6a59c700 1 -- 172.21.15.190:0/3030064369 learned_addr learned my addr 172.21.15.190:0/3030064369 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-07-21T22:27:09.559 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac6ad9d700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac6c100610 0x7fac6c19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:09.560 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac69d9b700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac6c0ffc80 0x7fac6c194de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:09.560 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac6a59c700 1 -- 172.21.15.190:0/3030064369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac6c0ffc80 msgr2=0x7fac6c194de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:09.560 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac6a59c700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac6c0ffc80 0x7fac6c194de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:09.560 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac6a59c700 1 -- 172.21.15.190:0/3030064369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac6c100610 msgr2=0x7fac6c19be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:09.560 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac6a59c700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac6c100610 0x7fac6c19be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:09.560 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac6a59c700 1 -- 172.21.15.190:0/3030064369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac64009d00 con 0x7fac6c0fea80 2023-07-21T22:27:09.560 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac6a59c700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac6c0fea80 0x7fac6c1948a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009920 tx=0x7fac5c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:09.561 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac637fe700 1 -- 172.21.15.190:0/3030064369 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c012070 con 0x7fac6c0fea80 2023-07-21T22:27:09.561 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac637fe700 1 -- 172.21.15.190:0/3030064369 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac5c0044e0 con 0x7fac6c0fea80 2023-07-21T22:27:09.561 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac70b91700 1 -- 172.21.15.190:0/3030064369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6c0688c0 con 0x7fac6c0fea80 2023-07-21T22:27:09.561 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac637fe700 1 -- 172.21.15.190:0/3030064369 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c003df0 con 0x7fac6c0fea80 2023-07-21T22:27:09.561 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.510+0000 7fac70b91700 1 -- 172.21.15.190:0/3030064369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6c068dc0 con 0x7fac6c0fea80 2023-07-21T22:27:09.561 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.511+0000 7fac70b91700 1 -- 172.21.15.190:0/3030064369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6c04e210 con 0x7fac6c0fea80 2023-07-21T22:27:09.562 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.513+0000 7fac637fe700 1 -- 172.21.15.190:0/3030064369 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fac5c016030 con 0x7fac6c0fea80 2023-07-21T22:27:09.562 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.513+0000 7fac637fe700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fac580a1150 0x7fac580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:09.562 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.513+0000 7fac637fe700 1 -- 172.21.15.190:0/3030064369 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fac5c0cd970 con 0x7fac6c0fea80 2023-07-21T22:27:09.562 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.513+0000 7fac69d9b700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fac580a1150 0x7fac580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:09.562 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.514+0000 7fac69d9b700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fac580a1150 0x7fac580a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fac54005950 tx=0x7fac5400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:09.563 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.517+0000 7fac637fe700 1 -- 172.21.15.190:0/3030064369 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fac5c097560 con 0x7fac6c0fea80 2023-07-21T22:27:09.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:09 smithi190 ceph-mon[98334]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:09.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:09 smithi132 ceph-mon[97863]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:09.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:09 smithi107 ceph-mon[96543]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:09.757 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.710+0000 7fac70b91700 1 -- 172.21.15.190:0/3030064369 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fac6c069300 con 0x7fac580a1150 2023-07-21T22:27:09.765 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.719+0000 7fac637fe700 1 -- 172.21.15.190:0/3030064369 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fac6c069300 con 0x7fac580a1150 2023-07-21T22:27:09.766 INFO:teuthology.orchestra.run.smithi190.stdout:Scheduled mds.all update... 2023-07-21T22:27:09.767 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.721+0000 7fac70b91700 1 -- 172.21.15.190:0/3030064369 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fac580a1150 msgr2=0x7fac580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:09.768 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.721+0000 7fac70b91700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fac580a1150 0x7fac580a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fac54005950 tx=0x7fac5400b410 comp rx=0 tx=0).stop 2023-07-21T22:27:09.768 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.721+0000 7fac70b91700 1 -- 172.21.15.190:0/3030064369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac6c0fea80 msgr2=0x7fac6c1948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:09.768 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.721+0000 7fac70b91700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac6c0fea80 0x7fac6c1948a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009920 tx=0x7fac5c009c30 comp rx=0 tx=0).stop 2023-07-21T22:27:09.768 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.721+0000 7fac70b91700 1 -- 172.21.15.190:0/3030064369 shutdown_connections 2023-07-21T22:27:09.768 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.721+0000 7fac70b91700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac6c100610 0x7fac6c19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:09.769 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.721+0000 7fac70b91700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fac580a1150 0x7fac580a3600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:09.769 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.721+0000 7fac70b91700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac6c0fea80 0x7fac6c1948a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:09.769 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.721+0000 7fac70b91700 1 --2- 172.21.15.190:0/3030064369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac6c0ffc80 0x7fac6c194de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:09.769 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.721+0000 7fac70b91700 1 -- 172.21.15.190:0/3030064369 >> 172.21.15.190:0/3030064369 conn(0x7fac6c0fa0b0 msgr2=0x7fac6c103170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:09.769 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.721+0000 7fac70b91700 1 -- 172.21.15.190:0/3030064369 shutdown_connections 2023-07-21T22:27:09.769 INFO:teuthology.orchestra.run.smithi190.stderr:2023-07-21T22:27:09.721+0000 7fac70b91700 1 -- 172.21.15.190:0/3030064369 wait complete. 2023-07-21T22:27:10.129 DEBUG:teuthology.orchestra.run.smithi107:mds.a> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.a.service 2023-07-21T22:27:10.132 DEBUG:teuthology.orchestra.run.smithi107:mds.d> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.d.service 2023-07-21T22:27:10.169 INFO:journalctl@ceph.mds.a.smithi107.stdout:-- Logs begin at Fri 2023-07-21 22:03:40 UTC. -- 2023-07-21T22:27:10.170 DEBUG:teuthology.orchestra.run.smithi107:mds.g> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.g.service 2023-07-21T22:27:10.199 INFO:journalctl@ceph.mds.d.smithi107.stdout:-- Logs begin at Fri 2023-07-21 22:03:40 UTC. -- 2023-07-21T22:27:10.200 DEBUG:teuthology.orchestra.run.smithi107:mds.j> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.j.service 2023-07-21T22:27:10.227 INFO:journalctl@ceph.mds.g.smithi107.stdout:-- Logs begin at Fri 2023-07-21 22:03:40 UTC. -- 2023-07-21T22:27:10.228 DEBUG:teuthology.orchestra.run.smithi132:mds.b> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.b.service 2023-07-21T22:27:10.231 DEBUG:teuthology.orchestra.run.smithi132:mds.e> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.e.service 2023-07-21T22:27:10.255 INFO:journalctl@ceph.mds.j.smithi107.stdout:-- Logs begin at Fri 2023-07-21 22:03:40 UTC. -- 2023-07-21T22:27:10.268 INFO:journalctl@ceph.mds.b.smithi132.stdout:-- Logs begin at Fri 2023-07-21 22:03:19 UTC. -- 2023-07-21T22:27:10.269 DEBUG:teuthology.orchestra.run.smithi132:mds.h> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.h.service 2023-07-21T22:27:10.298 INFO:journalctl@ceph.mds.e.smithi132.stdout:-- Logs begin at Fri 2023-07-21 22:03:19 UTC. -- 2023-07-21T22:27:10.299 DEBUG:teuthology.orchestra.run.smithi132:mds.k> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.k.service 2023-07-21T22:27:10.326 INFO:journalctl@ceph.mds.h.smithi132.stdout:-- Logs begin at Fri 2023-07-21 22:03:19 UTC. -- 2023-07-21T22:27:10.327 DEBUG:teuthology.orchestra.run.smithi190:mds.c> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.c.service 2023-07-21T22:27:10.329 DEBUG:teuthology.orchestra.run.smithi190:mds.f> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.f.service 2023-07-21T22:27:10.354 INFO:journalctl@ceph.mds.k.smithi132.stdout:-- Logs begin at Fri 2023-07-21 22:03:19 UTC. -- 2023-07-21T22:27:10.358 INFO:journalctl@ceph.mds.c.smithi190.stdout:-- Logs begin at Fri 2023-07-21 22:03:29 UTC. -- 2023-07-21T22:27:10.359 DEBUG:teuthology.orchestra.run.smithi190:mds.i> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.i.service 2023-07-21T22:27:10.384 INFO:journalctl@ceph.mds.f.smithi190.stdout:-- Logs begin at Fri 2023-07-21 22:03:29 UTC. -- 2023-07-21T22:27:10.385 DEBUG:teuthology.orchestra.run.smithi190:mds.l> sudo journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.l.service 2023-07-21T22:27:10.410 INFO:journalctl@ceph.mds.i.smithi190.stdout:-- Logs begin at Fri 2023-07-21 22:03:29 UTC. -- 2023-07-21T22:27:10.411 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-21T22:27:10.412 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:27:10.438 INFO:journalctl@ceph.mds.l.smithi190.stdout:-- Logs begin at Fri 2023-07-21 22:03:29 UTC. -- 2023-07-21T22:27:10.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.537+0000 7f0f033d1700 1 -- 172.21.15.107:0/3692554320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0efc100560 msgr2=0x7f0efc106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:10.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.537+0000 7f0f033d1700 1 --2- 172.21.15.107:0/3692554320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0efc100560 0x7f0efc106e20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400b0a0 tx=0x7f0ef400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.537+0000 7f0f033d1700 1 -- 172.21.15.107:0/3692554320 shutdown_connections 2023-07-21T22:27:10.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.537+0000 7f0f033d1700 1 --2- 172.21.15.107:0/3692554320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0efc100560 0x7f0efc106e20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.537+0000 7f0f033d1700 1 --2- 172.21.15.107:0/3692554320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0efc0fe9e0 0x7f0efc0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.537+0000 7f0f033d1700 1 --2- 172.21.15.107:0/3692554320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0efc0ffbd0 0x7f0efc100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.537+0000 7f0f033d1700 1 -- 172.21.15.107:0/3692554320 >> 172.21.15.107:0/3692554320 conn(0x7f0efc0fa000 msgr2=0x7f0efc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:10.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.537+0000 7f0f033d1700 1 -- 172.21.15.107:0/3692554320 shutdown_connections 2023-07-21T22:27:10.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.537+0000 7f0f033d1700 1 -- 172.21.15.107:0/3692554320 wait complete. 2023-07-21T22:27:10.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.537+0000 7f0f033d1700 1 Processor -- start 2023-07-21T22:27:10.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.537+0000 7f0f033d1700 1 -- start start 2023-07-21T22:27:10.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f033d1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0efc0fe9e0 0x7f0efc1958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:10.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f033d1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0efc0ffbd0 0x7f0efc195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:10.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f033d1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0efc100560 0x7f0efc19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:10.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f033d1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc19d420 con 0x7f0efc100560 2023-07-21T22:27:10.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f033d1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc19d560 con 0x7f0efc0fe9e0 2023-07-21T22:27:10.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f033d1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc19d6f0 con 0x7f0efc0ffbd0 2023-07-21T22:27:10.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f0196e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0efc100560 0x7f0efc19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:10.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f0196e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0efc100560 0x7f0efc19ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46104/0 (socket says 172.21.15.107:46104) 2023-07-21T22:27:10.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f0196e700 1 -- 172.21.15.107:0/3000087434 learned_addr learned my addr 172.21.15.107:0/3000087434 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:10.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f0116d700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0efc0fe9e0 0x7f0efc1958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:10.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f0096c700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0efc0ffbd0 0x7f0efc195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:10.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f0196e700 1 -- 172.21.15.107:0/3000087434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0efc0ffbd0 msgr2=0x7f0efc195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:10.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f0196e700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0efc0ffbd0 0x7f0efc195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f0196e700 1 -- 172.21.15.107:0/3000087434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0efc0fe9e0 msgr2=0x7f0efc1958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:10.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f0196e700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0efc0fe9e0 0x7f0efc1958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f0196e700 1 -- 172.21.15.107:0/3000087434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef4009d00 con 0x7f0efc100560 2023-07-21T22:27:10.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f0196e700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0efc100560 0x7f0efc19ce50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400b780 tx=0x7f0ef400bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:10.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0ef27fc700 1 -- 172.21.15.107:0/3000087434 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef400e040 con 0x7f0efc100560 2023-07-21T22:27:10.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f033d1700 1 -- 172.21.15.107:0/3000087434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0efc19d970 con 0x7f0efc100560 2023-07-21T22:27:10.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0ef27fc700 1 -- 172.21.15.107:0/3000087434 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ef4004e60 con 0x7f0efc100560 2023-07-21T22:27:10.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0ef27fc700 1 -- 172.21.15.107:0/3000087434 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef401b6b0 con 0x7f0efc100560 2023-07-21T22:27:10.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.538+0000 7f0f033d1700 1 -- 172.21.15.107:0/3000087434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0efc19ddf0 con 0x7f0efc100560 2023-07-21T22:27:10.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.539+0000 7f0f033d1700 1 -- 172.21.15.107:0/3000087434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0efc106990 con 0x7f0efc100560 2023-07-21T22:27:10.590 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.540+0000 7f0ef27fc700 1 -- 172.21.15.107:0/3000087434 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0ef401b810 con 0x7f0efc100560 2023-07-21T22:27:10.590 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.541+0000 7f0ef27fc700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0ee80a11a0 0x7f0ee80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:10.590 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.541+0000 7f0f0116d700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0ee80a11a0 0x7f0ee80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:10.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.541+0000 7f0ef27fc700 1 -- 172.21.15.107:0/3000087434 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0ef40ce1f0 con 0x7f0efc100560 2023-07-21T22:27:10.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.541+0000 7f0f0116d700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0ee80a11a0 0x7f0ee80a3650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0ffa30 tx=0x7f0ef8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:10.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.544+0000 7f0ef27fc700 1 -- 172.21.15.107:0/3000087434 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0ef4097e70 con 0x7f0efc100560 2023-07-21T22:27:10.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.735+0000 7f0f033d1700 1 -- 172.21.15.107:0/3000087434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0efc02c940 con 0x7f0efc100560 2023-07-21T22:27:10.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.736+0000 7f0ef27fc700 1 -- 172.21.15.107:0/3000087434 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18504 (secure 0 0 0) 0x7f0ef409b490 con 0x7f0efc100560 2023-07-21T22:27:10.782 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:10.782 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":78,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:27:02.233693+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:27:10.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.738+0000 7f0f033d1700 1 -- 172.21.15.107:0/3000087434 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0ee80a11a0 msgr2=0x7f0ee80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:10.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.738+0000 7f0f033d1700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0ee80a11a0 0x7f0ee80a3650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0ffa30 tx=0x7f0ef8009260 comp rx=0 tx=0).stop 2023-07-21T22:27:10.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.738+0000 7f0f033d1700 1 -- 172.21.15.107:0/3000087434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0efc100560 msgr2=0x7f0efc19ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:10.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.738+0000 7f0f033d1700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0efc100560 0x7f0efc19ce50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400b780 tx=0x7f0ef400bd00 comp rx=0 tx=0).stop 2023-07-21T22:27:10.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.738+0000 7f0f033d1700 1 -- 172.21.15.107:0/3000087434 shutdown_connections 2023-07-21T22:27:10.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.738+0000 7f0f033d1700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0efc100560 0x7f0efc19ce50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.738+0000 7f0f033d1700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0efc0fe9e0 0x7f0efc1958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.738+0000 7f0f033d1700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0ee80a11a0 0x7f0ee80a3650 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.738+0000 7f0f033d1700 1 --2- 172.21.15.107:0/3000087434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0efc0ffbd0 0x7f0efc195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.738+0000 7f0f033d1700 1 -- 172.21.15.107:0/3000087434 >> 172.21.15.107:0/3000087434 conn(0x7f0efc0fa000 msgr2=0x7f0efc0fc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:10.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.738+0000 7f0f033d1700 1 -- 172.21.15.107:0/3000087434 shutdown_connections 2023-07-21T22:27:10.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.738+0000 7f0f033d1700 1 -- 172.21.15.107:0/3000087434 wait complete. 2023-07-21T22:27:10.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:23:52.031819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:27:10.797 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:27:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.875+0000 7faa4e236700 1 -- 172.21.15.107:0/4018697190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa48101730 msgr2=0x7faa48107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.875+0000 7faa4e236700 1 --2- 172.21.15.107:0/4018697190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa48101730 0x7faa48107ff0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faa4000b0a0 tx=0x7faa4000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.922 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.875+0000 7faa4e236700 1 -- 172.21.15.107:0/4018697190 shutdown_connections 2023-07-21T22:27:10.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 --2- 172.21.15.107:0/4018697190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa48101730 0x7faa48107ff0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 --2- 172.21.15.107:0/4018697190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa480fddd0 0x7faa480fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 --2- 172.21.15.107:0/4018697190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa48100da0 0x7faa481011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 -- 172.21.15.107:0/4018697190 >> 172.21.15.107:0/4018697190 conn(0x7faa480f97b0 msgr2=0x7faa480fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:10.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 -- 172.21.15.107:0/4018697190 shutdown_connections 2023-07-21T22:27:10.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 -- 172.21.15.107:0/4018697190 wait complete. 2023-07-21T22:27:10.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 Processor -- start 2023-07-21T22:27:10.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 -- start start 2023-07-21T22:27:10.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa480fddd0 0x7faa48195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:10.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa48100da0 0x7faa48195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:10.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa48101730 0x7faa4819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:10.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4819d480 con 0x7faa480fddd0 2023-07-21T22:27:10.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4819d5c0 con 0x7faa48101730 2023-07-21T22:27:10.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.876+0000 7faa4e236700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4819d750 con 0x7faa48100da0 2023-07-21T22:27:10.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa477fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa480fddd0 0x7faa48195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:10.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa477fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa480fddd0 0x7faa48195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46114/0 (socket says 172.21.15.107:46114) 2023-07-21T22:27:10.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa477fe700 1 -- 172.21.15.107:0/1594648609 learned_addr learned my addr 172.21.15.107:0/1594648609 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:10.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa46ffd700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa48100da0 0x7faa48195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:10.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa47fff700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa48101730 0x7faa4819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:10.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa477fe700 1 -- 172.21.15.107:0/1594648609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa48100da0 msgr2=0x7faa48195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:10.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa477fe700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa48100da0 0x7faa48195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa477fe700 1 -- 172.21.15.107:0/1594648609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa48101730 msgr2=0x7faa4819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:10.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa477fe700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa48101730 0x7faa4819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:10.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa477fe700 1 -- 172.21.15.107:0/1594648609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa40009d00 con 0x7faa480fddd0 2023-07-21T22:27:10.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa47fff700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa48101730 0x7faa4819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:27:10.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa477fe700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa480fddd0 0x7faa48195900 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00d980 tx=0x7faa3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:10.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa44ff9700 1 -- 172.21.15.107:0/1594648609 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c004020 con 0x7faa480fddd0 2023-07-21T22:27:10.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa44ff9700 1 -- 172.21.15.107:0/1594648609 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa3c009730 con 0x7faa480fddd0 2023-07-21T22:27:10.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa4e236700 1 -- 172.21.15.107:0/1594648609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4819da30 con 0x7faa480fddd0 2023-07-21T22:27:10.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa44ff9700 1 -- 172.21.15.107:0/1594648609 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c003a80 con 0x7faa480fddd0 2023-07-21T22:27:10.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.877+0000 7faa4e236700 1 -- 172.21.15.107:0/1594648609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4819df40 con 0x7faa480fddd0 2023-07-21T22:27:10.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.878+0000 7faa4e236700 1 -- 172.21.15.107:0/1594648609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa481070b0 con 0x7faa480fddd0 2023-07-21T22:27:10.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.879+0000 7faa44ff9700 1 -- 172.21.15.107:0/1594648609 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7faa3c0098a0 con 0x7faa480fddd0 2023-07-21T22:27:10.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.880+0000 7faa44ff9700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faa300a1150 0x7faa300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:10.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.880+0000 7faa44ff9700 1 -- 172.21.15.107:0/1594648609 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faa3c0cc550 con 0x7faa480fddd0 2023-07-21T22:27:10.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.880+0000 7faa46ffd700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faa300a1150 0x7faa300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:10.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.880+0000 7faa46ffd700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faa300a1150 0x7faa300a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faa38005950 tx=0x7faa3800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:10.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:10.883+0000 7faa44ff9700 1 -- 172.21.15.107:0/1594648609 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7faa3c0960a0 con 0x7faa480fddd0 2023-07-21T22:27:11.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:10 smithi190 ceph-mon[98334]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:11.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:10 smithi190 ceph-mon[98334]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2023-07-21T22:27:11.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:10 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:11.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:10 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:11.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:10 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:11.018 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:10 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:11.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:10 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:11.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:10 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:11.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:10 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:11.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:10 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:11.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:10 smithi190 ceph-mon[98334]: Deploying daemon mds.l on smithi190 2023-07-21T22:27:11.019 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:10 smithi190 ceph-mon[98334]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:11.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.070+0000 7faa4e236700 1 -- 172.21.15.107:0/1594648609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa4802cac0 con 0x7faa480fddd0 2023-07-21T22:27:11.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.071+0000 7faa44ff9700 1 -- 172.21.15.107:0/1594648609 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa3c0996c0 con 0x7faa480fddd0 2023-07-21T22:27:11.117 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2023-07-21T22:27:11.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.073+0000 7faa4e236700 1 -- 172.21.15.107:0/1594648609 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faa300a1150 msgr2=0x7faa300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:11.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.073+0000 7faa4e236700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faa300a1150 0x7faa300a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faa38005950 tx=0x7faa3800b410 comp rx=0 tx=0).stop 2023-07-21T22:27:11.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.073+0000 7faa4e236700 1 -- 172.21.15.107:0/1594648609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa480fddd0 msgr2=0x7faa48195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.073+0000 7faa4e236700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa480fddd0 0x7faa48195900 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00d980 tx=0x7faa3c00dc90 comp rx=0 tx=0).stop 2023-07-21T22:27:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.073+0000 7faa4e236700 1 -- 172.21.15.107:0/1594648609 shutdown_connections 2023-07-21T22:27:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.073+0000 7faa4e236700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa48101730 0x7faa4819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.073+0000 7faa4e236700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa480fddd0 0x7faa48195900 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.073+0000 7faa4e236700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faa300a1150 0x7faa300a3600 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.073+0000 7faa4e236700 1 --2- 172.21.15.107:0/1594648609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa48100da0 0x7faa48195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.073+0000 7faa4e236700 1 -- 172.21.15.107:0/1594648609 >> 172.21.15.107:0/1594648609 conn(0x7faa480f97b0 msgr2=0x7faa481041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:11.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.073+0000 7faa4e236700 1 -- 172.21.15.107:0/1594648609 shutdown_connections 2023-07-21T22:27:11.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.073+0000 7faa4e236700 1 -- 172.21.15.107:0/1594648609 wait complete. 2023-07-21T22:27:11.130 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:10 smithi107 ceph-mon[96543]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:11.130 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:10 smithi107 ceph-mon[96543]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2023-07-21T22:27:11.131 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:10 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:11.131 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:10 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:11.131 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:10 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:11.131 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:10 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:11.131 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:10 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:11.131 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:10 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:11.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:10 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:11.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:10 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:11.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:10 smithi107 ceph-mon[96543]: Deploying daemon mds.l on smithi190 2023-07-21T22:27:11.132 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:10 smithi107 ceph-mon[96543]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:11.133 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:27:11.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:10 smithi132 ceph-mon[97863]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:11.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:10 smithi132 ceph-mon[97863]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2023-07-21T22:27:11.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:10 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:11.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:10 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:11.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:10 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:11.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:10 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:11.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:10 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:11.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:10 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:11.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:10 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:11.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:10 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:11.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:10 smithi132 ceph-mon[97863]: Deploying daemon mds.l on smithi190 2023-07-21T22:27:11.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:10 smithi132 ceph-mon[97863]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:11.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.207+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/336936859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a4100570 msgr2=0x7fd1a4106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:11.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.207+0000 7fd1a9b7c700 1 --2- 172.21.15.107:0/336936859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a4100570 0x7fd1a4106e30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00b0a0 tx=0x7fd19c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.207+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/336936859 shutdown_connections 2023-07-21T22:27:11.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.207+0000 7fd1a9b7c700 1 --2- 172.21.15.107:0/336936859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a4100570 0x7fd1a4106e30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.207+0000 7fd1a9b7c700 1 --2- 172.21.15.107:0/336936859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1a40fe9f0 0x7fd1a40fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.207+0000 7fd1a9b7c700 1 --2- 172.21.15.107:0/336936859 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a40ffbe0 0x7fd1a4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.207+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/336936859 >> 172.21.15.107:0/336936859 conn(0x7fd1a40f9fd0 msgr2=0x7fd1a40fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:11.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.207+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/336936859 shutdown_connections 2023-07-21T22:27:11.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.207+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/336936859 wait complete. 2023-07-21T22:27:11.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.207+0000 7fd1a9b7c700 1 Processor -- start 2023-07-21T22:27:11.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a9b7c700 1 -- start start 2023-07-21T22:27:11.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a9b7c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1a40fe9f0 0x7fd1a4069210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:11.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a9b7c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a40ffbe0 0x7fd1a419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:11.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a9b7c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a4100570 0x7fd1a419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:11.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a9b7c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a419d3c0 con 0x7fd1a4100570 2023-07-21T22:27:11.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a9b7c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a419d530 con 0x7fd1a40fe9f0 2023-07-21T22:27:11.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a9b7c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a419d6f0 con 0x7fd1a40ffbe0 2023-07-21T22:27:11.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a4100570 0x7fd1a419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:11.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a4100570 0x7fd1a419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46132/0 (socket says 172.21.15.107:46132) 2023-07-21T22:27:11.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a3fff700 1 -- 172.21.15.107:0/270755166 learned_addr learned my addr 172.21.15.107:0/270755166 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:11.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a2ffd700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a40ffbe0 0x7fd1a419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:11.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a37fe700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1a40fe9f0 0x7fd1a4069210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:11.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a3fff700 1 -- 172.21.15.107:0/270755166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a40ffbe0 msgr2=0x7fd1a419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:11.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a3fff700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a40ffbe0 0x7fd1a419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a3fff700 1 -- 172.21.15.107:0/270755166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1a40fe9f0 msgr2=0x7fd1a4069210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:11.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a3fff700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1a40fe9f0 0x7fd1a4069210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a3fff700 1 -- 172.21.15.107:0/270755166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c009d00 con 0x7fd1a4100570 2023-07-21T22:27:11.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.208+0000 7fd1a3fff700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a4100570 0x7fd1a419cc20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00b830 tx=0x7fd19c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:11.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.209+0000 7fd1a0ff9700 1 -- 172.21.15.107:0/270755166 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c00e040 con 0x7fd1a4100570 2023-07-21T22:27:11.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.209+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/270755166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a419d970 con 0x7fd1a4100570 2023-07-21T22:27:11.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.209+0000 7fd1a0ff9700 1 -- 172.21.15.107:0/270755166 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd19c01db50 con 0x7fd1a4100570 2023-07-21T22:27:11.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.209+0000 7fd1a0ff9700 1 -- 172.21.15.107:0/270755166 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c01bbd0 con 0x7fd1a4100570 2023-07-21T22:27:11.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.209+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/270755166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a419ddf0 con 0x7fd1a4100570 2023-07-21T22:27:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.209+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/270755166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a4105ea0 con 0x7fd1a4100570 2023-07-21T22:27:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.210+0000 7fd1a0ff9700 1 -- 172.21.15.107:0/270755166 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd19c01dcc0 con 0x7fd1a4100570 2023-07-21T22:27:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.211+0000 7fd1a0ff9700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd1900a11a0 0x7fd1900a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:11.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.211+0000 7fd1a37fe700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd1900a11a0 0x7fd1900a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.211+0000 7fd1a0ff9700 1 -- 172.21.15.107:0/270755166 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd19c0ce2f0 con 0x7fd1a4100570 2023-07-21T22:27:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.211+0000 7fd1a37fe700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd1900a11a0 0x7fd1900a3650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005950 tx=0x7fd18c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:11.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.215+0000 7fd1a0ff9700 1 -- 172.21.15.107:0/270755166 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd19c097f70 con 0x7fd1a4100570 2023-07-21T22:27:11.321 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:27:11 smithi190 systemd[1]: Starting Ceph mds.l for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:27:11.492 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.446+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/270755166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1a402c850 con 0x7fd1a4100570 2023-07-21T22:27:11.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.446+0000 7fd1a0ff9700 1 -- 172.21.15.107:0/270755166 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7fd19c09b590 con 0x7fd1a4100570 2023-07-21T22:27:11.493 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:11.494 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-07-21T22:27:11.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.448+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/270755166 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd1900a11a0 msgr2=0x7fd1900a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.448+0000 7fd1a9b7c700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd1900a11a0 0x7fd1900a3650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005950 tx=0x7fd18c0058e0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.448+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/270755166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a4100570 msgr2=0x7fd1a419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.448+0000 7fd1a9b7c700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a4100570 0x7fd1a419cc20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00b830 tx=0x7fd19c004c80 comp rx=0 tx=0).stop 2023-07-21T22:27:11.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.448+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/270755166 shutdown_connections 2023-07-21T22:27:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.448+0000 7fd1a9b7c700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a4100570 0x7fd1a419cc20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.448+0000 7fd1a9b7c700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1a40fe9f0 0x7fd1a4069210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.448+0000 7fd1a9b7c700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd1900a11a0 0x7fd1900a3650 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.448+0000 7fd1a9b7c700 1 --2- 172.21.15.107:0/270755166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1a40ffbe0 0x7fd1a419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.448+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/270755166 >> 172.21.15.107:0/270755166 conn(0x7fd1a40f9fd0 msgr2=0x7fd1a4102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.448+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/270755166 shutdown_connections 2023-07-21T22:27:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:11.448+0000 7fd1a9b7c700 1 -- 172.21.15.107:0/270755166 wait complete. 2023-07-21T22:27:11.498 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 1 2023-07-21T22:27:11.878 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:11 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3000087434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:11.879 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:11 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1594648609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:27:11.879 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:11 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/270755166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:11.879 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:27:11 smithi190 podman[123503]: 2023-07-21T22:27:11.879 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:27:11 smithi190 ceph-mds[123543]: -- 172.21.15.190:0/2698159691 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d9a2cf51e0 con 0x55d9a2ccb000 2023-07-21T22:27:12.140 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:27:11 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-l[123519]: starting mds.l at 2023-07-21T22:27:12.140 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:27:11 smithi190 bash[123503]: 388fa1c518750a423c8812f7d5f4361bc295510beb7f0c59e8affbdf074761bd 2023-07-21T22:27:12.140 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:27:11 smithi190 systemd[1]: Started Ceph mds.l for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:27:12.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:11 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3000087434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:12.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:11 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1594648609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:27:12.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:11 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/270755166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:12.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:11 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3000087434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:12.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:11 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1594648609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:27:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:11 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/270755166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:13.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:12 smithi132 ceph-mon[97863]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:13.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:12 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.190:6832/253562772,v1:172.21.15.190:6833/253562772] up:boot 2023-07-21T22:27:13.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:12 smithi132 ceph-mon[97863]: fsmap 1 up:standby 2023-07-21T22:27:13.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:12 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:27:13.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:12 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:13.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:12 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:13.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:12 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:13.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:12 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:13.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:12 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:13.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:12 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:13.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:12 smithi132 ceph-mon[97863]: Deploying daemon mds.k on smithi132 2023-07-21T22:27:13.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:12 smithi107 ceph-mon[96543]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:13.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:12 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.190:6832/253562772,v1:172.21.15.190:6833/253562772] up:boot 2023-07-21T22:27:13.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:12 smithi107 ceph-mon[96543]: fsmap 1 up:standby 2023-07-21T22:27:13.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:12 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:27:13.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:12 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:13.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:12 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:13.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:12 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:13.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:12 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:13.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:12 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:13.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:12 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:13.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:12 smithi107 ceph-mon[96543]: Deploying daemon mds.k on smithi132 2023-07-21T22:27:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:12 smithi190 ceph-mon[98334]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:12 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.190:6832/253562772,v1:172.21.15.190:6833/253562772] up:boot 2023-07-21T22:27:13.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:12 smithi190 ceph-mon[98334]: fsmap 1 up:standby 2023-07-21T22:27:13.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:12 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:27:13.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:12 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:13.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:12 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:13.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:12 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:13.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:12 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:13.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:12 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:13.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:12 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:13.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:12 smithi190 ceph-mon[98334]: Deploying daemon mds.k on smithi132 2023-07-21T22:27:13.508 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:27:13.514 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:27:13 smithi132 systemd[1]: Starting Ceph mds.k for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:27:13.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.595+0000 7ff3b9562700 1 -- 172.21.15.107:0/2665691853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3b41000b0 msgr2=0x7ff3b41004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:13.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.595+0000 7ff3b9562700 1 --2- 172.21.15.107:0/2665691853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3b41000b0 0x7ff3b41004c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8009990 tx=0x7ff3a8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:13.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.596+0000 7ff3b9562700 1 -- 172.21.15.107:0/2665691853 shutdown_connections 2023-07-21T22:27:13.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.596+0000 7ff3b9562700 1 --2- 172.21.15.107:0/2665691853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3b40ff6b0 0x7ff3b4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:13.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.596+0000 7ff3b9562700 1 --2- 172.21.15.107:0/2665691853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3b41000b0 0x7ff3b41004c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:13.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.596+0000 7ff3b9562700 1 --2- 172.21.15.107:0/2665691853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3b40fed00 0x7ff3b40ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:13.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.596+0000 7ff3b9562700 1 -- 172.21.15.107:0/2665691853 >> 172.21.15.107:0/2665691853 conn(0x7ff3b40f9860 msgr2=0x7ff3b40fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:13.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.596+0000 7ff3b9562700 1 -- 172.21.15.107:0/2665691853 shutdown_connections 2023-07-21T22:27:13.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.596+0000 7ff3b9562700 1 -- 172.21.15.107:0/2665691853 wait complete. 2023-07-21T22:27:13.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.596+0000 7ff3b9562700 1 Processor -- start 2023-07-21T22:27:13.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.596+0000 7ff3b9562700 1 -- start start 2023-07-21T22:27:13.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.596+0000 7ff3b9562700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3b40fed00 0x7ff3b41946e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:13.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b9562700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3b40ff6b0 0x7ff3b4194c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:13.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b9562700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3b41000b0 0x7ff3b419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b9562700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b419c260 con 0x7ff3b41000b0 2023-07-21T22:27:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b9562700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b419c3a0 con 0x7ff3b40ff6b0 2023-07-21T22:27:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b9562700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b419c530 con 0x7ff3b40fed00 2023-07-21T22:27:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3b41000b0 0x7ff3b419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3b41000b0 0x7ff3b419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46146/0 (socket says 172.21.15.107:46146) 2023-07-21T22:27:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b37fe700 1 -- 172.21.15.107:0/2034804131 learned_addr learned my addr 172.21.15.107:0/2034804131 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b27fc700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3b40ff6b0 0x7ff3b4194c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b2ffd700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3b40fed00 0x7ff3b41946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b37fe700 1 -- 172.21.15.107:0/2034804131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3b40fed00 msgr2=0x7ff3b41946e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b37fe700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3b40fed00 0x7ff3b41946e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b37fe700 1 -- 172.21.15.107:0/2034804131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3b40ff6b0 msgr2=0x7ff3b4194c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b37fe700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3b40ff6b0 0x7ff3b4194c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b37fe700 1 -- 172.21.15.107:0/2034804131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a80095f0 con 0x7ff3b41000b0 2023-07-21T22:27:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b37fe700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3b41000b0 0x7ff3b419bc90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac007ee0 tx=0x7ff3ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff39bfff700 1 -- 172.21.15.107:0/2034804131 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac00dd20 con 0x7ff3b41000b0 2023-07-21T22:27:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff39bfff700 1 -- 172.21.15.107:0/2034804131 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3ac00f040 con 0x7ff3b41000b0 2023-07-21T22:27:13.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b9562700 1 -- 172.21.15.107:0/2034804131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b419c810 con 0x7ff3b41000b0 2023-07-21T22:27:13.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff39bfff700 1 -- 172.21.15.107:0/2034804131 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0048a0 con 0x7ff3b41000b0 2023-07-21T22:27:13.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.597+0000 7ff3b9562700 1 -- 172.21.15.107:0/2034804131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b419ccd0 con 0x7ff3b41000b0 2023-07-21T22:27:13.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.598+0000 7ff3b9562700 1 -- 172.21.15.107:0/2034804131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b41060c0 con 0x7ff3b41000b0 2023-07-21T22:27:13.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.599+0000 7ff39bfff700 1 -- 172.21.15.107:0/2034804131 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff3ac003a00 con 0x7ff3b41000b0 2023-07-21T22:27:13.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.600+0000 7ff39bfff700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff39c0a11a0 0x7ff39c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:13.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.600+0000 7ff39bfff700 1 -- 172.21.15.107:0/2034804131 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff3ac0cc500 con 0x7ff3b41000b0 2023-07-21T22:27:13.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.600+0000 7ff3b2ffd700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff39c0a11a0 0x7ff39c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:13.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.600+0000 7ff3b2ffd700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff39c0a11a0 0x7ff39c0a3650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800b5c0 tx=0x7ff3a8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.603+0000 7ff39bfff700 1 -- 172.21.15.107:0/2034804131 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff3ac0960f0 con 0x7ff3b41000b0 2023-07-21T22:27:13.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.835+0000 7ff3b9562700 1 -- 172.21.15.107:0/2034804131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3b402c850 con 0x7ff3b41000b0 2023-07-21T22:27:13.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.835+0000 7ff39bfff700 1 -- 172.21.15.107:0/2034804131 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1257 (secure 0 0 0) 0x7ff3ac023500 con 0x7ff3b41000b0 2023-07-21T22:27:13.882 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:13.882 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-21T22:27:13.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.837+0000 7ff3b9562700 1 -- 172.21.15.107:0/2034804131 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff39c0a11a0 msgr2=0x7ff39c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:13.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.837+0000 7ff3b9562700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff39c0a11a0 0x7ff39c0a3650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800b5c0 tx=0x7ff3a8005fd0 comp rx=0 tx=0).stop 2023-07-21T22:27:13.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.837+0000 7ff3b9562700 1 -- 172.21.15.107:0/2034804131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3b41000b0 msgr2=0x7ff3b419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:13.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.837+0000 7ff3b9562700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3b41000b0 0x7ff3b419bc90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac007ee0 tx=0x7ff3ac00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:13.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.837+0000 7ff3b9562700 1 -- 172.21.15.107:0/2034804131 shutdown_connections 2023-07-21T22:27:13.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.837+0000 7ff3b9562700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3b41000b0 0x7ff3b419bc90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:13.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.837+0000 7ff3b9562700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff39c0a11a0 0x7ff39c0a3650 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:13.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.837+0000 7ff3b9562700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3b40fed00 0x7ff3b41946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:13.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.837+0000 7ff3b9562700 1 --2- 172.21.15.107:0/2034804131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3b40ff6b0 0x7ff3b4194c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:13.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.837+0000 7ff3b9562700 1 -- 172.21.15.107:0/2034804131 >> 172.21.15.107:0/2034804131 conn(0x7ff3b40f9860 msgr2=0x7ff3b4103200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:13.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.837+0000 7ff3b9562700 1 -- 172.21.15.107:0/2034804131 shutdown_connections 2023-07-21T22:27:13.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:13.837+0000 7ff3b9562700 1 -- 172.21.15.107:0/2034804131 wait complete. 2023-07-21T22:27:13.886 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 2 2023-07-21T22:27:14.034 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:27:13 smithi132 podman[122165]: 2023-07-21T22:27:14.034 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:13 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2034804131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:14.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:13 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2034804131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:14.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:13 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2034804131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:14.431 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:27:13 smithi132 ceph-mds[122205]: -- 172.21.15.132:0/3758257489 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c5837251e0 con 0x55c5836fa800 2023-07-21T22:27:14.431 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:27:13 smithi132 bash[122165]: 45df9a83c818c3028e140b43dd15dd1d0895b9bd2be7d14bfbab9c95d3c381e4 2023-07-21T22:27:14.432 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:27:13 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-k[122181]: starting mds.k at 2023-07-21T22:27:14.432 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:27:14 smithi132 systemd[1]: Started Ceph mds.k for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:27:15.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:15 smithi132 ceph-mon[97863]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:15.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:15.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:15.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:15.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:15.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:15.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:15.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:15 smithi132 ceph-mon[97863]: Deploying daemon mds.j on smithi107 2023-07-21T22:27:15.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:15 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] up:boot 2023-07-21T22:27:15.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:15 smithi132 ceph-mon[97863]: fsmap 2 up:standby 2023-07-21T22:27:15.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:27:15.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:15 smithi107 ceph-mon[96543]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:15.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:15.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:15.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:15.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:15.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:15.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:15.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:15 smithi107 ceph-mon[96543]: Deploying daemon mds.j on smithi107 2023-07-21T22:27:15.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:15 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] up:boot 2023-07-21T22:27:15.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:15 smithi107 ceph-mon[96543]: fsmap 2 up:standby 2023-07-21T22:27:15.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:27:15.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:15 smithi190 ceph-mon[98334]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:15.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:15.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:15.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:15.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:15.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:15.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:15.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:15 smithi190 ceph-mon[98334]: Deploying daemon mds.j on smithi107 2023-07-21T22:27:15.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:15 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] up:boot 2023-07-21T22:27:15.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:15 smithi190 ceph-mon[98334]: fsmap 2 up:standby 2023-07-21T22:27:15.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:27:15.825 INFO:journalctl@ceph.mds.j.smithi107.stdout:Jul 21 22:27:15 smithi107 systemd[1]: Starting Ceph mds.j for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:27:15.896 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:27:16.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.979+0000 7f814b51b700 1 -- 172.21.15.107:0/2893426809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81440fffb0 msgr2=0x7f81441003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:16.031 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.979+0000 7f814b51b700 1 --2- 172.21.15.107:0/2893426809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81440fffb0 0x7f81441003c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8140009990 tx=0x7f8140009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:16.031 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.979+0000 7f814b51b700 1 -- 172.21.15.107:0/2893426809 shutdown_connections 2023-07-21T22:27:16.031 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.979+0000 7f814b51b700 1 --2- 172.21.15.107:0/2893426809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81440ff5e0 0x7f8144106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:16.032 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.979+0000 7f814b51b700 1 --2- 172.21.15.107:0/2893426809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81440fffb0 0x7f81441003c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:16.032 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.979+0000 7f814b51b700 1 --2- 172.21.15.107:0/2893426809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81440fec50 0x7f81440ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:16.032 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.979+0000 7f814b51b700 1 -- 172.21.15.107:0/2893426809 >> 172.21.15.107:0/2893426809 conn(0x7f81440f9770 msgr2=0x7f81440fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:16.032 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.979+0000 7f814b51b700 1 -- 172.21.15.107:0/2893426809 shutdown_connections 2023-07-21T22:27:16.032 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.979+0000 7f814b51b700 1 -- 172.21.15.107:0/2893426809 wait complete. 2023-07-21T22:27:16.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f814b51b700 1 Processor -- start 2023-07-21T22:27:16.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f814b51b700 1 -- start start 2023-07-21T22:27:16.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f814b51b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81440fec50 0x7f8144194630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:16.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f814b51b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81440ff5e0 0x7f8144194b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:16.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f814b51b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81440fffb0 0x7f814419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:16.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f814b51b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814419c1b0 con 0x7f81440fec50 2023-07-21T22:27:16.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f814b51b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814419c2f0 con 0x7f81440fffb0 2023-07-21T22:27:16.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f814b51b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814419c480 con 0x7f81440ff5e0 2023-07-21T22:27:16.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f81492b7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81440fec50 0x7f8144194630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:16.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f81492b7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81440fec50 0x7f8144194630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46154/0 (socket says 172.21.15.107:46154) 2023-07-21T22:27:16.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f81492b7700 1 -- 172.21.15.107:0/4285037118 learned_addr learned my addr 172.21.15.107:0/4285037118 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:16.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f8148ab6700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81440ff5e0 0x7f8144194b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:16.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f8149ab8700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81440fffb0 0x7f814419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:16.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f81492b7700 1 -- 172.21.15.107:0/4285037118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81440ff5e0 msgr2=0x7f8144194b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:16.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f81492b7700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81440ff5e0 0x7f8144194b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:16.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f81492b7700 1 -- 172.21.15.107:0/4285037118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81440fffb0 msgr2=0x7f814419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:16.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f81492b7700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81440fffb0 0x7f814419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:16.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f81492b7700 1 -- 172.21.15.107:0/4285037118 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81400095f0 con 0x7f81440fec50 2023-07-21T22:27:16.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f8148ab6700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81440ff5e0 0x7f8144194b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:27:16.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f8149ab8700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81440fffb0 0x7f814419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:27:16.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f81492b7700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81440fec50 0x7f8144194630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f814001a8e0 tx=0x7f814001a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:16.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f813a7fc700 1 -- 172.21.15.107:0/4285037118 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814001f070 con 0x7f81440fec50 2023-07-21T22:27:16.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f813a7fc700 1 -- 172.21.15.107:0/4285037118 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f814001acd0 con 0x7f81440fec50 2023-07-21T22:27:16.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.980+0000 7f813a7fc700 1 -- 172.21.15.107:0/4285037118 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81400056a0 con 0x7f81440fec50 2023-07-21T22:27:16.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.981+0000 7f814b51b700 1 -- 172.21.15.107:0/4285037118 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81440fdb40 con 0x7f81440fec50 2023-07-21T22:27:16.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.981+0000 7f814b51b700 1 -- 172.21.15.107:0/4285037118 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81440fdf40 con 0x7f81440fec50 2023-07-21T22:27:16.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.981+0000 7f814b51b700 1 -- 172.21.15.107:0/4285037118 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81440fe9f0 con 0x7f81440fec50 2023-07-21T22:27:16.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.982+0000 7f813a7fc700 1 -- 172.21.15.107:0/4285037118 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8140023070 con 0x7f81440fec50 2023-07-21T22:27:16.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.983+0000 7f813a7fc700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81300a11a0 0x7f81300a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:16.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.983+0000 7f8148ab6700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81300a11a0 0x7f81300a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:16.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.983+0000 7f813a7fc700 1 -- 172.21.15.107:0/4285037118 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f81400cffb0 con 0x7f81440fec50 2023-07-21T22:27:16.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.983+0000 7f8148ab6700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81300a11a0 0x7f81300a3650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8134005fd0 tx=0x7f8134005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:16.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:15.986+0000 7f813a7fc700 1 -- 172.21.15.107:0/4285037118 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8140099c30 con 0x7f81440fec50 2023-07-21T22:27:16.075 INFO:journalctl@ceph.mds.j.smithi107.stdout:Jul 21 22:27:15 smithi107 podman[129821]: 2023-07-21T22:27:16.075 INFO:journalctl@ceph.mds.j.smithi107.stdout:Jul 21 22:27:16 smithi107 bash[129821]: 333e551620dea93e13d8cb91efe16b9ac74157ad0a52c2edee3a4079103dd76f 2023-07-21T22:27:16.075 INFO:journalctl@ceph.mds.j.smithi107.stdout:Jul 21 22:27:16 smithi107 ceph-mds[129904]: -- 172.21.15.107:0/3290728128 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55de5321f1e0 con 0x55de531f4800 2023-07-21T22:27:16.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.242+0000 7f814b51b700 1 -- 172.21.15.107:0/4285037118 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8144105db0 con 0x7f81440fec50 2023-07-21T22:27:16.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.242+0000 7f813a7fc700 1 -- 172.21.15.107:0/4285037118 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7f814009d250 con 0x7f81440fec50 2023-07-21T22:27:16.289 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:16.289 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-21T22:27:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.244+0000 7f814b51b700 1 -- 172.21.15.107:0/4285037118 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81300a11a0 msgr2=0x7f81300a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.244+0000 7f814b51b700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81300a11a0 0x7f81300a3650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8134005fd0 tx=0x7f8134005e30 comp rx=0 tx=0).stop 2023-07-21T22:27:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.244+0000 7f814b51b700 1 -- 172.21.15.107:0/4285037118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81440fec50 msgr2=0x7f8144194630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.244+0000 7f814b51b700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81440fec50 0x7f8144194630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f814001a8e0 tx=0x7f814001a9c0 comp rx=0 tx=0).stop 2023-07-21T22:27:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.245+0000 7f814b51b700 1 -- 172.21.15.107:0/4285037118 shutdown_connections 2023-07-21T22:27:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.245+0000 7f814b51b700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81440fffb0 0x7f814419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.245+0000 7f814b51b700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81440fec50 0x7f8144194630 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.245+0000 7f814b51b700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81300a11a0 0x7f81300a3650 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.245+0000 7f814b51b700 1 --2- 172.21.15.107:0/4285037118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81440ff5e0 0x7f8144194b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.245+0000 7f814b51b700 1 -- 172.21.15.107:0/4285037118 >> 172.21.15.107:0/4285037118 conn(0x7f81440f9770 msgr2=0x7f8144102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.245+0000 7f814b51b700 1 -- 172.21.15.107:0/4285037118 shutdown_connections 2023-07-21T22:27:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:16.245+0000 7f814b51b700 1 -- 172.21.15.107:0/4285037118 wait complete. 2023-07-21T22:27:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 3 2023-07-21T22:27:16.438 INFO:journalctl@ceph.mds.j.smithi107.stdout:Jul 21 22:27:16 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-j[129837]: starting mds.j at 2023-07-21T22:27:16.438 INFO:journalctl@ceph.mds.j.smithi107.stdout:Jul 21 22:27:16 smithi107 systemd[1]: Started Ceph mds.j for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:27:17.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:17 smithi107 ceph-mon[96543]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:17.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:17.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:17.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:17.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:17.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:17.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:17.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:17 smithi107 ceph-mon[96543]: Deploying daemon mds.d on smithi107 2023-07-21T22:27:17.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:17 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/4285037118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:17.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:17 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.107:6834/219129239,v1:172.21.15.107:6835/219129239] up:boot 2023-07-21T22:27:17.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:17 smithi107 ceph-mon[96543]: fsmap 3 up:standby 2023-07-21T22:27:17.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:17 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:27:17.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:17 smithi190 ceph-mon[98334]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:17.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:17 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:17.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:17 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:17 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:17 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:17 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:17 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:17 smithi190 ceph-mon[98334]: Deploying daemon mds.d on smithi107 2023-07-21T22:27:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:17 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/4285037118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:17 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.107:6834/219129239,v1:172.21.15.107:6835/219129239] up:boot 2023-07-21T22:27:17.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:17 smithi190 ceph-mon[98334]: fsmap 3 up:standby 2023-07-21T22:27:17.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:17 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:27:17.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:17 smithi132 ceph-mon[97863]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:17.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:17.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:17 smithi132 ceph-mon[97863]: Deploying daemon mds.d on smithi107 2023-07-21T22:27:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:17 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/4285037118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:17 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.107:6834/219129239,v1:172.21.15.107:6835/219129239] up:boot 2023-07-21T22:27:17.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:17 smithi132 ceph-mon[97863]: fsmap 3 up:standby 2023-07-21T22:27:17.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:17 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:27:17.938 INFO:journalctl@ceph.mds.d.smithi107.stdout:Jul 21 22:27:17 smithi107 systemd[1]: Starting Ceph mds.d for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:27:18.304 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:27:18.313 INFO:journalctl@ceph.mds.d.smithi107.stdout:Jul 21 22:27:17 smithi107 podman[130211]: 2023-07-21T22:27:18.314 INFO:journalctl@ceph.mds.d.smithi107.stdout:Jul 21 22:27:18 smithi107 ceph-mds[130251]: -- 172.21.15.107:0/864306100 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558641f391e0 con 0x558641f0e800 2023-07-21T22:27:18.314 INFO:journalctl@ceph.mds.d.smithi107.stdout:Jul 21 22:27:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-d[130227]: starting mds.d at 2023-07-21T22:27:18.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.388+0000 7f9aa9c46700 1 -- 172.21.15.107:0/1688615244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa40fea00 msgr2=0x7f9aa40fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:18.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.388+0000 7f9aa9c46700 1 --2- 172.21.15.107:0/1688615244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa40fea00 0x7f9aa40fee10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009990 tx=0x7f9a94009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:18.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.388+0000 7f9aa9c46700 1 -- 172.21.15.107:0/1688615244 shutdown_connections 2023-07-21T22:27:18.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.388+0000 7f9aa9c46700 1 --2- 172.21.15.107:0/1688615244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa4100580 0x7f9aa4106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:18.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.388+0000 7f9aa9c46700 1 --2- 172.21.15.107:0/1688615244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa40fea00 0x7f9aa40fee10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:18.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.388+0000 7f9aa9c46700 1 --2- 172.21.15.107:0/1688615244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9aa40ffbf0 0x7f9aa4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:18.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.388+0000 7f9aa9c46700 1 -- 172.21.15.107:0/1688615244 >> 172.21.15.107:0/1688615244 conn(0x7f9aa40fa020 msgr2=0x7f9aa40fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:18.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.388+0000 7f9aa9c46700 1 -- 172.21.15.107:0/1688615244 shutdown_connections 2023-07-21T22:27:18.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.388+0000 7f9aa9c46700 1 -- 172.21.15.107:0/1688615244 wait complete. 2023-07-21T22:27:18.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa9c46700 1 Processor -- start 2023-07-21T22:27:18.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa9c46700 1 -- start start 2023-07-21T22:27:18.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa9c46700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa40fea00 0x7f9aa4197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:18.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa9c46700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa40ffbf0 0x7f9aa4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:18.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa9c46700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9aa4100580 0x7f9aa419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:18.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa9c46700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa419f5a0 con 0x7f9aa40fea00 2023-07-21T22:27:18.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa9c46700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa419f6e0 con 0x7f9aa4100580 2023-07-21T22:27:18.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa9c46700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa419f870 con 0x7f9aa40ffbf0 2023-07-21T22:27:18.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa40fea00 0x7f9aa4197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:18.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa40fea00 0x7f9aa4197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46190/0 (socket says 172.21.15.107:46190) 2023-07-21T22:27:18.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa37fe700 1 -- 172.21.15.107:0/465633290 learned_addr learned my addr 172.21.15.107:0/465633290 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:18.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa3fff700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9aa4100580 0x7f9aa419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:18.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa2ffd700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa40ffbf0 0x7f9aa4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:18.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa37fe700 1 -- 172.21.15.107:0/465633290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa40ffbf0 msgr2=0x7f9aa4197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:18.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa37fe700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa40ffbf0 0x7f9aa4197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:18.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa37fe700 1 -- 172.21.15.107:0/465633290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9aa4100580 msgr2=0x7f9aa419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:18.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa37fe700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9aa4100580 0x7f9aa419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:18.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.389+0000 7f9aa37fe700 1 -- 172.21.15.107:0/465633290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a940095f0 con 0x7f9aa40fea00 2023-07-21T22:27:18.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.390+0000 7f9aa37fe700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa40fea00 0x7f9aa4197a20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9a94000c00 tx=0x7f9a940047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:18.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.390+0000 7f9aa0ff9700 1 -- 172.21.15.107:0/465633290 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9401d070 con 0x7f9aa40fea00 2023-07-21T22:27:18.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.390+0000 7f9aa0ff9700 1 -- 172.21.15.107:0/465633290 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a94022470 con 0x7f9aa40fea00 2023-07-21T22:27:18.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.390+0000 7f9aa9c46700 1 -- 172.21.15.107:0/465633290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa419faf0 con 0x7f9aa40fea00 2023-07-21T22:27:18.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.390+0000 7f9aa0ff9700 1 -- 172.21.15.107:0/465633290 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a94017650 con 0x7f9aa40fea00 2023-07-21T22:27:18.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.390+0000 7f9aa9c46700 1 -- 172.21.15.107:0/465633290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa419ff10 con 0x7f9aa40fea00 2023-07-21T22:27:18.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.391+0000 7f9aa9c46700 1 -- 172.21.15.107:0/465633290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa410a1d0 con 0x7f9aa40fea00 2023-07-21T22:27:18.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.392+0000 7f9aa0ff9700 1 -- 172.21.15.107:0/465633290 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9a940225e0 con 0x7f9aa40fea00 2023-07-21T22:27:18.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.393+0000 7f9aa0ff9700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a8c0a11a0 0x7f9a8c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:18.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.393+0000 7f9aa2ffd700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a8c0a11a0 0x7f9a8c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:18.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.393+0000 7f9aa0ff9700 1 -- 172.21.15.107:0/465633290 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9a940cdde0 con 0x7f9aa40fea00 2023-07-21T22:27:18.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.393+0000 7f9aa2ffd700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a8c0a11a0 0x7f9a8c0a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009c60 tx=0x7f9a98009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:18.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.396+0000 7f9aa0ff9700 1 -- 172.21.15.107:0/465633290 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9a94097a60 con 0x7f9aa40fea00 2023-07-21T22:27:18.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.634+0000 7f9aa9c46700 1 -- 172.21.15.107:0/465633290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9aa41a04c0 con 0x7f9aa40fea00 2023-07-21T22:27:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.635+0000 7f9aa0ff9700 1 -- 172.21.15.107:0/465633290 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2774 (secure 0 0 0) 0x7f9a9409b080 con 0x7f9aa40fea00 2023-07-21T22:27:18.681 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:18.681 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-21T22:27:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.637+0000 7f9aa9c46700 1 -- 172.21.15.107:0/465633290 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a8c0a11a0 msgr2=0x7f9a8c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.637+0000 7f9aa9c46700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a8c0a11a0 0x7f9a8c0a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009c60 tx=0x7f9a98009260 comp rx=0 tx=0).stop 2023-07-21T22:27:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.637+0000 7f9aa9c46700 1 -- 172.21.15.107:0/465633290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa40fea00 msgr2=0x7f9aa4197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.637+0000 7f9aa9c46700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa40fea00 0x7f9aa4197a20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9a94000c00 tx=0x7f9a940047a0 comp rx=0 tx=0).stop 2023-07-21T22:27:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.637+0000 7f9aa9c46700 1 -- 172.21.15.107:0/465633290 shutdown_connections 2023-07-21T22:27:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.637+0000 7f9aa9c46700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9aa4100580 0x7f9aa419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.637+0000 7f9aa9c46700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa40fea00 0x7f9aa4197a20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.637+0000 7f9aa9c46700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a8c0a11a0 0x7f9a8c0a3650 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.637+0000 7f9aa9c46700 1 --2- 172.21.15.107:0/465633290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa40ffbf0 0x7f9aa4197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.637+0000 7f9aa9c46700 1 -- 172.21.15.107:0/465633290 >> 172.21.15.107:0/465633290 conn(0x7f9aa40fa020 msgr2=0x7f9aa40688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.637+0000 7f9aa9c46700 1 -- 172.21.15.107:0/465633290 shutdown_connections 2023-07-21T22:27:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:18.637+0000 7f9aa9c46700 1 -- 172.21.15.107:0/465633290 wait complete. 2023-07-21T22:27:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 4 2023-07-21T22:27:18.687 INFO:journalctl@ceph.mds.d.smithi107.stdout:Jul 21 22:27:18 smithi107 bash[130211]: 05ed6aa1af961189b72bb140258eed90b5f19df584f5ea5d913ba912831e80e5 2023-07-21T22:27:18.687 INFO:journalctl@ceph.mds.d.smithi107.stdout:Jul 21 22:27:18 smithi107 systemd[1]: Started Ceph mds.d for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:27:19.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:19 smithi132 ceph-mon[97863]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:19.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:19 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:19.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:19 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:19.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:19 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:19.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:19 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:19.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:19 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:19.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:19 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:19.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:19 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/465633290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:19.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:19 smithi190 ceph-mon[98334]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:19.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:19 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:19.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:19 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:19.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:19 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:19.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:19 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:19.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:19 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:19.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:19 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:19.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:19 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/465633290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:19.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:19 smithi107 ceph-mon[96543]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:19.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:19.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:19.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:19.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:19.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:19.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:19 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:19.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:19 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/465633290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:20.182 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:27:19 smithi132 systemd[1]: Starting Ceph mds.e for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:27:20.542 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:27:20 smithi132 podman[122503]: 2023-07-21T22:27:20.542 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:20 smithi132 ceph-mon[97863]: Deploying daemon mds.e on smithi132 2023-07-21T22:27:20.542 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:20 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.107:6836/2102134479,v1:172.21.15.107:6837/2102134479] up:boot 2023-07-21T22:27:20.542 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:20 smithi132 ceph-mon[97863]: fsmap 4 up:standby 2023-07-21T22:27:20.543 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:20 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:27:20.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:20 smithi190 ceph-mon[98334]: Deploying daemon mds.e on smithi132 2023-07-21T22:27:20.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:20 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.107:6836/2102134479,v1:172.21.15.107:6837/2102134479] up:boot 2023-07-21T22:27:20.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:20 smithi190 ceph-mon[98334]: fsmap 4 up:standby 2023-07-21T22:27:20.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:20 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:27:20.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:20 smithi107 ceph-mon[96543]: Deploying daemon mds.e on smithi132 2023-07-21T22:27:20.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:20 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.107:6836/2102134479,v1:172.21.15.107:6837/2102134479] up:boot 2023-07-21T22:27:20.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:20 smithi107 ceph-mon[96543]: fsmap 4 up:standby 2023-07-21T22:27:20.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:20 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:27:20.695 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:27:20.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.779+0000 7f05530e7700 1 -- 172.21.15.107:0/1618731195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c1005a0 msgr2=0x7f054c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:20.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.779+0000 7f05530e7700 1 --2- 172.21.15.107:0/1618731195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c1005a0 0x7f054c106e60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f054400b0a0 tx=0x7f054400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.779+0000 7f05530e7700 1 -- 172.21.15.107:0/1618731195 shutdown_connections 2023-07-21T22:27:20.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.779+0000 7f05530e7700 1 --2- 172.21.15.107:0/1618731195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c1005a0 0x7f054c106e60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.779+0000 7f05530e7700 1 --2- 172.21.15.107:0/1618731195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f054c0fea20 0x7f054c0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.779+0000 7f05530e7700 1 --2- 172.21.15.107:0/1618731195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f054c0ffc10 0x7f054c100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.779+0000 7f05530e7700 1 -- 172.21.15.107:0/1618731195 >> 172.21.15.107:0/1618731195 conn(0x7f054c0fa020 msgr2=0x7f054c0fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:20.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.779+0000 7f05530e7700 1 -- 172.21.15.107:0/1618731195 shutdown_connections 2023-07-21T22:27:20.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.779+0000 7f05530e7700 1 -- 172.21.15.107:0/1618731195 wait complete. 2023-07-21T22:27:20.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.779+0000 7f05530e7700 1 Processor -- start 2023-07-21T22:27:20.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.779+0000 7f05530e7700 1 -- start start 2023-07-21T22:27:20.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f05530e7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f054c0fea20 0x7f054c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f05530e7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f054c0ffc10 0x7f054c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f05530e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c1005a0 0x7f054c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f05530e7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c19f690 con 0x7f054c1005a0 2023-07-21T22:27:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f05530e7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c19f7d0 con 0x7f054c0ffc10 2023-07-21T22:27:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f05530e7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c19f960 con 0x7f054c0fea20 2023-07-21T22:27:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f0551684700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c1005a0 0x7f054c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f0551684700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c1005a0 0x7f054c19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46210/0 (socket says 172.21.15.107:46210) 2023-07-21T22:27:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f0551684700 1 -- 172.21.15.107:0/1216875157 learned_addr learned my addr 172.21.15.107:0/1216875157 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:20.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f0550e83700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f054c0fea20 0x7f054c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f054bfff700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f054c0ffc10 0x7f054c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f0551684700 1 -- 172.21.15.107:0/1216875157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f054c0fea20 msgr2=0x7f054c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:20.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f0551684700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f054c0fea20 0x7f054c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f0551684700 1 -- 172.21.15.107:0/1216875157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f054c0ffc10 msgr2=0x7f054c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:20.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f0551684700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f054c0ffc10 0x7f054c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f0551684700 1 -- 172.21.15.107:0/1216875157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0544009d00 con 0x7f054c1005a0 2023-07-21T22:27:20.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f0551684700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c1005a0 0x7f054c19f0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0544004d50 tx=0x7f0544004d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:20.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f0549ffb700 1 -- 172.21.15.107:0/1216875157 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054400e040 con 0x7f054c1005a0 2023-07-21T22:27:20.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f05530e7700 1 -- 172.21.15.107:0/1216875157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054c19fbe0 con 0x7f054c1005a0 2023-07-21T22:27:20.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f0549ffb700 1 -- 172.21.15.107:0/1216875157 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f054401d940 con 0x7f054c1005a0 2023-07-21T22:27:20.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.780+0000 7f0549ffb700 1 -- 172.21.15.107:0/1216875157 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054401b850 con 0x7f054c1005a0 2023-07-21T22:27:20.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.781+0000 7f05530e7700 1 -- 172.21.15.107:0/1216875157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c1a00b0 con 0x7f054c1005a0 2023-07-21T22:27:20.832 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.781+0000 7f05530e7700 1 -- 172.21.15.107:0/1216875157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c10a200 con 0x7f054c1005a0 2023-07-21T22:27:20.832 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.782+0000 7f0549ffb700 1 -- 172.21.15.107:0/1216875157 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f054401dab0 con 0x7f054c1005a0 2023-07-21T22:27:20.832 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.783+0000 7f0549ffb700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f053c0a11a0 0x7f053c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.833 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.783+0000 7f0550e83700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f053c0a11a0 0x7f053c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.833 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.783+0000 7f0549ffb700 1 -- 172.21.15.107:0/1216875157 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f05440cdd70 con 0x7f054c1005a0 2023-07-21T22:27:20.833 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.783+0000 7f0550e83700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f053c0a11a0 0x7f053c0a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f054c0ffa70 tx=0x7f054000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:20.833 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:20.787+0000 7f0549ffb700 1 -- 172.21.15.107:0/1216875157 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f05440979f0 con 0x7f054c1005a0 2023-07-21T22:27:20.931 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:27:20 smithi132 bash[122503]: 77b76fdf4b5d566358afce03a1f4f176e5bd520f587efab064f821ca643eae63 2023-07-21T22:27:20.931 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:27:20 smithi132 ceph-mds[122543]: -- 172.21.15.132:0/4022224221 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ef3a4cd1e0 con 0x55ef3a4a2c00 2023-07-21T22:27:20.932 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:27:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-e[122519]: starting mds.e at 2023-07-21T22:27:20.932 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:27:20 smithi132 systemd[1]: Started Ceph mds.e for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:27:21.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.019+0000 7f05530e7700 1 -- 172.21.15.107:0/1216875157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f054c1a0660 con 0x7f054c1005a0 2023-07-21T22:27:21.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.020+0000 7f0549ffb700 1 -- 172.21.15.107:0/1216875157 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3534 (secure 0 0 0) 0x7f0544021de0 con 0x7f054c1005a0 2023-07-21T22:27:21.066 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:21.067 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-21T22:27:21.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.022+0000 7f05530e7700 1 -- 172.21.15.107:0/1216875157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f053c0a11a0 msgr2=0x7f053c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:21.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.022+0000 7f05530e7700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f053c0a11a0 0x7f053c0a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f054c0ffa70 tx=0x7f054000b410 comp rx=0 tx=0).stop 2023-07-21T22:27:21.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.022+0000 7f05530e7700 1 -- 172.21.15.107:0/1216875157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c1005a0 msgr2=0x7f054c19f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:21.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.022+0000 7f05530e7700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c1005a0 0x7f054c19f0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0544004d50 tx=0x7f0544004d80 comp rx=0 tx=0).stop 2023-07-21T22:27:21.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.022+0000 7f05530e7700 1 -- 172.21.15.107:0/1216875157 shutdown_connections 2023-07-21T22:27:21.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.022+0000 7f05530e7700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f054c1005a0 0x7f054c19f0c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:21.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.022+0000 7f05530e7700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f053c0a11a0 0x7f053c0a3650 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:21.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.022+0000 7f05530e7700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f054c0fea20 0x7f054c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:21.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.022+0000 7f05530e7700 1 --2- 172.21.15.107:0/1216875157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f054c0ffc10 0x7f054c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:21.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.022+0000 7f05530e7700 1 -- 172.21.15.107:0/1216875157 >> 172.21.15.107:0/1216875157 conn(0x7f054c0fa020 msgr2=0x7f054c068950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:21.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.022+0000 7f05530e7700 1 -- 172.21.15.107:0/1216875157 shutdown_connections 2023-07-21T22:27:21.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:21.022+0000 7f05530e7700 1 -- 172.21.15.107:0/1216875157 wait complete. 2023-07-21T22:27:21.070 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 5 2023-07-21T22:27:21.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:21 smithi107 ceph-mon[96543]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:21.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:21.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:21.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:21.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:21.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:21.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:21.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:21 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1216875157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:21.565 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:21 smithi190 ceph-mon[98334]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:21.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:21.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:21.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:21.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:21.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:21.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:21.566 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:21 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1216875157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:21.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:21 smithi132 ceph-mon[97863]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:21.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:21.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:21.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:21.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:21.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:21.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:21.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:21 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1216875157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:22.216 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:27:22 smithi190 systemd[1]: Starting Ceph mds.f for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:27:22.484 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:27:22 smithi190 podman[123842]: 2023-07-21T22:27:22.484 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:22 smithi190 ceph-mon[98334]: Deploying daemon mds.f on smithi190 2023-07-21T22:27:22.484 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:22 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] up:boot 2023-07-21T22:27:22.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:22 smithi190 ceph-mon[98334]: fsmap 5 up:standby 2023-07-21T22:27:22.485 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:22 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:27:22.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:22 smithi132 ceph-mon[97863]: Deploying daemon mds.f on smithi190 2023-07-21T22:27:22.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:22 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] up:boot 2023-07-21T22:27:22.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:22 smithi132 ceph-mon[97863]: fsmap 5 up:standby 2023-07-21T22:27:22.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:22 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:27:22.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:22 smithi107 ceph-mon[96543]: Deploying daemon mds.f on smithi190 2023-07-21T22:27:22.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:22 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] up:boot 2023-07-21T22:27:22.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:22 smithi107 ceph-mon[96543]: fsmap 5 up:standby 2023-07-21T22:27:22.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:22 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:27:22.890 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:27:22 smithi190 ceph-mds[123882]: -- 172.21.15.190:0/758908181 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556fd268f1e0 con 0x556fd2664800 2023-07-21T22:27:22.890 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:27:22 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: starting mds.f at 2023-07-21T22:27:22.890 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:27:22 smithi190 bash[123842]: 679d63805bc2c977c69d84ebca4ea7d0981fec456567a534b6e4c5e7de38c800 2023-07-21T22:27:22.890 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:27:22 smithi190 systemd[1]: Started Ceph mds.f for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:27:23.081 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:27:23.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.169+0000 7faaeb92e700 1 -- 172.21.15.107:0/1898762637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faae40ffbc0 msgr2=0x7faae4100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:23.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.169+0000 7faaeb92e700 1 --2- 172.21.15.107:0/1898762637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faae40ffbc0 0x7faae4100010 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faad4009990 tx=0x7faad4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:23.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.169+0000 7faaeb92e700 1 -- 172.21.15.107:0/1898762637 shutdown_connections 2023-07-21T22:27:23.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.169+0000 7faaeb92e700 1 --2- 172.21.15.107:0/1898762637 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faae4100550 0x7faae4106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:23.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.169+0000 7faaeb92e700 1 --2- 172.21.15.107:0/1898762637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faae40fe9d0 0x7faae40fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:23.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.169+0000 7faaeb92e700 1 --2- 172.21.15.107:0/1898762637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faae40ffbc0 0x7faae4100010 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:23.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.169+0000 7faaeb92e700 1 -- 172.21.15.107:0/1898762637 >> 172.21.15.107:0/1898762637 conn(0x7faae40f9fd0 msgr2=0x7faae40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:23.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.169+0000 7faaeb92e700 1 -- 172.21.15.107:0/1898762637 shutdown_connections 2023-07-21T22:27:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.169+0000 7faaeb92e700 1 -- 172.21.15.107:0/1898762637 wait complete. 2023-07-21T22:27:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.169+0000 7faaeb92e700 1 Processor -- start 2023-07-21T22:27:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.169+0000 7faaeb92e700 1 -- start start 2023-07-21T22:27:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.169+0000 7faaeb92e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faae40fe9d0 0x7faae4102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faaeb92e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faae40ffbc0 0x7faae4100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faaeb92e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faae4100550 0x7faae41014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faaeb92e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae4101c80 con 0x7faae40ffbc0 2023-07-21T22:27:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faaeb92e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae4101df0 con 0x7faae4100550 2023-07-21T22:27:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faaeb92e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae419f8a0 con 0x7faae40fe9d0 2023-07-21T22:27:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faae96ca700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faae40fe9d0 0x7faae4102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faae8ec9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faae40ffbc0 0x7faae4100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faae96ca700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faae40fe9d0 0x7faae4102960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:34776/0 (socket says 172.21.15.107:34776) 2023-07-21T22:27:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faae8ec9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faae40ffbc0 0x7faae4100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46222/0 (socket says 172.21.15.107:46222) 2023-07-21T22:27:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faae96ca700 1 -- 172.21.15.107:0/3941620235 learned_addr learned my addr 172.21.15.107:0/3941620235 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faae9ecb700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faae4100550 0x7faae41014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:23.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faae8ec9700 1 -- 172.21.15.107:0/3941620235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faae40fe9d0 msgr2=0x7faae4102960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:23.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faae8ec9700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faae40fe9d0 0x7faae4102960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:23.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faae8ec9700 1 -- 172.21.15.107:0/3941620235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faae4100550 msgr2=0x7faae41014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:23.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faae8ec9700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faae4100550 0x7faae41014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:23.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faae8ec9700 1 -- 172.21.15.107:0/3941620235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad40095f0 con 0x7faae40ffbc0 2023-07-21T22:27:23.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faae9ecb700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faae4100550 0x7faae41014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:27:23.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.170+0000 7faae8ec9700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faae40ffbc0 0x7faae4100fa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faad4009990 tx=0x7faad4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:23.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.173+0000 7faada7fc700 1 -- 172.21.15.107:0/3941620235 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad401d070 con 0x7faae40ffbc0 2023-07-21T22:27:23.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.173+0000 7faada7fc700 1 -- 172.21.15.107:0/3941620235 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faad4004020 con 0x7faae40ffbc0 2023-07-21T22:27:23.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.173+0000 7faaeb92e700 1 -- 172.21.15.107:0/3941620235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae419fb20 con 0x7faae40ffbc0 2023-07-21T22:27:23.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.173+0000 7faada7fc700 1 -- 172.21.15.107:0/3941620235 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4022c70 con 0x7faae40ffbc0 2023-07-21T22:27:23.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.173+0000 7faaeb92e700 1 -- 172.21.15.107:0/3941620235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae419ffa0 con 0x7faae40ffbc0 2023-07-21T22:27:23.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.174+0000 7faaeb92e700 1 -- 172.21.15.107:0/3941620235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae410a270 con 0x7faae40ffbc0 2023-07-21T22:27:23.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.175+0000 7faada7fc700 1 -- 172.21.15.107:0/3941620235 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7faad4003b60 con 0x7faae40ffbc0 2023-07-21T22:27:23.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.175+0000 7faada7fc700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faad00a1220 0x7faad00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:23.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.176+0000 7faae96ca700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faad00a1220 0x7faad00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:23.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.176+0000 7faada7fc700 1 -- 172.21.15.107:0/3941620235 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faad40cddd0 con 0x7faae40ffbc0 2023-07-21T22:27:23.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.176+0000 7faae96ca700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faad00a1220 0x7faad00a36d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faae000bd70 tx=0x7faae000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:23.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.179+0000 7faada7fc700 1 -- 172.21.15.107:0/3941620235 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7faad4097a50 con 0x7faae40ffbc0 2023-07-21T22:27:23.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:23 smithi107 ceph-mon[96543]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:23.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:23 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:23.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:23 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:23.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:23 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:23.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:23 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:23.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:23 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:23.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:23 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:23.458 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.412+0000 7faaeb92e700 1 -- 172.21.15.107:0/3941620235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faae41a0760 con 0x7faae40ffbc0 2023-07-21T22:27:23.459 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.413+0000 7faada7fc700 1 -- 172.21.15.107:0/3941620235 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5054 (secure 0 0 0) 0x7faad409b070 con 0x7faae40ffbc0 2023-07-21T22:27:23.459 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:23.459 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-07-21T22:27:23.461 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.415+0000 7faaeb92e700 1 -- 172.21.15.107:0/3941620235 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faad00a1220 msgr2=0x7faad00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:23.461 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.415+0000 7faaeb92e700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faad00a1220 0x7faad00a36d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faae000bd70 tx=0x7faae000b4a0 comp rx=0 tx=0).stop 2023-07-21T22:27:23.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.415+0000 7faaeb92e700 1 -- 172.21.15.107:0/3941620235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faae40ffbc0 msgr2=0x7faae4100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:23.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.415+0000 7faaeb92e700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faae40ffbc0 0x7faae4100fa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faad4009990 tx=0x7faad4009580 comp rx=0 tx=0).stop 2023-07-21T22:27:23.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.415+0000 7faaeb92e700 1 -- 172.21.15.107:0/3941620235 shutdown_connections 2023-07-21T22:27:23.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.415+0000 7faaeb92e700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faae4100550 0x7faae41014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:23.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.415+0000 7faaeb92e700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faad00a1220 0x7faad00a36d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:23.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.415+0000 7faaeb92e700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faae40fe9d0 0x7faae4102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:23.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.415+0000 7faaeb92e700 1 --2- 172.21.15.107:0/3941620235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faae40ffbc0 0x7faae4100fa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:23.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.415+0000 7faaeb92e700 1 -- 172.21.15.107:0/3941620235 >> 172.21.15.107:0/3941620235 conn(0x7faae40f9fd0 msgr2=0x7faae4067690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:23.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.415+0000 7faaeb92e700 1 -- 172.21.15.107:0/3941620235 shutdown_connections 2023-07-21T22:27:23.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:23.415+0000 7faaeb92e700 1 -- 172.21.15.107:0/3941620235 wait complete. 2023-07-21T22:27:23.463 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 7 2023-07-21T22:27:23.476 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:23 smithi190 ceph-mon[98334]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:23.476 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:23 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:23.477 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:23 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:23.477 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:23 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:23.477 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:23 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:23.477 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:23 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:23.477 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:23 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:23.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:23 smithi132 ceph-mon[97863]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:23.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:23 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:23.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:23 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:23.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:23 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:23.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:23 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:23.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:23 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:23.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:23 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:24.640 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:27:24 smithi190 systemd[1]: Starting Ceph mds.c for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:27:24.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:24 smithi190 ceph-mon[98334]: Deploying daemon mds.c on smithi190 2023-07-21T22:27:24.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:24 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] up:boot 2023-07-21T22:27:24.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:24 smithi190 ceph-mon[98334]: fsmap 6 up:standby 2023-07-21T22:27:24.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:24 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:27:24.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:24 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3941620235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:24.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:24 smithi132 ceph-mon[97863]: Deploying daemon mds.c on smithi190 2023-07-21T22:27:24.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:24 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] up:boot 2023-07-21T22:27:24.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:24 smithi132 ceph-mon[97863]: fsmap 6 up:standby 2023-07-21T22:27:24.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:24 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:27:24.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:24 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3941620235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:24.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:24 smithi107 ceph-mon[96543]: Deploying daemon mds.c on smithi190 2023-07-21T22:27:24.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:24 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] up:boot 2023-07-21T22:27:24.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:24 smithi107 ceph-mon[96543]: fsmap 6 up:standby 2023-07-21T22:27:24.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:24 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:27:24.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:24 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3941620235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:24.921 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:27:24 smithi190 podman[124184]: 2023-07-21T22:27:25.241 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:27:24 smithi190 bash[124184]: 2a7e428e6dae164dbb6c881c0e3cb39d2fbc462b9967e00ba649caa88a51da75 2023-07-21T22:27:25.241 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:27:24 smithi190 ceph-mds[124224]: -- 172.21.15.190:0/2644729343 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c12e70f1e0 con 0x55c12e6e5000 2023-07-21T22:27:25.241 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:27:24 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-c[124200]: starting mds.c at 2023-07-21T22:27:25.241 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:27:24 smithi190 systemd[1]: Started Ceph mds.c for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:27:25.474 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:27:25.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:25 smithi107 ceph-mon[96543]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:25.503 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:25 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:25.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:25 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:25.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:25 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:25.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:25 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:25.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:25 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:25.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:25 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:25.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.562+0000 7f8fefcf9700 1 -- 172.21.15.107:0/2020821621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe8100550 msgr2=0x7f8fe8106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:25.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.562+0000 7f8fefcf9700 1 --2- 172.21.15.107:0/2020821621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe8100550 0x7f8fe8106e10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000b0a0 tx=0x7f8fe000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:25.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 -- 172.21.15.107:0/2020821621 shutdown_connections 2023-07-21T22:27:25.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 --2- 172.21.15.107:0/2020821621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe8100550 0x7f8fe8106e10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:25.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 --2- 172.21.15.107:0/2020821621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fe80fe9d0 0x7f8fe80fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:25.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 --2- 172.21.15.107:0/2020821621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fe80ffbc0 0x7f8fe8100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:25.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 -- 172.21.15.107:0/2020821621 >> 172.21.15.107:0/2020821621 conn(0x7f8fe80f9fd0 msgr2=0x7f8fe80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:25.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 -- 172.21.15.107:0/2020821621 shutdown_connections 2023-07-21T22:27:25.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 -- 172.21.15.107:0/2020821621 wait complete. 2023-07-21T22:27:25.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 Processor -- start 2023-07-21T22:27:25.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 -- start start 2023-07-21T22:27:25.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fe80fe9d0 0x7f8fe80681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:25.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe80ffbc0 0x7f8fe819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:25.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fe8100550 0x7f8fe819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:25.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe819d330 con 0x7f8fe80ffbc0 2023-07-21T22:27:25.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe819d4a0 con 0x7f8fe80fe9d0 2023-07-21T22:27:25.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fefcf9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe819d660 con 0x7f8fe8100550 2023-07-21T22:27:25.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.563+0000 7f8fed294700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe80ffbc0 0x7f8fe819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:25.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fed294700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe80ffbc0 0x7f8fe819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46234/0 (socket says 172.21.15.107:46234) 2023-07-21T22:27:25.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fed294700 1 -- 172.21.15.107:0/3795845630 learned_addr learned my addr 172.21.15.107:0/3795845630 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:25.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8feda95700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fe80fe9d0 0x7f8fe80681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:25.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fed294700 1 -- 172.21.15.107:0/3795845630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fe8100550 msgr2=0x7f8fe819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:25.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fee296700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fe8100550 0x7f8fe819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:25.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fed294700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fe8100550 0x7f8fe819cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:25.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fed294700 1 -- 172.21.15.107:0/3795845630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fe80fe9d0 msgr2=0x7f8fe80681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:25.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fed294700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fe80fe9d0 0x7f8fe80681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:25.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fed294700 1 -- 172.21.15.107:0/3795845630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe0009d00 con 0x7f8fe80ffbc0 2023-07-21T22:27:25.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fed294700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe80ffbc0 0x7f8fe819a550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800e9a0 tx=0x7f8fd800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:25.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fdeffd700 1 -- 172.21.15.107:0/3795845630 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8009790 con 0x7f8fe80ffbc0 2023-07-21T22:27:25.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fdeffd700 1 -- 172.21.15.107:0/3795845630 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd800cbd0 con 0x7f8fe80ffbc0 2023-07-21T22:27:25.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fefcf9700 1 -- 172.21.15.107:0/3795845630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe819d940 con 0x7f8fe80ffbc0 2023-07-21T22:27:25.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fdeffd700 1 -- 172.21.15.107:0/3795845630 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8018680 con 0x7f8fe80ffbc0 2023-07-21T22:27:25.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.564+0000 7f8fefcf9700 1 -- 172.21.15.107:0/3795845630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe819de50 con 0x7f8fe80ffbc0 2023-07-21T22:27:25.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.565+0000 7f8fefcf9700 1 -- 172.21.15.107:0/3795845630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe810a1b0 con 0x7f8fe80ffbc0 2023-07-21T22:27:25.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.566+0000 7f8fdeffd700 1 -- 172.21.15.107:0/3795845630 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8fd800cd40 con 0x7f8fe80ffbc0 2023-07-21T22:27:25.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.567+0000 7f8fdeffd700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8fd40a11a0 0x7f8fd40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:25.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.567+0000 7f8feda95700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8fd40a11a0 0x7f8fd40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:25.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.567+0000 7f8fdeffd700 1 -- 172.21.15.107:0/3795845630 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8fd8014070 con 0x7f8fe80ffbc0 2023-07-21T22:27:25.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.567+0000 7f8feda95700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8fd40a11a0 0x7f8fd40a3650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005fd0 tx=0x7f8fe4005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:25.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.570+0000 7f8fdeffd700 1 -- 172.21.15.107:0/3795845630 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8fd8096850 con 0x7f8fe80ffbc0 2023-07-21T22:27:25.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:25 smithi190 ceph-mon[98334]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:25.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:25 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:25.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:25 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:25.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:25 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:25.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:25 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:25.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:25 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:25.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:25 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:25.658 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:25 smithi132 ceph-mon[97863]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:25.658 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:25 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:25.658 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:25 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:25.659 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:25 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:25.659 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:25 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:25.659 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:25 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:25.660 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:25 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:25.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.801+0000 7f8fefcf9700 1 -- 172.21.15.107:0/3795845630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fe810a3a0 con 0x7f8fe80ffbc0 2023-07-21T22:27:25.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.801+0000 7f8fdeffd700 1 -- 172.21.15.107:0/3795845630 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5814 (secure 0 0 0) 0x7f8fd8099e70 con 0x7f8fe80ffbc0 2023-07-21T22:27:25.847 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:25.847 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-21T22:27:25.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.803+0000 7f8fefcf9700 1 -- 172.21.15.107:0/3795845630 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8fd40a11a0 msgr2=0x7f8fd40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:25.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.803+0000 7f8fefcf9700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8fd40a11a0 0x7f8fd40a3650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005fd0 tx=0x7f8fe4005e50 comp rx=0 tx=0).stop 2023-07-21T22:27:25.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.803+0000 7f8fefcf9700 1 -- 172.21.15.107:0/3795845630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe80ffbc0 msgr2=0x7f8fe819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:25.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.803+0000 7f8fefcf9700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe80ffbc0 0x7f8fe819a550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800e9a0 tx=0x7f8fd800ed60 comp rx=0 tx=0).stop 2023-07-21T22:27:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.803+0000 7f8fefcf9700 1 -- 172.21.15.107:0/3795845630 shutdown_connections 2023-07-21T22:27:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.804+0000 7f8fefcf9700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8fd40a11a0 0x7f8fd40a3650 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.804+0000 7f8fefcf9700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8fe8100550 0x7f8fe819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.804+0000 7f8fefcf9700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fe80fe9d0 0x7f8fe80681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.804+0000 7f8fefcf9700 1 --2- 172.21.15.107:0/3795845630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fe80ffbc0 0x7f8fe819a550 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.804+0000 7f8fefcf9700 1 -- 172.21.15.107:0/3795845630 >> 172.21.15.107:0/3795845630 conn(0x7f8fe80f9fd0 msgr2=0x7f8fe80fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:25.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.804+0000 7f8fefcf9700 1 -- 172.21.15.107:0/3795845630 shutdown_connections 2023-07-21T22:27:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:25.804+0000 7f8fefcf9700 1 -- 172.21.15.107:0/3795845630 wait complete. 2023-07-21T22:27:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 8 2023-07-21T22:27:26.551 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:26 smithi132 ceph-mon[97863]: Deploying daemon mds.h on smithi132 2023-07-21T22:27:26.551 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:26 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.190:6836/3774629948,v1:172.21.15.190:6837/3774629948] up:boot 2023-07-21T22:27:26.551 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:26 smithi132 ceph-mon[97863]: fsmap 7 up:standby 2023-07-21T22:27:26.551 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:26 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:27:26.552 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:26 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3795845630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:26.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:26 smithi190 ceph-mon[98334]: Deploying daemon mds.h on smithi132 2023-07-21T22:27:26.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:26 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.190:6836/3774629948,v1:172.21.15.190:6837/3774629948] up:boot 2023-07-21T22:27:26.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:26 smithi190 ceph-mon[98334]: fsmap 7 up:standby 2023-07-21T22:27:26.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:26 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:27:26.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:26 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3795845630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:26.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:26 smithi107 ceph-mon[96543]: Deploying daemon mds.h on smithi132 2023-07-21T22:27:26.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:26 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.190:6836/3774629948,v1:172.21.15.190:6837/3774629948] up:boot 2023-07-21T22:27:26.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:26 smithi107 ceph-mon[96543]: fsmap 7 up:standby 2023-07-21T22:27:26.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:26 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:27:26.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:26 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3795845630' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:26.831 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:27:26 smithi132 systemd[1]: Starting Ceph mds.h for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:27:27.080 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:27:26 smithi132 podman[122851]: 2023-07-21T22:27:27.081 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:27:27 smithi132 ceph-mds[122891]: -- 172.21.15.132:0/2480131424 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5555eaba31e0 con 0x5555eab79000 2023-07-21T22:27:27.431 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:27:27 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-h[122867]: starting mds.h at 2023-07-21T22:27:27.431 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:27:27 smithi132 bash[122851]: 121f654ce162debe3a71a6035d88b626dd78746f85a3622a3cf44a6a9b150423 2023-07-21T22:27:27.432 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:27:27 smithi132 systemd[1]: Started Ceph mds.h for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:27:27.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:27 smithi132 ceph-mon[97863]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:27.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:27 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:27.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:27 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:27.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:27 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:27.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:27 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:27.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:27 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:27.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:27 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:27.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:27 smithi190 ceph-mon[98334]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:27.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:27.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:27.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:27.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:27.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:27.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:27 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:27.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:27 smithi107 ceph-mon[96543]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:27.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:27.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:27.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:27.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:27.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:27.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:27 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:27.862 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:27:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.949+0000 7f2b03c46700 1 -- 172.21.15.107:0/3124626221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2afc0fea20 msgr2=0x7f2afc0fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.949+0000 7f2b03c46700 1 --2- 172.21.15.107:0/3124626221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2afc0fea20 0x7f2afc0fee30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009990 tx=0x7f2af8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.949+0000 7f2b03c46700 1 -- 172.21.15.107:0/3124626221 shutdown_connections 2023-07-21T22:27:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.949+0000 7f2b03c46700 1 --2- 172.21.15.107:0/3124626221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2afc1005a0 0x7f2afc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.949+0000 7f2b03c46700 1 --2- 172.21.15.107:0/3124626221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2afc0fea20 0x7f2afc0fee30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.949+0000 7f2b03c46700 1 --2- 172.21.15.107:0/3124626221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc0ffc10 0x7f2afc100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.949+0000 7f2b03c46700 1 -- 172.21.15.107:0/3124626221 >> 172.21.15.107:0/3124626221 conn(0x7f2afc0fa060 msgr2=0x7f2afc0fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.949+0000 7f2b03c46700 1 -- 172.21.15.107:0/3124626221 shutdown_connections 2023-07-21T22:27:27.996 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.949+0000 7f2b03c46700 1 -- 172.21.15.107:0/3124626221 wait complete. 2023-07-21T22:27:27.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.950+0000 7f2b03c46700 1 Processor -- start 2023-07-21T22:27:27.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.950+0000 7f2b03c46700 1 -- start start 2023-07-21T22:27:27.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.950+0000 7f2b03c46700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2afc0fea20 0x7f2afc1957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.950+0000 7f2b03c46700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2afc0ffc10 0x7f2afc195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.950+0000 7f2b03c46700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc1005a0 0x7f2afc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:27.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.950+0000 7f2b03c46700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc19d370 con 0x7f2afc0ffc10 2023-07-21T22:27:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.950+0000 7f2b03c46700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc19d4b0 con 0x7f2afc0fea20 2023-07-21T22:27:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.950+0000 7f2b03c46700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc19d640 con 0x7f2afc1005a0 2023-07-21T22:27:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2b011e1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2afc0ffc10 0x7f2afc195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2b011e1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2afc0ffc10 0x7f2afc195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46248/0 (socket says 172.21.15.107:46248) 2023-07-21T22:27:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2b011e1700 1 -- 172.21.15.107:0/816309558 learned_addr learned my addr 172.21.15.107:0/816309558 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2b019e2700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2afc0fea20 0x7f2afc1957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2b021e3700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc1005a0 0x7f2afc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2b011e1700 1 -- 172.21.15.107:0/816309558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc1005a0 msgr2=0x7f2afc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2b011e1700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc1005a0 0x7f2afc19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2b011e1700 1 -- 172.21.15.107:0/816309558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2afc0fea20 msgr2=0x7f2afc1957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2b011e1700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2afc0fea20 0x7f2afc1957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2b011e1700 1 -- 172.21.15.107:0/816309558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80095f0 con 0x7f2afc0ffc10 2023-07-21T22:27:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2b011e1700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2afc0ffc10 0x7f2afc195d30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2af000d950 tx=0x7f2af000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2aeeffd700 1 -- 172.21.15.107:0/816309558 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000f890 con 0x7f2afc0ffc10 2023-07-21T22:27:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2aeeffd700 1 -- 172.21.15.107:0/816309558 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2af0010460 con 0x7f2afc0ffc10 2023-07-21T22:27:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2b03c46700 1 -- 172.21.15.107:0/816309558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc19d920 con 0x7f2afc0ffc10 2023-07-21T22:27:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2aeeffd700 1 -- 172.21.15.107:0/816309558 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000e610 con 0x7f2afc0ffc10 2023-07-21T22:27:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.951+0000 7f2b03c46700 1 -- 172.21.15.107:0/816309558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc19de30 con 0x7f2afc0ffc10 2023-07-21T22:27:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.952+0000 7f2b03c46700 1 -- 172.21.15.107:0/816309558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc10b340 con 0x7f2afc0ffc10 2023-07-21T22:27:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.953+0000 7f2aeeffd700 1 -- 172.21.15.107:0/816309558 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2af00108d0 con 0x7f2afc0ffc10 2023-07-21T22:27:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.954+0000 7f2aeeffd700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2ae80a11a0 0x7f2ae80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.954+0000 7f2b019e2700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2ae80a11a0 0x7f2ae80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.954+0000 7f2aeeffd700 1 -- 172.21.15.107:0/816309558 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2af00cc2d0 con 0x7f2afc0ffc10 2023-07-21T22:27:28.003 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.954+0000 7f2b019e2700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2ae80a11a0 0x7f2ae80a3650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2af800b5c0 tx=0x7f2af8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:28.003 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:27.957+0000 7f2aeeffd700 1 -- 172.21.15.107:0/816309558 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2af0095e20 con 0x7f2afc0ffc10 2023-07-21T22:27:28.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.192+0000 7f2b03c46700 1 -- 172.21.15.107:0/816309558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2afc1058a0 con 0x7f2afc0ffc10 2023-07-21T22:27:28.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.192+0000 7f2aeeffd700 1 -- 172.21.15.107:0/816309558 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6574 (secure 0 0 0) 0x7f2af0099440 con 0x7f2afc0ffc10 2023-07-21T22:27:28.238 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:28.238 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T22:27:28.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.194+0000 7f2b03c46700 1 -- 172.21.15.107:0/816309558 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2ae80a11a0 msgr2=0x7f2ae80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:28.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.194+0000 7f2b03c46700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2ae80a11a0 0x7f2ae80a3650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2af800b5c0 tx=0x7f2af8005fb0 comp rx=0 tx=0).stop 2023-07-21T22:27:28.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.194+0000 7f2b03c46700 1 -- 172.21.15.107:0/816309558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2afc0ffc10 msgr2=0x7f2afc195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:28.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.194+0000 7f2b03c46700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2afc0ffc10 0x7f2afc195d30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2af000d950 tx=0x7f2af000dc60 comp rx=0 tx=0).stop 2023-07-21T22:27:28.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.194+0000 7f2b03c46700 1 -- 172.21.15.107:0/816309558 shutdown_connections 2023-07-21T22:27:28.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.194+0000 7f2b03c46700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2ae80a11a0 0x7f2ae80a3650 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:28.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.194+0000 7f2b03c46700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc1005a0 0x7f2afc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:28.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.194+0000 7f2b03c46700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2afc0fea20 0x7f2afc1957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:28.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.194+0000 7f2b03c46700 1 --2- 172.21.15.107:0/816309558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2afc0ffc10 0x7f2afc195d30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:28.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.194+0000 7f2b03c46700 1 -- 172.21.15.107:0/816309558 >> 172.21.15.107:0/816309558 conn(0x7f2afc0fa060 msgr2=0x7f2afc0fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:28.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.194+0000 7f2b03c46700 1 -- 172.21.15.107:0/816309558 shutdown_connections 2023-07-21T22:27:28.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:28.194+0000 7f2b03c46700 1 -- 172.21.15.107:0/816309558 wait complete. 2023-07-21T22:27:28.242 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 9 2023-07-21T22:27:28.482 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:28 smithi190 ceph-mon[98334]: Deploying daemon mds.i on smithi190 2023-07-21T22:27:28.482 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:28 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.132:6836/1492770170,v1:172.21.15.132:6837/1492770170] up:boot 2023-07-21T22:27:28.482 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:28 smithi190 ceph-mon[98334]: fsmap 8 up:standby 2023-07-21T22:27:28.482 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:28 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:27:28.482 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:28 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/816309558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:28.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:28 smithi132 ceph-mon[97863]: Deploying daemon mds.i on smithi190 2023-07-21T22:27:28.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:28 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.132:6836/1492770170,v1:172.21.15.132:6837/1492770170] up:boot 2023-07-21T22:27:28.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:28 smithi132 ceph-mon[97863]: fsmap 8 up:standby 2023-07-21T22:27:28.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:28 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:27:28.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:28 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/816309558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:28.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:28 smithi107 ceph-mon[96543]: Deploying daemon mds.i on smithi190 2023-07-21T22:27:28.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:28 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.132:6836/1492770170,v1:172.21.15.132:6837/1492770170] up:boot 2023-07-21T22:27:28.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:28 smithi107 ceph-mon[96543]: fsmap 8 up:standby 2023-07-21T22:27:28.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:28 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:27:28.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:28 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/816309558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:28.891 INFO:journalctl@ceph.mds.i.smithi190.stdout:Jul 21 22:27:28 smithi190 systemd[1]: Starting Ceph mds.i for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:27:29.266 INFO:journalctl@ceph.mds.i.smithi190.stdout:Jul 21 22:27:28 smithi190 podman[124522]: 2023-07-21T22:27:29.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:29 smithi190 ceph-mon[98334]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:29.640 INFO:journalctl@ceph.mds.i.smithi190.stdout:Jul 21 22:27:29 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-i[124538]: starting mds.i at 2023-07-21T22:27:29.641 INFO:journalctl@ceph.mds.i.smithi190.stdout:Jul 21 22:27:29 smithi190 bash[124522]: b470e884a143fcf3aa57ad1249ebd72ce5da5ab387c85312f721d931add7098a 2023-07-21T22:27:29.641 INFO:journalctl@ceph.mds.i.smithi190.stdout:Jul 21 22:27:29 smithi190 systemd[1]: Started Ceph mds.i for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:27:29.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:29 smithi132 ceph-mon[97863]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:29.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:29 smithi107 ceph-mon[96543]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:30.252 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:27:30.384 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.338+0000 7faeffce0700 1 -- 172.21.15.107:0/840644008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faef80ffbc0 msgr2=0x7faef8100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:30.384 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.338+0000 7faeffce0700 1 --2- 172.21.15.107:0/840644008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faef80ffbc0 0x7faef8100010 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faef4009990 tx=0x7faef4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:30.384 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.338+0000 7faeffce0700 1 -- 172.21.15.107:0/840644008 shutdown_connections 2023-07-21T22:27:30.384 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.338+0000 7faeffce0700 1 --2- 172.21.15.107:0/840644008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faef8100550 0x7faef8106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:30.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.338+0000 7faeffce0700 1 --2- 172.21.15.107:0/840644008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faef80fe9d0 0x7faef80fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:30.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.338+0000 7faeffce0700 1 --2- 172.21.15.107:0/840644008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faef80ffbc0 0x7faef8100010 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:30.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.338+0000 7faeffce0700 1 -- 172.21.15.107:0/840644008 >> 172.21.15.107:0/840644008 conn(0x7faef80f9fd0 msgr2=0x7faef80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:30.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.338+0000 7faeffce0700 1 -- 172.21.15.107:0/840644008 shutdown_connections 2023-07-21T22:27:30.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.338+0000 7faeffce0700 1 -- 172.21.15.107:0/840644008 wait complete. 2023-07-21T22:27:30.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faeffce0700 1 Processor -- start 2023-07-21T22:27:30.387 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faeffce0700 1 -- start start 2023-07-21T22:27:30.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faeffce0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faef80fe9d0 0x7faef8102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:30.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faeffce0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faef80ffbc0 0x7faef8101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:30.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faeffce0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faef8100550 0x7faef8101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:30.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faeffce0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef8101ee0 con 0x7faef80fe9d0 2023-07-21T22:27:30.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faeffce0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef8102050 con 0x7faef80ffbc0 2023-07-21T22:27:30.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faeffce0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef819f840 con 0x7faef8100550 2023-07-21T22:27:30.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faefda7c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faef80fe9d0 0x7faef8102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:30.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faefda7c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faef80fe9d0 0x7faef8102bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46260/0 (socket says 172.21.15.107:46260) 2023-07-21T22:27:30.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faefda7c700 1 -- 172.21.15.107:0/2538506622 learned_addr learned my addr 172.21.15.107:0/2538506622 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:30.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faefd27b700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faef80ffbc0 0x7faef8101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:30.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faefe27d700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faef8100550 0x7faef8101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:30.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faefda7c700 1 -- 172.21.15.107:0/2538506622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faef8100550 msgr2=0x7faef8101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:30.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faefda7c700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faef8100550 0x7faef8101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:30.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faefda7c700 1 -- 172.21.15.107:0/2538506622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faef80ffbc0 msgr2=0x7faef8101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:30.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faefda7c700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faef80ffbc0 0x7faef8101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:30.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faefda7c700 1 -- 172.21.15.107:0/2538506622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef40095f0 con 0x7faef80fe9d0 2023-07-21T22:27:30.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faefd27b700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faef80ffbc0 0x7faef8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:27:30.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.339+0000 7faefe27d700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faef8100550 0x7faef8101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:27:30.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.340+0000 7faefda7c700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faef80fe9d0 0x7faef8102bc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faee800c970 tx=0x7faee800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:30.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.340+0000 7faeeeffd700 1 -- 172.21.15.107:0/2538506622 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8004220 con 0x7faef80fe9d0 2023-07-21T22:27:30.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.340+0000 7faeeeffd700 1 -- 172.21.15.107:0/2538506622 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee8007d60 con 0x7faef80fe9d0 2023-07-21T22:27:30.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.340+0000 7faeffce0700 1 -- 172.21.15.107:0/2538506622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef819fb20 con 0x7faef80fe9d0 2023-07-21T22:27:30.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.340+0000 7faeeeffd700 1 -- 172.21.15.107:0/2538506622 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8003d00 con 0x7faef80fe9d0 2023-07-21T22:27:30.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.340+0000 7faeffce0700 1 -- 172.21.15.107:0/2538506622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef81a0030 con 0x7faef80fe9d0 2023-07-21T22:27:30.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.341+0000 7faeffce0700 1 -- 172.21.15.107:0/2538506622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef8106a00 con 0x7faef80fe9d0 2023-07-21T22:27:30.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.342+0000 7faeeeffd700 1 -- 172.21.15.107:0/2538506622 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7faee8004380 con 0x7faef80fe9d0 2023-07-21T22:27:30.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.342+0000 7faeeeffd700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faee40a11a0 0x7faee40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:30.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.342+0000 7faefd27b700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faee40a11a0 0x7faee40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:30.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.342+0000 7faeeeffd700 1 -- 172.21.15.107:0/2538506622 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faee80cc410 con 0x7faef80fe9d0 2023-07-21T22:27:30.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.343+0000 7faefd27b700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faee40a11a0 0x7faee40a3650 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faef4006010 tx=0x7faef400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:30.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.346+0000 7faeeeffd700 1 -- 172.21.15.107:0/2538506622 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7faee8096000 con 0x7faef80fe9d0 2023-07-21T22:27:30.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.586+0000 7faeffce0700 1 -- 172.21.15.107:0/2538506622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faef802c850 con 0x7faef80fe9d0 2023-07-21T22:27:30.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.587+0000 7faeeeffd700 1 -- 172.21.15.107:0/2538506622 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7336 (secure 0 0 0) 0x7faee8099620 con 0x7faef80fe9d0 2023-07-21T22:27:30.632 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:30.633 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-21T22:27:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.589+0000 7faeffce0700 1 -- 172.21.15.107:0/2538506622 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faee40a11a0 msgr2=0x7faee40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.589+0000 7faeffce0700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faee40a11a0 0x7faee40a3650 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faef4006010 tx=0x7faef400b540 comp rx=0 tx=0).stop 2023-07-21T22:27:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.589+0000 7faeffce0700 1 -- 172.21.15.107:0/2538506622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faef80fe9d0 msgr2=0x7faef8102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.589+0000 7faeffce0700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faef80fe9d0 0x7faef8102bc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faee800c970 tx=0x7faee800cd30 comp rx=0 tx=0).stop 2023-07-21T22:27:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.589+0000 7faeffce0700 1 -- 172.21.15.107:0/2538506622 shutdown_connections 2023-07-21T22:27:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.589+0000 7faeffce0700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7faee40a11a0 0x7faee40a3650 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.589+0000 7faeffce0700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faef8100550 0x7faef8101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.589+0000 7faeffce0700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faef80fe9d0 0x7faef8102bc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.589+0000 7faeffce0700 1 --2- 172.21.15.107:0/2538506622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faef80ffbc0 0x7faef8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.589+0000 7faeffce0700 1 -- 172.21.15.107:0/2538506622 >> 172.21.15.107:0/2538506622 conn(0x7faef80f9fd0 msgr2=0x7faef80fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:30.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.589+0000 7faeffce0700 1 -- 172.21.15.107:0/2538506622 shutdown_connections 2023-07-21T22:27:30.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:30.589+0000 7faeffce0700 1 -- 172.21.15.107:0/2538506622 wait complete. 2023-07-21T22:27:30.636 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 10 2023-07-21T22:27:30.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:30.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:30.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:30.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:30.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:30.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:30.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:30 smithi190 ceph-mon[98334]: Deploying daemon mds.g on smithi107 2023-07-21T22:27:30.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:30 smithi190 ceph-mon[98334]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:30.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:30 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] up:boot 2023-07-21T22:27:30.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:30 smithi190 ceph-mon[98334]: fsmap 9 up:standby 2023-07-21T22:27:30.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:30 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:27:30.645 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:30.645 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:30.645 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:30.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:30.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:30.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:30.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:30 smithi107 ceph-mon[96543]: Deploying daemon mds.g on smithi107 2023-07-21T22:27:30.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:30 smithi107 ceph-mon[96543]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:30.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:30 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] up:boot 2023-07-21T22:27:30.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:30 smithi107 ceph-mon[96543]: fsmap 9 up:standby 2023-07-21T22:27:30.647 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:30 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:27:30.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:30.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:30.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:30.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:30.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:30.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:30.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:30 smithi132 ceph-mon[97863]: Deploying daemon mds.g on smithi107 2023-07-21T22:27:30.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:30 smithi132 ceph-mon[97863]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:30.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:30 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] up:boot 2023-07-21T22:27:30.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:30 smithi132 ceph-mon[97863]: fsmap 9 up:standby 2023-07-21T22:27:30.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:30 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:27:30.938 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:27:30 smithi107 systemd[1]: Starting Ceph mds.g for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:27:31.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:31 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2538506622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:31.474 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:27:31 smithi107 podman[130823]: 2023-07-21T22:27:31.474 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:27:31 smithi107 ceph-mds[130863]: -- 172.21.15.107:0/741862079 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555aa5b691e0 con 0x555aa5b3f000 2023-07-21T22:27:31.474 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:27:31 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-g[130839]: starting mds.g at 2023-07-21T22:27:31.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:31 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2538506622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:31.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:31 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2538506622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:31.937 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:27:31 smithi107 bash[130823]: 6e9e5728cecf85d5327d0495ec448c55c5d4c595ee36cf759e53a1e26e3f1010 2023-07-21T22:27:31.938 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:27:31 smithi107 systemd[1]: Started Ceph mds.g for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:27:32.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:27:32.777 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.731+0000 7fccba995700 1 -- 172.21.15.107:0/1242521584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccb40ffc20 msgr2=0x7fccb4100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:32.777 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.731+0000 7fccba995700 1 --2- 172.21.15.107:0/1242521584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccb40ffc20 0x7fccb4100090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009990 tx=0x7fcca4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:32.777 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.731+0000 7fccba995700 1 -- 172.21.15.107:0/1242521584 shutdown_connections 2023-07-21T22:27:32.777 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.731+0000 7fccba995700 1 --2- 172.21.15.107:0/1242521584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb41005d0 0x7fccb4106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:32.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.731+0000 7fccba995700 1 --2- 172.21.15.107:0/1242521584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb40fe990 0x7fccb40feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:32.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.731+0000 7fccba995700 1 --2- 172.21.15.107:0/1242521584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccb40ffc20 0x7fccb4100090 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:32.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.731+0000 7fccba995700 1 -- 172.21.15.107:0/1242521584 >> 172.21.15.107:0/1242521584 conn(0x7fccb40f9f90 msgr2=0x7fccb40fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:32.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.731+0000 7fccba995700 1 -- 172.21.15.107:0/1242521584 shutdown_connections 2023-07-21T22:27:32.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.731+0000 7fccba995700 1 -- 172.21.15.107:0/1242521584 wait complete. 2023-07-21T22:27:32.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.732+0000 7fccba995700 1 Processor -- start 2023-07-21T22:27:32.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.732+0000 7fccba995700 1 -- start start 2023-07-21T22:27:32.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.732+0000 7fccba995700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb40fe990 0x7fccb4194630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:32.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.732+0000 7fccba995700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccb40ffc20 0x7fccb4194b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:32.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.732+0000 7fccba995700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb41005d0 0x7fccb419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:32.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.732+0000 7fccba995700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb419c1b0 con 0x7fccb40ffc20 2023-07-21T22:27:32.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.732+0000 7fccba995700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb419c2f0 con 0x7fccb40fe990 2023-07-21T22:27:32.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.732+0000 7fccba995700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb419c480 con 0x7fccb41005d0 2023-07-21T22:27:32.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.732+0000 7fccb37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccb40ffc20 0x7fccb4194b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:32.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccb40ffc20 0x7fccb4194b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46290/0 (socket says 172.21.15.107:46290) 2023-07-21T22:27:32.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb37fe700 1 -- 172.21.15.107:0/1027620777 learned_addr learned my addr 172.21.15.107:0/1027620777 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:32.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb3fff700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb40fe990 0x7fccb4194630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:32.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb8f32700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb41005d0 0x7fccb419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:32.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb37fe700 1 -- 172.21.15.107:0/1027620777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb41005d0 msgr2=0x7fccb419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:32.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb37fe700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb41005d0 0x7fccb419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:32.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb37fe700 1 -- 172.21.15.107:0/1027620777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb40fe990 msgr2=0x7fccb4194630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:32.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb37fe700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb40fe990 0x7fccb4194630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:32.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb37fe700 1 -- 172.21.15.107:0/1027620777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca40095f0 con 0x7fccb40ffc20 2023-07-21T22:27:32.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb8f32700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb41005d0 0x7fccb419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:27:32.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb37fe700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccb40ffc20 0x7fccb4194b70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcca40059d0 tx=0x7fcca40047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:32.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb17fa700 1 -- 172.21.15.107:0/1027620777 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca401d070 con 0x7fccb40ffc20 2023-07-21T22:27:32.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb17fa700 1 -- 172.21.15.107:0/1027620777 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcca400fc40 con 0x7fccb40ffc20 2023-07-21T22:27:32.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccba995700 1 -- 172.21.15.107:0/1027620777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb419c700 con 0x7fccb40ffc20 2023-07-21T22:27:32.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccb17fa700 1 -- 172.21.15.107:0/1027620777 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4017650 con 0x7fccb40ffc20 2023-07-21T22:27:32.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.733+0000 7fccba995700 1 -- 172.21.15.107:0/1027620777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb419cb20 con 0x7fccb40ffc20 2023-07-21T22:27:32.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.734+0000 7fccba995700 1 -- 172.21.15.107:0/1027620777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb410a160 con 0x7fccb40ffc20 2023-07-21T22:27:32.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.735+0000 7fccb17fa700 1 -- 172.21.15.107:0/1027620777 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fcca4022470 con 0x7fccb40ffc20 2023-07-21T22:27:32.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.736+0000 7fccb17fa700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc9c0a11a0 0x7fcc9c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:32.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.736+0000 7fccb3fff700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc9c0a11a0 0x7fcc9c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:32.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.736+0000 7fccb17fa700 1 -- 172.21.15.107:0/1027620777 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcca40cdaa0 con 0x7fccb40ffc20 2023-07-21T22:27:32.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.736+0000 7fccb3fff700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc9c0a11a0 0x7fcc9c0a3650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009c60 tx=0x7fcca8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:32.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:32.740+0000 7fccb17fa700 1 -- 172.21.15.107:0/1027620777 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcca4097720 con 0x7fccb40ffc20 2023-07-21T22:27:32.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:32.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:32.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:32.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:32.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:32.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:32.864 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:32 smithi107 ceph-mon[96543]: Deploying daemon mds.a on smithi107 2023-07-21T22:27:32.864 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:32 smithi107 ceph-mon[96543]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:32.864 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:32 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.107:6838/1956222571,v1:172.21.15.107:6839/1956222571] up:boot 2023-07-21T22:27:32.864 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:32 smithi107 ceph-mon[96543]: fsmap 10 up:standby 2023-07-21T22:27:32.864 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:32 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:27:32.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:32.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:32.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:32.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:32.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:32.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:32.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:32 smithi190 ceph-mon[98334]: Deploying daemon mds.a on smithi107 2023-07-21T22:27:32.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:32 smithi190 ceph-mon[98334]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:32.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:32 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.107:6838/1956222571,v1:172.21.15.107:6839/1956222571] up:boot 2023-07-21T22:27:32.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:32 smithi190 ceph-mon[98334]: fsmap 10 up:standby 2023-07-21T22:27:32.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:32 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:27:32.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:32.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:32.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:32.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:32.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:32.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:32.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:32 smithi132 ceph-mon[97863]: Deploying daemon mds.a on smithi107 2023-07-21T22:27:32.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:32 smithi132 ceph-mon[97863]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:32.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:32 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.107:6838/1956222571,v1:172.21.15.107:6839/1956222571] up:boot 2023-07-21T22:27:32.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:32 smithi132 ceph-mon[97863]: fsmap 10 up:standby 2023-07-21T22:27:32.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:32 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:27:33.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.004+0000 7fccba995700 1 -- 172.21.15.107:0/1027620777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccb419d160 con 0x7fccb40ffc20 2023-07-21T22:27:33.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.005+0000 7fccb17fa700 1 -- 172.21.15.107:0/1027620777 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8097 (secure 0 0 0) 0x7fcca409ad40 con 0x7fccb40ffc20 2023-07-21T22:27:33.051 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:33.051 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-21T22:27:33.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.007+0000 7fccba995700 1 -- 172.21.15.107:0/1027620777 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc9c0a11a0 msgr2=0x7fcc9c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:33.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.007+0000 7fccba995700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc9c0a11a0 0x7fcc9c0a3650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009c60 tx=0x7fcca8009260 comp rx=0 tx=0).stop 2023-07-21T22:27:33.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.007+0000 7fccba995700 1 -- 172.21.15.107:0/1027620777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccb40ffc20 msgr2=0x7fccb4194b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:33.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.007+0000 7fccba995700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccb40ffc20 0x7fccb4194b70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcca40059d0 tx=0x7fcca40047a0 comp rx=0 tx=0).stop 2023-07-21T22:27:33.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.007+0000 7fccba995700 1 -- 172.21.15.107:0/1027620777 shutdown_connections 2023-07-21T22:27:33.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.007+0000 7fccba995700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc9c0a11a0 0x7fcc9c0a3650 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:33.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.007+0000 7fccba995700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb41005d0 0x7fccb419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:33.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.007+0000 7fccba995700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccb40fe990 0x7fccb4194630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:33.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.007+0000 7fccba995700 1 --2- 172.21.15.107:0/1027620777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccb40ffc20 0x7fccb4194b70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:33.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.007+0000 7fccba995700 1 -- 172.21.15.107:0/1027620777 >> 172.21.15.107:0/1027620777 conn(0x7fccb40f9f90 msgr2=0x7fccb40fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:33.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.008+0000 7fccba995700 1 -- 172.21.15.107:0/1027620777 shutdown_connections 2023-07-21T22:27:33.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:33.008+0000 7fccba995700 1 -- 172.21.15.107:0/1027620777 wait complete. 2023-07-21T22:27:33.055 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 11 2023-07-21T22:27:33.188 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:27:33 smithi107 systemd[1]: Starting Ceph mds.a for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:27:33.856 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:27:33 smithi107 podman[131208]: 2023-07-21T22:27:33.856 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:33 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1027620777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:33.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:33 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1027620777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:33.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:33 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1027620777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:34.187 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:27:33 smithi107 ceph-mds[131248]: -- 172.21.15.107:0/2891211840 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a2f371f1e0 con 0x55a2f36f5000 2023-07-21T22:27:34.188 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:27:33 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-a[131224]: starting mds.a at 2023-07-21T22:27:34.188 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:27:33 smithi107 bash[131208]: 4e99dd7a0a2c57fd977fd1172769415e7983bf118324041b7032623bc0a4620e 2023-07-21T22:27:34.188 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:27:33 smithi107 systemd[1]: Started Ceph mds.a for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:27:35.067 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:27:35.148 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:34 smithi132 ceph-mon[97863]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:35.149 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:35.149 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:35.149 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:35.149 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:35.149 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:35.149 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:35.150 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:34 smithi132 ceph-mon[97863]: Deploying daemon mds.b on smithi132 2023-07-21T22:27:35.150 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:34 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.107:6840/1878776680,v1:172.21.15.107:6841/1878776680] up:boot 2023-07-21T22:27:35.150 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:34 smithi132 ceph-mon[97863]: fsmap 11 up:standby 2023-07-21T22:27:35.150 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:34 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:27:35.206 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.161+0000 7f6db0ecf700 1 -- 172.21.15.107:0/1843836742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dac100550 msgr2=0x7f6dac106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:35.206 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.161+0000 7f6db0ecf700 1 --2- 172.21.15.107:0/1843836742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dac100550 0x7f6dac106e10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6da400b0a0 tx=0x7f6da400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:35.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.161+0000 7f6db0ecf700 1 -- 172.21.15.107:0/1843836742 shutdown_connections 2023-07-21T22:27:35.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.161+0000 7f6db0ecf700 1 --2- 172.21.15.107:0/1843836742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dac100550 0x7f6dac106e10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:35.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.161+0000 7f6db0ecf700 1 --2- 172.21.15.107:0/1843836742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6dac0fe9d0 0x7f6dac0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:35.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.161+0000 7f6db0ecf700 1 --2- 172.21.15.107:0/1843836742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dac0ffbc0 0x7f6dac100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:35.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.161+0000 7f6db0ecf700 1 -- 172.21.15.107:0/1843836742 >> 172.21.15.107:0/1843836742 conn(0x7f6dac0f9fd0 msgr2=0x7f6dac0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:35.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.161+0000 7f6db0ecf700 1 -- 172.21.15.107:0/1843836742 shutdown_connections 2023-07-21T22:27:35.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.161+0000 7f6db0ecf700 1 -- 172.21.15.107:0/1843836742 wait complete. 2023-07-21T22:27:35.209 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.162+0000 7f6db0ecf700 1 Processor -- start 2023-07-21T22:27:35.209 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.162+0000 7f6db0ecf700 1 -- start start 2023-07-21T22:27:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.162+0000 7f6db0ecf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dac0fe9d0 0x7f6dac1947d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.162+0000 7f6db0ecf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6dac0ffbc0 0x7f6dac194d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.162+0000 7f6db0ecf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dac100550 0x7f6dac19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.162+0000 7f6db0ecf700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac19c350 con 0x7f6dac100550 2023-07-21T22:27:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.162+0000 7f6db0ecf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac19c490 con 0x7f6dac0ffbc0 2023-07-21T22:27:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.162+0000 7f6db0ecf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac19c620 con 0x7f6dac0fe9d0 2023-07-21T22:27:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.162+0000 7f6daad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dac100550 0x7f6dac19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6daad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dac100550 0x7f6dac19bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46316/0 (socket says 172.21.15.107:46316) 2023-07-21T22:27:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6daad9d700 1 -- 172.21.15.107:0/434418359 learned_addr learned my addr 172.21.15.107:0/434418359 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6da9d9b700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6dac0ffbc0 0x7f6dac194d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6daa59c700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dac0fe9d0 0x7f6dac1947d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6daad9d700 1 -- 172.21.15.107:0/434418359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dac0fe9d0 msgr2=0x7f6dac1947d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6daad9d700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dac0fe9d0 0x7f6dac1947d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6daad9d700 1 -- 172.21.15.107:0/434418359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6dac0ffbc0 msgr2=0x7f6dac194d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6daad9d700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6dac0ffbc0 0x7f6dac194d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6daad9d700 1 -- 172.21.15.107:0/434418359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da4009d00 con 0x7f6dac100550 2023-07-21T22:27:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6daad9d700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dac100550 0x7f6dac19bd80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6da400d010 tx=0x7f6da400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6d9b7fe700 1 -- 172.21.15.107:0/434418359 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da400e040 con 0x7f6dac100550 2023-07-21T22:27:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6d9b7fe700 1 -- 172.21.15.107:0/434418359 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6da400bdb0 con 0x7f6dac100550 2023-07-21T22:27:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6d9b7fe700 1 -- 172.21.15.107:0/434418359 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4005260 con 0x7f6dac100550 2023-07-21T22:27:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6db0ecf700 1 -- 172.21.15.107:0/434418359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dac19c8a0 con 0x7f6dac100550 2023-07-21T22:27:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.163+0000 7f6db0ecf700 1 -- 172.21.15.107:0/434418359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dac19cc70 con 0x7f6dac100550 2023-07-21T22:27:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.164+0000 7f6db0ecf700 1 -- 172.21.15.107:0/434418359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dac10a200 con 0x7f6dac100550 2023-07-21T22:27:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.165+0000 7f6d9b7fe700 1 -- 172.21.15.107:0/434418359 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6da401d480 con 0x7f6dac100550 2023-07-21T22:27:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.166+0000 7f6d9b7fe700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6d940a11a0 0x7f6d940a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.166+0000 7f6daa59c700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6d940a11a0 0x7f6d940a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.166+0000 7f6d9b7fe700 1 -- 172.21.15.107:0/434418359 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6da40cd550 con 0x7f6dac100550 2023-07-21T22:27:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.166+0000 7f6daa59c700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6d940a11a0 0x7f6d940a3650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0ffa20 tx=0x7f6d9c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:35.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.170+0000 7f6d9b7fe700 1 -- 172.21.15.107:0/434418359 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6da4097290 con 0x7f6dac100550 2023-07-21T22:27:35.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:34 smithi190 ceph-mon[98334]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:35.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:35.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:35.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:35.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:35.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:35.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:35.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:34 smithi190 ceph-mon[98334]: Deploying daemon mds.b on smithi132 2023-07-21T22:27:35.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:34 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.107:6840/1878776680,v1:172.21.15.107:6841/1878776680] up:boot 2023-07-21T22:27:35.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:34 smithi190 ceph-mon[98334]: fsmap 11 up:standby 2023-07-21T22:27:35.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:34 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:27:35.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:34 smithi107 ceph-mon[96543]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:35.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:35.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:35.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:35.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:27:35.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:27:35.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:35.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:34 smithi107 ceph-mon[96543]: Deploying daemon mds.b on smithi132 2023-07-21T22:27:35.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:34 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.107:6840/1878776680,v1:172.21.15.107:6841/1878776680] up:boot 2023-07-21T22:27:35.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:34 smithi107 ceph-mon[96543]: fsmap 11 up:standby 2023-07-21T22:27:35.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:34 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:27:35.482 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.402+0000 7f6db0ecf700 1 -- 172.21.15.107:0/434418359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dac105850 con 0x7f6dac100550 2023-07-21T22:27:35.482 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.402+0000 7f6d9b7fe700 1 -- 172.21.15.107:0/434418359 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8858 (secure 0 0 0) 0x7f6da409a8b0 con 0x7f6dac100550 2023-07-21T22:27:35.482 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:35.483 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-21T22:27:35.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.404+0000 7f6db0ecf700 1 -- 172.21.15.107:0/434418359 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6d940a11a0 msgr2=0x7f6d940a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:35.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.404+0000 7f6db0ecf700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6d940a11a0 0x7f6d940a3650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0ffa20 tx=0x7f6d9c009260 comp rx=0 tx=0).stop 2023-07-21T22:27:35.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.404+0000 7f6db0ecf700 1 -- 172.21.15.107:0/434418359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dac100550 msgr2=0x7f6dac19bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:35.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.404+0000 7f6db0ecf700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dac100550 0x7f6dac19bd80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6da400d010 tx=0x7f6da400bc80 comp rx=0 tx=0).stop 2023-07-21T22:27:35.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.404+0000 7f6db0ecf700 1 -- 172.21.15.107:0/434418359 shutdown_connections 2023-07-21T22:27:35.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.404+0000 7f6db0ecf700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dac100550 0x7f6dac19bd80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:35.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.404+0000 7f6db0ecf700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6d940a11a0 0x7f6d940a3650 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:35.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.404+0000 7f6db0ecf700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6dac0fe9d0 0x7f6dac1947d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:35.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.404+0000 7f6db0ecf700 1 --2- 172.21.15.107:0/434418359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6dac0ffbc0 0x7f6dac194d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:35.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.404+0000 7f6db0ecf700 1 -- 172.21.15.107:0/434418359 >> 172.21.15.107:0/434418359 conn(0x7f6dac0f9fd0 msgr2=0x7f6dac0fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:35.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.404+0000 7f6db0ecf700 1 -- 172.21.15.107:0/434418359 shutdown_connections 2023-07-21T22:27:35.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:35.404+0000 7f6db0ecf700 1 -- 172.21.15.107:0/434418359 wait complete. 2023-07-21T22:27:35.485 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 12 2023-07-21T22:27:35.754 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:27:35 smithi132 systemd[1]: Starting Ceph mds.b for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:27:36.008 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:35 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/434418359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:36.008 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:27:35 smithi132 podman[123194]: 2023-07-21T22:27:36.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:35 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/434418359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:36.431 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:27:35 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-b[123210]: starting mds.b at 2023-07-21T22:27:36.431 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:27:35 smithi132 bash[123194]: 9c013ed619892161edca809f37c9641a6ca1626a8e8f5dceb6092d5b479e9e78 2023-07-21T22:27:36.432 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:27:36 smithi132 systemd[1]: Started Ceph mds.b for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:27:36.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:35 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/434418359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:37.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:37 smithi190 ceph-mon[98334]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:37.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:37 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:37 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:37 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:37 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:37 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:37 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:37.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:37 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] up:boot 2023-07-21T22:27:37.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:37 smithi190 ceph-mon[98334]: fsmap 12 up:standby 2023-07-21T22:27:37.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:37 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:27:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:37 smithi132 ceph-mon[97863]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:37.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:37 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] up:boot 2023-07-21T22:27:37.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:37 smithi132 ceph-mon[97863]: fsmap 12 up:standby 2023-07-21T22:27:37.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:37 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:27:37.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:37 smithi107 ceph-mon[96543]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:37.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:37 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:37 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:37 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:37 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:37 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:37.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:37 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:37.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:37 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] up:boot 2023-07-21T22:27:37.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:37 smithi107 ceph-mon[96543]: fsmap 12 up:standby 2023-07-21T22:27:37.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:37 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:27:37.487 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:27:37.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.575+0000 7f60bcc56700 1 -- 172.21.15.107:0/4165208694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b8100d60 msgr2=0x7f60b81011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:37.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.575+0000 7f60bcc56700 1 --2- 172.21.15.107:0/4165208694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b8100d60 0x7f60b81011b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009990 tx=0x7f60ac009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:37.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.575+0000 7f60bcc56700 1 -- 172.21.15.107:0/4165208694 shutdown_connections 2023-07-21T22:27:37.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.575+0000 7f60bcc56700 1 --2- 172.21.15.107:0/4165208694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81016f0 0x7f60b8107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:37.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.575+0000 7f60bcc56700 1 --2- 172.21.15.107:0/4165208694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60b80fdd90 0x7f60b80fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:37.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.575+0000 7f60bcc56700 1 --2- 172.21.15.107:0/4165208694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b8100d60 0x7f60b81011b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:37.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.575+0000 7f60bcc56700 1 -- 172.21.15.107:0/4165208694 >> 172.21.15.107:0/4165208694 conn(0x7f60b80f9770 msgr2=0x7f60b80fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:37.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.575+0000 7f60bcc56700 1 -- 172.21.15.107:0/4165208694 shutdown_connections 2023-07-21T22:27:37.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.575+0000 7f60bcc56700 1 -- 172.21.15.107:0/4165208694 wait complete. 2023-07-21T22:27:37.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60bcc56700 1 Processor -- start 2023-07-21T22:27:37.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60bcc56700 1 -- start start 2023-07-21T22:27:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60bcc56700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60b80fdd90 0x7f60b81006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60bcc56700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b8100d60 0x7f60b80fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60bcc56700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81016f0 0x7f60b80ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60bcc56700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b80ff9f0 con 0x7f60b8100d60 2023-07-21T22:27:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60bcc56700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b80ffb60 con 0x7f60b81016f0 2023-07-21T22:27:37.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60bcc56700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b819f7c0 con 0x7f60b80fdd90 2023-07-21T22:27:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60b5d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b8100d60 0x7f60b80fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60b5d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b8100d60 0x7f60b80fed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46326/0 (socket says 172.21.15.107:46326) 2023-07-21T22:27:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60b5d9b700 1 -- 172.21.15.107:0/4236142555 learned_addr learned my addr 172.21.15.107:0/4236142555 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60b6d9d700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81016f0 0x7f60b80ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60b659c700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60b80fdd90 0x7f60b81006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60b5d9b700 1 -- 172.21.15.107:0/4236142555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60b80fdd90 msgr2=0x7f60b81006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:37.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60b5d9b700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60b80fdd90 0x7f60b81006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:37.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60b5d9b700 1 -- 172.21.15.107:0/4236142555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81016f0 msgr2=0x7f60b80ff250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:37.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60b5d9b700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81016f0 0x7f60b80ff250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:37.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.576+0000 7f60b5d9b700 1 -- 172.21.15.107:0/4236142555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac0095f0 con 0x7f60b8100d60 2023-07-21T22:27:37.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.577+0000 7f60b5d9b700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b8100d60 0x7f60b80fed10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f60ac000c00 tx=0x7f60ac00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:37.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.577+0000 7f60a77fe700 1 -- 172.21.15.107:0/4236142555 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac01d070 con 0x7f60b8100d60 2023-07-21T22:27:37.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.577+0000 7f60a77fe700 1 -- 172.21.15.107:0/4236142555 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60ac00fb50 con 0x7f60b8100d60 2023-07-21T22:27:37.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.577+0000 7f60bcc56700 1 -- 172.21.15.107:0/4236142555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b819fa40 con 0x7f60b8100d60 2023-07-21T22:27:37.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.577+0000 7f60a77fe700 1 -- 172.21.15.107:0/4236142555 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac017650 con 0x7f60b8100d60 2023-07-21T22:27:37.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.577+0000 7f60bcc56700 1 -- 172.21.15.107:0/4236142555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b819fef0 con 0x7f60b8100d60 2023-07-21T22:27:37.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.578+0000 7f60bcc56700 1 -- 172.21.15.107:0/4236142555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b8107090 con 0x7f60b8100d60 2023-07-21T22:27:37.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.579+0000 7f60a77fe700 1 -- 172.21.15.107:0/4236142555 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f60ac00fcc0 con 0x7f60b8100d60 2023-07-21T22:27:37.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.579+0000 7f60a77fe700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f60a00a11a0 0x7f60a00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:37.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.579+0000 7f60b659c700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f60a00a11a0 0x7f60a00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:37.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.579+0000 7f60a77fe700 1 -- 172.21.15.107:0/4236142555 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f60ac0cdb30 con 0x7f60b8100d60 2023-07-21T22:27:37.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.580+0000 7f60b659c700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f60a00a11a0 0x7f60a00a3650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f60a8005fd0 tx=0x7f60a8005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:37.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.583+0000 7f60a77fe700 1 -- 172.21.15.107:0/4236142555 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f60ac0977b0 con 0x7f60b8100d60 2023-07-21T22:27:37.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.820+0000 7f60bcc56700 1 -- 172.21.15.107:0/4236142555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60b802c850 con 0x7f60b8100d60 2023-07-21T22:27:37.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.821+0000 7f60a77fe700 1 -- 172.21.15.107:0/4236142555 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9616 (secure 0 0 0) 0x7f60ac09add0 con 0x7f60b8100d60 2023-07-21T22:27:37.866 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:37.867 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14646,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14658,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24533,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-07-21T22:27:37.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.823+0000 7f60bcc56700 1 -- 172.21.15.107:0/4236142555 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f60a00a11a0 msgr2=0x7f60a00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.823+0000 7f60bcc56700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f60a00a11a0 0x7f60a00a3650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f60a8005fd0 tx=0x7f60a8005dd0 comp rx=0 tx=0).stop 2023-07-21T22:27:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.823+0000 7f60bcc56700 1 -- 172.21.15.107:0/4236142555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b8100d60 msgr2=0x7f60b80fed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.823+0000 7f60bcc56700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b8100d60 0x7f60b80fed10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f60ac000c00 tx=0x7f60ac00f710 comp rx=0 tx=0).stop 2023-07-21T22:27:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.823+0000 7f60bcc56700 1 -- 172.21.15.107:0/4236142555 shutdown_connections 2023-07-21T22:27:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.823+0000 7f60bcc56700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81016f0 0x7f60b80ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:37.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.823+0000 7f60bcc56700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f60a00a11a0 0x7f60a00a3650 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:37.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.823+0000 7f60bcc56700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60b80fdd90 0x7f60b81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:37.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.823+0000 7f60bcc56700 1 --2- 172.21.15.107:0/4236142555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b8100d60 0x7f60b80fed10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:37.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.823+0000 7f60bcc56700 1 -- 172.21.15.107:0/4236142555 >> 172.21.15.107:0/4236142555 conn(0x7f60b80f9770 msgr2=0x7f60b81041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:37.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.823+0000 7f60bcc56700 1 -- 172.21.15.107:0/4236142555 shutdown_connections 2023-07-21T22:27:37.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.823+0000 7f60bcc56700 1 -- 172.21.15.107:0/4236142555 wait complete. 2023-07-21T22:27:37.870 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 13 2023-07-21T22:27:37.884 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:27:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.963+0000 7fd6890c5700 1 -- 172.21.15.107:0/4093074686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd684102f00 msgr2=0x7fd6841052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.963+0000 7fd6890c5700 1 --2- 172.21.15.107:0/4093074686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd684102f00 0x7fd6841052e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd678009990 tx=0x7fd678009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.963+0000 7fd6890c5700 1 -- 172.21.15.107:0/4093074686 shutdown_connections 2023-07-21T22:27:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.963+0000 7fd6890c5700 1 --2- 172.21.15.107:0/4093074686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd684105820 0x7fd684107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.963+0000 7fd6890c5700 1 --2- 172.21.15.107:0/4093074686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6841005e0 0x7fd6841029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.963+0000 7fd6890c5700 1 --2- 172.21.15.107:0/4093074686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd684102f00 0x7fd6841052e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.963+0000 7fd6890c5700 1 -- 172.21.15.107:0/4093074686 >> 172.21.15.107:0/4093074686 conn(0x7fd6840f9fe0 msgr2=0x7fd6840fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.963+0000 7fd6890c5700 1 -- 172.21.15.107:0/4093074686 shutdown_connections 2023-07-21T22:27:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.963+0000 7fd6890c5700 1 -- 172.21.15.107:0/4093074686 wait complete. 2023-07-21T22:27:38.012 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.964+0000 7fd6890c5700 1 Processor -- start 2023-07-21T22:27:38.012 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.964+0000 7fd6890c5700 1 -- start start 2023-07-21T22:27:38.012 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.964+0000 7fd6890c5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6841005e0 0x7fd6841957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:38.012 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.964+0000 7fd6890c5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd684102f00 0x7fd684195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.964+0000 7fd6890c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd684105820 0x7fd68419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.964+0000 7fd6890c5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68419d320 con 0x7fd6841005e0 2023-07-21T22:27:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.964+0000 7fd6890c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68419d460 con 0x7fd684105820 2023-07-21T22:27:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.964+0000 7fd6890c5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68419d5f0 con 0x7fd684102f00 2023-07-21T22:27:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd682d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6841005e0 0x7fd6841957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd682d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6841005e0 0x7fd6841957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46338/0 (socket says 172.21.15.107:46338) 2023-07-21T22:27:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd682d9d700 1 -- 172.21.15.107:0/3375212750 learned_addr learned my addr 172.21.15.107:0/3375212750 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:38.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd68359e700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd684105820 0x7fd68419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:38.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd68259c700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd684102f00 0x7fd684195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:38.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd682d9d700 1 -- 172.21.15.107:0/3375212750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd684102f00 msgr2=0x7fd684195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd682d9d700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd684102f00 0x7fd684195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd682d9d700 1 -- 172.21.15.107:0/3375212750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd684105820 msgr2=0x7fd68419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd682d9d700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd684105820 0x7fd68419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd682d9d700 1 -- 172.21.15.107:0/3375212750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6780095f0 con 0x7fd6841005e0 2023-07-21T22:27:38.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd682d9d700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6841005e0 0x7fd6841957a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd67400e9a0 tx=0x7fd67400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:38.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd66bfff700 1 -- 172.21.15.107:0/3375212750 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674009790 con 0x7fd6841005e0 2023-07-21T22:27:38.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd66bfff700 1 -- 172.21.15.107:0/3375212750 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd674010430 con 0x7fd6841005e0 2023-07-21T22:27:38.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd66bfff700 1 -- 172.21.15.107:0/3375212750 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674018680 con 0x7fd6841005e0 2023-07-21T22:27:38.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd6890c5700 1 -- 172.21.15.107:0/3375212750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68419d8d0 con 0x7fd6841005e0 2023-07-21T22:27:38.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.965+0000 7fd6890c5700 1 -- 172.21.15.107:0/3375212750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68419dd90 con 0x7fd6841005e0 2023-07-21T22:27:38.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.966+0000 7fd6890c5700 1 -- 172.21.15.107:0/3375212750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6840fbb50 con 0x7fd6841005e0 2023-07-21T22:27:38.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.968+0000 7fd66bfff700 1 -- 172.21.15.107:0/3375212750 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd6740105a0 con 0x7fd6841005e0 2023-07-21T22:27:38.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.968+0000 7fd66bfff700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd66c0a1220 0x7fd66c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:38.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.969+0000 7fd66bfff700 1 -- 172.21.15.107:0/3375212750 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd674014070 con 0x7fd6841005e0 2023-07-21T22:27:38.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.969+0000 7fd68259c700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd66c0a1220 0x7fd66c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:38.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.969+0000 7fd68259c700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd66c0a1220 0x7fd66c0a36d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd67800b2e0 tx=0x7fd678005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:38.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:37.973+0000 7fd66bfff700 1 -- 172.21.15.107:0/3375212750 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd6740963d0 con 0x7fd6841005e0 2023-07-21T22:27:38.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:38 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/4236142555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:38.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.177+0000 7fd6890c5700 1 -- 172.21.15.107:0/3375212750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd68402cac0 con 0x7fd6841005e0 2023-07-21T22:27:38.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.178+0000 7fd66bfff700 1 -- 172.21.15.107:0/3375212750 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18504 (secure 0 0 0) 0x7fd6740999f0 con 0x7fd6841005e0 2023-07-21T22:27:38.224 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:38.224 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":78,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:27:02.233693+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:27:38.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.180+0000 7fd6890c5700 1 -- 172.21.15.107:0/3375212750 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd66c0a1220 msgr2=0x7fd66c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.180+0000 7fd6890c5700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd66c0a1220 0x7fd66c0a36d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd67800b2e0 tx=0x7fd678005fb0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.180+0000 7fd6890c5700 1 -- 172.21.15.107:0/3375212750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6841005e0 msgr2=0x7fd6841957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.180+0000 7fd6890c5700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6841005e0 0x7fd6841957a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd67400e9a0 tx=0x7fd67400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.180+0000 7fd6890c5700 1 -- 172.21.15.107:0/3375212750 shutdown_connections 2023-07-21T22:27:38.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.180+0000 7fd6890c5700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd684105820 0x7fd68419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.180+0000 7fd6890c5700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6841005e0 0x7fd6841957a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.180+0000 7fd6890c5700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd66c0a1220 0x7fd66c0a36d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.180+0000 7fd6890c5700 1 --2- 172.21.15.107:0/3375212750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd684102f00 0x7fd684195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.180+0000 7fd6890c5700 1 -- 172.21.15.107:0/3375212750 >> 172.21.15.107:0/3375212750 conn(0x7fd6840f9fe0 msgr2=0x7fd684101240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:38.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.181+0000 7fd6890c5700 1 -- 172.21.15.107:0/3375212750 shutdown_connections 2023-07-21T22:27:38.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.181+0000 7fd6890c5700 1 -- 172.21.15.107:0/3375212750 wait complete. 2023-07-21T22:27:38.238 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:23:52.031819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:27:38.238 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:27:38.375 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.328+0000 7f432fba8700 1 -- 172.21.15.107:0/620295536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43280ffb10 msgr2=0x7f43280fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.328+0000 7f432fba8700 1 --2- 172.21.15.107:0/620295536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43280ffb10 0x7f43280fff20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4324009990 tx=0x7f4324009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 -- 172.21.15.107:0/620295536 shutdown_connections 2023-07-21T22:27:38.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 --2- 172.21.15.107:0/620295536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4328101690 0x7f4328107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 --2- 172.21.15.107:0/620295536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43280ffb10 0x7f43280fff20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 --2- 172.21.15.107:0/620295536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4328100d00 0x7f4328101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.377 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 -- 172.21.15.107:0/620295536 >> 172.21.15.107:0/620295536 conn(0x7f43280fb110 msgr2=0x7f43280fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:38.377 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 -- 172.21.15.107:0/620295536 shutdown_connections 2023-07-21T22:27:38.377 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 -- 172.21.15.107:0/620295536 wait complete. 2023-07-21T22:27:38.377 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 Processor -- start 2023-07-21T22:27:38.377 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 -- start start 2023-07-21T22:27:38.377 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43280ffb10 0x7f4328195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:38.377 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4328100d00 0x7f4328195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:38.377 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4328101690 0x7f432819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:38.378 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432819d3e0 con 0x7f43280ffb10 2023-07-21T22:27:38.378 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432819d520 con 0x7f4328101690 2023-07-21T22:27:38.378 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432fba8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432819d6b0 con 0x7f4328100d00 2023-07-21T22:27:38.378 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432d944700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43280ffb10 0x7f4328195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:38.378 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432d944700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43280ffb10 0x7f4328195860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46352/0 (socket says 172.21.15.107:46352) 2023-07-21T22:27:38.378 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.329+0000 7f432d944700 1 -- 172.21.15.107:0/3773013497 learned_addr learned my addr 172.21.15.107:0/3773013497 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:38.378 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f432e145700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4328101690 0x7f432819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:38.379 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f432d143700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4328100d00 0x7f4328195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:38.379 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f432d944700 1 -- 172.21.15.107:0/3773013497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4328100d00 msgr2=0x7f4328195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.379 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f432d944700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4328100d00 0x7f4328195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.379 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f432d944700 1 -- 172.21.15.107:0/3773013497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4328101690 msgr2=0x7f432819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.379 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f432d944700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4328101690 0x7f432819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.379 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f432d944700 1 -- 172.21.15.107:0/3773013497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43240095f0 con 0x7f43280ffb10 2023-07-21T22:27:38.379 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f432d143700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4328100d00 0x7f4328195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:27:38.379 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f432d944700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43280ffb10 0x7f4328195860 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4324004760 tx=0x7f4324009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:38.380 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f431effd700 1 -- 172.21.15.107:0/3773013497 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432401d070 con 0x7f43280ffb10 2023-07-21T22:27:38.380 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f432fba8700 1 -- 172.21.15.107:0/3773013497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432819d930 con 0x7f43280ffb10 2023-07-21T22:27:38.380 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f431effd700 1 -- 172.21.15.107:0/3773013497 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4324004cd0 con 0x7f43280ffb10 2023-07-21T22:27:38.380 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f431effd700 1 -- 172.21.15.107:0/3773013497 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324005290 con 0x7f43280ffb10 2023-07-21T22:27:38.380 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.330+0000 7f432fba8700 1 -- 172.21.15.107:0/3773013497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432819dd50 con 0x7f43280ffb10 2023-07-21T22:27:38.381 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.331+0000 7f432fba8700 1 -- 172.21.15.107:0/3773013497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4328106a40 con 0x7f43280ffb10 2023-07-21T22:27:38.381 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.332+0000 7f431effd700 1 -- 172.21.15.107:0/3773013497 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4324003b60 con 0x7f43280ffb10 2023-07-21T22:27:38.381 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.333+0000 7f431effd700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f43140a11d0 0x7f43140a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:38.381 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.333+0000 7f432d143700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f43140a11d0 0x7f43140a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:38.382 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.333+0000 7f431effd700 1 -- 172.21.15.107:0/3773013497 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f43240cd550 con 0x7f43280ffb10 2023-07-21T22:27:38.382 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.333+0000 7f432d143700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f43140a11d0 0x7f43140a3680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4318005950 tx=0x7f43180058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:38.382 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.337+0000 7f431effd700 1 -- 172.21.15.107:0/3773013497 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4324097130 con 0x7f43280ffb10 2023-07-21T22:27:38.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:38 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/4236142555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:38.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:38 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/4236142555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:38.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.536+0000 7f432fba8700 1 -- 172.21.15.107:0/3773013497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4328061120 con 0x7f43280ffb10 2023-07-21T22:27:38.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.536+0000 7f431effd700 1 -- 172.21.15.107:0/3773013497 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f432409a750 con 0x7f43280ffb10 2023-07-21T22:27:38.582 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2023-07-21T22:27:38.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.538+0000 7f432fba8700 1 -- 172.21.15.107:0/3773013497 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f43140a11d0 msgr2=0x7f43140a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.538+0000 7f432fba8700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f43140a11d0 0x7f43140a3680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4318005950 tx=0x7f43180058e0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.538+0000 7f432fba8700 1 -- 172.21.15.107:0/3773013497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43280ffb10 msgr2=0x7f4328195860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.538+0000 7f432fba8700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43280ffb10 0x7f4328195860 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4324004760 tx=0x7f4324009580 comp rx=0 tx=0).stop 2023-07-21T22:27:38.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.538+0000 7f432fba8700 1 -- 172.21.15.107:0/3773013497 shutdown_connections 2023-07-21T22:27:38.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.538+0000 7f432fba8700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4328101690 0x7f432819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.538+0000 7f432fba8700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43280ffb10 0x7f4328195860 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.538+0000 7f432fba8700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f43140a11d0 0x7f43140a3680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.538+0000 7f432fba8700 1 --2- 172.21.15.107:0/3773013497 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4328100d00 0x7f4328195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.538+0000 7f432fba8700 1 -- 172.21.15.107:0/3773013497 >> 172.21.15.107:0/3773013497 conn(0x7f43280fb110 msgr2=0x7f43280fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:38.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.538+0000 7f432fba8700 1 -- 172.21.15.107:0/3773013497 shutdown_connections 2023-07-21T22:27:38.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.538+0000 7f432fba8700 1 -- 172.21.15.107:0/3773013497 wait complete. 2023-07-21T22:27:38.597 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-21T22:27:38.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.675+0000 7f91069d3700 1 -- 172.21.15.107:0/788262393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91000fed60 msgr2=0x7f91000ff1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.675+0000 7f91069d3700 1 --2- 172.21.15.107:0/788262393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91000fed60 0x7f91000ff1b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90f4009990 tx=0x7f90f4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.675+0000 7f91069d3700 1 -- 172.21.15.107:0/788262393 shutdown_connections 2023-07-21T22:27:38.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.675+0000 7f91069d3700 1 --2- 172.21.15.107:0/788262393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91000ff6f0 0x7f9100106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.675+0000 7f91069d3700 1 --2- 172.21.15.107:0/788262393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9100100030 0x7f9100100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.675+0000 7f91069d3700 1 --2- 172.21.15.107:0/788262393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91000fed60 0x7f91000ff1b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.675+0000 7f91069d3700 1 -- 172.21.15.107:0/788262393 >> 172.21.15.107:0/788262393 conn(0x7f91000f9860 msgr2=0x7f91000fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:38.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.675+0000 7f91069d3700 1 -- 172.21.15.107:0/788262393 shutdown_connections 2023-07-21T22:27:38.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.675+0000 7f91069d3700 1 -- 172.21.15.107:0/788262393 wait complete. 2023-07-21T22:27:38.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.676+0000 7f91069d3700 1 Processor -- start 2023-07-21T22:27:38.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.676+0000 7f91069d3700 1 -- start start 2023-07-21T22:27:38.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.676+0000 7f91069d3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91000fed60 0x7f91000fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:38.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.676+0000 7f91069d3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91000ff6f0 0x7f910019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:38.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.676+0000 7f91069d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9100100030 0x7f910019cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:38.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.676+0000 7f91069d3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910019d2f0 con 0x7f91000ff6f0 2023-07-21T22:27:38.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.676+0000 7f91069d3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910019d460 con 0x7f9100100030 2023-07-21T22:27:38.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.676+0000 7f91069d3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910019d620 con 0x7f91000fed60 2023-07-21T22:27:38.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.676+0000 7f90ff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91000ff6f0 0x7f910019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:38.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.676+0000 7f90ff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91000ff6f0 0x7f910019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46368/0 (socket says 172.21.15.107:46368) 2023-07-21T22:27:38.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f90ff7fe700 1 -- 172.21.15.107:0/2462827155 learned_addr learned my addr 172.21.15.107:0/2462827155 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:38.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f90fffff700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91000fed60 0x7f91000fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:38.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f9104f70700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9100100030 0x7f910019cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:38.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f90ff7fe700 1 -- 172.21.15.107:0/2462827155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91000fed60 msgr2=0x7f91000fe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f90ff7fe700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91000fed60 0x7f91000fe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f90ff7fe700 1 -- 172.21.15.107:0/2462827155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9100100030 msgr2=0x7f910019cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f90ff7fe700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9100100030 0x7f910019cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f90ff7fe700 1 -- 172.21.15.107:0/2462827155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f40095f0 con 0x7f91000ff6f0 2023-07-21T22:27:38.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f90ff7fe700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91000ff6f0 0x7f910019a4d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f90f400b680 tx=0x7f90f40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:38.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f90fd7fa700 1 -- 172.21.15.107:0/2462827155 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f401d070 con 0x7f91000ff6f0 2023-07-21T22:27:38.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f90fd7fa700 1 -- 172.21.15.107:0/2462827155 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90f4022470 con 0x7f91000ff6f0 2023-07-21T22:27:38.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f91069d3700 1 -- 172.21.15.107:0/2462827155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910019d8a0 con 0x7f91000ff6f0 2023-07-21T22:27:38.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f90fd7fa700 1 -- 172.21.15.107:0/2462827155 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4017650 con 0x7f91000ff6f0 2023-07-21T22:27:38.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.677+0000 7f91069d3700 1 -- 172.21.15.107:0/2462827155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910019dd20 con 0x7f91000ff6f0 2023-07-21T22:27:38.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.678+0000 7f91069d3700 1 -- 172.21.15.107:0/2462827155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910010a2b0 con 0x7f91000ff6f0 2023-07-21T22:27:38.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.680+0000 7f90fd7fa700 1 -- 172.21.15.107:0/2462827155 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f90f40225e0 con 0x7f91000ff6f0 2023-07-21T22:27:38.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.680+0000 7f90fd7fa700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f90e80a1220 0x7f90e80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:38.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.680+0000 7f90fffff700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f90e80a1220 0x7f90e80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:38.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.680+0000 7f90fd7fa700 1 -- 172.21.15.107:0/2462827155 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f90f40ce010 con 0x7f91000ff6f0 2023-07-21T22:27:38.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.681+0000 7f90fffff700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f90e80a1220 0x7f90e80a36d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9100068690 tx=0x7f90f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:38.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.683+0000 7f90fd7fa700 1 -- 172.21.15.107:0/2462827155 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f90f4097c90 con 0x7f91000ff6f0 2023-07-21T22:27:38.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.884+0000 7f91069d3700 1 -- 172.21.15.107:0/2462827155 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f910019e360 con 0x7f91000ff6f0 2023-07-21T22:27:38.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.884+0000 7f90fd7fa700 1 -- 172.21.15.107:0/2462827155 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7f90f409b2b0 con 0x7f91000ff6f0 2023-07-21T22:27:38.930 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:38.930 INFO:teuthology.orchestra.run.smithi107.stdout:[ 2023-07-21T22:27:38.931 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:27:38.931 INFO:teuthology.orchestra.run.smithi107.stdout: "poolnum": 1, 2023-07-21T22:27:38.931 INFO:teuthology.orchestra.run.smithi107.stdout: "poolname": ".mgr" 2023-07-21T22:27:38.931 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:27:38.931 INFO:teuthology.orchestra.run.smithi107.stdout:] 2023-07-21T22:27:38.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.887+0000 7f91069d3700 1 -- 172.21.15.107:0/2462827155 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f90e80a1220 msgr2=0x7f90e80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.887+0000 7f91069d3700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f90e80a1220 0x7f90e80a36d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9100068690 tx=0x7f90f0009260 comp rx=0 tx=0).stop 2023-07-21T22:27:38.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.887+0000 7f91069d3700 1 -- 172.21.15.107:0/2462827155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91000ff6f0 msgr2=0x7f910019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:38.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.887+0000 7f91069d3700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91000ff6f0 0x7f910019a4d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f90f400b680 tx=0x7f90f40057b0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.887+0000 7f91069d3700 1 -- 172.21.15.107:0/2462827155 shutdown_connections 2023-07-21T22:27:38.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.887+0000 7f91069d3700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9100100030 0x7f910019cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.887+0000 7f91069d3700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f90e80a1220 0x7f90e80a36d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.887+0000 7f91069d3700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91000fed60 0x7f91000fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.887+0000 7f91069d3700 1 --2- 172.21.15.107:0/2462827155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91000ff6f0 0x7f910019a4d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:38.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.887+0000 7f91069d3700 1 -- 172.21.15.107:0/2462827155 >> 172.21.15.107:0/2462827155 conn(0x7f91000f9860 msgr2=0x7f91000fb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:38.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.887+0000 7f91069d3700 1 -- 172.21.15.107:0/2462827155 shutdown_connections 2023-07-21T22:27:38.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:38.887+0000 7f91069d3700 1 -- 172.21.15.107:0/2462827155 wait complete. 2023-07-21T22:27:38.944 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-21T22:27:38.945 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-21T22:27:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.036+0000 7f641af67700 1 -- 172.21.15.107:0/2894349183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64140feca0 msgr2=0x7f64140ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.036+0000 7f641af67700 1 --2- 172.21.15.107:0/2894349183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64140feca0 0x7f64140ff110 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6400009990 tx=0x7f6400009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.036+0000 7f641af67700 1 -- 172.21.15.107:0/2894349183 shutdown_connections 2023-07-21T22:27:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.036+0000 7f641af67700 1 --2- 172.21.15.107:0/2894349183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64140ff650 0x7f64140ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.036+0000 7f641af67700 1 --2- 172.21.15.107:0/2894349183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64140fe350 0x7f64140fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.036+0000 7f641af67700 1 --2- 172.21.15.107:0/2894349183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64140feca0 0x7f64140ff110 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.036+0000 7f641af67700 1 -- 172.21.15.107:0/2894349183 >> 172.21.15.107:0/2894349183 conn(0x7f64140fa000 msgr2=0x7f64140fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.036+0000 7f641af67700 1 -- 172.21.15.107:0/2894349183 shutdown_connections 2023-07-21T22:27:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.036+0000 7f641af67700 1 -- 172.21.15.107:0/2894349183 wait complete. 2023-07-21T22:27:39.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.036+0000 7f641af67700 1 Processor -- start 2023-07-21T22:27:39.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.036+0000 7f641af67700 1 -- start start 2023-07-21T22:27:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f641af67700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64140fe350 0x7f6414197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f641af67700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64140feca0 0x7f6414197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f641af67700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64140ff650 0x7f641419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f641af67700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641419f600 con 0x7f64140ff650 2023-07-21T22:27:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f641af67700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641419f740 con 0x7f64140feca0 2023-07-21T22:27:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f641af67700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641419f8d0 con 0x7f64140fe350 2023-07-21T22:27:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6419504700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64140ff650 0x7f641419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6419504700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64140ff650 0x7f641419f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46384/0 (socket says 172.21.15.107:46384) 2023-07-21T22:27:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6419504700 1 -- 172.21.15.107:0/1296453832 learned_addr learned my addr 172.21.15.107:0/1296453832 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6418d03700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64140fe350 0x7f6414197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6413fff700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64140feca0 0x7f6414197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6419504700 1 -- 172.21.15.107:0/1296453832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64140fe350 msgr2=0x7f6414197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6419504700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64140fe350 0x7f6414197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6419504700 1 -- 172.21.15.107:0/1296453832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64140feca0 msgr2=0x7f6414197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6419504700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64140feca0 0x7f6414197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6419504700 1 -- 172.21.15.107:0/1296453832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64000095f0 con 0x7f64140ff650 2023-07-21T22:27:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6419504700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64140ff650 0x7f641419f030 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f640c007ee0 tx=0x7f640c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6411ffb700 1 -- 172.21.15.107:0/1296453832 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c00dd20 con 0x7f64140ff650 2023-07-21T22:27:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6411ffb700 1 -- 172.21.15.107:0/1296453832 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f640c00f040 con 0x7f64140ff650 2023-07-21T22:27:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f641af67700 1 -- 172.21.15.107:0/1296453832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641419fbb0 con 0x7f64140ff650 2023-07-21T22:27:39.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.037+0000 7f6411ffb700 1 -- 172.21.15.107:0/1296453832 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c0048a0 con 0x7f64140ff650 2023-07-21T22:27:39.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.038+0000 7f641af67700 1 -- 172.21.15.107:0/1296453832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64141a00c0 con 0x7f64140ff650 2023-07-21T22:27:39.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.039+0000 7f641af67700 1 -- 172.21.15.107:0/1296453832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6414109130 con 0x7f64140ff650 2023-07-21T22:27:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.040+0000 7f6411ffb700 1 -- 172.21.15.107:0/1296453832 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f640c003a00 con 0x7f64140ff650 2023-07-21T22:27:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.040+0000 7f6411ffb700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64040a1150 0x7f64040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.040+0000 7f6411ffb700 1 -- 172.21.15.107:0/1296453832 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f640c0cbc80 con 0x7f64140ff650 2023-07-21T22:27:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.040+0000 7f6418d03700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64040a1150 0x7f64040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.041+0000 7f6418d03700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64040a1150 0x7f64040a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6408009c60 tx=0x7f6408009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:39.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.044+0000 7f6411ffb700 1 -- 172.21.15.107:0/1296453832 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f640c04e770 con 0x7f64140ff650 2023-07-21T22:27:39.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:39.243+0000 7f641af67700 1 -- 172.21.15.107:0/1296453832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f641402c850 con 0x7f64140ff650 2023-07-21T22:27:39.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:39 smithi190 ceph-mon[98334]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:39.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:39 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3375212750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:39.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:39 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3773013497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:27:39.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:39 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2462827155' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T22:27:39.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:39 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:39 smithi132 ceph-mon[97863]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:39.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:39 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3375212750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:39.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:39 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3773013497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:27:39.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:39 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2462827155' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T22:27:39.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:39 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:39 smithi107 ceph-mon[96543]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:39.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:39 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3375212750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:39.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:39 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3773013497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:27:39.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:39 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2462827155' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T22:27:39.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:39.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:39 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:40.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.060+0000 7f6411ffb700 1 -- 172.21.15.107:0/1296453832 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7f640c095870 con 0x7f64140ff650 2023-07-21T22:27:40.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.064+0000 7f641af67700 1 -- 172.21.15.107:0/1296453832 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64040a1150 msgr2=0x7f64040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:40.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.064+0000 7f641af67700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64040a1150 0x7f64040a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6408009c60 tx=0x7f6408009260 comp rx=0 tx=0).stop 2023-07-21T22:27:40.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.064+0000 7f641af67700 1 -- 172.21.15.107:0/1296453832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64140ff650 msgr2=0x7f641419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:40.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.064+0000 7f641af67700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64140ff650 0x7f641419f030 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f640c007ee0 tx=0x7f640c00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.064+0000 7f641af67700 1 -- 172.21.15.107:0/1296453832 shutdown_connections 2023-07-21T22:27:40.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.064+0000 7f641af67700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64140ff650 0x7f641419f030 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.064+0000 7f641af67700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64040a1150 0x7f64040a3600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.064+0000 7f641af67700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64140fe350 0x7f6414197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.064+0000 7f641af67700 1 --2- 172.21.15.107:0/1296453832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64140feca0 0x7f6414197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.064+0000 7f641af67700 1 -- 172.21.15.107:0/1296453832 >> 172.21.15.107:0/1296453832 conn(0x7f64140fa000 msgr2=0x7f64140fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.066+0000 7f641af67700 1 -- 172.21.15.107:0/1296453832 shutdown_connections 2023-07-21T22:27:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.066+0000 7f641af67700 1 -- 172.21.15.107:0/1296453832 wait complete. 2023-07-21T22:27:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_metadata' created 2023-07-21T22:27:40.126 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-21T22:27:40.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.211+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/3621161083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e00fea50 msgr2=0x7fe4e00fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:40.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.211+0000 7fe4e5f5d700 1 --2- 172.21.15.107:0/3621161083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e00fea50 0x7fe4e00fee60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009990 tx=0x7fe4d0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.211+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/3621161083 shutdown_connections 2023-07-21T22:27:40.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.211+0000 7fe4e5f5d700 1 --2- 172.21.15.107:0/3621161083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4e01005d0 0x7fe4e0106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.211+0000 7fe4e5f5d700 1 --2- 172.21.15.107:0/3621161083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e00fea50 0x7fe4e00fee60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.211+0000 7fe4e5f5d700 1 --2- 172.21.15.107:0/3621161083 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e00ffc40 0x7fe4e0100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.211+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/3621161083 >> 172.21.15.107:0/3621161083 conn(0x7fe4e00fa070 msgr2=0x7fe4e00fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:40.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.211+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/3621161083 shutdown_connections 2023-07-21T22:27:40.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.211+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/3621161083 wait complete. 2023-07-21T22:27:40.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.212+0000 7fe4e5f5d700 1 Processor -- start 2023-07-21T22:27:40.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.212+0000 7fe4e5f5d700 1 -- start start 2023-07-21T22:27:40.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.212+0000 7fe4e5f5d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e00fea50 0x7fe4e0195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:40.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.212+0000 7fe4e5f5d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e00ffc40 0x7fe4e0195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:40.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.212+0000 7fe4e5f5d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4e01005d0 0x7fe4e019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:40.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.212+0000 7fe4e5f5d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e019d390 con 0x7fe4e00fea50 2023-07-21T22:27:40.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.212+0000 7fe4e5f5d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e019d4d0 con 0x7fe4e01005d0 2023-07-21T22:27:40.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.212+0000 7fe4e5f5d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e019d660 con 0x7fe4e00ffc40 2023-07-21T22:27:40.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.212+0000 7fe4df7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e00fea50 0x7fe4e0195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:40.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.212+0000 7fe4df7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e00fea50 0x7fe4e0195810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46394/0 (socket says 172.21.15.107:46394) 2023-07-21T22:27:40.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.212+0000 7fe4deffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e00ffc40 0x7fe4e0195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:40.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.212+0000 7fe4df7fe700 1 -- 172.21.15.107:0/2192928627 learned_addr learned my addr 172.21.15.107:0/2192928627 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:40.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.212+0000 7fe4dffff700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4e01005d0 0x7fe4e019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:40.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.213+0000 7fe4df7fe700 1 -- 172.21.15.107:0/2192928627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e00ffc40 msgr2=0x7fe4e0195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:40.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.213+0000 7fe4df7fe700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e00ffc40 0x7fe4e0195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.213+0000 7fe4df7fe700 1 -- 172.21.15.107:0/2192928627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4e01005d0 msgr2=0x7fe4e019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:40.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.213+0000 7fe4df7fe700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4e01005d0 0x7fe4e019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:40.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.213+0000 7fe4df7fe700 1 -- 172.21.15.107:0/2192928627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d00095f0 con 0x7fe4e00fea50 2023-07-21T22:27:40.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.213+0000 7fe4deffd700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e00ffc40 0x7fe4e0195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:40.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.213+0000 7fe4df7fe700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e00fea50 0x7fe4e0195810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000b620 tx=0x7fe4d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:40.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.213+0000 7fe4dcff9700 1 -- 172.21.15.107:0/2192928627 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d001d070 con 0x7fe4e00fea50 2023-07-21T22:27:40.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.213+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/2192928627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e019d8e0 con 0x7fe4e00fea50 2023-07-21T22:27:40.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.213+0000 7fe4dcff9700 1 -- 172.21.15.107:0/2192928627 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4d0005030 con 0x7fe4e00fea50 2023-07-21T22:27:40.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.213+0000 7fe4dcff9700 1 -- 172.21.15.107:0/2192928627 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0004b80 con 0x7fe4e00fea50 2023-07-21T22:27:40.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.213+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/2192928627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e019dd10 con 0x7fe4e00fea50 2023-07-21T22:27:40.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.215+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/2192928627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e010a1b0 con 0x7fe4e00fea50 2023-07-21T22:27:40.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.215+0000 7fe4dcff9700 1 -- 172.21.15.107:0/2192928627 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe4d00051a0 con 0x7fe4e00fea50 2023-07-21T22:27:40.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.216+0000 7fe4dcff9700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe4c80a1150 0x7fe4c80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:40.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.216+0000 7fe4deffd700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe4c80a1150 0x7fe4c80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:40.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.216+0000 7fe4dcff9700 1 -- 172.21.15.107:0/2192928627 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fe4d00cd5e0 con 0x7fe4e00fea50 2023-07-21T22:27:40.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.216+0000 7fe4deffd700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe4c80a1150 0x7fe4c80a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4011490 tx=0x7fe4d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:40.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.224+0000 7fe4dcff9700 1 -- 172.21.15.107:0/2192928627 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe4d00970f0 con 0x7fe4e00fea50 2023-07-21T22:27:40.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:40 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1296453832' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T22:27:40.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:40.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:40.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:40.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:40 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:40.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:40 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1296453832' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T22:27:40.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:40.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:40.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:40.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:40 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:40.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:40 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1296453832' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T22:27:40.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:40.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:40.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:40.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:40 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:40.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:40.426+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/2192928627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe4e010a370 con 0x7fe4e00fea50 2023-07-21T22:27:41.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.061+0000 7fe4dcff9700 1 -- 172.21.15.107:0/2192928627 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe4e010a370 con 0x7fe4e00fea50 2023-07-21T22:27:41.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.068+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/2192928627 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe4c80a1150 msgr2=0x7fe4c80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:41.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.068+0000 7fe4e5f5d700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe4c80a1150 0x7fe4c80a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4011490 tx=0x7fe4d4009260 comp rx=0 tx=0).stop 2023-07-21T22:27:41.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.068+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/2192928627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e00fea50 msgr2=0x7fe4e0195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:41.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.068+0000 7fe4e5f5d700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e00fea50 0x7fe4e0195810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000b620 tx=0x7fe4d0004730 comp rx=0 tx=0).stop 2023-07-21T22:27:41.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.070+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/2192928627 shutdown_connections 2023-07-21T22:27:41.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.070+0000 7fe4e5f5d700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4e01005d0 0x7fe4e019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.070+0000 7fe4e5f5d700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e00fea50 0x7fe4e0195810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.070+0000 7fe4e5f5d700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe4c80a1150 0x7fe4c80a3600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.070+0000 7fe4e5f5d700 1 --2- 172.21.15.107:0/2192928627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4e00ffc40 0x7fe4e0195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.070+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/2192928627 >> 172.21.15.107:0/2192928627 conn(0x7fe4e00fa070 msgr2=0x7fe4e00688a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:41.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.078+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/2192928627 shutdown_connections 2023-07-21T22:27:41.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.079+0000 7fe4e5f5d700 1 -- 172.21.15.107:0/2192928627 wait complete. 2023-07-21T22:27:41.126 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data' created 2023-07-21T22:27:41.140 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-21T22:27:41.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.222+0000 7f0401fc6700 1 -- 172.21.15.107:0/3890161544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03fc0fffb0 msgr2=0x7f03fc1003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:41.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.222+0000 7f0401fc6700 1 --2- 172.21.15.107:0/3890161544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03fc0fffb0 0x7f03fc1003c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f03f400b0a0 tx=0x7f03f400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.222+0000 7f0401fc6700 1 -- 172.21.15.107:0/3890161544 shutdown_connections 2023-07-21T22:27:41.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.222+0000 7f0401fc6700 1 --2- 172.21.15.107:0/3890161544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03fc0ff5e0 0x7f03fc106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.222+0000 7f0401fc6700 1 --2- 172.21.15.107:0/3890161544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03fc0fffb0 0x7f03fc1003c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.222+0000 7f0401fc6700 1 --2- 172.21.15.107:0/3890161544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03fc0fec50 0x7f03fc0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.222+0000 7f0401fc6700 1 -- 172.21.15.107:0/3890161544 >> 172.21.15.107:0/3890161544 conn(0x7f03fc0f9770 msgr2=0x7f03fc0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:41.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.222+0000 7f0401fc6700 1 -- 172.21.15.107:0/3890161544 shutdown_connections 2023-07-21T22:27:41.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.222+0000 7f0401fc6700 1 -- 172.21.15.107:0/3890161544 wait complete. 2023-07-21T22:27:41.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.222+0000 7f0401fc6700 1 Processor -- start 2023-07-21T22:27:41.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f0401fc6700 1 -- start start 2023-07-21T22:27:41.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f0401fc6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03fc0fec50 0x7f03fc102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f0401fc6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03fc0ff5e0 0x7f03fc101040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f0401fc6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03fc0fffb0 0x7f03fc101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f0401fc6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc101d20 con 0x7f03fc0fffb0 2023-07-21T22:27:41.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f0401fc6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc101e90 con 0x7f03fc0fec50 2023-07-21T22:27:41.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f0401fc6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc19fa10 con 0x7f03fc0ff5e0 2023-07-21T22:27:41.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f03fbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03fc0fffb0 0x7f03fc101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f03fbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03fc0fffb0 0x7f03fc101580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46414/0 (socket says 172.21.15.107:46414) 2023-07-21T22:27:41.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f03fbfff700 1 -- 172.21.15.107:0/3244349794 learned_addr learned my addr 172.21.15.107:0/3244349794 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:41.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f03faffd700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03fc0ff5e0 0x7f03fc101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f03fb7fe700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03fc0fec50 0x7f03fc102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f03fbfff700 1 -- 172.21.15.107:0/3244349794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03fc0ff5e0 msgr2=0x7f03fc101040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:41.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f03fbfff700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03fc0ff5e0 0x7f03fc101040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f03fbfff700 1 -- 172.21.15.107:0/3244349794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03fc0fec50 msgr2=0x7f03fc102a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:41.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f03fbfff700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03fc0fec50 0x7f03fc102a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:41.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f03fbfff700 1 -- 172.21.15.107:0/3244349794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f4009d00 con 0x7f03fc0fffb0 2023-07-21T22:27:41.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f03fb7fe700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03fc0fec50 0x7f03fc102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:27:41.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f03fbfff700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03fc0fffb0 0x7f03fc101580 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00d980 tx=0x7f03ec00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:41.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.223+0000 7f03f8ff9700 1 -- 172.21.15.107:0/3244349794 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec009940 con 0x7f03fc0fffb0 2023-07-21T22:27:41.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.224+0000 7f03f8ff9700 1 -- 172.21.15.107:0/3244349794 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03ec004830 con 0x7f03fc0fffb0 2023-07-21T22:27:41.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.224+0000 7f0401fc6700 1 -- 172.21.15.107:0/3244349794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc19fcf0 con 0x7f03fc0fffb0 2023-07-21T22:27:41.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.224+0000 7f03f8ff9700 1 -- 172.21.15.107:0/3244349794 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0054e0 con 0x7f03fc0fffb0 2023-07-21T22:27:41.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.224+0000 7f0401fc6700 1 -- 172.21.15.107:0/3244349794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc1a01f0 con 0x7f03fc0fffb0 2023-07-21T22:27:41.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.224+0000 7f0401fc6700 1 -- 172.21.15.107:0/3244349794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03fc10a1a0 con 0x7f03fc0fffb0 2023-07-21T22:27:41.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.226+0000 7f03f8ff9700 1 -- 172.21.15.107:0/3244349794 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f03ec004ca0 con 0x7f03fc0fffb0 2023-07-21T22:27:41.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.226+0000 7f03f8ff9700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f03e80a11d0 0x7f03e80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:41.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.226+0000 7f03fb7fe700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f03e80a11d0 0x7f03e80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:41.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.227+0000 7f03f8ff9700 1 -- 172.21.15.107:0/3244349794 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f03ec0cc990 con 0x7f03fc0fffb0 2023-07-21T22:27:41.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.227+0000 7f03fb7fe700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f03e80a11d0 0x7f03e80a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f03f4007ab0 tx=0x7f03f4009780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:41.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.230+0000 7f03f8ff9700 1 -- 172.21.15.107:0/3244349794 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f03ec096310 con 0x7f03fc0fffb0 2023-07-21T22:27:41.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:41 smithi190 ceph-mon[98334]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:41.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:41 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1296453832' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T22:27:41.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:41 smithi190 ceph-mon[98334]: osdmap e79: 12 total, 12 up, 12 in 2023-07-21T22:27:41.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:41 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2192928627' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T22:27:41.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:41 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:41.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:41 smithi132 ceph-mon[97863]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:41.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:41 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1296453832' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T22:27:41.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:41 smithi132 ceph-mon[97863]: osdmap e79: 12 total, 12 up, 12 in 2023-07-21T22:27:41.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:41 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2192928627' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T22:27:41.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:41 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:41.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:41 smithi107 ceph-mon[96543]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:41.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:41 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1296453832' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T22:27:41.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:41 smithi107 ceph-mon[96543]: osdmap e79: 12 total, 12 up, 12 in 2023-07-21T22:27:41.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:41 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2192928627' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T22:27:41.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:41 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:41.522 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:41.476+0000 7f0401fc6700 1 -- 172.21.15.107:0/3244349794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f03fc1a0800 con 0x7f03fc0fffb0 2023-07-21T22:27:42.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.074+0000 7f03f8ff9700 1 -- 172.21.15.107:0/3244349794 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f03ec017070 con 0x7f03fc0fffb0 2023-07-21T22:27:42.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.078+0000 7f0401fc6700 1 -- 172.21.15.107:0/3244349794 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f03e80a11d0 msgr2=0x7f03e80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.078+0000 7f0401fc6700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f03e80a11d0 0x7f03e80a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f03f4007ab0 tx=0x7f03f4009780 comp rx=0 tx=0).stop 2023-07-21T22:27:42.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.078+0000 7f0401fc6700 1 -- 172.21.15.107:0/3244349794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03fc0fffb0 msgr2=0x7f03fc101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.078+0000 7f0401fc6700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03fc0fffb0 0x7f03fc101580 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00d980 tx=0x7f03ec00dd40 comp rx=0 tx=0).stop 2023-07-21T22:27:42.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.079+0000 7f0401fc6700 1 -- 172.21.15.107:0/3244349794 shutdown_connections 2023-07-21T22:27:42.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.079+0000 7f0401fc6700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03fc0fffb0 0x7f03fc101580 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.079+0000 7f0401fc6700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03fc0fec50 0x7f03fc102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.079+0000 7f0401fc6700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f03e80a11d0 0x7f03e80a3680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.079+0000 7f0401fc6700 1 --2- 172.21.15.107:0/3244349794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03fc0ff5e0 0x7f03fc101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.079+0000 7f0401fc6700 1 -- 172.21.15.107:0/3244349794 >> 172.21.15.107:0/3244349794 conn(0x7f03fc0f9770 msgr2=0x7f03fc0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:42.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.084+0000 7f0401fc6700 1 -- 172.21.15.107:0/3244349794 shutdown_connections 2023-07-21T22:27:42.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.084+0000 7f0401fc6700 1 -- 172.21.15.107:0/3244349794 wait complete. 2023-07-21T22:27:42.131 INFO:teuthology.orchestra.run.smithi107.stderr:new fs with metadata pool 2 and data pool 3 2023-07-21T22:27:42.150 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:27:42.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.230+0000 7f55b86c3700 1 -- 172.21.15.107:0/1638315720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b00fe5c0 msgr2=0x7f55b00fea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.230+0000 7f55b86c3700 1 --2- 172.21.15.107:0/1638315720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b00fe5c0 0x7f55b00fea60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f55a800b0a0 tx=0x7f55a800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.230+0000 7f55b86c3700 1 -- 172.21.15.107:0/1638315720 shutdown_connections 2023-07-21T22:27:42.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.230+0000 7f55b86c3700 1 --2- 172.21.15.107:0/1638315720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b00fe5c0 0x7f55b00fea60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.230+0000 7f55b86c3700 1 --2- 172.21.15.107:0/1638315720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b01000b0 0x7f55b01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.230+0000 7f55b86c3700 1 --2- 172.21.15.107:0/1638315720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55b00fdc30 0x7f55b00fe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.230+0000 7f55b86c3700 1 -- 172.21.15.107:0/1638315720 >> 172.21.15.107:0/1638315720 conn(0x7f55b00f9860 msgr2=0x7f55b00fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:42.278 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.230+0000 7f55b86c3700 1 -- 172.21.15.107:0/1638315720 shutdown_connections 2023-07-21T22:27:42.278 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.230+0000 7f55b86c3700 1 -- 172.21.15.107:0/1638315720 wait complete. 2023-07-21T22:27:42.279 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b86c3700 1 Processor -- start 2023-07-21T22:27:42.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b86c3700 1 -- start start 2023-07-21T22:27:42.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b86c3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55b00fdc30 0x7f55b0197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:42.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b86c3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b00fe5c0 0x7f55b0197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:42.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b86c3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b01000b0 0x7f55b019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:42.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b86c3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b019f600 con 0x7f55b01000b0 2023-07-21T22:27:42.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b86c3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b019f740 con 0x7f55b00fe5c0 2023-07-21T22:27:42.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b86c3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b019f8d0 con 0x7f55b00fdc30 2023-07-21T22:27:42.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b6c60700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b01000b0 0x7f55b019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:42.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b645f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55b00fdc30 0x7f55b0197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:42.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b5c5e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b00fe5c0 0x7f55b0197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:42.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b645f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55b00fdc30 0x7f55b0197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:34984/0 (socket says 172.21.15.107:34984) 2023-07-21T22:27:42.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b645f700 1 -- 172.21.15.107:0/4201717548 learned_addr learned my addr 172.21.15.107:0/4201717548 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:42.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.231+0000 7f55b6c60700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b01000b0 0x7f55b019f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46430/0 (socket says 172.21.15.107:46430) 2023-07-21T22:27:42.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.232+0000 7f55b6c60700 1 -- 172.21.15.107:0/4201717548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55b00fdc30 msgr2=0x7f55b0197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.232+0000 7f55b6c60700 1 --2- 172.21.15.107:0/4201717548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55b00fdc30 0x7f55b0197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.232+0000 7f55b6c60700 1 -- 172.21.15.107:0/4201717548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b00fe5c0 msgr2=0x7f55b0197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.232+0000 7f55b6c60700 1 --2- 172.21.15.107:0/4201717548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b00fe5c0 0x7f55b0197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.232+0000 7f55b6c60700 1 -- 172.21.15.107:0/4201717548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a8009d00 con 0x7f55b01000b0 2023-07-21T22:27:42.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.232+0000 7f55b5c5e700 1 --2- 172.21.15.107:0/4201717548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b00fe5c0 0x7f55b0197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:42.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.232+0000 7f55b6c60700 1 --2- 172.21.15.107:0/4201717548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b01000b0 0x7f55b019f030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f55a800b070 tx=0x7f55a800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:42.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.234+0000 7f55a77fe700 1 -- 172.21.15.107:0/4201717548 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a800e040 con 0x7f55b01000b0 2023-07-21T22:27:42.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.234+0000 7f55a77fe700 1 -- 172.21.15.107:0/4201717548 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55a8004c50 con 0x7f55b01000b0 2023-07-21T22:27:42.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.234+0000 7f55a77fe700 1 -- 172.21.15.107:0/4201717548 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a801dea0 con 0x7f55b01000b0 2023-07-21T22:27:42.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.234+0000 7f55b86c3700 1 -- 172.21.15.107:0/4201717548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b019fb50 con 0x7f55b01000b0 2023-07-21T22:27:42.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.234+0000 7f55b86c3700 1 -- 172.21.15.107:0/4201717548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b019ffd0 con 0x7f55b01000b0 2023-07-21T22:27:42.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.235+0000 7f55b86c3700 1 -- 172.21.15.107:0/4201717548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b0109ae0 con 0x7f55b01000b0 2023-07-21T22:27:42.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.237+0000 7f55a77fe700 1 -- 172.21.15.107:0/4201717548 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f55a801d470 con 0x7f55b01000b0 2023-07-21T22:27:42.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.237+0000 7f55a77fe700 1 --2- 172.21.15.107:0/4201717548 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f559c0a1220 0x7f559c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:42.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.237+0000 7f55b645f700 1 --2- 172.21.15.107:0/4201717548 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f559c0a1220 0x7f559c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:42.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.238+0000 7f55b645f700 1 --2- 172.21.15.107:0/4201717548 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f559c0a1220 0x7f559c0a36d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f55ac005950 tx=0x7f55ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:42.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.239+0000 7f55a77fe700 1 -- 172.21.15.107:0/4201717548 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f55a809b4e0 con 0x7f55b01000b0 2023-07-21T22:27:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.243+0000 7f55a77fe700 1 -- 172.21.15.107:0/4201717548 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f55a8097820 con 0x7f55b01000b0 2023-07-21T22:27:42.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:42 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2192928627' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T22:27:42.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:42 smithi132 ceph-mon[97863]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T22:27:42.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:42 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3244349794' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T22:27:42.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:42 smithi132 ceph-mon[97863]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-21T22:27:42.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:42 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2192928627' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T22:27:42.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:42 smithi107 ceph-mon[96543]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T22:27:42.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:42 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3244349794' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T22:27:42.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:42 smithi107 ceph-mon[96543]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-21T22:27:42.492 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.445+0000 7f55b86c3700 1 -- 172.21.15.107:0/4201717548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55b002caa0 con 0x7f55b01000b0 2023-07-21T22:27:42.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.447+0000 7f55a77fe700 1 -- 172.21.15.107:0/4201717548 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22214 (secure 0 0 0) 0x7f55a809ae40 con 0x7f55b01000b0 2023-07-21T22:27:42.494 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:42.494 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":81,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:27:41.477985+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:27:42.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.449+0000 7f55b86c3700 1 -- 172.21.15.107:0/4201717548 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f559c0a1220 msgr2=0x7f559c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.449+0000 7f55b86c3700 1 --2- 172.21.15.107:0/4201717548 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f559c0a1220 0x7f559c0a36d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f55ac005950 tx=0x7f55ac009260 comp rx=0 tx=0).stop 2023-07-21T22:27:42.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.449+0000 7f55b86c3700 1 -- 172.21.15.107:0/4201717548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b01000b0 msgr2=0x7f55b019f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.449+0000 7f55b86c3700 1 --2- 172.21.15.107:0/4201717548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b01000b0 0x7f55b019f030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f55a800b070 tx=0x7f55a800bab0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.449+0000 7f55b86c3700 1 -- 172.21.15.107:0/4201717548 shutdown_connections 2023-07-21T22:27:42.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.449+0000 7f55b86c3700 1 --2- 172.21.15.107:0/4201717548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b01000b0 0x7f55b019f030 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.449+0000 7f55b86c3700 1 --2- 172.21.15.107:0/4201717548 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f559c0a1220 0x7f559c0a36d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.449+0000 7f55b86c3700 1 --2- 172.21.15.107:0/4201717548 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55b00fdc30 0x7f55b0197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.449+0000 7f55b86c3700 1 --2- 172.21.15.107:0/4201717548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55b00fe5c0 0x7f55b0197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.449+0000 7f55b86c3700 1 -- 172.21.15.107:0/4201717548 >> 172.21.15.107:0/4201717548 conn(0x7f55b00f9860 msgr2=0x7f55b00fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:42.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.449+0000 7f55b86c3700 1 -- 172.21.15.107:0/4201717548 shutdown_connections 2023-07-21T22:27:42.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.449+0000 7f55b86c3700 1 -- 172.21.15.107:0/4201717548 wait complete. 2023-07-21T22:27:42.508 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:27:42.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.581+0000 7f6f6cff5700 1 -- 172.21.15.107:0/2753355004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f68101660 msgr2=0x7f6f68107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.581+0000 7f6f6cff5700 1 --2- 172.21.15.107:0/2753355004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f68101660 0x7f6f68107f20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000b0a0 tx=0x7f6f6000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.581+0000 7f6f6cff5700 1 -- 172.21.15.107:0/2753355004 shutdown_connections 2023-07-21T22:27:42.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.581+0000 7f6f6cff5700 1 --2- 172.21.15.107:0/2753355004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f68101660 0x7f6f68107f20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.581+0000 7f6f6cff5700 1 --2- 172.21.15.107:0/2753355004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f680ffae0 0x7f6f680ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.581+0000 7f6f6cff5700 1 --2- 172.21.15.107:0/2753355004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f68100cd0 0x7f6f68101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.581+0000 7f6f6cff5700 1 -- 172.21.15.107:0/2753355004 >> 172.21.15.107:0/2753355004 conn(0x7f6f680fb100 msgr2=0x7f6f680fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:42.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.581+0000 7f6f6cff5700 1 -- 172.21.15.107:0/2753355004 shutdown_connections 2023-07-21T22:27:42.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.581+0000 7f6f6cff5700 1 -- 172.21.15.107:0/2753355004 wait complete. 2023-07-21T22:27:42.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.582+0000 7f6f6cff5700 1 Processor -- start 2023-07-21T22:27:42.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.582+0000 7f6f6cff5700 1 -- start start 2023-07-21T22:27:42.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.582+0000 7f6f6cff5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f680ffae0 0x7f6f68197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:42.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.582+0000 7f6f6cff5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f68100cd0 0x7f6f68197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:42.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.582+0000 7f6f6cff5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f68101660 0x7f6f6819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:42.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.582+0000 7f6f6cff5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6819f620 con 0x7f6f68101660 2023-07-21T22:27:42.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.582+0000 7f6f6cff5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6819f760 con 0x7f6f68100cd0 2023-07-21T22:27:42.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.582+0000 7f6f6cff5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6819f8f0 con 0x7f6f680ffae0 2023-07-21T22:27:42.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.582+0000 7f6f66d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f68101660 0x7f6f6819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:42.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.582+0000 7f6f6659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f680ffae0 0x7f6f68197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:42.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.582+0000 7f6f66d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f68101660 0x7f6f6819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46444/0 (socket says 172.21.15.107:46444) 2023-07-21T22:27:42.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.582+0000 7f6f66d9d700 1 -- 172.21.15.107:0/1391378355 learned_addr learned my addr 172.21.15.107:0/1391378355 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:42.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.582+0000 7f6f65d9b700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f68100cd0 0x7f6f68197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:42.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.583+0000 7f6f66d9d700 1 -- 172.21.15.107:0/1391378355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f680ffae0 msgr2=0x7f6f68197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.583+0000 7f6f66d9d700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f680ffae0 0x7f6f68197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.583+0000 7f6f66d9d700 1 -- 172.21.15.107:0/1391378355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f68100cd0 msgr2=0x7f6f68197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.583+0000 7f6f66d9d700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f68100cd0 0x7f6f68197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.583+0000 7f6f66d9d700 1 -- 172.21.15.107:0/1391378355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f60009d00 con 0x7f6f68101660 2023-07-21T22:27:42.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.583+0000 7f6f6659c700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f680ffae0 0x7f6f68197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:42.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.583+0000 7f6f66d9d700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f68101660 0x7f6f6819f050 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000b780 tx=0x7f6f6000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:42.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.583+0000 7f6f5f7fe700 1 -- 172.21.15.107:0/1391378355 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6000e040 con 0x7f6f68101660 2023-07-21T22:27:42.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.583+0000 7f6f5f7fe700 1 -- 172.21.15.107:0/1391378355 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f60004c60 con 0x7f6f68101660 2023-07-21T22:27:42.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.583+0000 7f6f6cff5700 1 -- 172.21.15.107:0/1391378355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6819fb70 con 0x7f6f68101660 2023-07-21T22:27:42.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.583+0000 7f6f5f7fe700 1 -- 172.21.15.107:0/1391378355 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6001b6b0 con 0x7f6f68101660 2023-07-21T22:27:42.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.583+0000 7f6f6cff5700 1 -- 172.21.15.107:0/1391378355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6819fff0 con 0x7f6f68101660 2023-07-21T22:27:42.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.584+0000 7f6f6cff5700 1 -- 172.21.15.107:0/1391378355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6810b2c0 con 0x7f6f68101660 2023-07-21T22:27:42.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.585+0000 7f6f5f7fe700 1 -- 172.21.15.107:0/1391378355 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6f6001d480 con 0x7f6f68101660 2023-07-21T22:27:42.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.585+0000 7f6f5f7fe700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f540a11a0 0x7f6f540a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:42.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.586+0000 7f6f6659c700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f540a11a0 0x7f6f540a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:42.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.586+0000 7f6f5f7fe700 1 -- 172.21.15.107:0/1391378355 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6f600ce410 con 0x7f6f68101660 2023-07-21T22:27:42.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.586+0000 7f6f6659c700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f540a11a0 0x7f6f540a3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6f68100b30 tx=0x7f6f58009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:42.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.589+0000 7f6f5f7fe700 1 -- 172.21.15.107:0/1391378355 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6f60097d20 con 0x7f6f68101660 2023-07-21T22:27:42.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:42 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2192928627' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T22:27:42.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:42 smithi190 ceph-mon[98334]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T22:27:42.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:42 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3244349794' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T22:27:42.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:42 smithi190 ceph-mon[98334]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-21T22:27:42.824 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.778+0000 7f6f6cff5700 1 -- 172.21.15.107:0/1391378355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f6810b4b0 con 0x7f6f68101660 2023-07-21T22:27:42.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.780+0000 7f6f5f7fe700 1 -- 172.21.15.107:0/1391378355 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22214 (secure 0 0 0) 0x7f6f60021090 con 0x7f6f68101660 2023-07-21T22:27:42.826 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:42.827 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":81,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:27:41.477985+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:27:42.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.782+0000 7f6f6cff5700 1 -- 172.21.15.107:0/1391378355 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f540a11a0 msgr2=0x7f6f540a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.782+0000 7f6f6cff5700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f540a11a0 0x7f6f540a3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6f68100b30 tx=0x7f6f58009260 comp rx=0 tx=0).stop 2023-07-21T22:27:42.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.782+0000 7f6f6cff5700 1 -- 172.21.15.107:0/1391378355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f68101660 msgr2=0x7f6f6819f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.782+0000 7f6f6cff5700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f68101660 0x7f6f6819f050 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000b780 tx=0x7f6f6000bd00 comp rx=0 tx=0).stop 2023-07-21T22:27:42.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.782+0000 7f6f6cff5700 1 -- 172.21.15.107:0/1391378355 shutdown_connections 2023-07-21T22:27:42.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.782+0000 7f6f6cff5700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f68101660 0x7f6f6819f050 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.782+0000 7f6f6cff5700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f540a11a0 0x7f6f540a3650 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.782+0000 7f6f6cff5700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f680ffae0 0x7f6f68197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.782+0000 7f6f6cff5700 1 --2- 172.21.15.107:0/1391378355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f68100cd0 0x7f6f68197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.782+0000 7f6f6cff5700 1 -- 172.21.15.107:0/1391378355 >> 172.21.15.107:0/1391378355 conn(0x7f6f680fb100 msgr2=0x7f6f680fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:42.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.782+0000 7f6f6cff5700 1 -- 172.21.15.107:0/1391378355 shutdown_connections 2023-07-21T22:27:42.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.782+0000 7f6f6cff5700 1 -- 172.21.15.107:0/1391378355 wait complete. 2023-07-21T22:27:42.841 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-21T22:27:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.919+0000 7f6022790700 1 -- 172.21.15.107:0/1929271914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f601c0fea40 msgr2=0x7f601c0fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.919+0000 7f6022790700 1 --2- 172.21.15.107:0/1929271914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f601c0fea40 0x7f601c0fee50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6010009990 tx=0x7f6010009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.919+0000 7f6022790700 1 -- 172.21.15.107:0/1929271914 shutdown_connections 2023-07-21T22:27:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.919+0000 7f6022790700 1 --2- 172.21.15.107:0/1929271914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f601c1005c0 0x7f601c106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.919+0000 7f6022790700 1 --2- 172.21.15.107:0/1929271914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f601c0fea40 0x7f601c0fee50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.919+0000 7f6022790700 1 --2- 172.21.15.107:0/1929271914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f601c0ffc30 0x7f601c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.919+0000 7f6022790700 1 -- 172.21.15.107:0/1929271914 >> 172.21.15.107:0/1929271914 conn(0x7f601c0fa060 msgr2=0x7f601c0fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.919+0000 7f6022790700 1 -- 172.21.15.107:0/1929271914 shutdown_connections 2023-07-21T22:27:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.919+0000 7f6022790700 1 -- 172.21.15.107:0/1929271914 wait complete. 2023-07-21T22:27:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.919+0000 7f6022790700 1 Processor -- start 2023-07-21T22:27:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.919+0000 7f6022790700 1 -- start start 2023-07-21T22:27:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6022790700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f601c0fea40 0x7f601c195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6022790700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f601c0ffc30 0x7f601c195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6022790700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f601c1005c0 0x7f601c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6022790700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c19d3c0 con 0x7f601c1005c0 2023-07-21T22:27:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6022790700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c19d500 con 0x7f601c0fea40 2023-07-21T22:27:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6022790700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c19d690 con 0x7f601c0ffc30 2023-07-21T22:27:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6020d2d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f601c1005c0 0x7f601c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6020d2d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f601c1005c0 0x7f601c19cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46458/0 (socket says 172.21.15.107:46458) 2023-07-21T22:27:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6020d2d700 1 -- 172.21.15.107:0/1435781350 learned_addr learned my addr 172.21.15.107:0/1435781350 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f601b7fe700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f601c0ffc30 0x7f601c195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f601bfff700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f601c0fea40 0x7f601c195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6020d2d700 1 -- 172.21.15.107:0/1435781350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f601c0ffc30 msgr2=0x7f601c195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6020d2d700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f601c0ffc30 0x7f601c195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6020d2d700 1 -- 172.21.15.107:0/1435781350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f601c0fea40 msgr2=0x7f601c195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6020d2d700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f601c0fea40 0x7f601c195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6020d2d700 1 -- 172.21.15.107:0/1435781350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60100095f0 con 0x7f601c1005c0 2023-07-21T22:27:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6020d2d700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f601c1005c0 0x7f601c19cdf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f601400ada0 tx=0x7f601400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f60197fa700 1 -- 172.21.15.107:0/1435781350 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601400f070 con 0x7f601c1005c0 2023-07-21T22:27:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.920+0000 7f6022790700 1 -- 172.21.15.107:0/1435781350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601c100eb0 con 0x7f601c1005c0 2023-07-21T22:27:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.921+0000 7f60197fa700 1 -- 172.21.15.107:0/1435781350 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6014007b30 con 0x7f601c1005c0 2023-07-21T22:27:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.921+0000 7f60197fa700 1 -- 172.21.15.107:0/1435781350 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6014005280 con 0x7f601c1005c0 2023-07-21T22:27:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.921+0000 7f6022790700 1 -- 172.21.15.107:0/1435781350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601c1013c0 con 0x7f601c1005c0 2023-07-21T22:27:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.922+0000 7f6022790700 1 -- 172.21.15.107:0/1435781350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601c10a1a0 con 0x7f601c1005c0 2023-07-21T22:27:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.923+0000 7f60197fa700 1 -- 172.21.15.107:0/1435781350 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f601400a090 con 0x7f601c1005c0 2023-07-21T22:27:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.923+0000 7f60197fa700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f60040a1150 0x7f60040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.923+0000 7f601bfff700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f60040a1150 0x7f60040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.923+0000 7f60197fa700 1 -- 172.21.15.107:0/1435781350 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f60140cc250 con 0x7f601c1005c0 2023-07-21T22:27:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.924+0000 7f601bfff700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f60040a1150 0x7f60040a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f601000b5c0 tx=0x7f6010005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:42.927+0000 7f60197fa700 1 -- 172.21.15.107:0/1435781350 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6014095b80 con 0x7f601c1005c0 2023-07-21T22:27:43.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:43.161+0000 7f6022790700 1 -- 172.21.15.107:0/1435781350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f601c101c30 con 0x7f601c1005c0 2023-07-21T22:27:43.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: pgmap v248: 97 pgs: 3 creating+activating, 26 creating+peering, 56 unknown, 12 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:43.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T22:27:43.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3244349794' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T22:27:43.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-21T22:27:43.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-21T22:27:43.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:43.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:43.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/4201717548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:43.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1391378355' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:43.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:43 smithi132 ceph-mon[97863]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T22:27:43.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: pgmap v248: 97 pgs: 3 creating+activating, 26 creating+peering, 56 unknown, 12 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:43.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T22:27:43.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3244349794' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T22:27:43.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-21T22:27:43.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-21T22:27:43.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:43.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:43.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/4201717548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:43.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1391378355' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:43.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:43 smithi107 ceph-mon[96543]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T22:27:43.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: pgmap v248: 97 pgs: 3 creating+activating, 26 creating+peering, 56 unknown, 12 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:43.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T22:27:43.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3244349794' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T22:27:43.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-21T22:27:43.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-21T22:27:43.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:43.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:43.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:27:43.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/4201717548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:43.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1391378355' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:43.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:43 smithi190 ceph-mon[98334]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T22:27:44.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.105+0000 7f60197fa700 1 -- 172.21.15.107:0/1435781350 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f60140991a0 con 0x7f601c1005c0 2023-07-21T22:27:44.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.107+0000 7f6022790700 1 -- 172.21.15.107:0/1435781350 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f60040a1150 msgr2=0x7f60040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:44.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.107+0000 7f6022790700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f60040a1150 0x7f60040a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f601000b5c0 tx=0x7f6010005960 comp rx=0 tx=0).stop 2023-07-21T22:27:44.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.107+0000 7f6022790700 1 -- 172.21.15.107:0/1435781350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f601c1005c0 msgr2=0x7f601c19cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:44.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.107+0000 7f6022790700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f601c1005c0 0x7f601c19cdf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f601400ada0 tx=0x7f601400c330 comp rx=0 tx=0).stop 2023-07-21T22:27:44.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.107+0000 7f6022790700 1 -- 172.21.15.107:0/1435781350 shutdown_connections 2023-07-21T22:27:44.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.107+0000 7f6022790700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f601c1005c0 0x7f601c19cdf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:44.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.107+0000 7f6022790700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f601c0fea40 0x7f601c195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:44.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.107+0000 7f6022790700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f60040a1150 0x7f60040a3600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:44.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.107+0000 7f6022790700 1 --2- 172.21.15.107:0/1435781350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f601c0ffc30 0x7f601c195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:44.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.107+0000 7f6022790700 1 -- 172.21.15.107:0/1435781350 >> 172.21.15.107:0/1435781350 conn(0x7f601c0fa060 msgr2=0x7f601c0688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:44.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.107+0000 7f6022790700 1 -- 172.21.15.107:0/1435781350 shutdown_connections 2023-07-21T22:27:44.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.107+0000 7f6022790700 1 -- 172.21.15.107:0/1435781350 wait complete. 2023-07-21T22:27:44.168 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-07-21T22:27:44.169 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-07-21T22:27:44.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.252+0000 7f263212b700 1 -- 172.21.15.107:0/999366856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f262c0fffa0 msgr2=0x7f262c1003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:44.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.252+0000 7f263212b700 1 --2- 172.21.15.107:0/999366856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f262c0fffa0 0x7f262c1003b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f261c009990 tx=0x7f261c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:44.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.252+0000 7f263212b700 1 -- 172.21.15.107:0/999366856 shutdown_connections 2023-07-21T22:27:44.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.252+0000 7f263212b700 1 --2- 172.21.15.107:0/999366856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f262c0ff5a0 0x7f262c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:44.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.252+0000 7f263212b700 1 --2- 172.21.15.107:0/999366856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f262c0fffa0 0x7f262c1003b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:44.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.252+0000 7f263212b700 1 --2- 172.21.15.107:0/999366856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f262c0febf0 0x7f262c0ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:44.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.252+0000 7f263212b700 1 -- 172.21.15.107:0/999366856 >> 172.21.15.107:0/999366856 conn(0x7f262c0f9770 msgr2=0x7f262c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:44.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.252+0000 7f263212b700 1 -- 172.21.15.107:0/999366856 shutdown_connections 2023-07-21T22:27:44.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.252+0000 7f263212b700 1 -- 172.21.15.107:0/999366856 wait complete. 2023-07-21T22:27:44.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.252+0000 7f263212b700 1 Processor -- start 2023-07-21T22:27:44.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.252+0000 7f263212b700 1 -- start start 2023-07-21T22:27:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f263212b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f262c0febf0 0x7f262c1029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f263212b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f262c0ff5a0 0x7f262c100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f263212b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f262c0fffa0 0x7f262c101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f263212b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c101cc0 con 0x7f262c0ff5a0 2023-07-21T22:27:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f263212b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c101e30 con 0x7f262c0fffa0 2023-07-21T22:27:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f263212b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c19f850 con 0x7f262c0febf0 2023-07-21T22:27:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f262affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f262c0ff5a0 0x7f262c100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f262affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f262c0ff5a0 0x7f262c100fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46470/0 (socket says 172.21.15.107:46470) 2023-07-21T22:27:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f262affd700 1 -- 172.21.15.107:0/3619122930 learned_addr learned my addr 172.21.15.107:0/3619122930 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f262b7fe700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f262c0febf0 0x7f262c1029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f262bfff700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f262c0fffa0 0x7f262c101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f262affd700 1 -- 172.21.15.107:0/3619122930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f262c0febf0 msgr2=0x7f262c1029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f262affd700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f262c0febf0 0x7f262c1029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f262affd700 1 -- 172.21.15.107:0/3619122930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f262c0fffa0 msgr2=0x7f262c101520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f262affd700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f262c0fffa0 0x7f262c101520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f262affd700 1 -- 172.21.15.107:0/3619122930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c0095f0 con 0x7f262c0ff5a0 2023-07-21T22:27:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f262bfff700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f262c0fffa0 0x7f262c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:27:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f262affd700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f262c0ff5a0 0x7f262c100fe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f262000c970 tx=0x7f262000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f2628ff9700 1 -- 172.21.15.107:0/3619122930 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620004220 con 0x7f262c0ff5a0 2023-07-21T22:27:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f263212b700 1 -- 172.21.15.107:0/3619122930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262c19fb30 con 0x7f262c0ff5a0 2023-07-21T22:27:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f2628ff9700 1 -- 172.21.15.107:0/3619122930 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2620007d60 con 0x7f262c0ff5a0 2023-07-21T22:27:44.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f2628ff9700 1 -- 172.21.15.107:0/3619122930 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620003d00 con 0x7f262c0ff5a0 2023-07-21T22:27:44.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.253+0000 7f263212b700 1 -- 172.21.15.107:0/3619122930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262c1a0040 con 0x7f262c0ff5a0 2023-07-21T22:27:44.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.254+0000 7f263212b700 1 -- 172.21.15.107:0/3619122930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262c10a1b0 con 0x7f262c0ff5a0 2023-07-21T22:27:44.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.256+0000 7f2628ff9700 1 -- 172.21.15.107:0/3619122930 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2620004380 con 0x7f262c0ff5a0 2023-07-21T22:27:44.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.256+0000 7f2628ff9700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f26140a1150 0x7f26140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:44.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.257+0000 7f262b7fe700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f26140a1150 0x7f26140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:44.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.257+0000 7f2628ff9700 1 -- 172.21.15.107:0/3619122930 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f26200cc7e0 con 0x7f262c0ff5a0 2023-07-21T22:27:44.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.257+0000 7f262b7fe700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f26140a1150 0x7f26140a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f261c006010 tx=0x7f261c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:44.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.260+0000 7f2628ff9700 1 -- 172.21.15.107:0/3619122930 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2620095fe0 con 0x7f262c0ff5a0 2023-07-21T22:27:44.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:44 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] up:active 2023-07-21T22:27:44.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:44 smithi132 ceph-mon[97863]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:27:44.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:44 smithi132 ceph-mon[97863]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:27:44.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:44 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1435781350' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T22:27:44.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:44 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] up:active 2023-07-21T22:27:44.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:44 smithi107 ceph-mon[96543]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:27:44.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:44 smithi107 ceph-mon[96543]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:27:44.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:44 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1435781350' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T22:27:44.537 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:44.491+0000 7f263212b700 1 -- 172.21.15.107:0/3619122930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f262c1a0650 con 0x7f262c0ff5a0 2023-07-21T22:27:44.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:44 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] up:active 2023-07-21T22:27:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:44 smithi190 ceph-mon[98334]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:27:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:44 smithi190 ceph-mon[98334]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:27:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:44 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1435781350' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T22:27:45.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.106+0000 7f2628ff9700 1 -- 172.21.15.107:0/3619122930 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f2620099600 con 0x7f262c0ff5a0 2023-07-21T22:27:45.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.109+0000 7f263212b700 1 -- 172.21.15.107:0/3619122930 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f26140a1150 msgr2=0x7f26140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:45.156 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.109+0000 7f263212b700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f26140a1150 0x7f26140a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f261c006010 tx=0x7f261c00b540 comp rx=0 tx=0).stop 2023-07-21T22:27:45.156 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.109+0000 7f263212b700 1 -- 172.21.15.107:0/3619122930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f262c0ff5a0 msgr2=0x7f262c100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:45.156 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.109+0000 7f263212b700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f262c0ff5a0 0x7f262c100fe0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f262000c970 tx=0x7f262000cd30 comp rx=0 tx=0).stop 2023-07-21T22:27:45.156 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.109+0000 7f263212b700 1 -- 172.21.15.107:0/3619122930 shutdown_connections 2023-07-21T22:27:45.156 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.109+0000 7f263212b700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f262c0fffa0 0x7f262c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:45.156 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.109+0000 7f263212b700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f26140a1150 0x7f26140a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:45.156 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.109+0000 7f263212b700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f262c0febf0 0x7f262c1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:45.157 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.109+0000 7f263212b700 1 --2- 172.21.15.107:0/3619122930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f262c0ff5a0 0x7f262c100fe0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:45.157 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.109+0000 7f263212b700 1 -- 172.21.15.107:0/3619122930 >> 172.21.15.107:0/3619122930 conn(0x7f262c0f9770 msgr2=0x7f262c0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:45.157 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.109+0000 7f263212b700 1 -- 172.21.15.107:0/3619122930 shutdown_connections 2023-07-21T22:27:45.157 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.109+0000 7f263212b700 1 -- 172.21.15.107:0/3619122930 wait complete. 2023-07-21T22:27:45.172 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-21T22:27:45.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.254+0000 7f91f3472700 1 -- 172.21.15.107:0/3432908404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91ec0ff5e0 msgr2=0x7f91ec106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:45.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.254+0000 7f91f3472700 1 --2- 172.21.15.107:0/3432908404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91ec0ff5e0 0x7f91ec106e90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f91e400b0a0 tx=0x7f91e400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:45.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.254+0000 7f91f3472700 1 -- 172.21.15.107:0/3432908404 shutdown_connections 2023-07-21T22:27:45.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.254+0000 7f91f3472700 1 --2- 172.21.15.107:0/3432908404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91ec0ff5e0 0x7f91ec106e90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:45.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.254+0000 7f91f3472700 1 --2- 172.21.15.107:0/3432908404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91ec0fffe0 0x7f91ec1003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:45.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.254+0000 7f91f3472700 1 --2- 172.21.15.107:0/3432908404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91ec0fec30 0x7f91ec0ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:45.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.254+0000 7f91f3472700 1 -- 172.21.15.107:0/3432908404 >> 172.21.15.107:0/3432908404 conn(0x7f91ec0f97b0 msgr2=0x7f91ec0fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:45.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.254+0000 7f91f3472700 1 -- 172.21.15.107:0/3432908404 shutdown_connections 2023-07-21T22:27:45.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.254+0000 7f91f3472700 1 -- 172.21.15.107:0/3432908404 wait complete. 2023-07-21T22:27:45.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.254+0000 7f91f3472700 1 Processor -- start 2023-07-21T22:27:45.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f3472700 1 -- start start 2023-07-21T22:27:45.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f3472700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91ec0fec30 0x7f91ec195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:45.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f3472700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91ec0ff5e0 0x7f91ec195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:45.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f3472700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91ec0fffe0 0x7f91ec19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:45.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f3472700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec19d490 con 0x7f91ec0fffe0 2023-07-21T22:27:45.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f3472700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec19d5d0 con 0x7f91ec0fec30 2023-07-21T22:27:45.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f3472700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec19d760 con 0x7f91ec0ff5e0 2023-07-21T22:27:45.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f1a0f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91ec0fffe0 0x7f91ec19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:45.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f1a0f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91ec0fffe0 0x7f91ec19cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46486/0 (socket says 172.21.15.107:46486) 2023-07-21T22:27:45.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f1a0f700 1 -- 172.21.15.107:0/2869848744 learned_addr learned my addr 172.21.15.107:0/2869848744 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:45.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f0a0d700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91ec0ff5e0 0x7f91ec195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:45.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f120e700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91ec0fec30 0x7f91ec195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:45.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f1a0f700 1 -- 172.21.15.107:0/2869848744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91ec0ff5e0 msgr2=0x7f91ec195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:45.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f1a0f700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91ec0ff5e0 0x7f91ec195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:45.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f1a0f700 1 -- 172.21.15.107:0/2869848744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91ec0fec30 msgr2=0x7f91ec195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:45.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f1a0f700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91ec0fec30 0x7f91ec195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:45.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f1a0f700 1 -- 172.21.15.107:0/2869848744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e4009d00 con 0x7f91ec0fffe0 2023-07-21T22:27:45.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f120e700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91ec0fec30 0x7f91ec195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:27:45.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f1a0f700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91ec0fffe0 0x7f91ec19cec0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f91e40062a0 tx=0x7f91e4004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:45.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91e27fc700 1 -- 172.21.15.107:0/2869848744 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e400e040 con 0x7f91ec0fffe0 2023-07-21T22:27:45.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91e27fc700 1 -- 172.21.15.107:0/2869848744 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91e401db50 con 0x7f91ec0fffe0 2023-07-21T22:27:45.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91f3472700 1 -- 172.21.15.107:0/2869848744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ec19d9e0 con 0x7f91ec0fffe0 2023-07-21T22:27:45.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.255+0000 7f91e27fc700 1 -- 172.21.15.107:0/2869848744 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e401bc40 con 0x7f91ec0fffe0 2023-07-21T22:27:45.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.256+0000 7f91f3472700 1 -- 172.21.15.107:0/2869848744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ec19de60 con 0x7f91ec0fffe0 2023-07-21T22:27:45.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.256+0000 7f91f3472700 1 -- 172.21.15.107:0/2869848744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ec105ea0 con 0x7f91ec0fffe0 2023-07-21T22:27:45.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.257+0000 7f91e27fc700 1 -- 172.21.15.107:0/2869848744 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f91e401d690 con 0x7f91ec0fffe0 2023-07-21T22:27:45.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.258+0000 7f91e27fc700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91d80a1150 0x7f91d80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:45.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.258+0000 7f91f120e700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91d80a1150 0x7f91d80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:45.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.258+0000 7f91e27fc700 1 -- 172.21.15.107:0/2869848744 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f91e40ce3c0 con 0x7f91ec0fffe0 2023-07-21T22:27:45.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.258+0000 7f91f120e700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91d80a1150 0x7f91d80a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91e8005fd0 tx=0x7f91e8005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:45.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.262+0000 7f91e27fc700 1 -- 172.21.15.107:0/2869848744 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f91e4097cf0 con 0x7f91ec0fffe0 2023-07-21T22:27:45.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:45 smithi132 ceph-mon[97863]: pgmap v251: 97 pgs: 3 creating+activating, 30 creating+peering, 42 unknown, 22 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:45.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:45 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1435781350' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T22:27:45.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:45 smithi132 ceph-mon[97863]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:27:45.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:45 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3619122930' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-21T22:27:45.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:45 smithi107 ceph-mon[96543]: pgmap v251: 97 pgs: 3 creating+activating, 30 creating+peering, 42 unknown, 22 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:45.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:45 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1435781350' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T22:27:45.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:45 smithi107 ceph-mon[96543]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:27:45.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:45 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3619122930' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-21T22:27:45.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:45.495+0000 7f91f3472700 1 -- 172.21.15.107:0/2869848744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f91ec02c850 con 0x7f91ec0fffe0 2023-07-21T22:27:45.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:45 smithi190 ceph-mon[98334]: pgmap v251: 97 pgs: 3 creating+activating, 30 creating+peering, 42 unknown, 22 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:27:45.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:45 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1435781350' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T22:27:45.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:45 smithi190 ceph-mon[98334]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:27:45.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:45 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3619122930' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-21T22:27:46.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.008+0000 7f91e27fc700 1 -- 172.21.15.107:0/2869848744 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f91e409b310 con 0x7f91ec0fffe0 2023-07-21T22:27:46.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.011+0000 7f91f3472700 1 -- 172.21.15.107:0/2869848744 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91d80a1150 msgr2=0x7f91d80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.011+0000 7f91f3472700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91d80a1150 0x7f91d80a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91e8005fd0 tx=0x7f91e8005d40 comp rx=0 tx=0).stop 2023-07-21T22:27:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.011+0000 7f91f3472700 1 -- 172.21.15.107:0/2869848744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91ec0fffe0 msgr2=0x7f91ec19cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.011+0000 7f91f3472700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91ec0fffe0 0x7f91ec19cec0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f91e40062a0 tx=0x7f91e4004db0 comp rx=0 tx=0).stop 2023-07-21T22:27:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.011+0000 7f91f3472700 1 -- 172.21.15.107:0/2869848744 shutdown_connections 2023-07-21T22:27:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.011+0000 7f91f3472700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91ec0fffe0 0x7f91ec19cec0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.011+0000 7f91f3472700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91ec0fec30 0x7f91ec195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.011+0000 7f91f3472700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91d80a1150 0x7f91d80a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:46.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.011+0000 7f91f3472700 1 --2- 172.21.15.107:0/2869848744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91ec0ff5e0 0x7f91ec195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:46.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.011+0000 7f91f3472700 1 -- 172.21.15.107:0/2869848744 >> 172.21.15.107:0/2869848744 conn(0x7f91ec0f97b0 msgr2=0x7f91ec102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:46.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.011+0000 7f91f3472700 1 -- 172.21.15.107:0/2869848744 shutdown_connections 2023-07-21T22:27:46.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.011+0000 7f91f3472700 1 -- 172.21.15.107:0/2869848744 wait complete. 2023-07-21T22:27:46.072 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-21T22:27:46.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.152+0000 7f6f12b56700 1 -- 172.21.15.107:0/3946307026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c100d70 msgr2=0x7f6f0c1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:46.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.152+0000 7f6f12b56700 1 --2- 172.21.15.107:0/3946307026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c100d70 0x7f6f0c1011e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009990 tx=0x7f6ef8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:46.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.152+0000 7f6f12b56700 1 -- 172.21.15.107:0/3946307026 shutdown_connections 2023-07-21T22:27:46.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.152+0000 7f6f12b56700 1 --2- 172.21.15.107:0/3946307026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c101720 0x7f6f0c107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:46.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.152+0000 7f6f12b56700 1 --2- 172.21.15.107:0/3946307026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f0c0ffae0 0x7f6f0c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:46.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.152+0000 7f6f12b56700 1 --2- 172.21.15.107:0/3946307026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c100d70 0x7f6f0c1011e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:46.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.152+0000 7f6f12b56700 1 -- 172.21.15.107:0/3946307026 >> 172.21.15.107:0/3946307026 conn(0x7f6f0c0fb0e0 msgr2=0x7f6f0c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:46.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.152+0000 7f6f12b56700 1 -- 172.21.15.107:0/3946307026 shutdown_connections 2023-07-21T22:27:46.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.152+0000 7f6f12b56700 1 -- 172.21.15.107:0/3946307026 wait complete. 2023-07-21T22:27:46.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f12b56700 1 Processor -- start 2023-07-21T22:27:46.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f12b56700 1 -- start start 2023-07-21T22:27:46.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f12b56700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c0ffae0 0x7f6f0c195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:46.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f12b56700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f0c100d70 0x7f6f0c195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:46.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f12b56700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c101720 0x7f6f0c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:46.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f12b56700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c19d3b0 con 0x7f6f0c0ffae0 2023-07-21T22:27:46.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f12b56700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c19d4f0 con 0x7f6f0c101720 2023-07-21T22:27:46.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f12b56700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c19d680 con 0x7f6f0c100d70 2023-07-21T22:27:46.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f108f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c0ffae0 0x7f6f0c195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:46.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f108f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c0ffae0 0x7f6f0c195830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46496/0 (socket says 172.21.15.107:46496) 2023-07-21T22:27:46.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f108f2700 1 -- 172.21.15.107:0/1656930123 learned_addr learned my addr 172.21.15.107:0/1656930123 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:46.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f0bfff700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f0c100d70 0x7f6f0c195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:46.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f110f3700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c101720 0x7f6f0c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:46.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f108f2700 1 -- 172.21.15.107:0/1656930123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f0c100d70 msgr2=0x7f6f0c195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:46.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f108f2700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f0c100d70 0x7f6f0c195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:46.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f108f2700 1 -- 172.21.15.107:0/1656930123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c101720 msgr2=0x7f6f0c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:46.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f108f2700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c101720 0x7f6f0c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:46.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f108f2700 1 -- 172.21.15.107:0/1656930123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef80095f0 con 0x7f6f0c0ffae0 2023-07-21T22:27:46.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f108f2700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c0ffae0 0x7f6f0c195830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000d950 tx=0x7f6f0000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:46.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f09ffb700 1 -- 172.21.15.107:0/1656930123 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0000f890 con 0x7f6f0c0ffae0 2023-07-21T22:27:46.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f09ffb700 1 -- 172.21.15.107:0/1656930123 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f00010460 con 0x7f6f0c0ffae0 2023-07-21T22:27:46.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.153+0000 7f6f12b56700 1 -- 172.21.15.107:0/1656930123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c19d960 con 0x7f6f0c0ffae0 2023-07-21T22:27:46.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.154+0000 7f6f09ffb700 1 -- 172.21.15.107:0/1656930123 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0000e610 con 0x7f6f0c0ffae0 2023-07-21T22:27:46.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.154+0000 7f6f12b56700 1 -- 172.21.15.107:0/1656930123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0c19de70 con 0x7f6f0c0ffae0 2023-07-21T22:27:46.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.155+0000 7f6f12b56700 1 -- 172.21.15.107:0/1656930123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0c106a60 con 0x7f6f0c0ffae0 2023-07-21T22:27:46.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.156+0000 7f6f09ffb700 1 -- 172.21.15.107:0/1656930123 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6f000108d0 con 0x7f6f0c0ffae0 2023-07-21T22:27:46.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.156+0000 7f6f09ffb700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6efc0a1150 0x7f6efc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:46.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.156+0000 7f6f0bfff700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6efc0a1150 0x7f6efc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:46.206 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.156+0000 7f6f09ffb700 1 -- 172.21.15.107:0/1656930123 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6f000cc5a0 con 0x7f6f0c0ffae0 2023-07-21T22:27:46.206 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.157+0000 7f6f0bfff700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6efc0a1150 0x7f6efc0a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b2e0 tx=0x7f6ef8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:46.206 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.161+0000 7f6f09ffb700 1 -- 172.21.15.107:0/1656930123 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6f00095e40 con 0x7f6f0c0ffae0 2023-07-21T22:27:46.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-mon[97863]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T22:27:46.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3619122930' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-21T22:27:46.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-mon[97863]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T22:27:46.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-mon[97863]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:27:46.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-mon[97863]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-21T22:27:46.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-mon[97863]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-21T22:27:46.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2869848744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T22:27:46.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] up:active 2023-07-21T22:27:46.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2869848744' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T22:27:46.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-mon[97863]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T22:27:46.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-mon[97863]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-07-21T22:27:46.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-mon[97863]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-07-21T22:27:46.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-mon[97863]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-07-21T22:27:46.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:46 smithi107 ceph-mon[96543]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T22:27:46.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:46 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3619122930' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-21T22:27:46.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:46 smithi107 ceph-mon[96543]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T22:27:46.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:46 smithi107 ceph-mon[96543]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:27:46.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:46 smithi107 ceph-mon[96543]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-21T22:27:46.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:46 smithi107 ceph-mon[96543]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-21T22:27:46.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:46 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2869848744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T22:27:46.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:46 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] up:active 2023-07-21T22:27:46.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:46 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2869848744' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T22:27:46.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:46 smithi107 ceph-mon[96543]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T22:27:46.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:46 smithi107 ceph-mon[96543]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-07-21T22:27:46.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:46 smithi107 ceph-mon[96543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-07-21T22:27:46.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:46 smithi107 ceph-mon[96543]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-07-21T22:27:46.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:46.394+0000 7f6f12b56700 1 -- 172.21.15.107:0/1656930123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f6f0c02cac0 con 0x7f6f0c0ffae0 2023-07-21T22:27:46.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:46 smithi190 ceph-mon[98334]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T22:27:46.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:46 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3619122930' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-21T22:27:46.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:46 smithi190 ceph-mon[98334]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T22:27:46.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:46 smithi190 ceph-mon[98334]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:27:46.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:46 smithi190 ceph-mon[98334]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-21T22:27:46.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:46 smithi190 ceph-mon[98334]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-21T22:27:46.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:46 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2869848744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T22:27:46.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:46 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] up:active 2023-07-21T22:27:46.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:46 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2869848744' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T22:27:46.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:46 smithi190 ceph-mon[98334]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T22:27:46.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:46 smithi190 ceph-mon[98334]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-07-21T22:27:46.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:46 smithi190 ceph-mon[98334]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2023-07-21T22:27:46.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:46 smithi190 ceph-mon[98334]: daemon mds.f is now active in filesystem cephfs as rank 2 2023-07-21T22:27:47.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.016+0000 7f6f09ffb700 1 -- 172.21.15.107:0/1656930123 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f6f00099460 con 0x7f6f0c0ffae0 2023-07-21T22:27:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.019+0000 7f6f12b56700 1 -- 172.21.15.107:0/1656930123 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6efc0a1150 msgr2=0x7f6efc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.019+0000 7f6f12b56700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6efc0a1150 0x7f6efc0a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800b2e0 tx=0x7f6ef8005fb0 comp rx=0 tx=0).stop 2023-07-21T22:27:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.019+0000 7f6f12b56700 1 -- 172.21.15.107:0/1656930123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c0ffae0 msgr2=0x7f6f0c195830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.019+0000 7f6f12b56700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c0ffae0 0x7f6f0c195830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000d950 tx=0x7f6f0000dc60 comp rx=0 tx=0).stop 2023-07-21T22:27:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.019+0000 7f6f12b56700 1 -- 172.21.15.107:0/1656930123 shutdown_connections 2023-07-21T22:27:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.019+0000 7f6f12b56700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f0c101720 0x7f6f0c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:47.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.019+0000 7f6f12b56700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f0c0ffae0 0x7f6f0c195830 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:47.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.019+0000 7f6f12b56700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6efc0a1150 0x7f6efc0a3600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:47.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.019+0000 7f6f12b56700 1 --2- 172.21.15.107:0/1656930123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f0c100d70 0x7f6f0c195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:47.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.019+0000 7f6f12b56700 1 -- 172.21.15.107:0/1656930123 >> 172.21.15.107:0/1656930123 conn(0x7f6f0c0fb0e0 msgr2=0x7f6f0c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:47.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.019+0000 7f6f12b56700 1 -- 172.21.15.107:0/1656930123 shutdown_connections 2023-07-21T22:27:47.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.019+0000 7f6f12b56700 1 -- 172.21.15.107:0/1656930123 wait complete. 2023-07-21T22:27:47.083 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-07-21T22:27:47.083 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-07-21T22:27:47.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.167+0000 7f0430994700 1 -- 172.21.15.107:0/534232562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f042c100560 msgr2=0x7f042c106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:47.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.167+0000 7f0430994700 1 --2- 172.21.15.107:0/534232562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f042c100560 0x7f042c106e20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f041c009990 tx=0x7f041c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:47.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.167+0000 7f0430994700 1 -- 172.21.15.107:0/534232562 shutdown_connections 2023-07-21T22:27:47.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.167+0000 7f0430994700 1 --2- 172.21.15.107:0/534232562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f042c100560 0x7f042c106e20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:47.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.167+0000 7f0430994700 1 --2- 172.21.15.107:0/534232562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f042c0fe9e0 0x7f042c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:47.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.167+0000 7f0430994700 1 --2- 172.21.15.107:0/534232562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f042c0ffbd0 0x7f042c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:47.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.167+0000 7f0430994700 1 -- 172.21.15.107:0/534232562 >> 172.21.15.107:0/534232562 conn(0x7f042c0fa000 msgr2=0x7f042c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:47.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.167+0000 7f0430994700 1 -- 172.21.15.107:0/534232562 shutdown_connections 2023-07-21T22:27:47.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.167+0000 7f0430994700 1 -- 172.21.15.107:0/534232562 wait complete. 2023-07-21T22:27:47.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.167+0000 7f0430994700 1 Processor -- start 2023-07-21T22:27:47.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.167+0000 7f0430994700 1 -- start start 2023-07-21T22:27:47.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f0430994700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f042c0fe9e0 0x7f042c194680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:47.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f0430994700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f042c0ffbd0 0x7f042c194bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:47.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f0430994700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f042c100560 0x7f042c19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:47.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f0430994700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c19c200 con 0x7f042c100560 2023-07-21T22:27:47.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f0430994700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c19c340 con 0x7f042c0ffbd0 2023-07-21T22:27:47.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f0430994700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c19c4d0 con 0x7f042c0fe9e0 2023-07-21T22:27:47.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f042ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f042c100560 0x7f042c19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:47.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f042ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f042c100560 0x7f042c19bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46514/0 (socket says 172.21.15.107:46514) 2023-07-21T22:27:47.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f042ad9d700 1 -- 172.21.15.107:0/722929259 learned_addr learned my addr 172.21.15.107:0/722929259 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:47.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f042a59c700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f042c0fe9e0 0x7f042c194680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:47.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f0429d9b700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f042c0ffbd0 0x7f042c194bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:47.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f042ad9d700 1 -- 172.21.15.107:0/722929259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f042c0fe9e0 msgr2=0x7f042c194680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:47.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f042ad9d700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f042c0fe9e0 0x7f042c194680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:47.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f042ad9d700 1 -- 172.21.15.107:0/722929259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f042c0ffbd0 msgr2=0x7f042c194bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:47.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f042ad9d700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f042c0ffbd0 0x7f042c194bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:47.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f042ad9d700 1 -- 172.21.15.107:0/722929259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c0095f0 con 0x7f042c100560 2023-07-21T22:27:47.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f042ad9d700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f042c100560 0x7f042c19bc30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f041c004760 tx=0x7f041c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:47.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f04237fe700 1 -- 172.21.15.107:0/722929259 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c01d070 con 0x7f042c100560 2023-07-21T22:27:47.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f04237fe700 1 -- 172.21.15.107:0/722929259 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f041c004cf0 con 0x7f042c100560 2023-07-21T22:27:47.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f04237fe700 1 -- 172.21.15.107:0/722929259 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c003d00 con 0x7f042c100560 2023-07-21T22:27:47.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f0430994700 1 -- 172.21.15.107:0/722929259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042c19c750 con 0x7f042c100560 2023-07-21T22:27:47.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.168+0000 7f0430994700 1 -- 172.21.15.107:0/722929259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042c19cac0 con 0x7f042c100560 2023-07-21T22:27:47.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.169+0000 7f0430994700 1 -- 172.21.15.107:0/722929259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042c105f00 con 0x7f042c100560 2023-07-21T22:27:47.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.170+0000 7f04237fe700 1 -- 172.21.15.107:0/722929259 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f041c004810 con 0x7f042c100560 2023-07-21T22:27:47.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.171+0000 7f04237fe700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f04180a1150 0x7f04180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:47.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.171+0000 7f042a59c700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f04180a1150 0x7f04180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:47.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.171+0000 7f04237fe700 1 -- 172.21.15.107:0/722929259 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f041c0cddd0 con 0x7f042c100560 2023-07-21T22:27:47.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.171+0000 7f042a59c700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f04180a1150 0x7f04180a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f042c0ffa30 tx=0x7f042400c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:47.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.174+0000 7f04237fe700 1 -- 172.21.15.107:0/722929259 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f041c097700 con 0x7f042c100560 2023-07-21T22:27:47.431 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-b[123210]: 2023-07-21T22:27:46.909+0000 7fc861074700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-21T22:27:47.432 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:27:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-b[123210]: 2023-07-21T22:27:46.909+0000 7fc861074700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-07-21T22:27:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:47 smithi132 ceph-mon[97863]: pgmap v252: 97 pgs: 3 creating+activating, 29 creating+peering, 8 unknown, 57 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 536 B/s wr, 2 op/s 2023-07-21T22:27:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:47 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1656930123' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T22:27:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:47 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] up:active 2023-07-21T22:27:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:47 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1656930123' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T22:27:47.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:47 smithi132 ceph-mon[97863]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-21T22:27:47.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:47 smithi132 ceph-mon[97863]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-07-21T22:27:47.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:47 smithi132 ceph-mon[97863]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2023-07-21T22:27:47.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:47 smithi132 ceph-mon[97863]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-07-21T22:27:47.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:47 smithi107 ceph-mon[96543]: pgmap v252: 97 pgs: 3 creating+activating, 29 creating+peering, 8 unknown, 57 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 536 B/s wr, 2 op/s 2023-07-21T22:27:47.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:47 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1656930123' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T22:27:47.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:47 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] up:active 2023-07-21T22:27:47.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:47 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1656930123' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T22:27:47.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:47 smithi107 ceph-mon[96543]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-21T22:27:47.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:47 smithi107 ceph-mon[96543]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-07-21T22:27:47.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:47 smithi107 ceph-mon[96543]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2023-07-21T22:27:47.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:47 smithi107 ceph-mon[96543]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-07-21T22:27:47.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:47.400+0000 7f0430994700 1 -- 172.21.15.107:0/722929259 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f042c0609e0 con 0x7f04180a1150 2023-07-21T22:27:47.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:47 smithi190 ceph-mon[98334]: pgmap v252: 97 pgs: 3 creating+activating, 29 creating+peering, 8 unknown, 57 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 536 B/s wr, 2 op/s 2023-07-21T22:27:47.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:47 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1656930123' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T22:27:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:47 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] up:active 2023-07-21T22:27:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:47 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1656930123' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T22:27:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:47 smithi190 ceph-mon[98334]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-21T22:27:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:47 smithi190 ceph-mon[98334]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2023-07-21T22:27:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:47 smithi190 ceph-mon[98334]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2023-07-21T22:27:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:47 smithi190 ceph-mon[98334]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-07-21T22:27:47.937 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:27:47 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:27:47.405+0000 7f92d86ed700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:27:47.938 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:27:47 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:27:47.406+0000 7f92d86ed700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:27:47.938 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:27:47 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:27:47.406+0000 7f92d86ed700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:27:47.938 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:27:47 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:27:47.406+0000 7f92d86ed700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:27:47.938 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:27:47 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:27:47.406+0000 7f92d86ed700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:27:48.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.062+0000 7f04237fe700 1 -- 172.21.15.107:0/722929259 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f042c0609e0 con 0x7f04180a1150 2023-07-21T22:27:48.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.064+0000 7f0430994700 1 -- 172.21.15.107:0/722929259 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f04180a1150 msgr2=0x7f04180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:48.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.064+0000 7f0430994700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f04180a1150 0x7f04180a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f042c0ffa30 tx=0x7f042400c040 comp rx=0 tx=0).stop 2023-07-21T22:27:48.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.064+0000 7f0430994700 1 -- 172.21.15.107:0/722929259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f042c100560 msgr2=0x7f042c19bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:48.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.064+0000 7f0430994700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f042c100560 0x7f042c19bc30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f041c004760 tx=0x7f041c009580 comp rx=0 tx=0).stop 2023-07-21T22:27:48.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.065+0000 7f0430994700 1 -- 172.21.15.107:0/722929259 shutdown_connections 2023-07-21T22:27:48.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.065+0000 7f0430994700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f042c100560 0x7f042c19bc30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.065+0000 7f0430994700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f04180a1150 0x7f04180a3600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.065+0000 7f0430994700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f042c0fe9e0 0x7f042c194680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.065+0000 7f0430994700 1 --2- 172.21.15.107:0/722929259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f042c0ffbd0 0x7f042c194bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.065+0000 7f0430994700 1 -- 172.21.15.107:0/722929259 >> 172.21.15.107:0/722929259 conn(0x7f042c0fa000 msgr2=0x7f042c103040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:48.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.065+0000 7f0430994700 1 -- 172.21.15.107:0/722929259 shutdown_connections 2023-07-21T22:27:48.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.065+0000 7f0430994700 1 -- 172.21.15.107:0/722929259 wait complete. 2023-07-21T22:27:48.124 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-07-21T22:27:48.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.204+0000 7f2936718700 1 -- 172.21.15.107:0/4118647764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29300febd0 msgr2=0x7f29300ff040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:48.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.204+0000 7f2936718700 1 --2- 172.21.15.107:0/4118647764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29300febd0 0x7f29300ff040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2924009990 tx=0x7f2924009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.204+0000 7f2936718700 1 -- 172.21.15.107:0/4118647764 shutdown_connections 2023-07-21T22:27:48.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.204+0000 7f2936718700 1 --2- 172.21.15.107:0/4118647764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29300ff580 0x7f2930106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.204+0000 7f2936718700 1 --2- 172.21.15.107:0/4118647764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29300fff80 0x7f2930100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.204+0000 7f2936718700 1 --2- 172.21.15.107:0/4118647764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29300febd0 0x7f29300ff040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.204+0000 7f2936718700 1 -- 172.21.15.107:0/4118647764 >> 172.21.15.107:0/4118647764 conn(0x7f29300f9770 msgr2=0x7f29300fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:48.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.204+0000 7f2936718700 1 -- 172.21.15.107:0/4118647764 shutdown_connections 2023-07-21T22:27:48.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.204+0000 7f2936718700 1 -- 172.21.15.107:0/4118647764 wait complete. 2023-07-21T22:27:48.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2936718700 1 Processor -- start 2023-07-21T22:27:48.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2936718700 1 -- start start 2023-07-21T22:27:48.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2936718700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29300febd0 0x7f2930194660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:48.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2936718700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29300ff580 0x7f2930194ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:48.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2936718700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29300fff80 0x7f293019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:48.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2936718700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293019c1e0 con 0x7f29300fff80 2023-07-21T22:27:48.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2936718700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293019c320 con 0x7f29300febd0 2023-07-21T22:27:48.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2936718700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293019c4b0 con 0x7f29300ff580 2023-07-21T22:27:48.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2934cb5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29300fff80 0x7f293019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:48.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2934cb5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29300fff80 0x7f293019bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46548/0 (socket says 172.21.15.107:46548) 2023-07-21T22:27:48.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2934cb5700 1 -- 172.21.15.107:0/3035284816 learned_addr learned my addr 172.21.15.107:0/3035284816 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:48.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f292f7fe700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29300ff580 0x7f2930194ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:48.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f292ffff700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29300febd0 0x7f2930194660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:48.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2934cb5700 1 -- 172.21.15.107:0/3035284816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29300ff580 msgr2=0x7f2930194ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:48.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2934cb5700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29300ff580 0x7f2930194ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2934cb5700 1 -- 172.21.15.107:0/3035284816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29300febd0 msgr2=0x7f2930194660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:48.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2934cb5700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29300febd0 0x7f2930194660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f2934cb5700 1 -- 172.21.15.107:0/3035284816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29240095f0 con 0x7f29300fff80 2023-07-21T22:27:48.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.205+0000 7f292ffff700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29300febd0 0x7f2930194660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:27:48.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.206+0000 7f2934cb5700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29300fff80 0x7f293019bc10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2928007ee0 tx=0x7f292800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:48.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.206+0000 7f292d7fa700 1 -- 172.21.15.107:0/3035284816 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928017070 con 0x7f29300fff80 2023-07-21T22:27:48.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.206+0000 7f2936718700 1 -- 172.21.15.107:0/3035284816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29300fdb20 con 0x7f29300fff80 2023-07-21T22:27:48.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.206+0000 7f292d7fa700 1 -- 172.21.15.107:0/3035284816 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f292800f040 con 0x7f29300fff80 2023-07-21T22:27:48.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.206+0000 7f292d7fa700 1 -- 172.21.15.107:0/3035284816 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928005740 con 0x7f29300fff80 2023-07-21T22:27:48.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.206+0000 7f2936718700 1 -- 172.21.15.107:0/3035284816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29300fdfe0 con 0x7f29300fff80 2023-07-21T22:27:48.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.207+0000 7f2936718700 1 -- 172.21.15.107:0/3035284816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293010a180 con 0x7f29300fff80 2023-07-21T22:27:48.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.208+0000 7f292d7fa700 1 -- 172.21.15.107:0/3035284816 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f292800dc60 con 0x7f29300fff80 2023-07-21T22:27:48.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.209+0000 7f292d7fa700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f29180a1150 0x7f29180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:48.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.209+0000 7f292ffff700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f29180a1150 0x7f29180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:48.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.209+0000 7f292d7fa700 1 -- 172.21.15.107:0/3035284816 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f29280cc670 con 0x7f29300fff80 2023-07-21T22:27:48.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.209+0000 7f292ffff700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f29180a1150 0x7f29180a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2920005fd0 tx=0x7f2920005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:48.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.212+0000 7f292d7fa700 1 -- 172.21.15.107:0/3035284816 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2928095f10 con 0x7f29300fff80 2023-07-21T22:27:48.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.437+0000 7f2936718700 1 -- 172.21.15.107:0/3035284816 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f2930060950 con 0x7f29180a1150 2023-07-21T22:27:48.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.441+0000 7f292d7fa700 1 -- 172.21.15.107:0/3035284816 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f2930060950 con 0x7f29180a1150 2023-07-21T22:27:48.488 INFO:teuthology.orchestra.run.smithi107.stdout:/volumes/_nogroup/sv_0/00cc3790-1e7e-405f-9c43-084782c8179f 2023-07-21T22:27:48.490 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.443+0000 7f2936718700 1 -- 172.21.15.107:0/3035284816 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f29180a1150 msgr2=0x7f29180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:48.490 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.443+0000 7f2936718700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f29180a1150 0x7f29180a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2920005fd0 tx=0x7f2920005e30 comp rx=0 tx=0).stop 2023-07-21T22:27:48.490 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.443+0000 7f2936718700 1 -- 172.21.15.107:0/3035284816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29300fff80 msgr2=0x7f293019bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:48.490 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.443+0000 7f2936718700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29300fff80 0x7f293019bc10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2928007ee0 tx=0x7f292800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.490 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.444+0000 7f2936718700 1 -- 172.21.15.107:0/3035284816 shutdown_connections 2023-07-21T22:27:48.491 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.444+0000 7f2936718700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29300fff80 0x7f293019bc10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.491 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.444+0000 7f2936718700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29300febd0 0x7f2930194660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.491 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.444+0000 7f2936718700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f29180a1150 0x7f29180a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.491 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.444+0000 7f2936718700 1 --2- 172.21.15.107:0/3035284816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29300ff580 0x7f2930194ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.491 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.444+0000 7f2936718700 1 -- 172.21.15.107:0/3035284816 >> 172.21.15.107:0/3035284816 conn(0x7f29300f9770 msgr2=0x7f29300fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:48.491 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.444+0000 7f2936718700 1 -- 172.21.15.107:0/3035284816 shutdown_connections 2023-07-21T22:27:48.491 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.444+0000 7f2936718700 1 -- 172.21.15.107:0/3035284816 wait complete. 2023-07-21T22:27:48.502 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-07-21T22:27:48.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.580+0000 7f4de4eb8700 1 -- 172.21.15.107:0/3357550401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de00feef0 msgr2=0x7f4de00ff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.580+0000 7f4de4eb8700 1 --2- 172.21.15.107:0/3357550401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de00feef0 0x7f4de00ff300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009990 tx=0x7f4dd4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.580+0000 7f4de4eb8700 1 -- 172.21.15.107:0/3357550401 shutdown_connections 2023-07-21T22:27:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.580+0000 7f4de4eb8700 1 --2- 172.21.15.107:0/3357550401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de01005b0 0x7f4de0106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.580+0000 7f4de4eb8700 1 --2- 172.21.15.107:0/3357550401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de00feef0 0x7f4de00ff300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.580+0000 7f4de4eb8700 1 --2- 172.21.15.107:0/3357550401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de00ffc00 0x7f4de0100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.580+0000 7f4de4eb8700 1 -- 172.21.15.107:0/3357550401 >> 172.21.15.107:0/3357550401 conn(0x7f4de00f9770 msgr2=0x7f4de00fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.580+0000 7f4de4eb8700 1 -- 172.21.15.107:0/3357550401 shutdown_connections 2023-07-21T22:27:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.580+0000 7f4de4eb8700 1 -- 172.21.15.107:0/3357550401 wait complete. 2023-07-21T22:27:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4de4eb8700 1 Processor -- start 2023-07-21T22:27:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4de4eb8700 1 -- start start 2023-07-21T22:27:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4de4eb8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de00feef0 0x7f4de00fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4de4eb8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de00ffc00 0x7f4de019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4de4eb8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de01005b0 0x7f4de019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4de4eb8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de019d450 con 0x7f4de01005b0 2023-07-21T22:27:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4de4eb8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de019d5c0 con 0x7f4de00feef0 2023-07-21T22:27:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4de4eb8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de019d780 con 0x7f4de00ffc00 2023-07-21T22:27:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4dded9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de01005b0 0x7f4de019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:48.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4dded9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de01005b0 0x7f4de019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46562/0 (socket says 172.21.15.107:46562) 2023-07-21T22:27:48.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4dded9d700 1 -- 172.21.15.107:0/48226843 learned_addr learned my addr 172.21.15.107:0/48226843 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:48.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4dde59c700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de00feef0 0x7f4de00fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:48.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4dddd9b700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de00ffc00 0x7f4de019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:48.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4dded9d700 1 -- 172.21.15.107:0/48226843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de00ffc00 msgr2=0x7f4de019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:48.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4dded9d700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de00ffc00 0x7f4de019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4dded9d700 1 -- 172.21.15.107:0/48226843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de00feef0 msgr2=0x7f4de00fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:48.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4dded9d700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de00feef0 0x7f4de00fe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4dded9d700 1 -- 172.21.15.107:0/48226843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd40095f0 con 0x7f4de01005b0 2023-07-21T22:27:48.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4dddd9b700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de00ffc00 0x7f4de019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:27:48.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.581+0000 7f4dded9d700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de01005b0 0x7f4de019cc80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800ada0 tx=0x7f4dd800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:48.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.582+0000 7f4dcf7fe700 1 -- 172.21.15.107:0/48226843 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800f070 con 0x7f4de01005b0 2023-07-21T22:27:48.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.582+0000 7f4dcf7fe700 1 -- 172.21.15.107:0/48226843 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dd8007b30 con 0x7f4de01005b0 2023-07-21T22:27:48.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.582+0000 7f4de4eb8700 1 -- 172.21.15.107:0/48226843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de019da60 con 0x7f4de01005b0 2023-07-21T22:27:48.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.582+0000 7f4dcf7fe700 1 -- 172.21.15.107:0/48226843 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd80053e0 con 0x7f4de01005b0 2023-07-21T22:27:48.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.582+0000 7f4de4eb8700 1 -- 172.21.15.107:0/48226843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de019df20 con 0x7f4de01005b0 2023-07-21T22:27:48.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.583+0000 7f4de4eb8700 1 -- 172.21.15.107:0/48226843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de01059c0 con 0x7f4de01005b0 2023-07-21T22:27:48.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.584+0000 7f4dcf7fe700 1 -- 172.21.15.107:0/48226843 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4dd800c980 con 0x7f4de01005b0 2023-07-21T22:27:48.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.584+0000 7f4dcf7fe700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4dc80a1150 0x7f4dc80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:48.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.585+0000 7f4dde59c700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4dc80a1150 0x7f4dc80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:48.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.585+0000 7f4dcf7fe700 1 -- 172.21.15.107:0/48226843 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4dd80cc370 con 0x7f4de01005b0 2023-07-21T22:27:48.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.585+0000 7f4dde59c700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4dc80a1150 0x7f4dc80a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400b5c0 tx=0x7f4dd4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:48.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.588+0000 7f4dcf7fe700 1 -- 172.21.15.107:0/48226843 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4dd8095ca0 con 0x7f4de01005b0 2023-07-21T22:27:48.854 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.808+0000 7f4de4eb8700 1 -- 172.21.15.107:0/48226843 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f4de00609e0 con 0x7f4dc80a1150 2023-07-21T22:27:48.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.834+0000 7f4dcf7fe700 1 -- 172.21.15.107:0/48226843 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4de00609e0 con 0x7f4dc80a1150 2023-07-21T22:27:48.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.837+0000 7f4de4eb8700 1 -- 172.21.15.107:0/48226843 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4dc80a1150 msgr2=0x7f4dc80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:48.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.837+0000 7f4de4eb8700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4dc80a1150 0x7f4dc80a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400b5c0 tx=0x7f4dd4005fd0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.837+0000 7f4de4eb8700 1 -- 172.21.15.107:0/48226843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de01005b0 msgr2=0x7f4de019cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:48.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.837+0000 7f4de4eb8700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de01005b0 0x7f4de019cc80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800ada0 tx=0x7f4dd800c330 comp rx=0 tx=0).stop 2023-07-21T22:27:48.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.837+0000 7f4de4eb8700 1 -- 172.21.15.107:0/48226843 shutdown_connections 2023-07-21T22:27:48.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.837+0000 7f4de4eb8700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4de01005b0 0x7f4de019cc80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.837+0000 7f4de4eb8700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4de00feef0 0x7f4de00fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.837+0000 7f4de4eb8700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4dc80a1150 0x7f4dc80a3600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.837+0000 7f4de4eb8700 1 --2- 172.21.15.107:0/48226843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4de00ffc00 0x7f4de019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:48.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.837+0000 7f4de4eb8700 1 -- 172.21.15.107:0/48226843 >> 172.21.15.107:0/48226843 conn(0x7f4de00f9770 msgr2=0x7f4de00fba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:48.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.837+0000 7f4de4eb8700 1 -- 172.21.15.107:0/48226843 shutdown_connections 2023-07-21T22:27:48.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.837+0000 7f4de4eb8700 1 -- 172.21.15.107:0/48226843 wait complete. 2023-07-21T22:27:48.895 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-07-21T22:27:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.972+0000 7f57c8535700 1 -- 172.21.15.107:0/577431310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57c0100d40 msgr2=0x7f57c0101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.972+0000 7f57c8535700 1 --2- 172.21.15.107:0/577431310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57c0100d40 0x7f57c0101190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009990 tx=0x7f57b4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.972+0000 7f57c8535700 1 -- 172.21.15.107:0/577431310 shutdown_connections 2023-07-21T22:27:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.972+0000 7f57c8535700 1 --2- 172.21.15.107:0/577431310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c01016d0 0x7f57c0107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.972+0000 7f57c8535700 1 --2- 172.21.15.107:0/577431310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57c00ffb50 0x7f57c00fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.972+0000 7f57c8535700 1 --2- 172.21.15.107:0/577431310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57c0100d40 0x7f57c0101190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.972+0000 7f57c8535700 1 -- 172.21.15.107:0/577431310 >> 172.21.15.107:0/577431310 conn(0x7f57c00fb170 msgr2=0x7f57c00fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.972+0000 7f57c8535700 1 -- 172.21.15.107:0/577431310 shutdown_connections 2023-07-21T22:27:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.972+0000 7f57c8535700 1 -- 172.21.15.107:0/577431310 wait complete. 2023-07-21T22:27:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c8535700 1 Processor -- start 2023-07-21T22:27:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c8535700 1 -- start start 2023-07-21T22:27:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c8535700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c00ffb50 0x7f57c0195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c8535700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57c0100d40 0x7f57c0195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c8535700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57c01016d0 0x7f57c019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c8535700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c019d3a0 con 0x7f57c0100d40 2023-07-21T22:27:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c8535700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c019d4e0 con 0x7f57c00ffb50 2023-07-21T22:27:49.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c8535700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c019d670 con 0x7f57c01016d0 2023-07-21T22:27:49.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c5ad0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57c0100d40 0x7f57c0195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:49.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c5ad0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57c0100d40 0x7f57c0195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46576/0 (socket says 172.21.15.107:46576) 2023-07-21T22:27:49.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c5ad0700 1 -- 172.21.15.107:0/3134414738 learned_addr learned my addr 172.21.15.107:0/3134414738 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:49.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c6ad2700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57c01016d0 0x7f57c019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:49.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c62d1700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c00ffb50 0x7f57c0195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:49.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c5ad0700 1 -- 172.21.15.107:0/3134414738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57c01016d0 msgr2=0x7f57c019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c5ad0700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57c01016d0 0x7f57c019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c5ad0700 1 -- 172.21.15.107:0/3134414738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c00ffb50 msgr2=0x7f57c0195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c5ad0700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c00ffb50 0x7f57c0195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.973+0000 7f57c5ad0700 1 -- 172.21.15.107:0/3134414738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b40095f0 con 0x7f57c0100d40 2023-07-21T22:27:49.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.974+0000 7f57c5ad0700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57c0100d40 0x7f57c0195d60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57b4005910 tx=0x7f57b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:49.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.974+0000 7f57b37fe700 1 -- 172.21.15.107:0/3134414738 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b401d070 con 0x7f57c0100d40 2023-07-21T22:27:49.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.974+0000 7f57c8535700 1 -- 172.21.15.107:0/3134414738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c019d8f0 con 0x7f57c0100d40 2023-07-21T22:27:49.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.974+0000 7f57b37fe700 1 -- 172.21.15.107:0/3134414738 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57b4003f60 con 0x7f57c0100d40 2023-07-21T22:27:49.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.974+0000 7f57b37fe700 1 -- 172.21.15.107:0/3134414738 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4022da0 con 0x7f57c0100d40 2023-07-21T22:27:49.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.974+0000 7f57c8535700 1 -- 172.21.15.107:0/3134414738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c019dd10 con 0x7f57c0100d40 2023-07-21T22:27:49.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.975+0000 7f57c8535700 1 -- 172.21.15.107:0/3134414738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c010b300 con 0x7f57c0100d40 2023-07-21T22:27:49.026 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.976+0000 7f57b37fe700 1 -- 172.21.15.107:0/3134414738 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f57b4003aa0 con 0x7f57c0100d40 2023-07-21T22:27:49.026 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.976+0000 7f57b37fe700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57ac0a1150 0x7f57ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:49.026 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.976+0000 7f57c62d1700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57ac0a1150 0x7f57ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:49.026 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.976+0000 7f57b37fe700 1 -- 172.21.15.107:0/3134414738 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f57b40ce3a0 con 0x7f57c0100d40 2023-07-21T22:27:49.026 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.977+0000 7f57c62d1700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57ac0a1150 0x7f57ac0a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f57bc005950 tx=0x7f57bc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:49.026 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:48.979+0000 7f57b37fe700 1 -- 172.21.15.107:0/3134414738 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f57b4097cd0 con 0x7f57c0100d40 2023-07-21T22:27:49.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:49 smithi107 ceph-mon[96543]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:49.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:49 smithi107 ceph-mon[96543]: pgmap v253: 97 pgs: 97 active+clean; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2023-07-21T22:27:49.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:49 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] up:active 2023-07-21T22:27:49.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:49 smithi107 ceph-mon[96543]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-21T22:27:49.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:49 smithi107 ceph-mon[96543]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T22:27:49.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:49 smithi107 ceph-mon[96543]: Cluster is now healthy 2023-07-21T22:27:49.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:49 smithi107 ceph-mon[96543]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2023-07-21T22:27:49.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:49 smithi107 ceph-mon[96543]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-07-21T22:27:49.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:49 smithi107 ceph-mon[96543]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-07-21T22:27:49.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.211+0000 7f57c8535700 1 -- 172.21.15.107:0/3134414738 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f57c0060950 con 0x7f57ac0a1150 2023-07-21T22:27:49.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.214+0000 7f57b37fe700 1 -- 172.21.15.107:0/3134414738 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f57c0060950 con 0x7f57ac0a1150 2023-07-21T22:27:49.261 INFO:teuthology.orchestra.run.smithi107.stdout:/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2 2023-07-21T22:27:49.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.216+0000 7f57c8535700 1 -- 172.21.15.107:0/3134414738 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57ac0a1150 msgr2=0x7f57ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.216+0000 7f57c8535700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57ac0a1150 0x7f57ac0a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f57bc005950 tx=0x7f57bc0058e0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.216+0000 7f57c8535700 1 -- 172.21.15.107:0/3134414738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57c0100d40 msgr2=0x7f57c0195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.216+0000 7f57c8535700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57c0100d40 0x7f57c0195d60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57b4005910 tx=0x7f57b4004730 comp rx=0 tx=0).stop 2023-07-21T22:27:49.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.216+0000 7f57c8535700 1 -- 172.21.15.107:0/3134414738 shutdown_connections 2023-07-21T22:27:49.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.216+0000 7f57c8535700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57ac0a1150 0x7f57ac0a3600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.216+0000 7f57c8535700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57c01016d0 0x7f57c019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.216+0000 7f57c8535700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57c00ffb50 0x7f57c0195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.216+0000 7f57c8535700 1 --2- 172.21.15.107:0/3134414738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57c0100d40 0x7f57c0195d60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.216+0000 7f57c8535700 1 -- 172.21.15.107:0/3134414738 >> 172.21.15.107:0/3134414738 conn(0x7f57c00fb170 msgr2=0x7f57c00fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:49.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.217+0000 7f57c8535700 1 -- 172.21.15.107:0/3134414738 shutdown_connections 2023-07-21T22:27:49.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.217+0000 7f57c8535700 1 -- 172.21.15.107:0/3134414738 wait complete. 2023-07-21T22:27:49.275 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:27:49.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:49 smithi190 ceph-mon[98334]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:49.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:49 smithi190 ceph-mon[98334]: pgmap v253: 97 pgs: 97 active+clean; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2023-07-21T22:27:49.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:49 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] up:active 2023-07-21T22:27:49.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:49 smithi190 ceph-mon[98334]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-21T22:27:49.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:49 smithi190 ceph-mon[98334]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T22:27:49.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:49 smithi190 ceph-mon[98334]: Cluster is now healthy 2023-07-21T22:27:49.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:49 smithi190 ceph-mon[98334]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2023-07-21T22:27:49.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:49 smithi190 ceph-mon[98334]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-07-21T22:27:49.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:49 smithi190 ceph-mon[98334]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-07-21T22:27:49.395 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.349+0000 7f25bda7a700 1 -- 172.21.15.107:0/51352318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b81005a0 msgr2=0x7f25b8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.395 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.349+0000 7f25bda7a700 1 --2- 172.21.15.107:0/51352318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b81005a0 0x7f25b8106e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f25b000b0a0 tx=0x7f25b000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.395 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.349+0000 7f25bda7a700 1 -- 172.21.15.107:0/51352318 shutdown_connections 2023-07-21T22:27:49.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.349+0000 7f25bda7a700 1 --2- 172.21.15.107:0/51352318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b81005a0 0x7f25b8106e60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.349+0000 7f25bda7a700 1 --2- 172.21.15.107:0/51352318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b80fea20 0x7f25b80fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.349+0000 7f25bda7a700 1 --2- 172.21.15.107:0/51352318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25b80ffc10 0x7f25b8100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.349+0000 7f25bda7a700 1 -- 172.21.15.107:0/51352318 >> 172.21.15.107:0/51352318 conn(0x7f25b80fa060 msgr2=0x7f25b80fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:49.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.349+0000 7f25bda7a700 1 -- 172.21.15.107:0/51352318 shutdown_connections 2023-07-21T22:27:49.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.349+0000 7f25bda7a700 1 -- 172.21.15.107:0/51352318 wait complete. 2023-07-21T22:27:49.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.349+0000 7f25bda7a700 1 Processor -- start 2023-07-21T22:27:49.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25bda7a700 1 -- start start 2023-07-21T22:27:49.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25bda7a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fea20 0x7f25b8195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:49.399 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25bda7a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25b80ffc10 0x7f25b8195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:49.399 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25bda7a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b81005a0 0x7f25b819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:49.399 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25bda7a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b819d3f0 con 0x7f25b80fea20 2023-07-21T22:27:49.399 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25bda7a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b819d530 con 0x7f25b80ffc10 2023-07-21T22:27:49.399 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25bda7a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b819d6c0 con 0x7f25b81005a0 2023-07-21T22:27:49.399 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fea20 0x7f25b8195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:49.399 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fea20 0x7f25b8195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46610/0 (socket says 172.21.15.107:46610) 2023-07-21T22:27:49.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b77fe700 1 -- 172.21.15.107:0/2216732447 learned_addr learned my addr 172.21.15.107:0/2216732447 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:49.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b7fff700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b81005a0 0x7f25b819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:49.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b77fe700 1 -- 172.21.15.107:0/2216732447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b81005a0 msgr2=0x7f25b819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b77fe700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b81005a0 0x7f25b819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b77fe700 1 -- 172.21.15.107:0/2216732447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25b80ffc10 msgr2=0x7f25b8195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b6ffd700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25b80ffc10 0x7f25b8195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:49.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b77fe700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25b80ffc10 0x7f25b8195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b77fe700 1 -- 172.21.15.107:0/2216732447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b0009d00 con 0x7f25b80fea20 2023-07-21T22:27:49.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b77fe700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fea20 0x7f25b8195870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c970 tx=0x7f25a000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:49.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b4ff9700 1 -- 172.21.15.107:0/2216732447 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a0004220 con 0x7f25b80fea20 2023-07-21T22:27:49.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25bda7a700 1 -- 172.21.15.107:0/2216732447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b819d9a0 con 0x7f25b80fea20 2023-07-21T22:27:49.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b4ff9700 1 -- 172.21.15.107:0/2216732447 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25a0007d60 con 0x7f25b80fea20 2023-07-21T22:27:49.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25b4ff9700 1 -- 172.21.15.107:0/2216732447 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a0003d00 con 0x7f25b80fea20 2023-07-21T22:27:49.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.350+0000 7f25bda7a700 1 -- 172.21.15.107:0/2216732447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b819deb0 con 0x7f25b80fea20 2023-07-21T22:27:49.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.351+0000 7f25bda7a700 1 -- 172.21.15.107:0/2216732447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b810a200 con 0x7f25b80fea20 2023-07-21T22:27:49.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.352+0000 7f25b4ff9700 1 -- 172.21.15.107:0/2216732447 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f25a0004380 con 0x7f25b80fea20 2023-07-21T22:27:49.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.353+0000 7f25b4ff9700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f25a40a11a0 0x7f25a40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:49.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.353+0000 7f25b4ff9700 1 -- 172.21.15.107:0/2216732447 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f25a00cc7a0 con 0x7f25b80fea20 2023-07-21T22:27:49.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.353+0000 7f25b6ffd700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f25a40a11a0 0x7f25a40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:49.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.353+0000 7f25b6ffd700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f25a40a11a0 0x7f25a40a3650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25a8005950 tx=0x7f25a8005cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:49.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.356+0000 7f25b4ff9700 1 -- 172.21.15.107:0/2216732447 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f25a0096040 con 0x7f25b80fea20 2023-07-21T22:27:49.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:49 smithi132 ceph-mon[97863]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:49.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:49 smithi132 ceph-mon[97863]: pgmap v253: 97 pgs: 97 active+clean; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2023-07-21T22:27:49.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:49 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] up:active 2023-07-21T22:27:49.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:49 smithi132 ceph-mon[97863]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-21T22:27:49.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:49 smithi132 ceph-mon[97863]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T22:27:49.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:49 smithi132 ceph-mon[97863]: Cluster is now healthy 2023-07-21T22:27:49.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:49 smithi132 ceph-mon[97863]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2023-07-21T22:27:49.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:49 smithi132 ceph-mon[97863]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-07-21T22:27:49.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:49 smithi132 ceph-mon[97863]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-07-21T22:27:49.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.587+0000 7f25bda7a700 1 -- 172.21.15.107:0/2216732447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25b819e4c0 con 0x7f25b80fea20 2023-07-21T22:27:49.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.587+0000 7f25b4ff9700 1 -- 172.21.15.107:0/2216732447 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10868 (secure 0 0 0) 0x7f25a0099660 con 0x7f25b80fea20 2023-07-21T22:27:49.634 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:49.634 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:27:49.034257+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:27:49.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.589+0000 7f25bda7a700 1 -- 172.21.15.107:0/2216732447 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f25a40a11a0 msgr2=0x7f25a40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.589+0000 7f25bda7a700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f25a40a11a0 0x7f25a40a3650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25a8005950 tx=0x7f25a8005cf0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.589+0000 7f25bda7a700 1 -- 172.21.15.107:0/2216732447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fea20 msgr2=0x7f25b8195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.589+0000 7f25bda7a700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fea20 0x7f25b8195870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c970 tx=0x7f25a000cd30 comp rx=0 tx=0).stop 2023-07-21T22:27:49.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.589+0000 7f25bda7a700 1 -- 172.21.15.107:0/2216732447 shutdown_connections 2023-07-21T22:27:49.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.589+0000 7f25bda7a700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f25a40a11a0 0x7f25a40a3650 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.589+0000 7f25bda7a700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25b81005a0 0x7f25b819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.589+0000 7f25bda7a700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fea20 0x7f25b8195870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.589+0000 7f25bda7a700 1 --2- 172.21.15.107:0/2216732447 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25b80ffc10 0x7f25b8195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.589+0000 7f25bda7a700 1 -- 172.21.15.107:0/2216732447 >> 172.21.15.107:0/2216732447 conn(0x7f25b80fa060 msgr2=0x7f25b80686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:49.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.589+0000 7f25bda7a700 1 -- 172.21.15.107:0/2216732447 shutdown_connections 2023-07-21T22:27:49.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.589+0000 7f25bda7a700 1 -- 172.21.15.107:0/2216732447 wait complete. 2023-07-21T22:27:49.637 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2023-07-21T22:27:49.648 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:27:49.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.723+0000 7f2ac379b700 1 -- 172.21.15.107:0/2872526268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc0ff5c0 msgr2=0x7f2abc106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.723+0000 7f2ac379b700 1 --2- 172.21.15.107:0/2872526268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc0ff5c0 0x7f2abc106e70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400b0a0 tx=0x7f2ab400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.723+0000 7f2ac379b700 1 -- 172.21.15.107:0/2872526268 shutdown_connections 2023-07-21T22:27:49.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.723+0000 7f2ac379b700 1 --2- 172.21.15.107:0/2872526268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc0ff5c0 0x7f2abc106e70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.723+0000 7f2ac379b700 1 --2- 172.21.15.107:0/2872526268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2abc100050 0x7f2abc100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.723+0000 7f2ac379b700 1 --2- 172.21.15.107:0/2872526268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2abc0fec10 0x7f2abc0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.723+0000 7f2ac379b700 1 -- 172.21.15.107:0/2872526268 >> 172.21.15.107:0/2872526268 conn(0x7f2abc0f9770 msgr2=0x7f2abc0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:49.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.723+0000 7f2ac379b700 1 -- 172.21.15.107:0/2872526268 shutdown_connections 2023-07-21T22:27:49.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.723+0000 7f2ac379b700 1 -- 172.21.15.107:0/2872526268 wait complete. 2023-07-21T22:27:49.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac379b700 1 Processor -- start 2023-07-21T22:27:49.773 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac379b700 1 -- start start 2023-07-21T22:27:49.773 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac379b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc0fec10 0x7f2abc195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:49.773 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac379b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2abc0ff5c0 0x7f2abc195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:49.774 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac379b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2abc100050 0x7f2abc19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:49.774 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac379b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc19d410 con 0x7f2abc0fec10 2023-07-21T22:27:49.774 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac379b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc19d550 con 0x7f2abc100050 2023-07-21T22:27:49.775 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac379b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc19d6e0 con 0x7f2abc0ff5c0 2023-07-21T22:27:49.775 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac1537700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc0fec10 0x7f2abc195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:49.776 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac1537700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc0fec10 0x7f2abc195890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46624/0 (socket says 172.21.15.107:46624) 2023-07-21T22:27:49.776 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac1537700 1 -- 172.21.15.107:0/2006131599 learned_addr learned my addr 172.21.15.107:0/2006131599 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:49.776 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac0d36700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2abc0ff5c0 0x7f2abc195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:49.776 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac1d38700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2abc100050 0x7f2abc19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:49.777 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac1537700 1 -- 172.21.15.107:0/2006131599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2abc0ff5c0 msgr2=0x7f2abc195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.777 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac1537700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2abc0ff5c0 0x7f2abc195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.777 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac1537700 1 -- 172.21.15.107:0/2006131599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2abc100050 msgr2=0x7f2abc19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac1537700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2abc100050 0x7f2abc19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac1537700 1 -- 172.21.15.107:0/2006131599 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab4009d00 con 0x7f2abc0fec10 2023-07-21T22:27:49.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac1537700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc0fec10 0x7f2abc195890 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00d950 tx=0x7f2aac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:49.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ab27fc700 1 -- 172.21.15.107:0/2006131599 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00f890 con 0x7f2abc0fec10 2023-07-21T22:27:49.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ab27fc700 1 -- 172.21.15.107:0/2006131599 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aac010460 con 0x7f2abc0fec10 2023-07-21T22:27:49.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac379b700 1 -- 172.21.15.107:0/2006131599 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2abc19d9c0 con 0x7f2abc0fec10 2023-07-21T22:27:49.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ab27fc700 1 -- 172.21.15.107:0/2006131599 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00e610 con 0x7f2abc0fec10 2023-07-21T22:27:49.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.724+0000 7f2ac379b700 1 -- 172.21.15.107:0/2006131599 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc19ded0 con 0x7f2abc0fec10 2023-07-21T22:27:49.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.725+0000 7f2ac379b700 1 -- 172.21.15.107:0/2006131599 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2abc106a60 con 0x7f2abc0fec10 2023-07-21T22:27:49.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.726+0000 7f2ab27fc700 1 -- 172.21.15.107:0/2006131599 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2aac0108d0 con 0x7f2abc0fec10 2023-07-21T22:27:49.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.727+0000 7f2ab27fc700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2aa80a11a0 0x7f2aa80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:49.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.727+0000 7f2ac0d36700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2aa80a11a0 0x7f2aa80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:49.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.727+0000 7f2ab27fc700 1 -- 172.21.15.107:0/2006131599 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2aac0cc580 con 0x7f2abc0fec10 2023-07-21T22:27:49.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.727+0000 7f2ac0d36700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2aa80a11a0 0x7f2aa80a3650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009c60 tx=0x7f2ab8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:49.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.730+0000 7f2ab27fc700 1 -- 172.21.15.107:0/2006131599 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2aac095e20 con 0x7f2abc0fec10 2023-07-21T22:27:49.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.921+0000 7f2ac379b700 1 -- 172.21.15.107:0/2006131599 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2abc02c850 con 0x7f2abc0fec10 2023-07-21T22:27:49.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.923+0000 7f2ab27fc700 1 -- 172.21.15.107:0/2006131599 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22201 (secure 0 0 0) 0x7f2aac016090 con 0x7f2abc0fec10 2023-07-21T22:27:49.970 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:49.970 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:27:43.071566+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:27:49.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.925+0000 7f2ac379b700 1 -- 172.21.15.107:0/2006131599 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2aa80a11a0 msgr2=0x7f2aa80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.925+0000 7f2ac379b700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2aa80a11a0 0x7f2aa80a3650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009c60 tx=0x7f2ab8009260 comp rx=0 tx=0).stop 2023-07-21T22:27:49.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.925+0000 7f2ac379b700 1 -- 172.21.15.107:0/2006131599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc0fec10 msgr2=0x7f2abc195890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:49.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.925+0000 7f2ac379b700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc0fec10 0x7f2abc195890 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00d950 tx=0x7f2aac00dc60 comp rx=0 tx=0).stop 2023-07-21T22:27:49.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.925+0000 7f2ac379b700 1 -- 172.21.15.107:0/2006131599 shutdown_connections 2023-07-21T22:27:49.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.925+0000 7f2ac379b700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2abc100050 0x7f2abc19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.925+0000 7f2ac379b700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc0fec10 0x7f2abc195890 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.925+0000 7f2ac379b700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2aa80a11a0 0x7f2aa80a3650 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.925+0000 7f2ac379b700 1 --2- 172.21.15.107:0/2006131599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2abc0ff5c0 0x7f2abc195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:49.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.925+0000 7f2ac379b700 1 -- 172.21.15.107:0/2006131599 >> 172.21.15.107:0/2006131599 conn(0x7f2abc0f9770 msgr2=0x7f2abc0fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:49.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.925+0000 7f2ac379b700 1 -- 172.21.15.107:0/2006131599 shutdown_connections 2023-07-21T22:27:49.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:49.925+0000 7f2ac379b700 1 -- 172.21.15.107:0/2006131599 wait complete. 2023-07-21T22:27:49.983 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-21T22:27:49.984 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:27:50.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.067+0000 7fae1bd13700 1 -- 172.21.15.107:0/2268212189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae140fe9f0 msgr2=0x7fae140fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.067+0000 7fae1bd13700 1 --2- 172.21.15.107:0/2268212189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae140fe9f0 0x7fae140fee00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fae04009990 tx=0x7fae04009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.067+0000 7fae1bd13700 1 -- 172.21.15.107:0/2268212189 shutdown_connections 2023-07-21T22:27:50.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.067+0000 7fae1bd13700 1 --2- 172.21.15.107:0/2268212189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae14100570 0x7fae14106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.067+0000 7fae1bd13700 1 --2- 172.21.15.107:0/2268212189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae140fe9f0 0x7fae140fee00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.067+0000 7fae1bd13700 1 --2- 172.21.15.107:0/2268212189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae140ffbe0 0x7fae14100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.067+0000 7fae1bd13700 1 -- 172.21.15.107:0/2268212189 >> 172.21.15.107:0/2268212189 conn(0x7fae140f9fd0 msgr2=0x7fae140fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:50.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.067+0000 7fae1bd13700 1 -- 172.21.15.107:0/2268212189 shutdown_connections 2023-07-21T22:27:50.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.067+0000 7fae1bd13700 1 -- 172.21.15.107:0/2268212189 wait complete. 2023-07-21T22:27:50.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.068+0000 7fae1bd13700 1 Processor -- start 2023-07-21T22:27:50.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.068+0000 7fae1bd13700 1 -- start start 2023-07-21T22:27:50.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.068+0000 7fae1bd13700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae140fe9f0 0x7fae141946c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.068+0000 7fae1bd13700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae140ffbe0 0x7fae14194c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.068+0000 7fae1bd13700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae14100570 0x7fae1419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.068+0000 7fae1bd13700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1419c240 con 0x7fae14100570 2023-07-21T22:27:50.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.068+0000 7fae1bd13700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1419c380 con 0x7fae140fe9f0 2023-07-21T22:27:50.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.068+0000 7fae1bd13700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1419c510 con 0x7fae140ffbe0 2023-07-21T22:27:50.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.068+0000 7fae1a2b0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae14100570 0x7fae1419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.068+0000 7fae1a2b0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae14100570 0x7fae1419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46642/0 (socket says 172.21.15.107:46642) 2023-07-21T22:27:50.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.068+0000 7fae1a2b0700 1 -- 172.21.15.107:0/3407647124 learned_addr learned my addr 172.21.15.107:0/3407647124 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:50.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.068+0000 7fae19aaf700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae140fe9f0 0x7fae141946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.068+0000 7fae192ae700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae140ffbe0 0x7fae14194c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.069+0000 7fae1a2b0700 1 -- 172.21.15.107:0/3407647124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae140ffbe0 msgr2=0x7fae14194c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.069+0000 7fae1a2b0700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae140ffbe0 0x7fae14194c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.069+0000 7fae1a2b0700 1 -- 172.21.15.107:0/3407647124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae140fe9f0 msgr2=0x7fae141946c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.069+0000 7fae1a2b0700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae140fe9f0 0x7fae141946c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.069+0000 7fae1a2b0700 1 -- 172.21.15.107:0/3407647124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae040095f0 con 0x7fae14100570 2023-07-21T22:27:50.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.069+0000 7fae1a2b0700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae14100570 0x7fae1419bc70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fae0c007ee0 tx=0x7fae0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:50.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.069+0000 7fae0affd700 1 -- 172.21.15.107:0/3407647124 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c017070 con 0x7fae14100570 2023-07-21T22:27:50.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.069+0000 7fae0affd700 1 -- 172.21.15.107:0/3407647124 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae0c00f040 con 0x7fae14100570 2023-07-21T22:27:50.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.069+0000 7fae1bd13700 1 -- 172.21.15.107:0/3407647124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1419c7f0 con 0x7fae14100570 2023-07-21T22:27:50.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.069+0000 7fae0affd700 1 -- 172.21.15.107:0/3407647124 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c005580 con 0x7fae14100570 2023-07-21T22:27:50.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.069+0000 7fae1bd13700 1 -- 172.21.15.107:0/3407647124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1419cd00 con 0x7fae14100570 2023-07-21T22:27:50.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.070+0000 7fae1bd13700 1 -- 172.21.15.107:0/3407647124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae14105ee0 con 0x7fae14100570 2023-07-21T22:27:50.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.071+0000 7fae0affd700 1 -- 172.21.15.107:0/3407647124 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fae0c00db20 con 0x7fae14100570 2023-07-21T22:27:50.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.071+0000 7fae0affd700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fae000a12a0 0x7fae000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.072+0000 7fae19aaf700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fae000a12a0 0x7fae000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.072+0000 7fae0affd700 1 -- 172.21.15.107:0/3407647124 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fae0c0cc0f0 con 0x7fae14100570 2023-07-21T22:27:50.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.072+0000 7fae19aaf700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fae000a12a0 0x7fae000a3750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fae0400b5c0 tx=0x7fae04005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:50.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.076+0000 7fae0affd700 1 -- 172.21.15.107:0/3407647124 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fae0c095790 con 0x7fae14100570 2023-07-21T22:27:50.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.265+0000 7fae1bd13700 1 -- 172.21.15.107:0/3407647124 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae14060950 con 0x7fae000a12a0 2023-07-21T22:27:50.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.275+0000 7fae0affd700 1 -- 172.21.15.107:0/3407647124 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336110 (secure 0 0 0) 0x7fae14060950 con 0x7fae000a12a0 2023-07-21T22:27:50.325 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.278+0000 7fae1bd13700 1 -- 172.21.15.107:0/3407647124 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fae000a12a0 msgr2=0x7fae000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.278+0000 7fae1bd13700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fae000a12a0 0x7fae000a3750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fae0400b5c0 tx=0x7fae04005960 comp rx=0 tx=0).stop 2023-07-21T22:27:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.278+0000 7fae1bd13700 1 -- 172.21.15.107:0/3407647124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae14100570 msgr2=0x7fae1419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.278+0000 7fae1bd13700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae14100570 0x7fae1419bc70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fae0c007ee0 tx=0x7fae0c00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.278+0000 7fae1bd13700 1 -- 172.21.15.107:0/3407647124 shutdown_connections 2023-07-21T22:27:50.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.278+0000 7fae1bd13700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae14100570 0x7fae1419bc70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.278+0000 7fae1bd13700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae140fe9f0 0x7fae141946c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.278+0000 7fae1bd13700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fae000a12a0 0x7fae000a3750 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.278+0000 7fae1bd13700 1 --2- 172.21.15.107:0/3407647124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae140ffbe0 0x7fae14194c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.278+0000 7fae1bd13700 1 -- 172.21.15.107:0/3407647124 >> 172.21.15.107:0/3407647124 conn(0x7fae140f9fd0 msgr2=0x7fae14103020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.278+0000 7fae1bd13700 1 -- 172.21.15.107:0/3407647124 shutdown_connections 2023-07-21T22:27:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.278+0000 7fae1bd13700 1 -- 172.21.15.107:0/3407647124 wait complete. 2023-07-21T22:27:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2023-07-21T22:27:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:50 smithi107 ceph-mon[96543]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:50 smithi107 ceph-mon[96543]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:50 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] up:active 2023-07-21T22:27:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:50 smithi107 ceph-mon[96543]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-07-21T22:27:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:50 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:27:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:50 smithi107 ceph-mon[96543]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:50.337 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:50 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2216732447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:50.337 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:50 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2006131599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:50.338 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2023-07-21T22:27:49.789756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594346,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":145,"num_read_kb":126,"num_write":299,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":279,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335588,"kb_used_data":12164,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606236,"statfs":{"total":1151940427776,"available":1151596785664,"internally_reserved":0,"allocated":12455936,"data_stored":6834145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3978,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":19,"num_read_kb":17,"num_write":118,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002297"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075449+0000","last_change":"2023-07-21T22:27:42.091397+0000","last_active":"2023-07-21T22:27:43.075449+0000","last_peered":"2023-07-21T22:27:43.075449+0000","last_clean":"2023-07-21T22:27:43.075449+0000","last_became_active":"2023-07-21T22:27:42.090836+0000","last_became_peered":"2023-07-21T22:27:42.090836+0000","last_unstale":"2023-07-21T22:27:43.075449+0000","last_undegraded":"2023-07-21T22:27:43.075449+0000","last_fullsized":"2023-07-21T22:27:43.075449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:32:29.476744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957881+0000","last_change":"2023-07-21T22:27:42.097003+0000","last_active":"2023-07-21T22:27:43.957881+0000","last_peered":"2023-07-21T22:27:43.957881+0000","last_clean":"2023-07-21T22:27:43.957881+0000","last_became_active":"2023-07-21T22:27:42.096652+0000","last_became_peered":"2023-07-21T22:27:42.096652+0000","last_unstale":"2023-07-21T22:27:43.957881+0000","last_undegraded":"2023-07-21T22:27:43.957881+0000","last_fullsized":"2023-07-21T22:27:43.957881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:46:04.474133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957863+0000","last_change":"2023-07-21T22:27:42.090090+0000","last_active":"2023-07-21T22:27:43.957863+0000","last_peered":"2023-07-21T22:27:43.957863+0000","last_clean":"2023-07-21T22:27:43.957863+0000","last_became_active":"2023-07-21T22:27:42.089802+0000","last_became_peered":"2023-07-21T22:27:42.089802+0000","last_unstale":"2023-07-21T22:27:43.957863+0000","last_undegraded":"2023-07-21T22:27:43.957863+0000","last_fullsized":"2023-07-21T22:27:43.957863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:48:12.562625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075565+0000","last_change":"2023-07-21T22:27:42.096291+0000","last_active":"2023-07-21T22:27:43.075565+0000","last_peered":"2023-07-21T22:27:43.075565+0000","last_clean":"2023-07-21T22:27:43.075565+0000","last_became_active":"2023-07-21T22:27:42.095982+0000","last_became_peered":"2023-07-21T22:27:42.095982+0000","last_unstale":"2023-07-21T22:27:43.075565+0000","last_undegraded":"2023-07-21T22:27:43.075565+0000","last_fullsized":"2023-07-21T22:27:43.075565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:26:52.394622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375575+0000","last_change":"2023-07-21T22:27:42.097495+0000","last_active":"2023-07-21T22:27:43.375575+0000","last_peered":"2023-07-21T22:27:43.375575+0000","last_clean":"2023-07-21T22:27:43.375575+0000","last_became_active":"2023-07-21T22:27:42.095112+0000","last_became_peered":"2023-07-21T22:27:42.095112+0000","last_unstale":"2023-07-21T22:27:43.375575+0000","last_undegraded":"2023-07-21T22:27:43.375575+0000","last_fullsized":"2023-07-21T22:27:43.375575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:10:05.357425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802495+0000","last_change":"2023-07-21T22:27:42.089880+0000","last_active":"2023-07-21T22:27:44.802495+0000","last_peered":"2023-07-21T22:27:44.802495+0000","last_clean":"2023-07-21T22:27:44.802495+0000","last_became_active":"2023-07-21T22:27:42.088614+0000","last_became_peered":"2023-07-21T22:27:42.088614+0000","last_unstale":"2023-07-21T22:27:44.802495+0000","last_undegraded":"2023-07-21T22:27:44.802495+0000","last_fullsized":"2023-07-21T22:27:44.802495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:11.392499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171618+0000","last_change":"2023-07-21T22:27:42.085457+0000","last_active":"2023-07-21T22:27:44.171618+0000","last_peered":"2023-07-21T22:27:44.171618+0000","last_clean":"2023-07-21T22:27:44.171618+0000","last_became_active":"2023-07-21T22:27:42.085146+0000","last_became_peered":"2023-07-21T22:27:42.085146+0000","last_unstale":"2023-07-21T22:27:44.171618+0000","last_undegraded":"2023-07-21T22:27:44.171618+0000","last_fullsized":"2023-07-21T22:27:44.171618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:33:12.389852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390533+0000","last_change":"2023-07-21T22:27:42.089414+0000","last_active":"2023-07-21T22:27:44.390533+0000","last_peered":"2023-07-21T22:27:44.390533+0000","last_clean":"2023-07-21T22:27:44.390533+0000","last_became_active":"2023-07-21T22:27:42.089099+0000","last_became_peered":"2023-07-21T22:27:42.089099+0000","last_unstale":"2023-07-21T22:27:44.390533+0000","last_undegraded":"2023-07-21T22:27:44.390533+0000","last_fullsized":"2023-07-21T22:27:44.390533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:22:36.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802605+0000","last_change":"2023-07-21T22:27:42.094223+0000","last_active":"2023-07-21T22:27:44.802605+0000","last_peered":"2023-07-21T22:27:44.802605+0000","last_clean":"2023-07-21T22:27:44.802605+0000","last_became_active":"2023-07-21T22:27:42.093875+0000","last_became_peered":"2023-07-21T22:27:42.093875+0000","last_unstale":"2023-07-21T22:27:44.802605+0000","last_undegraded":"2023-07-21T22:27:44.802605+0000","last_fullsized":"2023-07-21T22:27:44.802605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:42:27.835710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.803127+0000","last_change":"2023-07-21T22:27:42.090442+0000","last_active":"2023-07-21T22:27:44.803127+0000","last_peered":"2023-07-21T22:27:44.803127+0000","last_clean":"2023-07-21T22:27:44.803127+0000","last_became_active":"2023-07-21T22:27:42.090103+0000","last_became_peered":"2023-07-21T22:27:42.090103+0000","last_unstale":"2023-07-21T22:27:44.803127+0000","last_undegraded":"2023-07-21T22:27:44.803127+0000","last_fullsized":"2023-07-21T22:27:44.803127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:48:58.891154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171944+0000","last_change":"2023-07-21T22:27:42.098364+0000","last_active":"2023-07-21T22:27:44.171944+0000","last_peered":"2023-07-21T22:27:44.171944+0000","last_clean":"2023-07-21T22:27:44.171944+0000","last_became_active":"2023-07-21T22:27:42.097378+0000","last_became_peered":"2023-07-21T22:27:42.097378+0000","last_unstale":"2023-07-21T22:27:44.171944+0000","last_undegraded":"2023-07-21T22:27:44.171944+0000","last_fullsized":"2023-07-21T22:27:44.171944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:04:59.958741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075627+0000","last_change":"2023-07-21T22:27:42.103110+0000","last_active":"2023-07-21T22:27:43.075627+0000","last_peered":"2023-07-21T22:27:43.075627+0000","last_clean":"2023-07-21T22:27:43.075627+0000","last_became_active":"2023-07-21T22:27:42.102790+0000","last_became_peered":"2023-07-21T22:27:42.102790+0000","last_unstale":"2023-07-21T22:27:43.075627+0000","last_undegraded":"2023-07-21T22:27:43.075627+0000","last_fullsized":"2023-07-21T22:27:43.075627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:15:45.908285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074861+0000","last_change":"2023-07-21T22:27:42.085176+0000","last_active":"2023-07-21T22:27:43.074861+0000","last_peered":"2023-07-21T22:27:43.074861+0000","last_clean":"2023-07-21T22:27:43.074861+0000","last_became_active":"2023-07-21T22:27:42.084721+0000","last_became_peered":"2023-07-21T22:27:42.084721+0000","last_unstale":"2023-07-21T22:27:43.074861+0000","last_undegraded":"2023-07-21T22:27:43.074861+0000","last_fullsized":"2023-07-21T22:27:43.074861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:54:19.144955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957681+0000","last_change":"2023-07-21T22:27:42.084423+0000","last_active":"2023-07-21T22:27:43.957681+0000","last_peered":"2023-07-21T22:27:43.957681+0000","last_clean":"2023-07-21T22:27:43.957681+0000","last_became_active":"2023-07-21T22:27:42.084074+0000","last_became_peered":"2023-07-21T22:27:42.084074+0000","last_unstale":"2023-07-21T22:27:43.957681+0000","last_undegraded":"2023-07-21T22:27:43.957681+0000","last_fullsized":"2023-07-21T22:27:43.957681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:41:12.520325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957649+0000","last_change":"2023-07-21T22:27:42.091542+0000","last_active":"2023-07-21T22:27:43.957649+0000","last_peered":"2023-07-21T22:27:43.957649+0000","last_clean":"2023-07-21T22:27:43.957649+0000","last_became_active":"2023-07-21T22:27:42.091217+0000","last_became_peered":"2023-07-21T22:27:42.091217+0000","last_unstale":"2023-07-21T22:27:43.957649+0000","last_undegraded":"2023-07-21T22:27:43.957649+0000","last_fullsized":"2023-07-21T22:27:43.957649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:45:58.131728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802320+0000","last_change":"2023-07-21T22:27:42.087505+0000","last_active":"2023-07-21T22:27:44.802320+0000","last_peered":"2023-07-21T22:27:44.802320+0000","last_clean":"2023-07-21T22:27:44.802320+0000","last_became_active":"2023-07-21T22:27:42.087177+0000","last_became_peered":"2023-07-21T22:27:42.087177+0000","last_unstale":"2023-07-21T22:27:44.802320+0000","last_undegraded":"2023-07-21T22:27:44.802320+0000","last_fullsized":"2023-07-21T22:27:44.802320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:58:06.502960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075614+0000","last_change":"2023-07-21T22:27:42.091490+0000","last_active":"2023-07-21T22:27:43.075614+0000","last_peered":"2023-07-21T22:27:43.075614+0000","last_clean":"2023-07-21T22:27:43.075614+0000","last_became_active":"2023-07-21T22:27:42.090782+0000","last_became_peered":"2023-07-21T22:27:42.090782+0000","last_unstale":"2023-07-21T22:27:43.075614+0000","last_undegraded":"2023-07-21T22:27:43.075614+0000","last_fullsized":"2023-07-21T22:27:43.075614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:59:46.774654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.303235+0000","last_change":"2023-07-21T22:27:42.092362+0000","last_active":"2023-07-21T22:27:43.303235+0000","last_peered":"2023-07-21T22:27:43.303235+0000","last_clean":"2023-07-21T22:27:43.303235+0000","last_became_active":"2023-07-21T22:27:42.091915+0000","last_became_peered":"2023-07-21T22:27:42.091915+0000","last_unstale":"2023-07-21T22:27:43.303235+0000","last_undegraded":"2023-07-21T22:27:43.303235+0000","last_fullsized":"2023-07-21T22:27:43.303235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:24:03.180394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701430+0000","last_change":"2023-07-21T22:27:42.098959+0000","last_active":"2023-07-21T22:27:43.701430+0000","last_peered":"2023-07-21T22:27:43.701430+0000","last_clean":"2023-07-21T22:27:43.701430+0000","last_became_active":"2023-07-21T22:27:42.094463+0000","last_became_peered":"2023-07-21T22:27:42.094463+0000","last_unstale":"2023-07-21T22:27:43.701430+0000","last_undegraded":"2023-07-21T22:27:43.701430+0000","last_fullsized":"2023-07-21T22:27:43.701430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:51:18.646837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.303212+0000","last_change":"2023-07-21T22:27:41.074299+0000","last_active":"2023-07-21T22:27:43.303212+0000","last_peered":"2023-07-21T22:27:43.303212+0000","last_clean":"2023-07-21T22:27:43.303212+0000","last_became_active":"2023-07-21T22:27:41.072895+0000","last_became_peered":"2023-07-21T22:27:41.072895+0000","last_unstale":"2023-07-21T22:27:43.303212+0000","last_undegraded":"2023-07-21T22:27:43.303212+0000","last_fullsized":"2023-07-21T22:27:43.303212+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:56:42.897768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.391035+0000","last_change":"2023-07-21T22:27:42.090477+0000","last_active":"2023-07-21T22:27:44.391035+0000","last_peered":"2023-07-21T22:27:44.391035+0000","last_clean":"2023-07-21T22:27:44.391035+0000","last_became_active":"2023-07-21T22:27:42.090144+0000","last_became_peered":"2023-07-21T22:27:42.090144+0000","last_unstale":"2023-07-21T22:27:44.391035+0000","last_undegraded":"2023-07-21T22:27:44.391035+0000","last_fullsized":"2023-07-21T22:27:44.391035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:33:32.362360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.391141+0000","last_change":"2023-07-21T22:27:41.075178+0000","last_active":"2023-07-21T22:27:44.391141+0000","last_peered":"2023-07-21T22:27:44.391141+0000","last_clean":"2023-07-21T22:27:44.391141+0000","last_became_active":"2023-07-21T22:27:41.074861+0000","last_became_peered":"2023-07-21T22:27:41.074861+0000","last_unstale":"2023-07-21T22:27:44.391141+0000","last_undegraded":"2023-07-21T22:27:44.391141+0000","last_fullsized":"2023-07-21T22:27:44.391141+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:24:04.625686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.958039+0000","last_change":"2023-07-21T22:27:42.093135+0000","last_active":"2023-07-21T22:27:43.958039+0000","last_peered":"2023-07-21T22:27:43.958039+0000","last_clean":"2023-07-21T22:27:43.958039+0000","last_became_active":"2023-07-21T22:27:42.092457+0000","last_became_peered":"2023-07-21T22:27:42.092457+0000","last_unstale":"2023-07-21T22:27:43.958039+0000","last_undegraded":"2023-07-21T22:27:43.958039+0000","last_fullsized":"2023-07-21T22:27:43.958039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:26:26.440753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.025828+0000","last_change":"2023-07-21T22:27:41.076759+0000","last_active":"2023-07-21T22:27:46.025828+0000","last_peered":"2023-07-21T22:27:46.025828+0000","last_clean":"2023-07-21T22:27:46.025828+0000","last_became_active":"2023-07-21T22:27:41.076376+0000","last_became_peered":"2023-07-21T22:27:41.076376+0000","last_unstale":"2023-07-21T22:27:46.025828+0000","last_undegraded":"2023-07-21T22:27:46.025828+0000","last_fullsized":"2023-07-21T22:27:46.025828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:39:40.849097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802286+0000","last_change":"2023-07-21T22:27:42.093643+0000","last_active":"2023-07-21T22:27:44.802286+0000","last_peered":"2023-07-21T22:27:44.802286+0000","last_clean":"2023-07-21T22:27:44.802286+0000","last_became_active":"2023-07-21T22:27:42.093227+0000","last_became_peered":"2023-07-21T22:27:42.093227+0000","last_unstale":"2023-07-21T22:27:44.802286+0000","last_undegraded":"2023-07-21T22:27:44.802286+0000","last_fullsized":"2023-07-21T22:27:44.802286+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:13:02.515776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375770+0000","last_change":"2023-07-21T22:27:41.073119+0000","last_active":"2023-07-21T22:27:43.375770+0000","last_peered":"2023-07-21T22:27:43.375770+0000","last_clean":"2023-07-21T22:27:43.375770+0000","last_became_active":"2023-07-21T22:27:41.072813+0000","last_became_peered":"2023-07-21T22:27:41.072813+0000","last_unstale":"2023-07-21T22:27:43.375770+0000","last_undegraded":"2023-07-21T22:27:43.375770+0000","last_fullsized":"2023-07-21T22:27:43.375770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:53:43.622644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176697+0000","last_change":"2023-07-21T22:27:42.096474+0000","last_active":"2023-07-21T22:27:44.176697+0000","last_peered":"2023-07-21T22:27:44.176697+0000","last_clean":"2023-07-21T22:27:44.176697+0000","last_became_active":"2023-07-21T22:27:42.096187+0000","last_became_peered":"2023-07-21T22:27:42.096187+0000","last_unstale":"2023-07-21T22:27:44.176697+0000","last_undegraded":"2023-07-21T22:27:44.176697+0000","last_fullsized":"2023-07-21T22:27:44.176697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:47:09.211204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176548+0000","last_change":"2023-07-21T22:27:41.074125+0000","last_active":"2023-07-21T22:27:44.176548+0000","last_peered":"2023-07-21T22:27:44.176548+0000","last_clean":"2023-07-21T22:27:44.176548+0000","last_became_active":"2023-07-21T22:27:41.073829+0000","last_became_peered":"2023-07-21T22:27:41.073829+0000","last_unstale":"2023-07-21T22:27:44.176548+0000","last_undegraded":"2023-07-21T22:27:44.176548+0000","last_fullsized":"2023-07-21T22:27:44.176548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:53:14.194907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375574+0000","last_change":"2023-07-21T22:27:42.082970+0000","last_active":"2023-07-21T22:27:43.375574+0000","last_peered":"2023-07-21T22:27:43.375574+0000","last_clean":"2023-07-21T22:27:43.375574+0000","last_became_active":"2023-07-21T22:27:42.082676+0000","last_became_peered":"2023-07-21T22:27:42.082676+0000","last_unstale":"2023-07-21T22:27:43.375574+0000","last_undegraded":"2023-07-21T22:27:43.375574+0000","last_fullsized":"2023-07-21T22:27:43.375574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:47.852129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.026283+0000","last_change":"2023-07-21T22:27:41.076844+0000","last_active":"2023-07-21T22:27:46.026283+0000","last_peered":"2023-07-21T22:27:46.026283+0000","last_clean":"2023-07-21T22:27:46.026283+0000","last_became_active":"2023-07-21T22:27:41.076487+0000","last_became_peered":"2023-07-21T22:27:41.076487+0000","last_unstale":"2023-07-21T22:27:46.026283+0000","last_undegraded":"2023-07-21T22:27:46.026283+0000","last_fullsized":"2023-07-21T22:27:46.026283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:22:04.537934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176398+0000","last_change":"2023-07-21T22:27:42.089996+0000","last_active":"2023-07-21T22:27:44.176398+0000","last_peered":"2023-07-21T22:27:44.176398+0000","last_clean":"2023-07-21T22:27:44.176398+0000","last_became_active":"2023-07-21T22:27:42.089668+0000","last_became_peered":"2023-07-21T22:27:42.089668+0000","last_unstale":"2023-07-21T22:27:44.176398+0000","last_undegraded":"2023-07-21T22:27:44.176398+0000","last_fullsized":"2023-07-21T22:27:44.176398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:31:09.221434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.024685+0000","last_change":"2023-07-21T22:27:41.075864+0000","last_active":"2023-07-21T22:27:46.024685+0000","last_peered":"2023-07-21T22:27:46.024685+0000","last_clean":"2023-07-21T22:27:46.024685+0000","last_became_active":"2023-07-21T22:27:41.075577+0000","last_became_peered":"2023-07-21T22:27:41.075577+0000","last_unstale":"2023-07-21T22:27:46.024685+0000","last_undegraded":"2023-07-21T22:27:46.024685+0000","last_fullsized":"2023-07-21T22:27:46.024685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:54.831786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390617+0000","last_change":"2023-07-21T22:27:42.091730+0000","last_active":"2023-07-21T22:27:44.390617+0000","last_peered":"2023-07-21T22:27:44.390617+0000","last_clean":"2023-07-21T22:27:44.390617+0000","last_became_active":"2023-07-21T22:27:42.091394+0000","last_became_peered":"2023-07-21T22:27:42.091394+0000","last_unstale":"2023-07-21T22:27:44.390617+0000","last_undegraded":"2023-07-21T22:27:44.390617+0000","last_fullsized":"2023-07-21T22:27:44.390617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:17:17.092150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176132+0000","last_change":"2023-07-21T22:27:41.074822+0000","last_active":"2023-07-21T22:27:44.176132+0000","last_peered":"2023-07-21T22:27:44.176132+0000","last_clean":"2023-07-21T22:27:44.176132+0000","last_became_active":"2023-07-21T22:27:41.074542+0000","last_became_peered":"2023-07-21T22:27:41.074542+0000","last_unstale":"2023-07-21T22:27:44.176132+0000","last_undegraded":"2023-07-21T22:27:44.176132+0000","last_fullsized":"2023-07-21T22:27:44.176132+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:44:23.601406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.302989+0000","last_change":"2023-07-21T22:27:42.094351+0000","last_active":"2023-07-21T22:27:43.302989+0000","last_peered":"2023-07-21T22:27:43.302989+0000","last_clean":"2023-07-21T22:27:43.302989+0000","last_became_active":"2023-07-21T22:27:42.094068+0000","last_became_peered":"2023-07-21T22:27:42.094068+0000","last_unstale":"2023-07-21T22:27:43.302989+0000","last_undegraded":"2023-07-21T22:27:43.302989+0000","last_fullsized":"2023-07-21T22:27:43.302989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:27:50.111150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.122968+0000","last_change":"2023-07-21T22:27:41.079188+0000","last_active":"2023-07-21T22:27:45.122968+0000","last_peered":"2023-07-21T22:27:45.122968+0000","last_clean":"2023-07-21T22:27:45.122968+0000","last_became_active":"2023-07-21T22:27:41.078899+0000","last_became_peered":"2023-07-21T22:27:41.078899+0000","last_unstale":"2023-07-21T22:27:45.122968+0000","last_undegraded":"2023-07-21T22:27:45.122968+0000","last_fullsized":"2023-07-21T22:27:45.122968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:00:37.074692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802626+0000","last_change":"2023-07-21T22:27:42.091194+0000","last_active":"2023-07-21T22:27:44.802626+0000","last_peered":"2023-07-21T22:27:44.802626+0000","last_clean":"2023-07-21T22:27:44.802626+0000","last_became_active":"2023-07-21T22:27:42.090869+0000","last_became_peered":"2023-07-21T22:27:42.090869+0000","last_unstale":"2023-07-21T22:27:44.802626+0000","last_undegraded":"2023-07-21T22:27:44.802626+0000","last_fullsized":"2023-07-21T22:27:44.802626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:26:26.169905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.121596+0000","last_change":"2023-07-21T22:27:41.072701+0000","last_active":"2023-07-21T22:27:45.121596+0000","last_peered":"2023-07-21T22:27:45.121596+0000","last_clean":"2023-07-21T22:27:45.121596+0000","last_became_active":"2023-07-21T22:27:41.072358+0000","last_became_peered":"2023-07-21T22:27:41.072358+0000","last_unstale":"2023-07-21T22:27:45.121596+0000","last_undegraded":"2023-07-21T22:27:45.121596+0000","last_fullsized":"2023-07-21T22:27:45.121596+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:51:15.728117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957974+0000","last_change":"2023-07-21T22:27:42.090808+0000","last_active":"2023-07-21T22:27:43.957974+0000","last_peered":"2023-07-21T22:27:43.957974+0000","last_clean":"2023-07-21T22:27:43.957974+0000","last_became_active":"2023-07-21T22:27:42.090506+0000","last_became_peered":"2023-07-21T22:27:42.090506+0000","last_unstale":"2023-07-21T22:27:43.957974+0000","last_undegraded":"2023-07-21T22:27:43.957974+0000","last_fullsized":"2023-07-21T22:27:43.957974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:59:17.254273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176669+0000","last_change":"2023-07-21T22:27:41.075643+0000","last_active":"2023-07-21T22:27:44.176669+0000","last_peered":"2023-07-21T22:27:44.176669+0000","last_clean":"2023-07-21T22:27:44.176669+0000","last_became_active":"2023-07-21T22:27:41.075336+0000","last_became_peered":"2023-07-21T22:27:41.075336+0000","last_unstale":"2023-07-21T22:27:44.176669+0000","last_undegraded":"2023-07-21T22:27:44.176669+0000","last_fullsized":"2023-07-21T22:27:44.176669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:12:56.006860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802567+0000","last_change":"2023-07-21T22:27:42.096722+0000","last_active":"2023-07-21T22:27:44.802567+0000","last_peered":"2023-07-21T22:27:44.802567+0000","last_clean":"2023-07-21T22:27:44.802567+0000","last_became_active":"2023-07-21T22:27:42.096472+0000","last_became_peered":"2023-07-21T22:27:42.096472+0000","last_unstale":"2023-07-21T22:27:44.802567+0000","last_undegraded":"2023-07-21T22:27:44.802567+0000","last_fullsized":"2023-07-21T22:27:44.802567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:25:16.425955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.015785+0000","last_change":"2023-07-21T22:27:41.072297+0000","last_active":"2023-07-21T22:27:46.015785+0000","last_peered":"2023-07-21T22:27:46.015785+0000","last_clean":"2023-07-21T22:27:46.015785+0000","last_became_active":"2023-07-21T22:27:41.071971+0000","last_became_peered":"2023-07-21T22:27:41.071971+0000","last_unstale":"2023-07-21T22:27:46.015785+0000","last_undegraded":"2023-07-21T22:27:46.015785+0000","last_fullsized":"2023-07-21T22:27:46.015785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:39:54.361110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375731+0000","last_change":"2023-07-21T22:27:42.097130+0000","last_active":"2023-07-21T22:27:43.375731+0000","last_peered":"2023-07-21T22:27:43.375731+0000","last_clean":"2023-07-21T22:27:43.375731+0000","last_became_active":"2023-07-21T22:27:42.096853+0000","last_became_peered":"2023-07-21T22:27:42.096853+0000","last_unstale":"2023-07-21T22:27:43.375731+0000","last_undegraded":"2023-07-21T22:27:43.375731+0000","last_fullsized":"2023-07-21T22:27:43.375731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:36:06.680575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.024570+0000","last_change":"2023-07-21T22:27:41.071755+0000","last_active":"2023-07-21T22:27:46.024570+0000","last_peered":"2023-07-21T22:27:46.024570+0000","last_clean":"2023-07-21T22:27:46.024570+0000","last_became_active":"2023-07-21T22:27:41.071245+0000","last_became_peered":"2023-07-21T22:27:41.071245+0000","last_unstale":"2023-07-21T22:27:46.024570+0000","last_undegraded":"2023-07-21T22:27:46.024570+0000","last_fullsized":"2023-07-21T22:27:46.024570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:09:41.990748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074639+0000","last_change":"2023-07-21T22:27:42.097433+0000","last_active":"2023-07-21T22:27:43.074639+0000","last_peered":"2023-07-21T22:27:43.074639+0000","last_clean":"2023-07-21T22:27:43.074639+0000","last_became_active":"2023-07-21T22:27:42.097100+0000","last_became_peered":"2023-07-21T22:27:42.097100+0000","last_unstale":"2023-07-21T22:27:43.074639+0000","last_undegraded":"2023-07-21T22:27:43.074639+0000","last_fullsized":"2023-07-21T22:27:43.074639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:02:40.214198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176432+0000","last_change":"2023-07-21T22:27:41.074019+0000","last_active":"2023-07-21T22:27:44.176432+0000","last_peered":"2023-07-21T22:27:44.176432+0000","last_clean":"2023-07-21T22:27:44.176432+0000","last_became_active":"2023-07-21T22:27:41.073699+0000","last_became_peered":"2023-07-21T22:27:41.073699+0000","last_unstale":"2023-07-21T22:27:44.176432+0000","last_undegraded":"2023-07-21T22:27:44.176432+0000","last_fullsized":"2023-07-21T22:27:44.176432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:58:24.226793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172010+0000","last_change":"2023-07-21T22:27:42.098010+0000","last_active":"2023-07-21T22:27:44.172010+0000","last_peered":"2023-07-21T22:27:44.172010+0000","last_clean":"2023-07-21T22:27:44.172010+0000","last_became_active":"2023-07-21T22:27:42.097227+0000","last_became_peered":"2023-07-21T22:27:42.097227+0000","last_unstale":"2023-07-21T22:27:44.172010+0000","last_undegraded":"2023-07-21T22:27:44.172010+0000","last_fullsized":"2023-07-21T22:27:44.172010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:49.592034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:49.042497+0000","last_change":"2023-07-21T22:27:41.072298+0000","last_active":"2023-07-21T22:27:49.042497+0000","last_peered":"2023-07-21T22:27:49.042497+0000","last_clean":"2023-07-21T22:27:49.042497+0000","last_became_active":"2023-07-21T22:27:41.071963+0000","last_became_peered":"2023-07-21T22:27:41.071963+0000","last_unstale":"2023-07-21T22:27:49.042497+0000","last_undegraded":"2023-07-21T22:27:49.042497+0000","last_fullsized":"2023-07-21T22:27:49.042497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:47:00.466196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171795+0000","last_change":"2023-07-21T22:27:42.091276+0000","last_active":"2023-07-21T22:27:44.171795+0000","last_peered":"2023-07-21T22:27:44.171795+0000","last_clean":"2023-07-21T22:27:44.171795+0000","last_became_active":"2023-07-21T22:27:42.090918+0000","last_became_peered":"2023-07-21T22:27:42.090918+0000","last_unstale":"2023-07-21T22:27:44.171795+0000","last_undegraded":"2023-07-21T22:27:44.171795+0000","last_fullsized":"2023-07-21T22:27:44.171795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:07:22.664750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.026679+0000","last_change":"2023-07-21T22:27:41.075633+0000","last_active":"2023-07-21T22:27:46.026679+0000","last_peered":"2023-07-21T22:27:46.026679+0000","last_clean":"2023-07-21T22:27:46.026679+0000","last_became_active":"2023-07-21T22:27:41.075331+0000","last_became_peered":"2023-07-21T22:27:41.075331+0000","last_unstale":"2023-07-21T22:27:46.026679+0000","last_undegraded":"2023-07-21T22:27:46.026679+0000","last_fullsized":"2023-07-21T22:27:46.026679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:28:14.600527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176511+0000","last_change":"2023-07-21T22:27:42.089934+0000","last_active":"2023-07-21T22:27:44.176511+0000","last_peered":"2023-07-21T22:27:44.176511+0000","last_clean":"2023-07-21T22:27:44.176511+0000","last_became_active":"2023-07-21T22:27:42.089652+0000","last_became_peered":"2023-07-21T22:27:42.089652+0000","last_unstale":"2023-07-21T22:27:44.176511+0000","last_undegraded":"2023-07-21T22:27:44.176511+0000","last_fullsized":"2023-07-21T22:27:44.176511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:10:40.658561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375856+0000","last_change":"2023-07-21T22:27:41.070862+0000","last_active":"2023-07-21T22:27:43.375856+0000","last_peered":"2023-07-21T22:27:43.375856+0000","last_clean":"2023-07-21T22:27:43.375856+0000","last_became_active":"2023-07-21T22:27:41.070507+0000","last_became_peered":"2023-07-21T22:27:41.070507+0000","last_unstale":"2023-07-21T22:27:43.375856+0000","last_undegraded":"2023-07-21T22:27:43.375856+0000","last_fullsized":"2023-07-21T22:27:43.375856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:28:00.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075397+0000","last_change":"2023-07-21T22:27:42.092046+0000","last_active":"2023-07-21T22:27:43.075397+0000","last_peered":"2023-07-21T22:27:43.075397+0000","last_clean":"2023-07-21T22:27:43.075397+0000","last_became_active":"2023-07-21T22:27:42.091069+0000","last_became_peered":"2023-07-21T22:27:42.091069+0000","last_unstale":"2023-07-21T22:27:43.075397+0000","last_undegraded":"2023-07-21T22:27:43.075397+0000","last_fullsized":"2023-07-21T22:27:43.075397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:39:03.277929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.303093+0000","last_change":"2023-07-21T22:27:41.074847+0000","last_active":"2023-07-21T22:27:43.303093+0000","last_peered":"2023-07-21T22:27:43.303093+0000","last_clean":"2023-07-21T22:27:43.303093+0000","last_became_active":"2023-07-21T22:27:41.074530+0000","last_became_peered":"2023-07-21T22:27:41.074530+0000","last_unstale":"2023-07-21T22:27:43.303093+0000","last_undegraded":"2023-07-21T22:27:43.303093+0000","last_fullsized":"2023-07-21T22:27:43.303093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:54:10.420130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802961+0000","last_change":"2023-07-21T22:27:42.089285+0000","last_active":"2023-07-21T22:27:44.802961+0000","last_peered":"2023-07-21T22:27:44.802961+0000","last_clean":"2023-07-21T22:27:44.802961+0000","last_became_active":"2023-07-21T22:27:42.088970+0000","last_became_peered":"2023-07-21T22:27:42.088970+0000","last_unstale":"2023-07-21T22:27:44.802961+0000","last_undegraded":"2023-07-21T22:27:44.802961+0000","last_fullsized":"2023-07-21T22:27:44.802961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:45:46.101767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171721+0000","last_change":"2023-07-21T22:27:41.077433+0000","last_active":"2023-07-21T22:27:44.171721+0000","last_peered":"2023-07-21T22:27:44.171721+0000","last_clean":"2023-07-21T22:27:44.171721+0000","last_became_active":"2023-07-21T22:27:41.077133+0000","last_became_peered":"2023-07-21T22:27:41.077133+0000","last_unstale":"2023-07-21T22:27:44.171721+0000","last_undegraded":"2023-07-21T22:27:44.171721+0000","last_fullsized":"2023-07-21T22:27:44.171721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:48:14.982483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802503+0000","last_change":"2023-07-21T22:27:42.089833+0000","last_active":"2023-07-21T22:27:44.802503+0000","last_peered":"2023-07-21T22:27:44.802503+0000","last_clean":"2023-07-21T22:27:44.802503+0000","last_became_active":"2023-07-21T22:27:42.089502+0000","last_became_peered":"2023-07-21T22:27:42.089502+0000","last_unstale":"2023-07-21T22:27:44.802503+0000","last_undegraded":"2023-07-21T22:27:44.802503+0000","last_fullsized":"2023-07-21T22:27:44.802503+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:48:49.350741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176463+0000","last_change":"2023-07-21T22:27:42.090033+0000","last_active":"2023-07-21T22:27:44.176463+0000","last_peered":"2023-07-21T22:27:44.176463+0000","last_clean":"2023-07-21T22:27:44.176463+0000","last_became_active":"2023-07-21T22:27:42.089708+0000","last_became_peered":"2023-07-21T22:27:42.089708+0000","last_unstale":"2023-07-21T22:27:44.176463+0000","last_undegraded":"2023-07-21T22:27:44.176463+0000","last_fullsized":"2023-07-21T22:27:44.176463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:52:53.948825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.025934+0000","last_change":"2023-07-21T22:27:41.076784+0000","last_active":"2023-07-21T22:27:46.025934+0000","last_peered":"2023-07-21T22:27:46.025934+0000","last_clean":"2023-07-21T22:27:46.025934+0000","last_became_active":"2023-07-21T22:27:41.076453+0000","last_became_peered":"2023-07-21T22:27:41.076453+0000","last_unstale":"2023-07-21T22:27:46.025934+0000","last_undegraded":"2023-07-21T22:27:46.025934+0000","last_fullsized":"2023-07-21T22:27:46.025934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:32:32.953367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.391049+0000","last_change":"2023-07-21T22:27:42.091286+0000","last_active":"2023-07-21T22:27:44.391049+0000","last_peered":"2023-07-21T22:27:44.391049+0000","last_clean":"2023-07-21T22:27:44.391049+0000","last_became_active":"2023-07-21T22:27:42.090986+0000","last_became_peered":"2023-07-21T22:27:42.090986+0000","last_unstale":"2023-07-21T22:27:44.391049+0000","last_undegraded":"2023-07-21T22:27:44.391049+0000","last_fullsized":"2023-07-21T22:27:44.391049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:07:53.301709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:49.040869+0000","last_change":"2023-07-21T22:27:41.074075+0000","last_active":"2023-07-21T22:27:49.040869+0000","last_peered":"2023-07-21T22:27:49.040869+0000","last_clean":"2023-07-21T22:27:49.040869+0000","last_became_active":"2023-07-21T22:27:41.073757+0000","last_became_peered":"2023-07-21T22:27:41.073757+0000","last_unstale":"2023-07-21T22:27:49.040869+0000","last_undegraded":"2023-07-21T22:27:49.040869+0000","last_fullsized":"2023-07-21T22:27:49.040869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:35:27.586578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171569+0000","last_change":"2023-07-21T22:26:43.405788+0000","last_active":"2023-07-21T22:27:44.171569+0000","last_peered":"2023-07-21T22:27:44.171569+0000","last_clean":"2023-07-21T22:27:44.171569+0000","last_became_active":"2023-07-21T22:26:43.405351+0000","last_became_peered":"2023-07-21T22:26:43.405351+0000","last_unstale":"2023-07-21T22:27:44.171569+0000","last_undegraded":"2023-07-21T22:27:44.171569+0000","last_fullsized":"2023-07-21T22:27:44.171569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_clean_scrub_stamp":"2023-07-21T22:23:52.032210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:54:33.446817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074509+0000","last_change":"2023-07-21T22:27:42.097967+0000","last_active":"2023-07-21T22:27:43.074509+0000","last_peered":"2023-07-21T22:27:43.074509+0000","last_clean":"2023-07-21T22:27:43.074509+0000","last_became_active":"2023-07-21T22:27:42.097708+0000","last_became_peered":"2023-07-21T22:27:42.097708+0000","last_unstale":"2023-07-21T22:27:43.074509+0000","last_undegraded":"2023-07-21T22:27:43.074509+0000","last_fullsized":"2023-07-21T22:27:43.074509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:13:15.609283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.026635+0000","last_change":"2023-07-21T22:27:41.077853+0000","last_active":"2023-07-21T22:27:46.026635+0000","last_peered":"2023-07-21T22:27:46.026635+0000","last_clean":"2023-07-21T22:27:46.026635+0000","last_became_active":"2023-07-21T22:27:41.077539+0000","last_became_peered":"2023-07-21T22:27:41.077539+0000","last_unstale":"2023-07-21T22:27:46.026635+0000","last_undegraded":"2023-07-21T22:27:46.026635+0000","last_fullsized":"2023-07-21T22:27:46.026635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:09:25.539370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.118977+0000","last_change":"2023-07-21T22:27:41.076529+0000","last_active":"2023-07-21T22:27:45.118977+0000","last_peered":"2023-07-21T22:27:45.118977+0000","last_clean":"2023-07-21T22:27:45.118977+0000","last_became_active":"2023-07-21T22:27:41.076208+0000","last_became_peered":"2023-07-21T22:27:41.076208+0000","last_unstale":"2023-07-21T22:27:45.118977+0000","last_undegraded":"2023-07-21T22:27:45.118977+0000","last_fullsized":"2023-07-21T22:27:45.118977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:54:46.984912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074958+0000","last_change":"2023-07-21T22:27:42.082587+0000","last_active":"2023-07-21T22:27:43.074958+0000","last_peered":"2023-07-21T22:27:43.074958+0000","last_clean":"2023-07-21T22:27:43.074958+0000","last_became_active":"2023-07-21T22:27:42.082212+0000","last_became_peered":"2023-07-21T22:27:42.082212+0000","last_unstale":"2023-07-21T22:27:43.074958+0000","last_undegraded":"2023-07-21T22:27:43.074958+0000","last_fullsized":"2023-07-21T22:27:43.074958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:38.746984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.303237+0000","last_change":"2023-07-21T22:27:41.073736+0000","last_active":"2023-07-21T22:27:43.303237+0000","last_peered":"2023-07-21T22:27:43.303237+0000","last_clean":"2023-07-21T22:27:43.303237+0000","last_became_active":"2023-07-21T22:27:41.073213+0000","last_became_peered":"2023-07-21T22:27:41.073213+0000","last_unstale":"2023-07-21T22:27:43.303237+0000","last_undegraded":"2023-07-21T22:27:43.303237+0000","last_fullsized":"2023-07-21T22:27:43.303237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:47:44.847798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390871+0000","last_change":"2023-07-21T22:27:42.091313+0000","last_active":"2023-07-21T22:27:44.390871+0000","last_peered":"2023-07-21T22:27:44.390871+0000","last_clean":"2023-07-21T22:27:44.390871+0000","last_became_active":"2023-07-21T22:27:42.091012+0000","last_became_peered":"2023-07-21T22:27:42.091012+0000","last_unstale":"2023-07-21T22:27:44.390871+0000","last_undegraded":"2023-07-21T22:27:44.390871+0000","last_fullsized":"2023-07-21T22:27:44.390871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:27:43.360142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.041975+0000","last_change":"2023-07-21T22:27:41.074644+0000","last_active":"2023-07-21T22:27:48.041975+0000","last_peered":"2023-07-21T22:27:48.041975+0000","last_clean":"2023-07-21T22:27:48.041975+0000","last_became_active":"2023-07-21T22:27:41.074364+0000","last_became_peered":"2023-07-21T22:27:41.074364+0000","last_unstale":"2023-07-21T22:27:48.041975+0000","last_undegraded":"2023-07-21T22:27:48.041975+0000","last_fullsized":"2023-07-21T22:27:48.041975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:31:09.704935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957978+0000","last_change":"2023-07-21T22:27:42.089452+0000","last_active":"2023-07-21T22:27:43.957978+0000","last_peered":"2023-07-21T22:27:43.957978+0000","last_clean":"2023-07-21T22:27:43.957978+0000","last_became_active":"2023-07-21T22:27:42.088859+0000","last_became_peered":"2023-07-21T22:27:42.088859+0000","last_unstale":"2023-07-21T22:27:43.957978+0000","last_undegraded":"2023-07-21T22:27:43.957978+0000","last_fullsized":"2023-07-21T22:27:43.957978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:17:15.973380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.120703+0000","last_change":"2023-07-21T22:27:41.077067+0000","last_active":"2023-07-21T22:27:45.120703+0000","last_peered":"2023-07-21T22:27:45.120703+0000","last_clean":"2023-07-21T22:27:45.120703+0000","last_became_active":"2023-07-21T22:27:41.076768+0000","last_became_peered":"2023-07-21T22:27:41.076768+0000","last_unstale":"2023-07-21T22:27:45.120703+0000","last_undegraded":"2023-07-21T22:27:45.120703+0000","last_fullsized":"2023-07-21T22:27:45.120703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:28:19.457823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375465+0000","last_change":"2023-07-21T22:27:42.088789+0000","last_active":"2023-07-21T22:27:43.375465+0000","last_peered":"2023-07-21T22:27:43.375465+0000","last_clean":"2023-07-21T22:27:43.375465+0000","last_became_active":"2023-07-21T22:27:42.088361+0000","last_became_peered":"2023-07-21T22:27:42.088361+0000","last_unstale":"2023-07-21T22:27:43.375465+0000","last_undegraded":"2023-07-21T22:27:43.375465+0000","last_fullsized":"2023-07-21T22:27:43.375465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:55:58.658869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.121918+0000","last_change":"2023-07-21T22:27:41.069595+0000","last_active":"2023-07-21T22:27:45.121918+0000","last_peered":"2023-07-21T22:27:45.121918+0000","last_clean":"2023-07-21T22:27:45.121918+0000","last_became_active":"2023-07-21T22:27:41.068733+0000","last_became_peered":"2023-07-21T22:27:41.068733+0000","last_unstale":"2023-07-21T22:27:45.121918+0000","last_undegraded":"2023-07-21T22:27:45.121918+0000","last_fullsized":"2023-07-21T22:27:45.121918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:45:30.220542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075228+0000","last_change":"2023-07-21T22:27:42.085701+0000","last_active":"2023-07-21T22:27:43.075228+0000","last_peered":"2023-07-21T22:27:43.075228+0000","last_clean":"2023-07-21T22:27:43.075228+0000","last_became_active":"2023-07-21T22:27:42.085429+0000","last_became_peered":"2023-07-21T22:27:42.085429+0000","last_unstale":"2023-07-21T22:27:43.075228+0000","last_undegraded":"2023-07-21T22:27:43.075228+0000","last_fullsized":"2023-07-21T22:27:43.075228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:48:06.310953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701092+0000","last_change":"2023-07-21T22:27:41.073233+0000","last_active":"2023-07-21T22:27:43.701092+0000","last_peered":"2023-07-21T22:27:43.701092+0000","last_clean":"2023-07-21T22:27:43.701092+0000","last_became_active":"2023-07-21T22:27:41.072866+0000","last_became_peered":"2023-07-21T22:27:41.072866+0000","last_unstale":"2023-07-21T22:27:43.701092+0000","last_undegraded":"2023-07-21T22:27:43.701092+0000","last_fullsized":"2023-07-21T22:27:43.701092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:24:22.198579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390951+0000","last_change":"2023-07-21T22:27:42.100093+0000","last_active":"2023-07-21T22:27:44.390951+0000","last_peered":"2023-07-21T22:27:44.390951+0000","last_clean":"2023-07-21T22:27:44.390951+0000","last_became_active":"2023-07-21T22:27:42.099813+0000","last_became_peered":"2023-07-21T22:27:42.099813+0000","last_unstale":"2023-07-21T22:27:44.390951+0000","last_undegraded":"2023-07-21T22:27:44.390951+0000","last_fullsized":"2023-07-21T22:27:44.390951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:37:26.901693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.121256+0000","last_change":"2023-07-21T22:27:41.075641+0000","last_active":"2023-07-21T22:27:45.121256+0000","last_peered":"2023-07-21T22:27:45.121256+0000","last_clean":"2023-07-21T22:27:45.121256+0000","last_became_active":"2023-07-21T22:27:41.075268+0000","last_became_peered":"2023-07-21T22:27:41.075268+0000","last_unstale":"2023-07-21T22:27:45.121256+0000","last_undegraded":"2023-07-21T22:27:45.121256+0000","last_fullsized":"2023-07-21T22:27:45.121256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:32.377902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172002+0000","last_change":"2023-07-21T22:27:42.097736+0000","last_active":"2023-07-21T22:27:44.172002+0000","last_peered":"2023-07-21T22:27:44.172002+0000","last_clean":"2023-07-21T22:27:44.172002+0000","last_became_active":"2023-07-21T22:27:42.097419+0000","last_became_peered":"2023-07-21T22:27:42.097419+0000","last_unstale":"2023-07-21T22:27:44.172002+0000","last_undegraded":"2023-07-21T22:27:44.172002+0000","last_fullsized":"2023-07-21T22:27:44.172002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:24:11.626848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.026597+0000","last_change":"2023-07-21T22:27:41.074922+0000","last_active":"2023-07-21T22:27:46.026597+0000","last_peered":"2023-07-21T22:27:46.026597+0000","last_clean":"2023-07-21T22:27:46.026597+0000","last_became_active":"2023-07-21T22:27:41.074610+0000","last_became_peered":"2023-07-21T22:27:41.074610+0000","last_unstale":"2023-07-21T22:27:46.026597+0000","last_undegraded":"2023-07-21T22:27:46.026597+0000","last_fullsized":"2023-07-21T22:27:46.026597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:52:46.091639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172068+0000","last_change":"2023-07-21T22:27:42.097312+0000","last_active":"2023-07-21T22:27:44.172068+0000","last_peered":"2023-07-21T22:27:44.172068+0000","last_clean":"2023-07-21T22:27:44.172068+0000","last_became_active":"2023-07-21T22:27:42.096835+0000","last_became_peered":"2023-07-21T22:27:42.096835+0000","last_unstale":"2023-07-21T22:27:44.172068+0000","last_undegraded":"2023-07-21T22:27:44.172068+0000","last_fullsized":"2023-07-21T22:27:44.172068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:46:51.718896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176528+0000","last_change":"2023-07-21T22:27:41.071971+0000","last_active":"2023-07-21T22:27:44.176528+0000","last_peered":"2023-07-21T22:27:44.176528+0000","last_clean":"2023-07-21T22:27:44.176528+0000","last_became_active":"2023-07-21T22:27:41.071550+0000","last_became_peered":"2023-07-21T22:27:41.071550+0000","last_unstale":"2023-07-21T22:27:44.176528+0000","last_undegraded":"2023-07-21T22:27:44.176528+0000","last_fullsized":"2023-07-21T22:27:44.176528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:21:47.017891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171781+0000","last_change":"2023-07-21T22:27:42.098039+0000","last_active":"2023-07-21T22:27:44.171781+0000","last_peered":"2023-07-21T22:27:44.171781+0000","last_clean":"2023-07-21T22:27:44.171781+0000","last_became_active":"2023-07-21T22:27:42.097737+0000","last_became_peered":"2023-07-21T22:27:42.097737+0000","last_unstale":"2023-07-21T22:27:44.171781+0000","last_undegraded":"2023-07-21T22:27:44.171781+0000","last_fullsized":"2023-07-21T22:27:44.171781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:36:20.727587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075028+0000","last_change":"2023-07-21T22:27:41.076768+0000","last_active":"2023-07-21T22:27:43.075028+0000","last_peered":"2023-07-21T22:27:43.075028+0000","last_clean":"2023-07-21T22:27:43.075028+0000","last_became_active":"2023-07-21T22:27:41.076451+0000","last_became_peered":"2023-07-21T22:27:41.076451+0000","last_unstale":"2023-07-21T22:27:43.075028+0000","last_undegraded":"2023-07-21T22:27:43.075028+0000","last_fullsized":"2023-07-21T22:27:43.075028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:27:42.924064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172018+0000","last_change":"2023-07-21T22:27:42.097891+0000","last_active":"2023-07-21T22:27:44.172018+0000","last_peered":"2023-07-21T22:27:44.172018+0000","last_clean":"2023-07-21T22:27:44.172018+0000","last_became_active":"2023-07-21T22:27:42.097022+0000","last_became_peered":"2023-07-21T22:27:42.097022+0000","last_unstale":"2023-07-21T22:27:44.172018+0000","last_undegraded":"2023-07-21T22:27:44.172018+0000","last_fullsized":"2023-07-21T22:27:44.172018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:04:32.167698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075273+0000","last_change":"2023-07-21T22:27:42.085774+0000","last_active":"2023-07-21T22:27:43.075273+0000","last_peered":"2023-07-21T22:27:43.075273+0000","last_clean":"2023-07-21T22:27:43.075273+0000","last_became_active":"2023-07-21T22:27:42.085460+0000","last_became_peered":"2023-07-21T22:27:42.085460+0000","last_unstale":"2023-07-21T22:27:43.075273+0000","last_undegraded":"2023-07-21T22:27:43.075273+0000","last_fullsized":"2023-07-21T22:27:43.075273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:20.846694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701118+0000","last_change":"2023-07-21T22:27:42.091630+0000","last_active":"2023-07-21T22:27:43.701118+0000","last_peered":"2023-07-21T22:27:43.701118+0000","last_clean":"2023-07-21T22:27:43.701118+0000","last_became_active":"2023-07-21T22:27:42.090538+0000","last_became_peered":"2023-07-21T22:27:42.090538+0000","last_unstale":"2023-07-21T22:27:43.701118+0000","last_undegraded":"2023-07-21T22:27:43.701118+0000","last_fullsized":"2023-07-21T22:27:43.701118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:05:25.473040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701072+0000","last_change":"2023-07-21T22:27:42.092918+0000","last_active":"2023-07-21T22:27:43.701072+0000","last_peered":"2023-07-21T22:27:43.701072+0000","last_clean":"2023-07-21T22:27:43.701072+0000","last_became_active":"2023-07-21T22:27:42.092484+0000","last_became_peered":"2023-07-21T22:27:42.092484+0000","last_unstale":"2023-07-21T22:27:43.701072+0000","last_undegraded":"2023-07-21T22:27:43.701072+0000","last_fullsized":"2023-07-21T22:27:43.701072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:40:43.563394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.958024+0000","last_change":"2023-07-21T22:27:42.096306+0000","last_active":"2023-07-21T22:27:43.958024+0000","last_peered":"2023-07-21T22:27:43.958024+0000","last_clean":"2023-07-21T22:27:43.958024+0000","last_became_active":"2023-07-21T22:27:42.096010+0000","last_became_peered":"2023-07-21T22:27:42.096010+0000","last_unstale":"2023-07-21T22:27:43.958024+0000","last_undegraded":"2023-07-21T22:27:43.958024+0000","last_fullsized":"2023-07-21T22:27:43.958024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:36:42.362478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074992+0000","last_change":"2023-07-21T22:27:42.097964+0000","last_active":"2023-07-21T22:27:43.074992+0000","last_peered":"2023-07-21T22:27:43.074992+0000","last_clean":"2023-07-21T22:27:43.074992+0000","last_became_active":"2023-07-21T22:27:42.097666+0000","last_became_peered":"2023-07-21T22:27:42.097666+0000","last_unstale":"2023-07-21T22:27:43.074992+0000","last_undegraded":"2023-07-21T22:27:43.074992+0000","last_fullsized":"2023-07-21T22:27:43.074992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:42:54.073891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075488+0000","last_change":"2023-07-21T22:27:42.083225+0000","last_active":"2023-07-21T22:27:43.075488+0000","last_peered":"2023-07-21T22:27:43.075488+0000","last_clean":"2023-07-21T22:27:43.075488+0000","last_became_active":"2023-07-21T22:27:42.082892+0000","last_became_peered":"2023-07-21T22:27:42.082892+0000","last_unstale":"2023-07-21T22:27:43.075488+0000","last_undegraded":"2023-07-21T22:27:43.075488+0000","last_fullsized":"2023-07-21T22:27:43.075488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:17:08.372691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701648+0000","last_change":"2023-07-21T22:27:42.090854+0000","last_active":"2023-07-21T22:27:43.701648+0000","last_peered":"2023-07-21T22:27:43.701648+0000","last_clean":"2023-07-21T22:27:43.701648+0000","last_became_active":"2023-07-21T22:27:42.090501+0000","last_became_peered":"2023-07-21T22:27:42.090501+0000","last_unstale":"2023-07-21T22:27:43.701648+0000","last_undegraded":"2023-07-21T22:27:43.701648+0000","last_fullsized":"2023-07-21T22:27:43.701648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:52:51.633954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390858+0000","last_change":"2023-07-21T22:27:42.094986+0000","last_active":"2023-07-21T22:27:44.390858+0000","last_peered":"2023-07-21T22:27:44.390858+0000","last_clean":"2023-07-21T22:27:44.390858+0000","last_became_active":"2023-07-21T22:27:42.094668+0000","last_became_peered":"2023-07-21T22:27:42.094668+0000","last_unstale":"2023-07-21T22:27:44.390858+0000","last_undegraded":"2023-07-21T22:27:44.390858+0000","last_fullsized":"2023-07-21T22:27:44.390858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:27:19.466686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176209+0000","last_change":"2023-07-21T22:27:42.098033+0000","last_active":"2023-07-21T22:27:44.176209+0000","last_peered":"2023-07-21T22:27:44.176209+0000","last_clean":"2023-07-21T22:27:44.176209+0000","last_became_active":"2023-07-21T22:27:42.097219+0000","last_became_peered":"2023-07-21T22:27:42.097219+0000","last_unstale":"2023-07-21T22:27:44.176209+0000","last_undegraded":"2023-07-21T22:27:44.176209+0000","last_fullsized":"2023-07-21T22:27:44.176209+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:12:40.233218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390721+0000","last_change":"2023-07-21T22:27:42.092136+0000","last_active":"2023-07-21T22:27:44.390721+0000","last_peered":"2023-07-21T22:27:44.390721+0000","last_clean":"2023-07-21T22:27:44.390721+0000","last_became_active":"2023-07-21T22:27:42.091559+0000","last_became_peered":"2023-07-21T22:27:42.091559+0000","last_unstale":"2023-07-21T22:27:44.390721+0000","last_undegraded":"2023-07-21T22:27:44.390721+0000","last_fullsized":"2023-07-21T22:27:44.390721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:30:28.654604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802891+0000","last_change":"2023-07-21T22:27:42.093227+0000","last_active":"2023-07-21T22:27:44.802891+0000","last_peered":"2023-07-21T22:27:44.802891+0000","last_clean":"2023-07-21T22:27:44.802891+0000","last_became_active":"2023-07-21T22:27:42.092918+0000","last_became_peered":"2023-07-21T22:27:42.092918+0000","last_unstale":"2023-07-21T22:27:44.802891+0000","last_undegraded":"2023-07-21T22:27:44.802891+0000","last_fullsized":"2023-07-21T22:27:44.802891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:21:39.475897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074784+0000","last_change":"2023-07-21T22:27:42.086494+0000","last_active":"2023-07-21T22:27:43.074784+0000","last_peered":"2023-07-21T22:27:43.074784+0000","last_clean":"2023-07-21T22:27:43.074784+0000","last_became_active":"2023-07-21T22:27:42.086189+0000","last_became_peered":"2023-07-21T22:27:42.086189+0000","last_unstale":"2023-07-21T22:27:43.074784+0000","last_undegraded":"2023-07-21T22:27:43.074784+0000","last_fullsized":"2023-07-21T22:27:43.074784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:24:31.736443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075473+0000","last_change":"2023-07-21T22:27:42.090874+0000","last_active":"2023-07-21T22:27:43.075473+0000","last_peered":"2023-07-21T22:27:43.075473+0000","last_clean":"2023-07-21T22:27:43.075473+0000","last_became_active":"2023-07-21T22:27:42.090597+0000","last_became_peered":"2023-07-21T22:27:42.090597+0000","last_unstale":"2023-07-21T22:27:43.075473+0000","last_undegraded":"2023-07-21T22:27:43.075473+0000","last_fullsized":"2023-07-21T22:27:43.075473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:10:24.103529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3978,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":19,"num_read_kb":17,"num_write":118,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":13016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481804,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":421667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":10,"up_from":70,"seq":300647710736,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1503232,"data_stored":1013642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":1,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.315},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.40100000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.497}]},{"osd":6,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.51800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.52600000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.55400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":9,"up_from":65,"seq":279172874260,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":421667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.252},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.31}]},{"osd":1,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.33300000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.37}]},{"osd":7,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63800000000000001}]},{"osd":8,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.57299999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.65300000000000002}]}]},{"osd":8,"up_from":57,"seq":244813135896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":421711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79300000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.85399999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.93400000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.874},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.89000000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.97699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":7,"up_from":50,"seq":214748364828,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28400,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716752,"statfs":{"total":95995035648,"available":95965954048,"internally_reserved":0,"allocated":1482752,"data_stored":1012609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53800000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.433,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46200000000000002,"15min":0.46400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58699999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.33900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.86299999999999999,"15min":0.90500000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81999999999999995,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.81100000000000005,"15min":0.83899999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.6859999999999999,"5min":5.7480000000000002,"15min":5.7480000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.83399999999999996,"15min":0.872},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85799999999999998,"5min":5.7649999999999997,"15min":5.7649999999999997},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.38900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.77200000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.497}]},{"osd":5,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67600000000000005,"15min":0.68300000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86599999999999999,"5min":0.873,"15min":0.873},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.42799999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.7800000000000002,"5min":5.7800000000000002,"15min":5.7800000000000002},"last":0.627},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.71699999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.65000000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.58899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":421085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.26300000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.371},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.36899999999999999,"15min":0.35999999999999999},"min":{"1min":0.29699999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.27300000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.55700000000000005,"15min":0.54800000000000004},"min":{"1min":0.40200000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.55700000000000005}]},{"osd":2,"last update":"Fri Jul 21 22:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52600000000000002,"15min":0.51500000000000001},"min":{"1min":0.39200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.55900000000000005,"15min":0.54600000000000004},"min":{"1min":0.47199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57499999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.33600000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.45700000000000002,"15min":0.44400000000000001},"min":{"1min":0.34300000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.40400000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.46800000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.5}]},{"osd":5,"last update":"Fri Jul 21 22:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.51800000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52700000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.621},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.63500000000000001,"15min":0.61799999999999999},"min":{"1min":0.47099999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.72799999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.60799999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.63}]},{"osd":10,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.69999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":422649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.502,"15min":0.47999999999999998},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.50800000000000001,"15min":0.47999999999999998},"min":{"1min":0.47099999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.44400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.36899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82599999999999996,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.432,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.436,"15min":0.41899999999999998},"min":{"1min":0.28599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44600000000000001,"15min":0.43099999999999999},"min":{"1min":0.31,"5min":0.248,"15min":0.248},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.76000000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.41899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.40000000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.54600000000000004,"15min":0.52600000000000002},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.77700000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.747},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.83199999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.82099999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.88100000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":421175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.50700000000000001,"15min":0.48499999999999999},"min":{"1min":0.40899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.746},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.497},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81399999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52300000000000002,"15min":0.52000000000000002},"min":{"1min":0.42299999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52300000000000002,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.89500000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39400000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.44,"15min":0.432},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.66900000000000004,"5min":0.753,"15min":0.753},"last":0.45400000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.83199999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:27:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.499,"15min":0.497},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.71099999999999997,"5min":0.749,"15min":0.749},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.33500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80000000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57499999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66500000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85499999999999998,"5min":1.04,"15min":1.04},"last":0.48499999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:27:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.88},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.84599999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.65500000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.77300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":423951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.46899999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.46999999999999997,"15min":0.45400000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.629}]},{"osd":1,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.5,"15min":0.47599999999999998},"min":{"1min":0.39900000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.80500000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.41699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.69599999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55800000000000005,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.44900000000000001,"15min":0.45200000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46600000000000003}]},{"osd":6,"last update":"Fri Jul 21 22:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84799999999999998,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.754}]},{"osd":7,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.435,"5min":0.432,"15min":0.432},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46899999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.56599999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.72299999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.88500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":420948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.33200000000000002},"min":{"1min":0.29399999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.371,"15min":0.35999999999999999},"min":{"1min":0.30299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57799999999999996,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.47799999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71499999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.41099999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.45300000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.622},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.48299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.80200000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.41899999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.49099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.77300000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.51500000000000001,"15min":0.503},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69999999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66300000000000003,"15min":0.65800000000000003},"min":{"1min":0.498,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.85499999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.64300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:27:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.66400000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.83699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920809,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":421737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.47899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.49299999999999999},"min":{"1min":0.41799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30599999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.37},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.40500000000000003,"15min":0.38800000000000001},"min":{"1min":0.32600000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.59199999999999997,"15min":0.58299999999999996},"min":{"1min":0.38300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.46700000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53100000000000003,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.435,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.75800000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58999999999999997,"15min":0.58099999999999996},"min":{"1min":0.47999999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84399999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68999999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.48399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90300000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.621,"15min":0.60599999999999998},"min":{"1min":0.48299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.61099999999999999,"15min":0.59299999999999997},"min":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.59199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.58299999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.751},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.73899999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.64500000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.70599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757285,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28396,"kb_used_data":1444,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716756,"statfs":{"total":95995035648,"available":95965958144,"internally_reserved":0,"allocated":1478656,"data_stored":1011304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.309,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.374,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.38600000000000001,"15min":0.378},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.70299999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.39900000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94599999999999995,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.42299999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.502,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.89400000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.57199999999999995,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.92800000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.72499999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.95199999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.018},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.96999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:27:50.347 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:27:50.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:50 smithi190 ceph-mon[98334]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:50.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:50 smithi190 ceph-mon[98334]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:50.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:50 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] up:active 2023-07-21T22:27:50.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:50 smithi190 ceph-mon[98334]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-07-21T22:27:50.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:50 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:27:50.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:50 smithi190 ceph-mon[98334]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:50.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:50 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2216732447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:50.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:50 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2006131599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:50.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:50 smithi132 ceph-mon[97863]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:50 smithi132 ceph-mon[97863]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:50 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] up:active 2023-07-21T22:27:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:50 smithi132 ceph-mon[97863]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-07-21T22:27:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:50 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:27:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:50 smithi132 ceph-mon[97863]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:27:50.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:50 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2216732447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:27:50.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:50 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2006131599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:50.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.453+0000 7f2c874da700 1 -- 172.21.15.107:0/3559027421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c80105850 msgr2=0x7f2c80107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.453+0000 7f2c874da700 1 --2- 172.21.15.107:0/3559027421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c80105850 0x7f2c80107c30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800b0a0 tx=0x7f2c7800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.453+0000 7f2c874da700 1 -- 172.21.15.107:0/3559027421 shutdown_connections 2023-07-21T22:27:50.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.453+0000 7f2c874da700 1 --2- 172.21.15.107:0/3559027421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c80105850 0x7f2c80107c30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.453+0000 7f2c874da700 1 --2- 172.21.15.107:0/3559027421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c80100610 0x7f2c801029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.453+0000 7f2c874da700 1 --2- 172.21.15.107:0/3559027421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c80102f30 0x7f2c80105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.453+0000 7f2c874da700 1 -- 172.21.15.107:0/3559027421 >> 172.21.15.107:0/3559027421 conn(0x7f2c800f9fd0 msgr2=0x7f2c800fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:50.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.453+0000 7f2c874da700 1 -- 172.21.15.107:0/3559027421 shutdown_connections 2023-07-21T22:27:50.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.453+0000 7f2c874da700 1 -- 172.21.15.107:0/3559027421 wait complete. 2023-07-21T22:27:50.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c874da700 1 Processor -- start 2023-07-21T22:27:50.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c874da700 1 -- start start 2023-07-21T22:27:50.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c874da700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c80100610 0x7f2c80195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c874da700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c80102f30 0x7f2c80195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c874da700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c80105850 0x7f2c8019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c874da700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8019d380 con 0x7f2c80102f30 2023-07-21T22:27:50.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c874da700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8019d4c0 con 0x7f2c80105850 2023-07-21T22:27:50.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c874da700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8019d650 con 0x7f2c80100610 2023-07-21T22:27:50.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c84a75700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c80102f30 0x7f2c80195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c84a75700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c80102f30 0x7f2c80195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46654/0 (socket says 172.21.15.107:46654) 2023-07-21T22:27:50.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c84a75700 1 -- 172.21.15.107:0/1077147770 learned_addr learned my addr 172.21.15.107:0/1077147770 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:50.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c85276700 1 --2- 172.21.15.107:0/1077147770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c80100610 0x7f2c80195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c84a75700 1 -- 172.21.15.107:0/1077147770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c80100610 msgr2=0x7f2c80195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c84a75700 1 --2- 172.21.15.107:0/1077147770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c80100610 0x7f2c80195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c84a75700 1 -- 172.21.15.107:0/1077147770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c80105850 msgr2=0x7f2c8019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c84a75700 1 --2- 172.21.15.107:0/1077147770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c80105850 0x7f2c8019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c84a75700 1 -- 172.21.15.107:0/1077147770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c78009d00 con 0x7f2c80102f30 2023-07-21T22:27:50.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c84a75700 1 --2- 172.21.15.107:0/1077147770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c80102f30 0x7f2c80195d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00c9b0 tx=0x7f2c7c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:50.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.454+0000 7f2c767fc700 1 -- 172.21.15.107:0/1077147770 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c007b00 con 0x7f2c80102f30 2023-07-21T22:27:50.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.455+0000 7f2c874da700 1 -- 172.21.15.107:0/1077147770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8019d930 con 0x7f2c80102f30 2023-07-21T22:27:50.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.455+0000 7f2c767fc700 1 -- 172.21.15.107:0/1077147770 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c7c007c60 con 0x7f2c80102f30 2023-07-21T22:27:50.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.455+0000 7f2c767fc700 1 -- 172.21.15.107:0/1077147770 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c018660 con 0x7f2c80102f30 2023-07-21T22:27:50.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.455+0000 7f2c874da700 1 -- 172.21.15.107:0/1077147770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8019de40 con 0x7f2c80102f30 2023-07-21T22:27:50.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.456+0000 7f2c874da700 1 -- 172.21.15.107:0/1077147770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c800fbb40 con 0x7f2c80102f30 2023-07-21T22:27:50.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.457+0000 7f2c767fc700 1 -- 172.21.15.107:0/1077147770 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f2c7c00f450 con 0x7f2c80102f30 2023-07-21T22:27:50.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.457+0000 7f2c767fc700 1 --2- 172.21.15.107:0/1077147770 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2c6c0a1220 0x7f2c6c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.457+0000 7f2c85276700 1 --2- 172.21.15.107:0/1077147770 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2c6c0a1220 0x7f2c6c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.457+0000 7f2c767fc700 1 -- 172.21.15.107:0/1077147770 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2c7c0cc8e0 con 0x7f2c80102f30 2023-07-21T22:27:50.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.458+0000 7f2c85276700 1 --2- 172.21.15.107:0/1077147770 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2c6c0a1220 0x7f2c6c0a36d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c70005950 tx=0x7f2c7000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:50.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.462+0000 7f2c767fc700 1 -- 172.21.15.107:0/1077147770 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2c7c096080 con 0x7f2c80102f30 2023-07-21T22:27:50.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.649+0000 7f2c874da700 1 -- 172.21.15.107:0/1077147770 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c80060950 con 0x7f2c6c0a1220 2023-07-21T22:27:50.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.659+0000 7f2c767fc700 1 -- 172.21.15.107:0/1077147770 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336110 (secure 0 0 0) 0x7f2c80060950 con 0x7f2c6c0a1220 2023-07-21T22:27:50.709 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:50.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.662+0000 7f2c874da700 1 -- 172.21.15.107:0/1077147770 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2c6c0a1220 msgr2=0x7f2c6c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.662+0000 7f2c874da700 1 --2- 172.21.15.107:0/1077147770 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2c6c0a1220 0x7f2c6c0a36d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c70005950 tx=0x7f2c7000b410 comp rx=0 tx=0).stop 2023-07-21T22:27:50.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.662+0000 7f2c874da700 1 -- 172.21.15.107:0/1077147770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c80102f30 msgr2=0x7f2c80195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.662+0000 7f2c874da700 1 --2- 172.21.15.107:0/1077147770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c80102f30 0x7f2c80195d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00c9b0 tx=0x7f2c7c00ccc0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.662+0000 7f2c874da700 1 -- 172.21.15.107:0/1077147770 shutdown_connections 2023-07-21T22:27:50.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.662+0000 7f2c874da700 1 --2- 172.21.15.107:0/1077147770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c80105850 0x7f2c8019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.662+0000 7f2c874da700 1 --2- 172.21.15.107:0/1077147770 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2c6c0a1220 0x7f2c6c0a36d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.662+0000 7f2c874da700 1 --2- 172.21.15.107:0/1077147770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c80100610 0x7f2c80195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.662+0000 7f2c874da700 1 --2- 172.21.15.107:0/1077147770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c80102f30 0x7f2c80195d40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.662+0000 7f2c874da700 1 -- 172.21.15.107:0/1077147770 >> 172.21.15.107:0/1077147770 conn(0x7f2c800f9fd0 msgr2=0x7f2c80101270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:50.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.662+0000 7f2c874da700 1 -- 172.21.15.107:0/1077147770 shutdown_connections 2023-07-21T22:27:50.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:50.662+0000 7f2c874da700 1 -- 172.21.15.107:0/1077147770 wait complete. 2023-07-21T22:27:50.713 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2023-07-21T22:27:50.721 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2023-07-21T22:27:49.789756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594346,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":145,"num_read_kb":126,"num_write":299,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":279,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335588,"kb_used_data":12164,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606236,"statfs":{"total":1151940427776,"available":1151596785664,"internally_reserved":0,"allocated":12455936,"data_stored":6834145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3978,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":19,"num_read_kb":17,"num_write":118,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002297"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075449+0000","last_change":"2023-07-21T22:27:42.091397+0000","last_active":"2023-07-21T22:27:43.075449+0000","last_peered":"2023-07-21T22:27:43.075449+0000","last_clean":"2023-07-21T22:27:43.075449+0000","last_became_active":"2023-07-21T22:27:42.090836+0000","last_became_peered":"2023-07-21T22:27:42.090836+0000","last_unstale":"2023-07-21T22:27:43.075449+0000","last_undegraded":"2023-07-21T22:27:43.075449+0000","last_fullsized":"2023-07-21T22:27:43.075449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:32:29.476744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957881+0000","last_change":"2023-07-21T22:27:42.097003+0000","last_active":"2023-07-21T22:27:43.957881+0000","last_peered":"2023-07-21T22:27:43.957881+0000","last_clean":"2023-07-21T22:27:43.957881+0000","last_became_active":"2023-07-21T22:27:42.096652+0000","last_became_peered":"2023-07-21T22:27:42.096652+0000","last_unstale":"2023-07-21T22:27:43.957881+0000","last_undegraded":"2023-07-21T22:27:43.957881+0000","last_fullsized":"2023-07-21T22:27:43.957881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:46:04.474133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957863+0000","last_change":"2023-07-21T22:27:42.090090+0000","last_active":"2023-07-21T22:27:43.957863+0000","last_peered":"2023-07-21T22:27:43.957863+0000","last_clean":"2023-07-21T22:27:43.957863+0000","last_became_active":"2023-07-21T22:27:42.089802+0000","last_became_peered":"2023-07-21T22:27:42.089802+0000","last_unstale":"2023-07-21T22:27:43.957863+0000","last_undegraded":"2023-07-21T22:27:43.957863+0000","last_fullsized":"2023-07-21T22:27:43.957863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:48:12.562625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075565+0000","last_change":"2023-07-21T22:27:42.096291+0000","last_active":"2023-07-21T22:27:43.075565+0000","last_peered":"2023-07-21T22:27:43.075565+0000","last_clean":"2023-07-21T22:27:43.075565+0000","last_became_active":"2023-07-21T22:27:42.095982+0000","last_became_peered":"2023-07-21T22:27:42.095982+0000","last_unstale":"2023-07-21T22:27:43.075565+0000","last_undegraded":"2023-07-21T22:27:43.075565+0000","last_fullsized":"2023-07-21T22:27:43.075565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:26:52.394622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375575+0000","last_change":"2023-07-21T22:27:42.097495+0000","last_active":"2023-07-21T22:27:43.375575+0000","last_peered":"2023-07-21T22:27:43.375575+0000","last_clean":"2023-07-21T22:27:43.375575+0000","last_became_active":"2023-07-21T22:27:42.095112+0000","last_became_peered":"2023-07-21T22:27:42.095112+0000","last_unstale":"2023-07-21T22:27:43.375575+0000","last_undegraded":"2023-07-21T22:27:43.375575+0000","last_fullsized":"2023-07-21T22:27:43.375575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:10:05.357425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802495+0000","last_change":"2023-07-21T22:27:42.089880+0000","last_active":"2023-07-21T22:27:44.802495+0000","last_peered":"2023-07-21T22:27:44.802495+0000","last_clean":"2023-07-21T22:27:44.802495+0000","last_became_active":"2023-07-21T22:27:42.088614+0000","last_became_peered":"2023-07-21T22:27:42.088614+0000","last_unstale":"2023-07-21T22:27:44.802495+0000","last_undegraded":"2023-07-21T22:27:44.802495+0000","last_fullsized":"2023-07-21T22:27:44.802495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:11.392499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171618+0000","last_change":"2023-07-21T22:27:42.085457+0000","last_active":"2023-07-21T22:27:44.171618+0000","last_peered":"2023-07-21T22:27:44.171618+0000","last_clean":"2023-07-21T22:27:44.171618+0000","last_became_active":"2023-07-21T22:27:42.085146+0000","last_became_peered":"2023-07-21T22:27:42.085146+0000","last_unstale":"2023-07-21T22:27:44.171618+0000","last_undegraded":"2023-07-21T22:27:44.171618+0000","last_fullsized":"2023-07-21T22:27:44.171618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:33:12.389852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390533+0000","last_change":"2023-07-21T22:27:42.089414+0000","last_active":"2023-07-21T22:27:44.390533+0000","last_peered":"2023-07-21T22:27:44.390533+0000","last_clean":"2023-07-21T22:27:44.390533+0000","last_became_active":"2023-07-21T22:27:42.089099+0000","last_became_peered":"2023-07-21T22:27:42.089099+0000","last_unstale":"2023-07-21T22:27:44.390533+0000","last_undegraded":"2023-07-21T22:27:44.390533+0000","last_fullsized":"2023-07-21T22:27:44.390533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:22:36.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802605+0000","last_change":"2023-07-21T22:27:42.094223+0000","last_active":"2023-07-21T22:27:44.802605+0000","last_peered":"2023-07-21T22:27:44.802605+0000","last_clean":"2023-07-21T22:27:44.802605+0000","last_became_active":"2023-07-21T22:27:42.093875+0000","last_became_peered":"2023-07-21T22:27:42.093875+0000","last_unstale":"2023-07-21T22:27:44.802605+0000","last_undegraded":"2023-07-21T22:27:44.802605+0000","last_fullsized":"2023-07-21T22:27:44.802605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:42:27.835710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.803127+0000","last_change":"2023-07-21T22:27:42.090442+0000","last_active":"2023-07-21T22:27:44.803127+0000","last_peered":"2023-07-21T22:27:44.803127+0000","last_clean":"2023-07-21T22:27:44.803127+0000","last_became_active":"2023-07-21T22:27:42.090103+0000","last_became_peered":"2023-07-21T22:27:42.090103+0000","last_unstale":"2023-07-21T22:27:44.803127+0000","last_undegraded":"2023-07-21T22:27:44.803127+0000","last_fullsized":"2023-07-21T22:27:44.803127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:48:58.891154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171944+0000","last_change":"2023-07-21T22:27:42.098364+0000","last_active":"2023-07-21T22:27:44.171944+0000","last_peered":"2023-07-21T22:27:44.171944+0000","last_clean":"2023-07-21T22:27:44.171944+0000","last_became_active":"2023-07-21T22:27:42.097378+0000","last_became_peered":"2023-07-21T22:27:42.097378+0000","last_unstale":"2023-07-21T22:27:44.171944+0000","last_undegraded":"2023-07-21T22:27:44.171944+0000","last_fullsized":"2023-07-21T22:27:44.171944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:04:59.958741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075627+0000","last_change":"2023-07-21T22:27:42.103110+0000","last_active":"2023-07-21T22:27:43.075627+0000","last_peered":"2023-07-21T22:27:43.075627+0000","last_clean":"2023-07-21T22:27:43.075627+0000","last_became_active":"2023-07-21T22:27:42.102790+0000","last_became_peered":"2023-07-21T22:27:42.102790+0000","last_unstale":"2023-07-21T22:27:43.075627+0000","last_undegraded":"2023-07-21T22:27:43.075627+0000","last_fullsized":"2023-07-21T22:27:43.075627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:15:45.908285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074861+0000","last_change":"2023-07-21T22:27:42.085176+0000","last_active":"2023-07-21T22:27:43.074861+0000","last_peered":"2023-07-21T22:27:43.074861+0000","last_clean":"2023-07-21T22:27:43.074861+0000","last_became_active":"2023-07-21T22:27:42.084721+0000","last_became_peered":"2023-07-21T22:27:42.084721+0000","last_unstale":"2023-07-21T22:27:43.074861+0000","last_undegraded":"2023-07-21T22:27:43.074861+0000","last_fullsized":"2023-07-21T22:27:43.074861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:54:19.144955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957681+0000","last_change":"2023-07-21T22:27:42.084423+0000","last_active":"2023-07-21T22:27:43.957681+0000","last_peered":"2023-07-21T22:27:43.957681+0000","last_clean":"2023-07-21T22:27:43.957681+0000","last_became_active":"2023-07-21T22:27:42.084074+0000","last_became_peered":"2023-07-21T22:27:42.084074+0000","last_unstale":"2023-07-21T22:27:43.957681+0000","last_undegraded":"2023-07-21T22:27:43.957681+0000","last_fullsized":"2023-07-21T22:27:43.957681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:41:12.520325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957649+0000","last_change":"2023-07-21T22:27:42.091542+0000","last_active":"2023-07-21T22:27:43.957649+0000","last_peered":"2023-07-21T22:27:43.957649+0000","last_clean":"2023-07-21T22:27:43.957649+0000","last_became_active":"2023-07-21T22:27:42.091217+0000","last_became_peered":"2023-07-21T22:27:42.091217+0000","last_unstale":"2023-07-21T22:27:43.957649+0000","last_undegraded":"2023-07-21T22:27:43.957649+0000","last_fullsized":"2023-07-21T22:27:43.957649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:45:58.131728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802320+0000","last_change":"2023-07-21T22:27:42.087505+0000","last_active":"2023-07-21T22:27:44.802320+0000","last_peered":"2023-07-21T22:27:44.802320+0000","last_clean":"2023-07-21T22:27:44.802320+0000","last_became_active":"2023-07-21T22:27:42.087177+0000","last_became_peered":"2023-07-21T22:27:42.087177+0000","last_unstale":"2023-07-21T22:27:44.802320+0000","last_undegraded":"2023-07-21T22:27:44.802320+0000","last_fullsized":"2023-07-21T22:27:44.802320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:58:06.502960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075614+0000","last_change":"2023-07-21T22:27:42.091490+0000","last_active":"2023-07-21T22:27:43.075614+0000","last_peered":"2023-07-21T22:27:43.075614+0000","last_clean":"2023-07-21T22:27:43.075614+0000","last_became_active":"2023-07-21T22:27:42.090782+0000","last_became_peered":"2023-07-21T22:27:42.090782+0000","last_unstale":"2023-07-21T22:27:43.075614+0000","last_undegraded":"2023-07-21T22:27:43.075614+0000","last_fullsized":"2023-07-21T22:27:43.075614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:59:46.774654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.303235+0000","last_change":"2023-07-21T22:27:42.092362+0000","last_active":"2023-07-21T22:27:43.303235+0000","last_peered":"2023-07-21T22:27:43.303235+0000","last_clean":"2023-07-21T22:27:43.303235+0000","last_became_active":"2023-07-21T22:27:42.091915+0000","last_became_peered":"2023-07-21T22:27:42.091915+0000","last_unstale":"2023-07-21T22:27:43.303235+0000","last_undegraded":"2023-07-21T22:27:43.303235+0000","last_fullsized":"2023-07-21T22:27:43.303235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:24:03.180394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701430+0000","last_change":"2023-07-21T22:27:42.098959+0000","last_active":"2023-07-21T22:27:43.701430+0000","last_peered":"2023-07-21T22:27:43.701430+0000","last_clean":"2023-07-21T22:27:43.701430+0000","last_became_active":"2023-07-21T22:27:42.094463+0000","last_became_peered":"2023-07-21T22:27:42.094463+0000","last_unstale":"2023-07-21T22:27:43.701430+0000","last_undegraded":"2023-07-21T22:27:43.701430+0000","last_fullsized":"2023-07-21T22:27:43.701430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:51:18.646837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.303212+0000","last_change":"2023-07-21T22:27:41.074299+0000","last_active":"2023-07-21T22:27:43.303212+0000","last_peered":"2023-07-21T22:27:43.303212+0000","last_clean":"2023-07-21T22:27:43.303212+0000","last_became_active":"2023-07-21T22:27:41.072895+0000","last_became_peered":"2023-07-21T22:27:41.072895+0000","last_unstale":"2023-07-21T22:27:43.303212+0000","last_undegraded":"2023-07-21T22:27:43.303212+0000","last_fullsized":"2023-07-21T22:27:43.303212+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:56:42.897768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.391035+0000","last_change":"2023-07-21T22:27:42.090477+0000","last_active":"2023-07-21T22:27:44.391035+0000","last_peered":"2023-07-21T22:27:44.391035+0000","last_clean":"2023-07-21T22:27:44.391035+0000","last_became_active":"2023-07-21T22:27:42.090144+0000","last_became_peered":"2023-07-21T22:27:42.090144+0000","last_unstale":"2023-07-21T22:27:44.391035+0000","last_undegraded":"2023-07-21T22:27:44.391035+0000","last_fullsized":"2023-07-21T22:27:44.391035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:33:32.362360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.391141+0000","last_change":"2023-07-21T22:27:41.075178+0000","last_active":"2023-07-21T22:27:44.391141+0000","last_peered":"2023-07-21T22:27:44.391141+0000","last_clean":"2023-07-21T22:27:44.391141+0000","last_became_active":"2023-07-21T22:27:41.074861+0000","last_became_peered":"2023-07-21T22:27:41.074861+0000","last_unstale":"2023-07-21T22:27:44.391141+0000","last_undegraded":"2023-07-21T22:27:44.391141+0000","last_fullsized":"2023-07-21T22:27:44.391141+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:24:04.625686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.958039+0000","last_change":"2023-07-21T22:27:42.093135+0000","last_active":"2023-07-21T22:27:43.958039+0000","last_peered":"2023-07-21T22:27:43.958039+0000","last_clean":"2023-07-21T22:27:43.958039+0000","last_became_active":"2023-07-21T22:27:42.092457+0000","last_became_peered":"2023-07-21T22:27:42.092457+0000","last_unstale":"2023-07-21T22:27:43.958039+0000","last_undegraded":"2023-07-21T22:27:43.958039+0000","last_fullsized":"2023-07-21T22:27:43.958039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:26:26.440753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.025828+0000","last_change":"2023-07-21T22:27:41.076759+0000","last_active":"2023-07-21T22:27:46.025828+0000","last_peered":"2023-07-21T22:27:46.025828+0000","last_clean":"2023-07-21T22:27:46.025828+0000","last_became_active":"2023-07-21T22:27:41.076376+0000","last_became_peered":"2023-07-21T22:27:41.076376+0000","last_unstale":"2023-07-21T22:27:46.025828+0000","last_undegraded":"2023-07-21T22:27:46.025828+0000","last_fullsized":"2023-07-21T22:27:46.025828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:39:40.849097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802286+0000","last_change":"2023-07-21T22:27:42.093643+0000","last_active":"2023-07-21T22:27:44.802286+0000","last_peered":"2023-07-21T22:27:44.802286+0000","last_clean":"2023-07-21T22:27:44.802286+0000","last_became_active":"2023-07-21T22:27:42.093227+0000","last_became_peered":"2023-07-21T22:27:42.093227+0000","last_unstale":"2023-07-21T22:27:44.802286+0000","last_undegraded":"2023-07-21T22:27:44.802286+0000","last_fullsized":"2023-07-21T22:27:44.802286+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:13:02.515776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375770+0000","last_change":"2023-07-21T22:27:41.073119+0000","last_active":"2023-07-21T22:27:43.375770+0000","last_peered":"2023-07-21T22:27:43.375770+0000","last_clean":"2023-07-21T22:27:43.375770+0000","last_became_active":"2023-07-21T22:27:41.072813+0000","last_became_peered":"2023-07-21T22:27:41.072813+0000","last_unstale":"2023-07-21T22:27:43.375770+0000","last_undegraded":"2023-07-21T22:27:43.375770+0000","last_fullsized":"2023-07-21T22:27:43.375770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:53:43.622644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176697+0000","last_change":"2023-07-21T22:27:42.096474+0000","last_active":"2023-07-21T22:27:44.176697+0000","last_peered":"2023-07-21T22:27:44.176697+0000","last_clean":"2023-07-21T22:27:44.176697+0000","last_became_active":"2023-07-21T22:27:42.096187+0000","last_became_peered":"2023-07-21T22:27:42.096187+0000","last_unstale":"2023-07-21T22:27:44.176697+0000","last_undegraded":"2023-07-21T22:27:44.176697+0000","last_fullsized":"2023-07-21T22:27:44.176697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:47:09.211204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176548+0000","last_change":"2023-07-21T22:27:41.074125+0000","last_active":"2023-07-21T22:27:44.176548+0000","last_peered":"2023-07-21T22:27:44.176548+0000","last_clean":"2023-07-21T22:27:44.176548+0000","last_became_active":"2023-07-21T22:27:41.073829+0000","last_became_peered":"2023-07-21T22:27:41.073829+0000","last_unstale":"2023-07-21T22:27:44.176548+0000","last_undegraded":"2023-07-21T22:27:44.176548+0000","last_fullsized":"2023-07-21T22:27:44.176548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:53:14.194907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375574+0000","last_change":"2023-07-21T22:27:42.082970+0000","last_active":"2023-07-21T22:27:43.375574+0000","last_peered":"2023-07-21T22:27:43.375574+0000","last_clean":"2023-07-21T22:27:43.375574+0000","last_became_active":"2023-07-21T22:27:42.082676+0000","last_became_peered":"2023-07-21T22:27:42.082676+0000","last_unstale":"2023-07-21T22:27:43.375574+0000","last_undegraded":"2023-07-21T22:27:43.375574+0000","last_fullsized":"2023-07-21T22:27:43.375574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:47.852129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.026283+0000","last_change":"2023-07-21T22:27:41.076844+0000","last_active":"2023-07-21T22:27:46.026283+0000","last_peered":"2023-07-21T22:27:46.026283+0000","last_clean":"2023-07-21T22:27:46.026283+0000","last_became_active":"2023-07-21T22:27:41.076487+0000","last_became_peered":"2023-07-21T22:27:41.076487+0000","last_unstale":"2023-07-21T22:27:46.026283+0000","last_undegraded":"2023-07-21T22:27:46.026283+0000","last_fullsized":"2023-07-21T22:27:46.026283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:22:04.537934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176398+0000","last_change":"2023-07-21T22:27:42.089996+0000","last_active":"2023-07-21T22:27:44.176398+0000","last_peered":"2023-07-21T22:27:44.176398+0000","last_clean":"2023-07-21T22:27:44.176398+0000","last_became_active":"2023-07-21T22:27:42.089668+0000","last_became_peered":"2023-07-21T22:27:42.089668+0000","last_unstale":"2023-07-21T22:27:44.176398+0000","last_undegraded":"2023-07-21T22:27:44.176398+0000","last_fullsized":"2023-07-21T22:27:44.176398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:31:09.221434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.024685+0000","last_change":"2023-07-21T22:27:41.075864+0000","last_active":"2023-07-21T22:27:46.024685+0000","last_peered":"2023-07-21T22:27:46.024685+0000","last_clean":"2023-07-21T22:27:46.024685+0000","last_became_active":"2023-07-21T22:27:41.075577+0000","last_became_peered":"2023-07-21T22:27:41.075577+0000","last_unstale":"2023-07-21T22:27:46.024685+0000","last_undegraded":"2023-07-21T22:27:46.024685+0000","last_fullsized":"2023-07-21T22:27:46.024685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:54.831786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390617+0000","last_change":"2023-07-21T22:27:42.091730+0000","last_active":"2023-07-21T22:27:44.390617+0000","last_peered":"2023-07-21T22:27:44.390617+0000","last_clean":"2023-07-21T22:27:44.390617+0000","last_became_active":"2023-07-21T22:27:42.091394+0000","last_became_peered":"2023-07-21T22:27:42.091394+0000","last_unstale":"2023-07-21T22:27:44.390617+0000","last_undegraded":"2023-07-21T22:27:44.390617+0000","last_fullsized":"2023-07-21T22:27:44.390617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:17:17.092150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176132+0000","last_change":"2023-07-21T22:27:41.074822+0000","last_active":"2023-07-21T22:27:44.176132+0000","last_peered":"2023-07-21T22:27:44.176132+0000","last_clean":"2023-07-21T22:27:44.176132+0000","last_became_active":"2023-07-21T22:27:41.074542+0000","last_became_peered":"2023-07-21T22:27:41.074542+0000","last_unstale":"2023-07-21T22:27:44.176132+0000","last_undegraded":"2023-07-21T22:27:44.176132+0000","last_fullsized":"2023-07-21T22:27:44.176132+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:44:23.601406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.302989+0000","last_change":"2023-07-21T22:27:42.094351+0000","last_active":"2023-07-21T22:27:43.302989+0000","last_peered":"2023-07-21T22:27:43.302989+0000","last_clean":"2023-07-21T22:27:43.302989+0000","last_became_active":"2023-07-21T22:27:42.094068+0000","last_became_peered":"2023-07-21T22:27:42.094068+0000","last_unstale":"2023-07-21T22:27:43.302989+0000","last_undegraded":"2023-07-21T22:27:43.302989+0000","last_fullsized":"2023-07-21T22:27:43.302989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:27:50.111150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.122968+0000","last_change":"2023-07-21T22:27:41.079188+0000","last_active":"2023-07-21T22:27:45.122968+0000","last_peered":"2023-07-21T22:27:45.122968+0000","last_clean":"2023-07-21T22:27:45.122968+0000","last_became_active":"2023-07-21T22:27:41.078899+0000","last_became_peered":"2023-07-21T22:27:41.078899+0000","last_unstale":"2023-07-21T22:27:45.122968+0000","last_undegraded":"2023-07-21T22:27:45.122968+0000","last_fullsized":"2023-07-21T22:27:45.122968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:00:37.074692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802626+0000","last_change":"2023-07-21T22:27:42.091194+0000","last_active":"2023-07-21T22:27:44.802626+0000","last_peered":"2023-07-21T22:27:44.802626+0000","last_clean":"2023-07-21T22:27:44.802626+0000","last_became_active":"2023-07-21T22:27:42.090869+0000","last_became_peered":"2023-07-21T22:27:42.090869+0000","last_unstale":"2023-07-21T22:27:44.802626+0000","last_undegraded":"2023-07-21T22:27:44.802626+0000","last_fullsized":"2023-07-21T22:27:44.802626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:26:26.169905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.121596+0000","last_change":"2023-07-21T22:27:41.072701+0000","last_active":"2023-07-21T22:27:45.121596+0000","last_peered":"2023-07-21T22:27:45.121596+0000","last_clean":"2023-07-21T22:27:45.121596+0000","last_became_active":"2023-07-21T22:27:41.072358+0000","last_became_peered":"2023-07-21T22:27:41.072358+0000","last_unstale":"2023-07-21T22:27:45.121596+0000","last_undegraded":"2023-07-21T22:27:45.121596+0000","last_fullsized":"2023-07-21T22:27:45.121596+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:51:15.728117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957974+0000","last_change":"2023-07-21T22:27:42.090808+0000","last_active":"2023-07-21T22:27:43.957974+0000","last_peered":"2023-07-21T22:27:43.957974+0000","last_clean":"2023-07-21T22:27:43.957974+0000","last_became_active":"2023-07-21T22:27:42.090506+0000","last_became_peered":"2023-07-21T22:27:42.090506+0000","last_unstale":"2023-07-21T22:27:43.957974+0000","last_undegraded":"2023-07-21T22:27:43.957974+0000","last_fullsized":"2023-07-21T22:27:43.957974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:59:17.254273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176669+0000","last_change":"2023-07-21T22:27:41.075643+0000","last_active":"2023-07-21T22:27:44.176669+0000","last_peered":"2023-07-21T22:27:44.176669+0000","last_clean":"2023-07-21T22:27:44.176669+0000","last_became_active":"2023-07-21T22:27:41.075336+0000","last_became_peered":"2023-07-21T22:27:41.075336+0000","last_unstale":"2023-07-21T22:27:44.176669+0000","last_undegraded":"2023-07-21T22:27:44.176669+0000","last_fullsized":"2023-07-21T22:27:44.176669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:12:56.006860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802567+0000","last_change":"2023-07-21T22:27:42.096722+0000","last_active":"2023-07-21T22:27:44.802567+0000","last_peered":"2023-07-21T22:27:44.802567+0000","last_clean":"2023-07-21T22:27:44.802567+0000","last_became_active":"2023-07-21T22:27:42.096472+0000","last_became_peered":"2023-07-21T22:27:42.096472+0000","last_unstale":"2023-07-21T22:27:44.802567+0000","last_undegraded":"2023-07-21T22:27:44.802567+0000","last_fullsized":"2023-07-21T22:27:44.802567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:25:16.425955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.015785+0000","last_change":"2023-07-21T22:27:41.072297+0000","last_active":"2023-07-21T22:27:46.015785+0000","last_peered":"2023-07-21T22:27:46.015785+0000","last_clean":"2023-07-21T22:27:46.015785+0000","last_became_active":"2023-07-21T22:27:41.071971+0000","last_became_peered":"2023-07-21T22:27:41.071971+0000","last_unstale":"2023-07-21T22:27:46.015785+0000","last_undegraded":"2023-07-21T22:27:46.015785+0000","last_fullsized":"2023-07-21T22:27:46.015785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:39:54.361110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375731+0000","last_change":"2023-07-21T22:27:42.097130+0000","last_active":"2023-07-21T22:27:43.375731+0000","last_peered":"2023-07-21T22:27:43.375731+0000","last_clean":"2023-07-21T22:27:43.375731+0000","last_became_active":"2023-07-21T22:27:42.096853+0000","last_became_peered":"2023-07-21T22:27:42.096853+0000","last_unstale":"2023-07-21T22:27:43.375731+0000","last_undegraded":"2023-07-21T22:27:43.375731+0000","last_fullsized":"2023-07-21T22:27:43.375731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:36:06.680575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.024570+0000","last_change":"2023-07-21T22:27:41.071755+0000","last_active":"2023-07-21T22:27:46.024570+0000","last_peered":"2023-07-21T22:27:46.024570+0000","last_clean":"2023-07-21T22:27:46.024570+0000","last_became_active":"2023-07-21T22:27:41.071245+0000","last_became_peered":"2023-07-21T22:27:41.071245+0000","last_unstale":"2023-07-21T22:27:46.024570+0000","last_undegraded":"2023-07-21T22:27:46.024570+0000","last_fullsized":"2023-07-21T22:27:46.024570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:09:41.990748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074639+0000","last_change":"2023-07-21T22:27:42.097433+0000","last_active":"2023-07-21T22:27:43.074639+0000","last_peered":"2023-07-21T22:27:43.074639+0000","last_clean":"2023-07-21T22:27:43.074639+0000","last_became_active":"2023-07-21T22:27:42.097100+0000","last_became_peered":"2023-07-21T22:27:42.097100+0000","last_unstale":"2023-07-21T22:27:43.074639+0000","last_undegraded":"2023-07-21T22:27:43.074639+0000","last_fullsized":"2023-07-21T22:27:43.074639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:02:40.214198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176432+0000","last_change":"2023-07-21T22:27:41.074019+0000","last_active":"2023-07-21T22:27:44.176432+0000","last_peered":"2023-07-21T22:27:44.176432+0000","last_clean":"2023-07-21T22:27:44.176432+0000","last_became_active":"2023-07-21T22:27:41.073699+0000","last_became_peered":"2023-07-21T22:27:41.073699+0000","last_unstale":"2023-07-21T22:27:44.176432+0000","last_undegraded":"2023-07-21T22:27:44.176432+0000","last_fullsized":"2023-07-21T22:27:44.176432+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:58:24.226793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172010+0000","last_change":"2023-07-21T22:27:42.098010+0000","last_active":"2023-07-21T22:27:44.172010+0000","last_peered":"2023-07-21T22:27:44.172010+0000","last_clean":"2023-07-21T22:27:44.172010+0000","last_became_active":"2023-07-21T22:27:42.097227+0000","last_became_peered":"2023-07-21T22:27:42.097227+0000","last_unstale":"2023-07-21T22:27:44.172010+0000","last_undegraded":"2023-07-21T22:27:44.172010+0000","last_fullsized":"2023-07-21T22:27:44.172010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:49.592034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:49.042497+0000","last_change":"2023-07-21T22:27:41.072298+0000","last_active":"2023-07-21T22:27:49.042497+0000","last_peered":"2023-07-21T22:27:49.042497+0000","last_clean":"2023-07-21T22:27:49.042497+0000","last_became_active":"2023-07-21T22:27:41.071963+0000","last_became_peered":"2023-07-21T22:27:41.071963+0000","last_unstale":"2023-07-21T22:27:49.042497+0000","last_undegraded":"2023-07-21T22:27:49.042497+0000","last_fullsized":"2023-07-21T22:27:49.042497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:47:00.466196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171795+0000","last_change":"2023-07-21T22:27:42.091276+0000","last_active":"2023-07-21T22:27:44.171795+0000","last_peered":"2023-07-21T22:27:44.171795+0000","last_clean":"2023-07-21T22:27:44.171795+0000","last_became_active":"2023-07-21T22:27:42.090918+0000","last_became_peered":"2023-07-21T22:27:42.090918+0000","last_unstale":"2023-07-21T22:27:44.171795+0000","last_undegraded":"2023-07-21T22:27:44.171795+0000","last_fullsized":"2023-07-21T22:27:44.171795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:07:22.664750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.026679+0000","last_change":"2023-07-21T22:27:41.075633+0000","last_active":"2023-07-21T22:27:46.026679+0000","last_peered":"2023-07-21T22:27:46.026679+0000","last_clean":"2023-07-21T22:27:46.026679+0000","last_became_active":"2023-07-21T22:27:41.075331+0000","last_became_peered":"2023-07-21T22:27:41.075331+0000","last_unstale":"2023-07-21T22:27:46.026679+0000","last_undegraded":"2023-07-21T22:27:46.026679+0000","last_fullsized":"2023-07-21T22:27:46.026679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:28:14.600527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176511+0000","last_change":"2023-07-21T22:27:42.089934+0000","last_active":"2023-07-21T22:27:44.176511+0000","last_peered":"2023-07-21T22:27:44.176511+0000","last_clean":"2023-07-21T22:27:44.176511+0000","last_became_active":"2023-07-21T22:27:42.089652+0000","last_became_peered":"2023-07-21T22:27:42.089652+0000","last_unstale":"2023-07-21T22:27:44.176511+0000","last_undegraded":"2023-07-21T22:27:44.176511+0000","last_fullsized":"2023-07-21T22:27:44.176511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:10:40.658561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375856+0000","last_change":"2023-07-21T22:27:41.070862+0000","last_active":"2023-07-21T22:27:43.375856+0000","last_peered":"2023-07-21T22:27:43.375856+0000","last_clean":"2023-07-21T22:27:43.375856+0000","last_became_active":"2023-07-21T22:27:41.070507+0000","last_became_peered":"2023-07-21T22:27:41.070507+0000","last_unstale":"2023-07-21T22:27:43.375856+0000","last_undegraded":"2023-07-21T22:27:43.375856+0000","last_fullsized":"2023-07-21T22:27:43.375856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:28:00.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075397+0000","last_change":"2023-07-21T22:27:42.092046+0000","last_active":"2023-07-21T22:27:43.075397+0000","last_peered":"2023-07-21T22:27:43.075397+0000","last_clean":"2023-07-21T22:27:43.075397+0000","last_became_active":"2023-07-21T22:27:42.091069+0000","last_became_peered":"2023-07-21T22:27:42.091069+0000","last_unstale":"2023-07-21T22:27:43.075397+0000","last_undegraded":"2023-07-21T22:27:43.075397+0000","last_fullsized":"2023-07-21T22:27:43.075397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:39:03.277929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.303093+0000","last_change":"2023-07-21T22:27:41.074847+0000","last_active":"2023-07-21T22:27:43.303093+0000","last_peered":"2023-07-21T22:27:43.303093+0000","last_clean":"2023-07-21T22:27:43.303093+0000","last_became_active":"2023-07-21T22:27:41.074530+0000","last_became_peered":"2023-07-21T22:27:41.074530+0000","last_unstale":"2023-07-21T22:27:43.303093+0000","last_undegraded":"2023-07-21T22:27:43.303093+0000","last_fullsized":"2023-07-21T22:27:43.303093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:54:10.420130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802961+0000","last_change":"2023-07-21T22:27:42.089285+0000","last_active":"2023-07-21T22:27:44.802961+0000","last_peered":"2023-07-21T22:27:44.802961+0000","last_clean":"2023-07-21T22:27:44.802961+0000","last_became_active":"2023-07-21T22:27:42.088970+0000","last_became_peered":"2023-07-21T22:27:42.088970+0000","last_unstale":"2023-07-21T22:27:44.802961+0000","last_undegraded":"2023-07-21T22:27:44.802961+0000","last_fullsized":"2023-07-21T22:27:44.802961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:45:46.101767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171721+0000","last_change":"2023-07-21T22:27:41.077433+0000","last_active":"2023-07-21T22:27:44.171721+0000","last_peered":"2023-07-21T22:27:44.171721+0000","last_clean":"2023-07-21T22:27:44.171721+0000","last_became_active":"2023-07-21T22:27:41.077133+0000","last_became_peered":"2023-07-21T22:27:41.077133+0000","last_unstale":"2023-07-21T22:27:44.171721+0000","last_undegraded":"2023-07-21T22:27:44.171721+0000","last_fullsized":"2023-07-21T22:27:44.171721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:48:14.982483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802503+0000","last_change":"2023-07-21T22:27:42.089833+0000","last_active":"2023-07-21T22:27:44.802503+0000","last_peered":"2023-07-21T22:27:44.802503+0000","last_clean":"2023-07-21T22:27:44.802503+0000","last_became_active":"2023-07-21T22:27:42.089502+0000","last_became_peered":"2023-07-21T22:27:42.089502+0000","last_unstale":"2023-07-21T22:27:44.802503+0000","last_undegraded":"2023-07-21T22:27:44.802503+0000","last_fullsized":"2023-07-21T22:27:44.802503+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:48:49.350741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176463+0000","last_change":"2023-07-21T22:27:42.090033+0000","last_active":"2023-07-21T22:27:44.176463+0000","last_peered":"2023-07-21T22:27:44.176463+0000","last_clean":"2023-07-21T22:27:44.176463+0000","last_became_active":"2023-07-21T22:27:42.089708+0000","last_became_peered":"2023-07-21T22:27:42.089708+0000","last_unstale":"2023-07-21T22:27:44.176463+0000","last_undegraded":"2023-07-21T22:27:44.176463+0000","last_fullsized":"2023-07-21T22:27:44.176463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:52:53.948825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.025934+0000","last_change":"2023-07-21T22:27:41.076784+0000","last_active":"2023-07-21T22:27:46.025934+0000","last_peered":"2023-07-21T22:27:46.025934+0000","last_clean":"2023-07-21T22:27:46.025934+0000","last_became_active":"2023-07-21T22:27:41.076453+0000","last_became_peered":"2023-07-21T22:27:41.076453+0000","last_unstale":"2023-07-21T22:27:46.025934+0000","last_undegraded":"2023-07-21T22:27:46.025934+0000","last_fullsized":"2023-07-21T22:27:46.025934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:32:32.953367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.391049+0000","last_change":"2023-07-21T22:27:42.091286+0000","last_active":"2023-07-21T22:27:44.391049+0000","last_peered":"2023-07-21T22:27:44.391049+0000","last_clean":"2023-07-21T22:27:44.391049+0000","last_became_active":"2023-07-21T22:27:42.090986+0000","last_became_peered":"2023-07-21T22:27:42.090986+0000","last_unstale":"2023-07-21T22:27:44.391049+0000","last_undegraded":"2023-07-21T22:27:44.391049+0000","last_fullsized":"2023-07-21T22:27:44.391049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:07:53.301709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:49.040869+0000","last_change":"2023-07-21T22:27:41.074075+0000","last_active":"2023-07-21T22:27:49.040869+0000","last_peered":"2023-07-21T22:27:49.040869+0000","last_clean":"2023-07-21T22:27:49.040869+0000","last_became_active":"2023-07-21T22:27:41.073757+0000","last_became_peered":"2023-07-21T22:27:41.073757+0000","last_unstale":"2023-07-21T22:27:49.040869+0000","last_undegraded":"2023-07-21T22:27:49.040869+0000","last_fullsized":"2023-07-21T22:27:49.040869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:35:27.586578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171569+0000","last_change":"2023-07-21T22:26:43.405788+0000","last_active":"2023-07-21T22:27:44.171569+0000","last_peered":"2023-07-21T22:27:44.171569+0000","last_clean":"2023-07-21T22:27:44.171569+0000","last_became_active":"2023-07-21T22:26:43.405351+0000","last_became_peered":"2023-07-21T22:26:43.405351+0000","last_unstale":"2023-07-21T22:27:44.171569+0000","last_undegraded":"2023-07-21T22:27:44.171569+0000","last_fullsized":"2023-07-21T22:27:44.171569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_clean_scrub_stamp":"2023-07-21T22:23:52.032210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:54:33.446817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074509+0000","last_change":"2023-07-21T22:27:42.097967+0000","last_active":"2023-07-21T22:27:43.074509+0000","last_peered":"2023-07-21T22:27:43.074509+0000","last_clean":"2023-07-21T22:27:43.074509+0000","last_became_active":"2023-07-21T22:27:42.097708+0000","last_became_peered":"2023-07-21T22:27:42.097708+0000","last_unstale":"2023-07-21T22:27:43.074509+0000","last_undegraded":"2023-07-21T22:27:43.074509+0000","last_fullsized":"2023-07-21T22:27:43.074509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:13:15.609283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.026635+0000","last_change":"2023-07-21T22:27:41.077853+0000","last_active":"2023-07-21T22:27:46.026635+0000","last_peered":"2023-07-21T22:27:46.026635+0000","last_clean":"2023-07-21T22:27:46.026635+0000","last_became_active":"2023-07-21T22:27:41.077539+0000","last_became_peered":"2023-07-21T22:27:41.077539+0000","last_unstale":"2023-07-21T22:27:46.026635+0000","last_undegraded":"2023-07-21T22:27:46.026635+0000","last_fullsized":"2023-07-21T22:27:46.026635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:09:25.539370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.118977+0000","last_change":"2023-07-21T22:27:41.076529+0000","last_active":"2023-07-21T22:27:45.118977+0000","last_peered":"2023-07-21T22:27:45.118977+0000","last_clean":"2023-07-21T22:27:45.118977+0000","last_became_active":"2023-07-21T22:27:41.076208+0000","last_became_peered":"2023-07-21T22:27:41.076208+0000","last_unstale":"2023-07-21T22:27:45.118977+0000","last_undegraded":"2023-07-21T22:27:45.118977+0000","last_fullsized":"2023-07-21T22:27:45.118977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:54:46.984912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074958+0000","last_change":"2023-07-21T22:27:42.082587+0000","last_active":"2023-07-21T22:27:43.074958+0000","last_peered":"2023-07-21T22:27:43.074958+0000","last_clean":"2023-07-21T22:27:43.074958+0000","last_became_active":"2023-07-21T22:27:42.082212+0000","last_became_peered":"2023-07-21T22:27:42.082212+0000","last_unstale":"2023-07-21T22:27:43.074958+0000","last_undegraded":"2023-07-21T22:27:43.074958+0000","last_fullsized":"2023-07-21T22:27:43.074958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:38.746984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.303237+0000","last_change":"2023-07-21T22:27:41.073736+0000","last_active":"2023-07-21T22:27:43.303237+0000","last_peered":"2023-07-21T22:27:43.303237+0000","last_clean":"2023-07-21T22:27:43.303237+0000","last_became_active":"2023-07-21T22:27:41.073213+0000","last_became_peered":"2023-07-21T22:27:41.073213+0000","last_unstale":"2023-07-21T22:27:43.303237+0000","last_undegraded":"2023-07-21T22:27:43.303237+0000","last_fullsized":"2023-07-21T22:27:43.303237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:47:44.847798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390871+0000","last_change":"2023-07-21T22:27:42.091313+0000","last_active":"2023-07-21T22:27:44.390871+0000","last_peered":"2023-07-21T22:27:44.390871+0000","last_clean":"2023-07-21T22:27:44.390871+0000","last_became_active":"2023-07-21T22:27:42.091012+0000","last_became_peered":"2023-07-21T22:27:42.091012+0000","last_unstale":"2023-07-21T22:27:44.390871+0000","last_undegraded":"2023-07-21T22:27:44.390871+0000","last_fullsized":"2023-07-21T22:27:44.390871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:27:43.360142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.041975+0000","last_change":"2023-07-21T22:27:41.074644+0000","last_active":"2023-07-21T22:27:48.041975+0000","last_peered":"2023-07-21T22:27:48.041975+0000","last_clean":"2023-07-21T22:27:48.041975+0000","last_became_active":"2023-07-21T22:27:41.074364+0000","last_became_peered":"2023-07-21T22:27:41.074364+0000","last_unstale":"2023-07-21T22:27:48.041975+0000","last_undegraded":"2023-07-21T22:27:48.041975+0000","last_fullsized":"2023-07-21T22:27:48.041975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:31:09.704935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957978+0000","last_change":"2023-07-21T22:27:42.089452+0000","last_active":"2023-07-21T22:27:43.957978+0000","last_peered":"2023-07-21T22:27:43.957978+0000","last_clean":"2023-07-21T22:27:43.957978+0000","last_became_active":"2023-07-21T22:27:42.088859+0000","last_became_peered":"2023-07-21T22:27:42.088859+0000","last_unstale":"2023-07-21T22:27:43.957978+0000","last_undegraded":"2023-07-21T22:27:43.957978+0000","last_fullsized":"2023-07-21T22:27:43.957978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:17:15.973380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.120703+0000","last_change":"2023-07-21T22:27:41.077067+0000","last_active":"2023-07-21T22:27:45.120703+0000","last_peered":"2023-07-21T22:27:45.120703+0000","last_clean":"2023-07-21T22:27:45.120703+0000","last_became_active":"2023-07-21T22:27:41.076768+0000","last_became_peered":"2023-07-21T22:27:41.076768+0000","last_unstale":"2023-07-21T22:27:45.120703+0000","last_undegraded":"2023-07-21T22:27:45.120703+0000","last_fullsized":"2023-07-21T22:27:45.120703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:28:19.457823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375465+0000","last_change":"2023-07-21T22:27:42.088789+0000","last_active":"2023-07-21T22:27:43.375465+0000","last_peered":"2023-07-21T22:27:43.375465+0000","last_clean":"2023-07-21T22:27:43.375465+0000","last_became_active":"2023-07-21T22:27:42.088361+0000","last_became_peered":"2023-07-21T22:27:42.088361+0000","last_unstale":"2023-07-21T22:27:43.375465+0000","last_undegraded":"2023-07-21T22:27:43.375465+0000","last_fullsized":"2023-07-21T22:27:43.375465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:55:58.658869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.121918+0000","last_change":"2023-07-21T22:27:41.069595+0000","last_active":"2023-07-21T22:27:45.121918+0000","last_peered":"2023-07-21T22:27:45.121918+0000","last_clean":"2023-07-21T22:27:45.121918+0000","last_became_active":"2023-07-21T22:27:41.068733+0000","last_became_peered":"2023-07-21T22:27:41.068733+0000","last_unstale":"2023-07-21T22:27:45.121918+0000","last_undegraded":"2023-07-21T22:27:45.121918+0000","last_fullsized":"2023-07-21T22:27:45.121918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:45:30.220542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075228+0000","last_change":"2023-07-21T22:27:42.085701+0000","last_active":"2023-07-21T22:27:43.075228+0000","last_peered":"2023-07-21T22:27:43.075228+0000","last_clean":"2023-07-21T22:27:43.075228+0000","last_became_active":"2023-07-21T22:27:42.085429+0000","last_became_peered":"2023-07-21T22:27:42.085429+0000","last_unstale":"2023-07-21T22:27:43.075228+0000","last_undegraded":"2023-07-21T22:27:43.075228+0000","last_fullsized":"2023-07-21T22:27:43.075228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:48:06.310953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701092+0000","last_change":"2023-07-21T22:27:41.073233+0000","last_active":"2023-07-21T22:27:43.701092+0000","last_peered":"2023-07-21T22:27:43.701092+0000","last_clean":"2023-07-21T22:27:43.701092+0000","last_became_active":"2023-07-21T22:27:41.072866+0000","last_became_peered":"2023-07-21T22:27:41.072866+0000","last_unstale":"2023-07-21T22:27:43.701092+0000","last_undegraded":"2023-07-21T22:27:43.701092+0000","last_fullsized":"2023-07-21T22:27:43.701092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:24:22.198579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390951+0000","last_change":"2023-07-21T22:27:42.100093+0000","last_active":"2023-07-21T22:27:44.390951+0000","last_peered":"2023-07-21T22:27:44.390951+0000","last_clean":"2023-07-21T22:27:44.390951+0000","last_became_active":"2023-07-21T22:27:42.099813+0000","last_became_peered":"2023-07-21T22:27:42.099813+0000","last_unstale":"2023-07-21T22:27:44.390951+0000","last_undegraded":"2023-07-21T22:27:44.390951+0000","last_fullsized":"2023-07-21T22:27:44.390951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:37:26.901693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.121256+0000","last_change":"2023-07-21T22:27:41.075641+0000","last_active":"2023-07-21T22:27:45.121256+0000","last_peered":"2023-07-21T22:27:45.121256+0000","last_clean":"2023-07-21T22:27:45.121256+0000","last_became_active":"2023-07-21T22:27:41.075268+0000","last_became_peered":"2023-07-21T22:27:41.075268+0000","last_unstale":"2023-07-21T22:27:45.121256+0000","last_undegraded":"2023-07-21T22:27:45.121256+0000","last_fullsized":"2023-07-21T22:27:45.121256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:32.377902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172002+0000","last_change":"2023-07-21T22:27:42.097736+0000","last_active":"2023-07-21T22:27:44.172002+0000","last_peered":"2023-07-21T22:27:44.172002+0000","last_clean":"2023-07-21T22:27:44.172002+0000","last_became_active":"2023-07-21T22:27:42.097419+0000","last_became_peered":"2023-07-21T22:27:42.097419+0000","last_unstale":"2023-07-21T22:27:44.172002+0000","last_undegraded":"2023-07-21T22:27:44.172002+0000","last_fullsized":"2023-07-21T22:27:44.172002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:24:11.626848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.026597+0000","last_change":"2023-07-21T22:27:41.074922+0000","last_active":"2023-07-21T22:27:46.026597+0000","last_peered":"2023-07-21T22:27:46.026597+0000","last_clean":"2023-07-21T22:27:46.026597+0000","last_became_active":"2023-07-21T22:27:41.074610+0000","last_became_peered":"2023-07-21T22:27:41.074610+0000","last_unstale":"2023-07-21T22:27:46.026597+0000","last_undegraded":"2023-07-21T22:27:46.026597+0000","last_fullsized":"2023-07-21T22:27:46.026597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:52:46.091639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172068+0000","last_change":"2023-07-21T22:27:42.097312+0000","last_active":"2023-07-21T22:27:44.172068+0000","last_peered":"2023-07-21T22:27:44.172068+0000","last_clean":"2023-07-21T22:27:44.172068+0000","last_became_active":"2023-07-21T22:27:42.096835+0000","last_became_peered":"2023-07-21T22:27:42.096835+0000","last_unstale":"2023-07-21T22:27:44.172068+0000","last_undegraded":"2023-07-21T22:27:44.172068+0000","last_fullsized":"2023-07-21T22:27:44.172068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:46:51.718896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176528+0000","last_change":"2023-07-21T22:27:41.071971+0000","last_active":"2023-07-21T22:27:44.176528+0000","last_peered":"2023-07-21T22:27:44.176528+0000","last_clean":"2023-07-21T22:27:44.176528+0000","last_became_active":"2023-07-21T22:27:41.071550+0000","last_became_peered":"2023-07-21T22:27:41.071550+0000","last_unstale":"2023-07-21T22:27:44.176528+0000","last_undegraded":"2023-07-21T22:27:44.176528+0000","last_fullsized":"2023-07-21T22:27:44.176528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:21:47.017891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171781+0000","last_change":"2023-07-21T22:27:42.098039+0000","last_active":"2023-07-21T22:27:44.171781+0000","last_peered":"2023-07-21T22:27:44.171781+0000","last_clean":"2023-07-21T22:27:44.171781+0000","last_became_active":"2023-07-21T22:27:42.097737+0000","last_became_peered":"2023-07-21T22:27:42.097737+0000","last_unstale":"2023-07-21T22:27:44.171781+0000","last_undegraded":"2023-07-21T22:27:44.171781+0000","last_fullsized":"2023-07-21T22:27:44.171781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:36:20.727587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075028+0000","last_change":"2023-07-21T22:27:41.076768+0000","last_active":"2023-07-21T22:27:43.075028+0000","last_peered":"2023-07-21T22:27:43.075028+0000","last_clean":"2023-07-21T22:27:43.075028+0000","last_became_active":"2023-07-21T22:27:41.076451+0000","last_became_peered":"2023-07-21T22:27:41.076451+0000","last_unstale":"2023-07-21T22:27:43.075028+0000","last_undegraded":"2023-07-21T22:27:43.075028+0000","last_fullsized":"2023-07-21T22:27:43.075028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:27:42.924064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172018+0000","last_change":"2023-07-21T22:27:42.097891+0000","last_active":"2023-07-21T22:27:44.172018+0000","last_peered":"2023-07-21T22:27:44.172018+0000","last_clean":"2023-07-21T22:27:44.172018+0000","last_became_active":"2023-07-21T22:27:42.097022+0000","last_became_peered":"2023-07-21T22:27:42.097022+0000","last_unstale":"2023-07-21T22:27:44.172018+0000","last_undegraded":"2023-07-21T22:27:44.172018+0000","last_fullsized":"2023-07-21T22:27:44.172018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:04:32.167698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075273+0000","last_change":"2023-07-21T22:27:42.085774+0000","last_active":"2023-07-21T22:27:43.075273+0000","last_peered":"2023-07-21T22:27:43.075273+0000","last_clean":"2023-07-21T22:27:43.075273+0000","last_became_active":"2023-07-21T22:27:42.085460+0000","last_became_peered":"2023-07-21T22:27:42.085460+0000","last_unstale":"2023-07-21T22:27:43.075273+0000","last_undegraded":"2023-07-21T22:27:43.075273+0000","last_fullsized":"2023-07-21T22:27:43.075273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:20.846694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701118+0000","last_change":"2023-07-21T22:27:42.091630+0000","last_active":"2023-07-21T22:27:43.701118+0000","last_peered":"2023-07-21T22:27:43.701118+0000","last_clean":"2023-07-21T22:27:43.701118+0000","last_became_active":"2023-07-21T22:27:42.090538+0000","last_became_peered":"2023-07-21T22:27:42.090538+0000","last_unstale":"2023-07-21T22:27:43.701118+0000","last_undegraded":"2023-07-21T22:27:43.701118+0000","last_fullsized":"2023-07-21T22:27:43.701118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:05:25.473040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701072+0000","last_change":"2023-07-21T22:27:42.092918+0000","last_active":"2023-07-21T22:27:43.701072+0000","last_peered":"2023-07-21T22:27:43.701072+0000","last_clean":"2023-07-21T22:27:43.701072+0000","last_became_active":"2023-07-21T22:27:42.092484+0000","last_became_peered":"2023-07-21T22:27:42.092484+0000","last_unstale":"2023-07-21T22:27:43.701072+0000","last_undegraded":"2023-07-21T22:27:43.701072+0000","last_fullsized":"2023-07-21T22:27:43.701072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:40:43.563394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.958024+0000","last_change":"2023-07-21T22:27:42.096306+0000","last_active":"2023-07-21T22:27:43.958024+0000","last_peered":"2023-07-21T22:27:43.958024+0000","last_clean":"2023-07-21T22:27:43.958024+0000","last_became_active":"2023-07-21T22:27:42.096010+0000","last_became_peered":"2023-07-21T22:27:42.096010+0000","last_unstale":"2023-07-21T22:27:43.958024+0000","last_undegraded":"2023-07-21T22:27:43.958024+0000","last_fullsized":"2023-07-21T22:27:43.958024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:36:42.362478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074992+0000","last_change":"2023-07-21T22:27:42.097964+0000","last_active":"2023-07-21T22:27:43.074992+0000","last_peered":"2023-07-21T22:27:43.074992+0000","last_clean":"2023-07-21T22:27:43.074992+0000","last_became_active":"2023-07-21T22:27:42.097666+0000","last_became_peered":"2023-07-21T22:27:42.097666+0000","last_unstale":"2023-07-21T22:27:43.074992+0000","last_undegraded":"2023-07-21T22:27:43.074992+0000","last_fullsized":"2023-07-21T22:27:43.074992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:42:54.073891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075488+0000","last_change":"2023-07-21T22:27:42.083225+0000","last_active":"2023-07-21T22:27:43.075488+0000","last_peered":"2023-07-21T22:27:43.075488+0000","last_clean":"2023-07-21T22:27:43.075488+0000","last_became_active":"2023-07-21T22:27:42.082892+0000","last_became_peered":"2023-07-21T22:27:42.082892+0000","last_unstale":"2023-07-21T22:27:43.075488+0000","last_undegraded":"2023-07-21T22:27:43.075488+0000","last_fullsized":"2023-07-21T22:27:43.075488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:17:08.372691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701648+0000","last_change":"2023-07-21T22:27:42.090854+0000","last_active":"2023-07-21T22:27:43.701648+0000","last_peered":"2023-07-21T22:27:43.701648+0000","last_clean":"2023-07-21T22:27:43.701648+0000","last_became_active":"2023-07-21T22:27:42.090501+0000","last_became_peered":"2023-07-21T22:27:42.090501+0000","last_unstale":"2023-07-21T22:27:43.701648+0000","last_undegraded":"2023-07-21T22:27:43.701648+0000","last_fullsized":"2023-07-21T22:27:43.701648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:52:51.633954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390858+0000","last_change":"2023-07-21T22:27:42.094986+0000","last_active":"2023-07-21T22:27:44.390858+0000","last_peered":"2023-07-21T22:27:44.390858+0000","last_clean":"2023-07-21T22:27:44.390858+0000","last_became_active":"2023-07-21T22:27:42.094668+0000","last_became_peered":"2023-07-21T22:27:42.094668+0000","last_unstale":"2023-07-21T22:27:44.390858+0000","last_undegraded":"2023-07-21T22:27:44.390858+0000","last_fullsized":"2023-07-21T22:27:44.390858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:27:19.466686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176209+0000","last_change":"2023-07-21T22:27:42.098033+0000","last_active":"2023-07-21T22:27:44.176209+0000","last_peered":"2023-07-21T22:27:44.176209+0000","last_clean":"2023-07-21T22:27:44.176209+0000","last_became_active":"2023-07-21T22:27:42.097219+0000","last_became_peered":"2023-07-21T22:27:42.097219+0000","last_unstale":"2023-07-21T22:27:44.176209+0000","last_undegraded":"2023-07-21T22:27:44.176209+0000","last_fullsized":"2023-07-21T22:27:44.176209+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:12:40.233218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390721+0000","last_change":"2023-07-21T22:27:42.092136+0000","last_active":"2023-07-21T22:27:44.390721+0000","last_peered":"2023-07-21T22:27:44.390721+0000","last_clean":"2023-07-21T22:27:44.390721+0000","last_became_active":"2023-07-21T22:27:42.091559+0000","last_became_peered":"2023-07-21T22:27:42.091559+0000","last_unstale":"2023-07-21T22:27:44.390721+0000","last_undegraded":"2023-07-21T22:27:44.390721+0000","last_fullsized":"2023-07-21T22:27:44.390721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:30:28.654604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802891+0000","last_change":"2023-07-21T22:27:42.093227+0000","last_active":"2023-07-21T22:27:44.802891+0000","last_peered":"2023-07-21T22:27:44.802891+0000","last_clean":"2023-07-21T22:27:44.802891+0000","last_became_active":"2023-07-21T22:27:42.092918+0000","last_became_peered":"2023-07-21T22:27:42.092918+0000","last_unstale":"2023-07-21T22:27:44.802891+0000","last_undegraded":"2023-07-21T22:27:44.802891+0000","last_fullsized":"2023-07-21T22:27:44.802891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:21:39.475897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074784+0000","last_change":"2023-07-21T22:27:42.086494+0000","last_active":"2023-07-21T22:27:43.074784+0000","last_peered":"2023-07-21T22:27:43.074784+0000","last_clean":"2023-07-21T22:27:43.074784+0000","last_became_active":"2023-07-21T22:27:42.086189+0000","last_became_peered":"2023-07-21T22:27:42.086189+0000","last_unstale":"2023-07-21T22:27:43.074784+0000","last_undegraded":"2023-07-21T22:27:43.074784+0000","last_fullsized":"2023-07-21T22:27:43.074784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:24:31.736443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075473+0000","last_change":"2023-07-21T22:27:42.090874+0000","last_active":"2023-07-21T22:27:43.075473+0000","last_peered":"2023-07-21T22:27:43.075473+0000","last_clean":"2023-07-21T22:27:43.075473+0000","last_became_active":"2023-07-21T22:27:42.090597+0000","last_became_peered":"2023-07-21T22:27:42.090597+0000","last_unstale":"2023-07-21T22:27:43.075473+0000","last_undegraded":"2023-07-21T22:27:43.075473+0000","last_fullsized":"2023-07-21T22:27:43.075473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:10:24.103529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3978,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":19,"num_read_kb":17,"num_write":118,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":13016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481804,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":421667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":10,"up_from":70,"seq":300647710736,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1503232,"data_stored":1013642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":1,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.315},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.40100000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.497}]},{"osd":6,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.51800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.52600000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.55400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":9,"up_from":65,"seq":279172874260,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":421667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.252},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.31}]},{"osd":1,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.33300000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.37}]},{"osd":7,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63800000000000001}]},{"osd":8,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.57299999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.65300000000000002}]}]},{"osd":8,"up_from":57,"seq":244813135896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":421711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79300000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.85399999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.93400000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.874},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.89000000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.97699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":7,"up_from":50,"seq":214748364828,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28400,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716752,"statfs":{"total":95995035648,"available":95965954048,"internally_reserved":0,"allocated":1482752,"data_stored":1012609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53800000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.433,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46200000000000002,"15min":0.46400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58699999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.33900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.86299999999999999,"15min":0.90500000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81999999999999995,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.81100000000000005,"15min":0.83899999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.6859999999999999,"5min":5.7480000000000002,"15min":5.7480000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.83399999999999996,"15min":0.872},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85799999999999998,"5min":5.7649999999999997,"15min":5.7649999999999997},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.38900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.77200000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.497}]},{"osd":5,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67600000000000005,"15min":0.68300000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86599999999999999,"5min":0.873,"15min":0.873},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.42799999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.7800000000000002,"5min":5.7800000000000002,"15min":5.7800000000000002},"last":0.627},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.71699999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.65000000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.58899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":421085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.36199999999999999,"15min":0.35299999999999998},"min":{"1min":0.26300000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.371},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.36899999999999999,"15min":0.35999999999999999},"min":{"1min":0.29699999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.27300000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.55700000000000005,"15min":0.54800000000000004},"min":{"1min":0.40200000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.55700000000000005}]},{"osd":2,"last update":"Fri Jul 21 22:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52600000000000002,"15min":0.51500000000000001},"min":{"1min":0.39200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.55900000000000005,"15min":0.54600000000000004},"min":{"1min":0.47199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57499999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.33600000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.45700000000000002,"15min":0.44400000000000001},"min":{"1min":0.34300000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.40400000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.46800000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.5}]},{"osd":5,"last update":"Fri Jul 21 22:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.51800000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.46700000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52700000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.621},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.63500000000000001,"15min":0.61799999999999999},"min":{"1min":0.47099999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.72799999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.60799999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.63}]},{"osd":10,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.69999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":422649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.502,"15min":0.47999999999999998},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.50800000000000001,"15min":0.47999999999999998},"min":{"1min":0.47099999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.44400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.36899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82599999999999996,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59199999999999997,"15min":0.58699999999999997},"min":{"1min":0.432,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.436,"15min":0.41899999999999998},"min":{"1min":0.28599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44600000000000001,"15min":0.43099999999999999},"min":{"1min":0.31,"5min":0.248,"15min":0.248},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.76000000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53300000000000003,"15min":0.52200000000000002},"min":{"1min":0.41899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.40000000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.54600000000000004,"15min":0.52600000000000002},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.77700000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.747},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.83199999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.82099999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.88100000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.80400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":421175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.50700000000000001,"15min":0.48499999999999999},"min":{"1min":0.40899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.746},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.497},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81399999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52300000000000002,"15min":0.52000000000000002},"min":{"1min":0.42299999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52300000000000002,"15min":0.51800000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.89500000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39400000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.44,"15min":0.432},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.66900000000000004,"5min":0.753,"15min":0.753},"last":0.45400000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.83199999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:27:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.499,"15min":0.497},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.71099999999999997,"5min":0.749,"15min":0.749},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54800000000000004,"15min":0.54400000000000004},"min":{"1min":0.33500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80000000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57499999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66500000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85499999999999998,"5min":1.04,"15min":1.04},"last":0.48499999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:27:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.88},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.84599999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.65500000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.77300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":423951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.46899999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.46999999999999997,"15min":0.45400000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.629}]},{"osd":1,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.5,"15min":0.47599999999999998},"min":{"1min":0.39900000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.80500000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.41699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.69599999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55800000000000005,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61299999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.44900000000000001,"15min":0.45200000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.46800000000000003,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46600000000000003}]},{"osd":6,"last update":"Fri Jul 21 22:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84799999999999998,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.754}]},{"osd":7,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.435,"5min":0.432,"15min":0.432},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46899999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.56599999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.72299999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.88500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":420948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.33200000000000002},"min":{"1min":0.29399999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.371,"15min":0.35999999999999999},"min":{"1min":0.30299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57799999999999996,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.47799999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71499999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.41099999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.45300000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82599999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.622},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.48299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.80200000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.41899999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.49099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.77300000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.51500000000000001,"15min":0.503},"min":{"1min":0.40200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69999999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66300000000000003,"15min":0.65800000000000003},"min":{"1min":0.498,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.85499999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.64300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:27:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.66400000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.83699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920809,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":421737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.47899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.49299999999999999},"min":{"1min":0.41799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30599999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.37},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.40500000000000003,"15min":0.38800000000000001},"min":{"1min":0.32600000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.59199999999999997,"15min":0.58299999999999996},"min":{"1min":0.38300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.46700000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53100000000000003,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.435,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.75800000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58999999999999997,"15min":0.58099999999999996},"min":{"1min":0.47999999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84399999999999997,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68999999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:27:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.48399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90300000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.621,"15min":0.60599999999999998},"min":{"1min":0.48299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.61099999999999999,"15min":0.59299999999999997},"min":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.59199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.58299999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.751},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.73899999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.64500000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.70599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757285,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28396,"kb_used_data":1444,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716756,"statfs":{"total":95995035648,"available":95965958144,"internally_reserved":0,"allocated":1478656,"data_stored":1011304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.309,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.374,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.38600000000000001,"15min":0.378},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.39200000000000002},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.70299999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.39900000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94599999999999995,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.42299999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.502,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.89400000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.57199999999999995,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.92800000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.72499999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.95199999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.018},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.96999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:27:50.729 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-21T22:27:50.730 INFO:tasks.cephadm:Setting up client nodes... 2023-07-21T22:27:50.731 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-21T22:27:51.306 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:27:51.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:51 smithi190 ceph-mon[98334]: pgmap v254: 97 pgs: 97 active+clean; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.6 KiB/s wr, 17 op/s 2023-07-21T22:27:51.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:51 smithi190 ceph-mon[98334]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-21T22:27:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:51 smithi132 ceph-mon[97863]: pgmap v254: 97 pgs: 97 active+clean; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.6 KiB/s wr, 17 op/s 2023-07-21T22:27:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:51 smithi132 ceph-mon[97863]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-21T22:27:51.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:51 smithi107 ceph-mon[96543]: pgmap v254: 97 pgs: 97 active+clean; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.6 KiB/s wr, 17 op/s 2023-07-21T22:27:51.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:51 smithi107 ceph-mon[96543]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-21T22:27:52.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:52 smithi190 ceph-mon[98334]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:27:52.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:52 smithi190 ceph-mon[98334]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:27:52.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:52 smithi107 ceph-mon[96543]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:27:52.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:52 smithi107 ceph-mon[96543]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:27:52.431 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:27:51 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-b[123210]: 2023-07-21T22:27:51.909+0000 7fc861074700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-07-21T22:27:52.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:52 smithi132 ceph-mon[97863]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:27:52.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:52 smithi132 ceph-mon[97863]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:27:52.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.559+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2134828561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af40ffd30 msgr2=0x7f7af4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:52.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.559+0000 7f7af9cd9700 1 --2- 172.21.15.107:0/2134828561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af40ffd30 0x7f7af4100180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009990 tx=0x7f7adc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:52.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.560+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2134828561 shutdown_connections 2023-07-21T22:27:52.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.560+0000 7f7af9cd9700 1 --2- 172.21.15.107:0/2134828561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af41006c0 0x7f7af4106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.560+0000 7f7af9cd9700 1 --2- 172.21.15.107:0/2134828561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af40feb30 0x7f7af40fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.560+0000 7f7af9cd9700 1 --2- 172.21.15.107:0/2134828561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af40ffd30 0x7f7af4100180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.560+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2134828561 >> 172.21.15.107:0/2134828561 conn(0x7f7af40fa1a0 msgr2=0x7f7af40fc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.560+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2134828561 shutdown_connections 2023-07-21T22:27:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.560+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2134828561 wait complete. 2023-07-21T22:27:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.560+0000 7f7af9cd9700 1 Processor -- start 2023-07-21T22:27:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.560+0000 7f7af9cd9700 1 -- start start 2023-07-21T22:27:52.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.560+0000 7f7af9cd9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af40feb30 0x7f7af4195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:52.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.560+0000 7f7af9cd9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af40ffd30 0x7f7af4195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:52.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.560+0000 7f7af9cd9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af41006c0 0x7f7af419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:52.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af9cd9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af419d400 con 0x7f7af41006c0 2023-07-21T22:27:52.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af9cd9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af419d540 con 0x7f7af40feb30 2023-07-21T22:27:52.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af9cd9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af419d6d0 con 0x7f7af40ffd30 2023-07-21T22:27:52.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af41006c0 0x7f7af419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:52.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af41006c0 0x7f7af419ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46670/0 (socket says 172.21.15.107:46670) 2023-07-21T22:27:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af3fff700 1 -- 172.21.15.107:0/2864367932 learned_addr learned my addr 172.21.15.107:0/2864367932 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af2ffd700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af40ffd30 0x7f7af4195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af37fe700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af40feb30 0x7f7af4195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af3fff700 1 -- 172.21.15.107:0/2864367932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af40ffd30 msgr2=0x7f7af4195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af3fff700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af40ffd30 0x7f7af4195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af3fff700 1 -- 172.21.15.107:0/2864367932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af40feb30 msgr2=0x7f7af4195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af3fff700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af40feb30 0x7f7af4195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:52.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af3fff700 1 -- 172.21.15.107:0/2864367932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc0095f0 con 0x7f7af41006c0 2023-07-21T22:27:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af37fe700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af40feb30 0x7f7af4195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:27:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af3fff700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af41006c0 0x7f7af419ce30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7aec007f10 tx=0x7f7aec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af0ff9700 1 -- 172.21.15.107:0/2864367932 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec017070 con 0x7f7af41006c0 2023-07-21T22:27:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2864367932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af4100fb0 con 0x7f7af41006c0 2023-07-21T22:27:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af0ff9700 1 -- 172.21.15.107:0/2864367932 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7aec00f040 con 0x7f7af41006c0 2023-07-21T22:27:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af0ff9700 1 -- 172.21.15.107:0/2864367932 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec005740 con 0x7f7af41006c0 2023-07-21T22:27:52.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.561+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2864367932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af41014b0 con 0x7f7af41006c0 2023-07-21T22:27:52.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.562+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2864367932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af410a2b0 con 0x7f7af41006c0 2023-07-21T22:27:52.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.563+0000 7f7af0ff9700 1 -- 172.21.15.107:0/2864367932 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f7aec0058a0 con 0x7f7af41006c0 2023-07-21T22:27:52.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.564+0000 7f7af0ff9700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7ae00a11e0 0x7f7ae00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:52.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.564+0000 7f7af37fe700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7ae00a11e0 0x7f7ae00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:52.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.564+0000 7f7af0ff9700 1 -- 172.21.15.107:0/2864367932 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7aec0cc0c0 con 0x7f7af41006c0 2023-07-21T22:27:52.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.564+0000 7f7af37fe700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7ae00a11e0 0x7f7ae00a3690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7af40ffb90 tx=0x7f7ae4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:52.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.568+0000 7f7af0ff9700 1 -- 172.21.15.107:0/2864367932 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7aec095860 con 0x7f7af41006c0 2023-07-21T22:27:52.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.833+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2864367932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f7af410a4a0 con 0x7f7af41006c0 2023-07-21T22:27:52.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.838+0000 7f7af0ff9700 1 -- 172.21.15.107:0/2864367932 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f7aec098e80 con 0x7f7af41006c0 2023-07-21T22:27:52.884 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2023-07-21T22:27:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBoBrtkbvPLMRAAyXh2d8mm/SOVXFgHq9RmtA== 2023-07-21T22:27:52.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.840+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2864367932 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7ae00a11e0 msgr2=0x7f7ae00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:52.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.840+0000 7f7af9cd9700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7ae00a11e0 0x7f7ae00a3690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7af40ffb90 tx=0x7f7ae4005f90 comp rx=0 tx=0).stop 2023-07-21T22:27:52.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.840+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2864367932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af41006c0 msgr2=0x7f7af419ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:52.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.840+0000 7f7af9cd9700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af41006c0 0x7f7af419ce30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7aec007f10 tx=0x7f7aec00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:52.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.840+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2864367932 shutdown_connections 2023-07-21T22:27:52.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.840+0000 7f7af9cd9700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af41006c0 0x7f7af419ce30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:52.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.840+0000 7f7af9cd9700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7af40feb30 0x7f7af4195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:52.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.840+0000 7f7af9cd9700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7ae00a11e0 0x7f7ae00a3690 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:52.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.840+0000 7f7af9cd9700 1 --2- 172.21.15.107:0/2864367932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7af40ffd30 0x7f7af4195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:52.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.840+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2864367932 >> 172.21.15.107:0/2864367932 conn(0x7f7af40fa1a0 msgr2=0x7f7af4068970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:52.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.840+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2864367932 shutdown_connections 2023-07-21T22:27:52.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:52.840+0000 7f7af9cd9700 1 -- 172.21.15.107:0/2864367932 wait complete. 2023-07-21T22:27:53.174 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:53 smithi107 ceph-mon[96543]: pgmap v255: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 KiB/s wr, 28 op/s 2023-07-21T22:27:53.174 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:53 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2864367932' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T22:27:53.174 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:53 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2864367932' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T22:27:53.249 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:27:53.249 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-21T22:27:53.249 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-21T22:27:53.290 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-21T22:27:53.290 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-21T22:27:53.291 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph mgr dump --format=json 2023-07-21T22:27:53.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:53 smithi190 ceph-mon[98334]: pgmap v255: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 KiB/s wr, 28 op/s 2023-07-21T22:27:53.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:53 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2864367932' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T22:27:53.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:53 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2864367932' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T22:27:53.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:53 smithi132 ceph-mon[97863]: pgmap v255: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 KiB/s wr, 28 op/s 2023-07-21T22:27:53.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:53 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2864367932' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T22:27:53.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:53 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2864367932' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T22:27:53.879 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:27:55.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.020+0000 7fd4fa80f700 1 -- 172.21.15.107:0/2124668229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4f41015e0 msgr2=0x7fd4f4109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:55.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.020+0000 7fd4fa80f700 1 --2- 172.21.15.107:0/2124668229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4f41015e0 0x7fd4f4109cb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009990 tx=0x7fd4dc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:27:55.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.020+0000 7fd4fa80f700 1 -- 172.21.15.107:0/2124668229 shutdown_connections 2023-07-21T22:27:55.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.020+0000 7fd4fa80f700 1 --2- 172.21.15.107:0/2124668229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f410a1f0 0x7fd4f410c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:55.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.020+0000 7fd4fa80f700 1 --2- 172.21.15.107:0/2124668229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f4100cd0 0x7fd4f41010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:55.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.020+0000 7fd4fa80f700 1 --2- 172.21.15.107:0/2124668229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4f41015e0 0x7fd4f4109cb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:55.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.020+0000 7fd4fa80f700 1 -- 172.21.15.107:0/2124668229 >> 172.21.15.107:0/2124668229 conn(0x7fd4f4075980 msgr2=0x7fd4f4075d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:55.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.020+0000 7fd4fa80f700 1 -- 172.21.15.107:0/2124668229 shutdown_connections 2023-07-21T22:27:55.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.020+0000 7fd4fa80f700 1 -- 172.21.15.107:0/2124668229 wait complete. 2023-07-21T22:27:55.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.021+0000 7fd4fa80f700 1 Processor -- start 2023-07-21T22:27:55.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.021+0000 7fd4fa80f700 1 -- start start 2023-07-21T22:27:55.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.021+0000 7fd4fa80f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f4100cd0 0x7fd4f419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:55.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.021+0000 7fd4fa80f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4f41015e0 0x7fd4f419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:55.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.021+0000 7fd4fa80f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f410a1f0 0x7fd4f4195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:55.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.021+0000 7fd4fa80f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f4196140 con 0x7fd4f41015e0 2023-07-21T22:27:55.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.021+0000 7fd4fa80f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f4196280 con 0x7fd4f410a1f0 2023-07-21T22:27:55.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.021+0000 7fd4fa80f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f4196440 con 0x7fd4f4100cd0 2023-07-21T22:27:55.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.021+0000 7fd4ebfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4f41015e0 0x7fd4f419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:55.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.021+0000 7fd4ebfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4f41015e0 0x7fd4f419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46682/0 (socket says 172.21.15.107:46682) 2023-07-21T22:27:55.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.021+0000 7fd4ebfff700 1 -- 172.21.15.107:0/3096367309 learned_addr learned my addr 172.21.15.107:0/3096367309 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:55.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.021+0000 7fd4f8dac700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f410a1f0 0x7fd4f4195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:55.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.021+0000 7fd4f3fff700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f4100cd0 0x7fd4f419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:55.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.022+0000 7fd4ebfff700 1 -- 172.21.15.107:0/3096367309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f4100cd0 msgr2=0x7fd4f419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:55.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.022+0000 7fd4ebfff700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f4100cd0 0x7fd4f419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:55.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.022+0000 7fd4ebfff700 1 -- 172.21.15.107:0/3096367309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f410a1f0 msgr2=0x7fd4f4195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:55.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.022+0000 7fd4ebfff700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f410a1f0 0x7fd4f4195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:55.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.022+0000 7fd4ebfff700 1 -- 172.21.15.107:0/3096367309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc0095f0 con 0x7fd4f41015e0 2023-07-21T22:27:55.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.022+0000 7fd4ebfff700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4f41015e0 0x7fd4f419bed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0095c0 tx=0x7fd4dc00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:55.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.022+0000 7fd4f1ffb700 1 -- 172.21.15.107:0/3096367309 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc01d070 con 0x7fd4f41015e0 2023-07-21T22:27:55.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.022+0000 7fd4f1ffb700 1 -- 172.21.15.107:0/3096367309 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4dc0054f0 con 0x7fd4f41015e0 2023-07-21T22:27:55.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.022+0000 7fd4fa80f700 1 -- 172.21.15.107:0/3096367309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f41966c0 con 0x7fd4f41015e0 2023-07-21T22:27:55.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.022+0000 7fd4f1ffb700 1 -- 172.21.15.107:0/3096367309 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc004b00 con 0x7fd4f41015e0 2023-07-21T22:27:55.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.022+0000 7fd4fa80f700 1 -- 172.21.15.107:0/3096367309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f41a2a30 con 0x7fd4f41015e0 2023-07-21T22:27:55.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.023+0000 7fd4eaffd700 1 -- 172.21.15.107:0/3096367309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d80052f0 con 0x7fd4f41015e0 2023-07-21T22:27:55.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.024+0000 7fd4f1ffb700 1 -- 172.21.15.107:0/3096367309 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd4dc004c60 con 0x7fd4f41015e0 2023-07-21T22:27:55.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.024+0000 7fd4f1ffb700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd4e00a1260 0x7fd4e00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:55.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.025+0000 7fd4f3fff700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd4e00a1260 0x7fd4e00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:55.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.025+0000 7fd4f1ffb700 1 -- 172.21.15.107:0/3096367309 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd4dc0cd610 con 0x7fd4f41015e0 2023-07-21T22:27:55.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.025+0000 7fd4f3fff700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd4e00a1260 0x7fd4e00a3710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4005950 tx=0x7fd4e400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:55.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.029+0000 7fd4f1ffb700 1 -- 172.21.15.107:0/3096367309 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd4dc096e40 con 0x7fd4f41015e0 2023-07-21T22:27:55.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.273+0000 7fd4eaffd700 1 -- 172.21.15.107:0/3096367309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd4d8005c90 con 0x7fd4f41015e0 2023-07-21T22:27:55.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.277+0000 7fd4f1ffb700 1 -- 172.21.15.107:0/3096367309 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260133 (secure 0 0 0) 0x7fd4dc09a460 con 0x7fd4f41015e0 2023-07-21T22:27:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:55.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.280+0000 7fd4eaffd700 1 -- 172.21.15.107:0/3096367309 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd4e00a1260 msgr2=0x7fd4e00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:55.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.280+0000 7fd4eaffd700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd4e00a1260 0x7fd4e00a3710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4005950 tx=0x7fd4e400a300 comp rx=0 tx=0).stop 2023-07-21T22:27:55.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.280+0000 7fd4eaffd700 1 -- 172.21.15.107:0/3096367309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4f41015e0 msgr2=0x7fd4f419bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:55.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.280+0000 7fd4eaffd700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4f41015e0 0x7fd4f419bed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0095c0 tx=0x7fd4dc00f710 comp rx=0 tx=0).stop 2023-07-21T22:27:55.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.280+0000 7fd4eaffd700 1 -- 172.21.15.107:0/3096367309 shutdown_connections 2023-07-21T22:27:55.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.280+0000 7fd4eaffd700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4f410a1f0 0x7fd4f4195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:55.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.280+0000 7fd4eaffd700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd4e00a1260 0x7fd4e00a3710 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:55.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.280+0000 7fd4eaffd700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f4100cd0 0x7fd4f419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:55.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.280+0000 7fd4eaffd700 1 --2- 172.21.15.107:0/3096367309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4f41015e0 0x7fd4f419bed0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:55.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.280+0000 7fd4eaffd700 1 -- 172.21.15.107:0/3096367309 >> 172.21.15.107:0/3096367309 conn(0x7fd4f4075980 msgr2=0x7fd4f40fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:55.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.280+0000 7fd4eaffd700 1 -- 172.21.15.107:0/3096367309 shutdown_connections 2023-07-21T22:27:55.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:55.280+0000 7fd4eaffd700 1 -- 172.21.15.107:0/3096367309 wait complete. 2023-07-21T22:27:55.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:55 smithi190 ceph-mon[98334]: pgmap v256: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.9 KiB/s wr, 27 op/s 2023-07-21T22:27:55.419 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:55 smithi107 ceph-mon[96543]: pgmap v256: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.9 KiB/s wr, 27 op/s 2023-07-21T22:27:55.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:55 smithi132 ceph-mon[97863]: pgmap v256: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.9 KiB/s wr, 27 op/s 2023-07-21T22:27:55.632 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1657248266},{"type":"v1","addr":"172.21.15.107:6801","nonce":1657248266}]},"active_addr":"172.21.15.107:6801/1657248266","active_change":"2023-07-21T22:21:45.714467+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.107:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2021632209}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":1251147271}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":322944938}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":846018660}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2490385280}]}]} 2023-07-21T22:27:55.640 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-21T22:27:55.640 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-21T22:27:55.640 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T22:27:56.193 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:27:56.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:56 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3096367309' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T22:27:56.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:56 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3096367309' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T22:27:56.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:56 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3096367309' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T22:27:57.210 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:57 smithi107 ceph-mon[96543]: pgmap v257: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.9 KiB/s wr, 25 op/s 2023-07-21T22:27:57.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:57 smithi190 ceph-mon[98334]: pgmap v257: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.9 KiB/s wr, 25 op/s 2023-07-21T22:27:57.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.361+0000 7f618d662700 1 -- 172.21.15.107:0/3892464312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6188104ea0 msgr2=0x7f618810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:57.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.361+0000 7f618d662700 1 --2- 172.21.15.107:0/3892464312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6188104ea0 0x7f618810ca20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f618000b0d0 tx=0x7f618000b3e0 comp rx=0 tx=0).stop 2023-07-21T22:27:57.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.361+0000 7f618d662700 1 -- 172.21.15.107:0/3892464312 shutdown_connections 2023-07-21T22:27:57.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.361+0000 7f618d662700 1 --2- 172.21.15.107:0/3892464312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6188104ea0 0x7f618810ca20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:57.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.361+0000 7f618d662700 1 --2- 172.21.15.107:0/3892464312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61880ffe00 0x7f61881001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:57.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.361+0000 7f618d662700 1 --2- 172.21.15.107:0/3892464312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61881007a0 0x7f6188104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:57.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.361+0000 7f618d662700 1 -- 172.21.15.107:0/3892464312 >> 172.21.15.107:0/3892464312 conn(0x7f61880fb6a0 msgr2=0x7f61880fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:57.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.361+0000 7f618d662700 1 -- 172.21.15.107:0/3892464312 shutdown_connections 2023-07-21T22:27:57.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.361+0000 7f618d662700 1 -- 172.21.15.107:0/3892464312 wait complete. 2023-07-21T22:27:57.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.362+0000 7f618d662700 1 Processor -- start 2023-07-21T22:27:57.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.362+0000 7f618d662700 1 -- start start 2023-07-21T22:27:57.411 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.362+0000 7f618d662700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61880ffe00 0x7f6188195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:57.411 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.362+0000 7f618d662700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61881007a0 0x7f6188196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:57.411 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.362+0000 7f618d662700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6188104ea0 0x7f618819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:57.411 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.362+0000 7f618d662700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618819ad50 con 0x7f6188104ea0 2023-07-21T22:27:57.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.362+0000 7f618d662700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618819aec0 con 0x7f61880ffe00 2023-07-21T22:27:57.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.362+0000 7f618d662700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618819b080 con 0x7f61881007a0 2023-07-21T22:27:57.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.362+0000 7f61877fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6188104ea0 0x7f618819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:57.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.362+0000 7f61877fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6188104ea0 0x7f618819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46698/0 (socket says 172.21.15.107:46698) 2023-07-21T22:27:57.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.362+0000 7f61877fe700 1 -- 172.21.15.107:0/2721610205 learned_addr learned my addr 172.21.15.107:0/2721610205 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:57.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.362+0000 7f6186ffd700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61880ffe00 0x7f6188195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:57.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.362+0000 7f61867fc700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61881007a0 0x7f6188196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:57.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.363+0000 7f61877fe700 1 -- 172.21.15.107:0/2721610205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61881007a0 msgr2=0x7f6188196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:57.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.363+0000 7f61877fe700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61881007a0 0x7f6188196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:57.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.363+0000 7f61877fe700 1 -- 172.21.15.107:0/2721610205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61880ffe00 msgr2=0x7f6188195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:57.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.363+0000 7f61877fe700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61880ffe00 0x7f6188195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:57.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.363+0000 7f61877fe700 1 -- 172.21.15.107:0/2721610205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6180009d00 con 0x7f6188104ea0 2023-07-21T22:27:57.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.363+0000 7f61867fc700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61881007a0 0x7f6188196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:27:57.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.363+0000 7f61877fe700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6188104ea0 0x7f618819a740 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6180007870 tx=0x7f61800078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:57.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.363+0000 7f616ffff700 1 -- 172.21.15.107:0/2721610205 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000e040 con 0x7f6188104ea0 2023-07-21T22:27:57.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.363+0000 7f618d662700 1 -- 172.21.15.107:0/2721610205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618819b300 con 0x7f6188104ea0 2023-07-21T22:27:57.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.363+0000 7f616ffff700 1 -- 172.21.15.107:0/2721610205 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6180003e10 con 0x7f6188104ea0 2023-07-21T22:27:57.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.363+0000 7f616ffff700 1 -- 172.21.15.107:0/2721610205 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618001b980 con 0x7f6188104ea0 2023-07-21T22:27:57.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.363+0000 7f618d662700 1 -- 172.21.15.107:0/2721610205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61881a29f0 con 0x7f6188104ea0 2023-07-21T22:27:57.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.364+0000 7f618d662700 1 -- 172.21.15.107:0/2721610205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618804e210 con 0x7f6188104ea0 2023-07-21T22:27:57.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.365+0000 7f616ffff700 1 -- 172.21.15.107:0/2721610205 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6180024400 con 0x7f6188104ea0 2023-07-21T22:27:57.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.365+0000 7f616ffff700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f61700a11e0 0x7f61700a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:57.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.366+0000 7f6186ffd700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f61700a11e0 0x7f61700a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:57.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.366+0000 7f616ffff700 1 -- 172.21.15.107:0/2721610205 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f61800ce060 con 0x7f6188104ea0 2023-07-21T22:27:57.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.366+0000 7f6186ffd700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f61700a11e0 0x7f61700a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f617c009a10 tx=0x7f617c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:57.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.370+0000 7f616ffff700 1 -- 172.21.15.107:0/2721610205 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6180097890 con 0x7f6188104ea0 2023-07-21T22:27:57.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:57 smithi132 ceph-mon[97863]: pgmap v257: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.9 KiB/s wr, 25 op/s 2023-07-21T22:27:57.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.554+0000 7f618d662700 1 -- 172.21.15.107:0/2721610205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61881a2c80 con 0x7f6188104ea0 2023-07-21T22:27:57.602 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.556+0000 7f616ffff700 1 -- 172.21.15.107:0/2721610205 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22201 (secure 0 0 0) 0x7f618009aeb0 con 0x7f6188104ea0 2023-07-21T22:27:57.602 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:27:57.603 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:27:43.071566+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:27:57.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.558+0000 7f618d662700 1 -- 172.21.15.107:0/2721610205 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f61700a11e0 msgr2=0x7f61700a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:57.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.558+0000 7f618d662700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f61700a11e0 0x7f61700a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f617c009a10 tx=0x7f617c009190 comp rx=0 tx=0).stop 2023-07-21T22:27:57.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.558+0000 7f618d662700 1 -- 172.21.15.107:0/2721610205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6188104ea0 msgr2=0x7f618819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:57.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.558+0000 7f618d662700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6188104ea0 0x7f618819a740 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6180007870 tx=0x7f61800078a0 comp rx=0 tx=0).stop 2023-07-21T22:27:57.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.558+0000 7f618d662700 1 -- 172.21.15.107:0/2721610205 shutdown_connections 2023-07-21T22:27:57.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.558+0000 7f618d662700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6188104ea0 0x7f618819a740 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:57.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.558+0000 7f618d662700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61880ffe00 0x7f6188195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:57.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.558+0000 7f618d662700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f61700a11e0 0x7f61700a3690 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:57.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.558+0000 7f618d662700 1 --2- 172.21.15.107:0/2721610205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61881007a0 0x7f6188196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:57.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.558+0000 7f618d662700 1 -- 172.21.15.107:0/2721610205 >> 172.21.15.107:0/2721610205 conn(0x7f61880fb6a0 msgr2=0x7f61880fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:57.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.558+0000 7f618d662700 1 -- 172.21.15.107:0/2721610205 shutdown_connections 2023-07-21T22:27:57.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:57.558+0000 7f618d662700 1 -- 172.21.15.107:0/2721610205 wait complete. 2023-07-21T22:27:58.039 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-21T22:27:58.039 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T22:27:58.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:58 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2721610205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:58.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:58 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2721610205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:58.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:58 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2721610205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:27:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:27:59.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:27:59 smithi107 ceph-mon[96543]: pgmap v258: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 27 op/s 2023-07-21T22:27:59.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:27:59 smithi190 ceph-mon[98334]: pgmap v258: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 27 op/s 2023-07-21T22:27:59.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:27:59 smithi132 ceph-mon[97863]: pgmap v258: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 27 op/s 2023-07-21T22:27:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.877+0000 7f0124922700 1 -- 172.21.15.107:0/1323107333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0120103a70 msgr2=0x7f0120103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.877+0000 7f0124922700 1 --2- 172.21.15.107:0/1323107333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0120103a70 0x7f0120103e40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f01100099c0 tx=0x7f0110009cd0 comp rx=0 tx=0).stop 2023-07-21T22:27:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.877+0000 7f0124922700 1 -- 172.21.15.107:0/1323107333 shutdown_connections 2023-07-21T22:27:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.877+0000 7f0124922700 1 --2- 172.21.15.107:0/1323107333 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01200ffbe0 0x7f012010b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:59.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.877+0000 7f0124922700 1 --2- 172.21.15.107:0/1323107333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0120103a70 0x7f0120103e40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:59.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.877+0000 7f0124922700 1 --2- 172.21.15.107:0/1323107333 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01200ff230 0x7f01200ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:59.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.877+0000 7f0124922700 1 -- 172.21.15.107:0/1323107333 >> 172.21.15.107:0/1323107333 conn(0x7f0120075150 msgr2=0x7f0120075550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:59.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.877+0000 7f0124922700 1 -- 172.21.15.107:0/1323107333 shutdown_connections 2023-07-21T22:27:59.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.877+0000 7f0124922700 1 -- 172.21.15.107:0/1323107333 wait complete. 2023-07-21T22:27:59.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f0124922700 1 Processor -- start 2023-07-21T22:27:59.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f0124922700 1 -- start start 2023-07-21T22:27:59.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f0124922700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01200ff230 0x7f012019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:59.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f0124922700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01200ffbe0 0x7f012019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:59.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f0124922700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0120103a70 0x7f0120194940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:59.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f0124922700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0120194fb0 con 0x7f01200ff230 2023-07-21T22:27:59.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f0124922700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0120195120 con 0x7f0120103a70 2023-07-21T22:27:59.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f0124922700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01201952e0 con 0x7f01200ffbe0 2023-07-21T22:27:59.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f011e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01200ff230 0x7f012019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:59.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f011e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01200ff230 0x7f012019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46704/0 (socket says 172.21.15.107:46704) 2023-07-21T22:27:59.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f011e59c700 1 -- 172.21.15.107:0/2301246940 learned_addr learned my addr 172.21.15.107:0/2301246940 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:27:59.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f011dd9b700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01200ffbe0 0x7f012019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:59.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f011ed9d700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0120103a70 0x7f0120194940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:59.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f011e59c700 1 -- 172.21.15.107:0/2301246940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01200ffbe0 msgr2=0x7f012019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:59.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f011e59c700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01200ffbe0 0x7f012019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:59.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f011e59c700 1 -- 172.21.15.107:0/2301246940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0120103a70 msgr2=0x7f0120194940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:59.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f011e59c700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0120103a70 0x7f0120194940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:59.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.878+0000 7f011e59c700 1 -- 172.21.15.107:0/2301246940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01100095f0 con 0x7f01200ff230 2023-07-21T22:27:59.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.879+0000 7f011e59c700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01200ff230 0x7f012019a8c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0110009990 tx=0x7f0110009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:59.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.879+0000 7f010f7fe700 1 -- 172.21.15.107:0/2301246940 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110025070 con 0x7f01200ff230 2023-07-21T22:27:59.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.879+0000 7f010f7fe700 1 -- 172.21.15.107:0/2301246940 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0110004cb0 con 0x7f01200ff230 2023-07-21T22:27:59.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.879+0000 7f0124922700 1 -- 172.21.15.107:0/2301246940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0120195560 con 0x7f01200ff230 2023-07-21T22:27:59.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.879+0000 7f010f7fe700 1 -- 172.21.15.107:0/2301246940 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110003920 con 0x7f01200ff230 2023-07-21T22:27:59.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.879+0000 7f0124922700 1 -- 172.21.15.107:0/2301246940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01200fe2f0 con 0x7f01200ff230 2023-07-21T22:27:59.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.880+0000 7f0124922700 1 -- 172.21.15.107:0/2301246940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012004e210 con 0x7f01200ff230 2023-07-21T22:27:59.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.881+0000 7f010f7fe700 1 -- 172.21.15.107:0/2301246940 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f0110029070 con 0x7f01200ff230 2023-07-21T22:27:59.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.881+0000 7f010f7fe700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f01080a11e0 0x7f01080a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:59.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.882+0000 7f011dd9b700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f01080a11e0 0x7f01080a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:59.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.882+0000 7f010f7fe700 1 -- 172.21.15.107:0/2301246940 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f01100d6260 con 0x7f01200ff230 2023-07-21T22:27:59.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.882+0000 7f011dd9b700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f01080a11e0 0x7f01080a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f01201051f0 tx=0x7f011400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:59.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:27:59.886+0000 7f010f7fe700 1 -- 172.21.15.107:0/2301246940 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f011009f990 con 0x7f01200ff230 2023-07-21T22:28:00.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.075+0000 7f0124922700 1 -- 172.21.15.107:0/2301246940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01200fe570 con 0x7f01200ff230 2023-07-21T22:28:00.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.076+0000 7f010f7fe700 1 -- 172.21.15.107:0/2301246940 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22201 (secure 0 0 0) 0x7f01100a2fb0 con 0x7f01200ff230 2023-07-21T22:28:00.123 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:00.123 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:27:43.071566+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:28:00.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.078+0000 7f0124922700 1 -- 172.21.15.107:0/2301246940 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f01080a11e0 msgr2=0x7f01080a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:00.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.078+0000 7f0124922700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f01080a11e0 0x7f01080a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f01201051f0 tx=0x7f011400b3c0 comp rx=0 tx=0).stop 2023-07-21T22:28:00.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.078+0000 7f0124922700 1 -- 172.21.15.107:0/2301246940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01200ff230 msgr2=0x7f012019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:00.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.078+0000 7f0124922700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01200ff230 0x7f012019a8c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0110009990 tx=0x7f0110009580 comp rx=0 tx=0).stop 2023-07-21T22:28:00.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.078+0000 7f0124922700 1 -- 172.21.15.107:0/2301246940 shutdown_connections 2023-07-21T22:28:00.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.078+0000 7f0124922700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0120103a70 0x7f0120194940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:00.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.078+0000 7f0124922700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01200ff230 0x7f012019a8c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:00.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.078+0000 7f0124922700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f01080a11e0 0x7f01080a3690 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:00.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.078+0000 7f0124922700 1 --2- 172.21.15.107:0/2301246940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f01200ffbe0 0x7f012019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:00.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.078+0000 7f0124922700 1 -- 172.21.15.107:0/2301246940 >> 172.21.15.107:0/2301246940 conn(0x7f0120075150 msgr2=0x7f01200fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:00.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.078+0000 7f0124922700 1 -- 172.21.15.107:0/2301246940 shutdown_connections 2023-07-21T22:28:00.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:00.078+0000 7f0124922700 1 -- 172.21.15.107:0/2301246940 wait complete. 2023-07-21T22:28:00.450 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-21T22:28:00.451 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-21T22:28:00.451 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-21T22:28:00.452 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-21T22:28:00.452 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-21T22:28:00.453 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-21T22:28:00.453 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-21T22:28:00.454 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-21T22:28:00.454 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-21T22:28:00.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-21T22:28:00.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-21T22:28:00.456 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-21T22:28:01.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:01 smithi190 ceph-mon[98334]: pgmap v259: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 KiB/s wr, 20 op/s 2023-07-21T22:28:01.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:01 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2301246940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:01.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:01 smithi132 ceph-mon[97863]: pgmap v259: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 KiB/s wr, 20 op/s 2023-07-21T22:28:01.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:01 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2301246940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:01.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:01 smithi107 ceph-mon[96543]: pgmap v259: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 KiB/s wr, 20 op/s 2023-07-21T22:28:01.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:01 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2301246940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:01.754 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:01.772 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:02.157 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:02.901 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:02.918 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:03.178 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:03.180 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:03.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:03 smithi190 ceph-mon[98334]: pgmap v260: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2023-07-21T22:28:03.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:03 smithi132 ceph-mon[97863]: pgmap v260: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2023-07-21T22:28:03.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:03 smithi107 ceph-mon[96543]: pgmap v260: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2023-07-21T22:28:03.446 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:03.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.838+0000 7f566afa5700 1 -- 172.21.15.107:0/1894109586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f566410a140 msgr2=0x7f566410c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:03.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.838+0000 7f566afa5700 1 --2- 172.21.15.107:0/1894109586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f566410a140 0x7f566410c580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f565c00b0a0 tx=0x7f565c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:03.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.838+0000 7f566afa5700 1 -- 172.21.15.107:0/1894109586 shutdown_connections 2023-07-21T22:28:03.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.838+0000 7f566afa5700 1 --2- 172.21.15.107:0/1894109586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f566410a140 0x7f566410c580 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:03.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.838+0000 7f566afa5700 1 --2- 172.21.15.107:0/1894109586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5664100df0 0x7f56641011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.838+0000 7f566afa5700 1 --2- 172.21.15.107:0/1894109586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5664101700 0x7f5664109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.838+0000 7f566afa5700 1 -- 172.21.15.107:0/1894109586 >> 172.21.15.107:0/1894109586 conn(0x7f5664075b70 msgr2=0x7f5664075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.838+0000 7f566afa5700 1 -- 172.21.15.107:0/1894109586 shutdown_connections 2023-07-21T22:28:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.838+0000 7f566afa5700 1 -- 172.21.15.107:0/1894109586 wait complete. 2023-07-21T22:28:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.839+0000 7f566afa5700 1 Processor -- start 2023-07-21T22:28:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.839+0000 7f566afa5700 1 -- start start 2023-07-21T22:28:03.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.839+0000 7f566afa5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5664100df0 0x7f5664195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:03.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.839+0000 7f566afa5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5664101700 0x7f56641961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:03.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.839+0000 7f566afa5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f566410a140 0x7f56641995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:03.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.839+0000 7f566afa5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5664199ba0 con 0x7f5664101700 2023-07-21T22:28:03.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.839+0000 7f566afa5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5664199d10 con 0x7f566410a140 2023-07-21T22:28:03.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.839+0000 7f566afa5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5664199ed0 con 0x7f5664100df0 2023-07-21T22:28:03.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.839+0000 7f5663fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5664101700 0x7f56641961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:03.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.839+0000 7f5663fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5664101700 0x7f56641961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46718/0 (socket says 172.21.15.107:46718) 2023-07-21T22:28:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.839+0000 7f5663fff700 1 -- 172.21.15.107:0/3678626054 learned_addr learned my addr 172.21.15.107:0/3678626054 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.839+0000 7f5669542700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f566410a140 0x7f56641995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.839+0000 7f5668d41700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5664100df0 0x7f5664195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.840+0000 7f5663fff700 1 -- 172.21.15.107:0/3678626054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5664100df0 msgr2=0x7f5664195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.840+0000 7f5663fff700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5664100df0 0x7f5664195ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.840+0000 7f5663fff700 1 -- 172.21.15.107:0/3678626054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f566410a140 msgr2=0x7f56641995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:03.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.840+0000 7f5663fff700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f566410a140 0x7f56641995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.840+0000 7f5663fff700 1 -- 172.21.15.107:0/3678626054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c009d00 con 0x7f5664101700 2023-07-21T22:28:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.840+0000 7f5668d41700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5664100df0 0x7f5664195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.840+0000 7f5663fff700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5664101700 0x7f56641961e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f565000d950 tx=0x7f565000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.840+0000 7f5661ffb700 1 -- 172.21.15.107:0/3678626054 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5650009940 con 0x7f5664101700 2023-07-21T22:28:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.840+0000 7f5661ffb700 1 -- 172.21.15.107:0/3678626054 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5650010460 con 0x7f5664101700 2023-07-21T22:28:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.840+0000 7f566afa5700 1 -- 172.21.15.107:0/3678626054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566419a1b0 con 0x7f5664101700 2023-07-21T22:28:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.840+0000 7f5661ffb700 1 -- 172.21.15.107:0/3678626054 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565000f620 con 0x7f5664101700 2023-07-21T22:28:03.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.840+0000 7f566afa5700 1 -- 172.21.15.107:0/3678626054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56641a2b00 con 0x7f5664101700 2023-07-21T22:28:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.841+0000 7f566afa5700 1 -- 172.21.15.107:0/3678626054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f566410fcf0 con 0x7f5664101700 2023-07-21T22:28:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.842+0000 7f5661ffb700 1 -- 172.21.15.107:0/3678626054 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f56500108d0 con 0x7f5664101700 2023-07-21T22:28:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.842+0000 7f5661ffb700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56540a11e0 0x7f56540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.843+0000 7f5668d41700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56540a11e0 0x7f56540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.843+0000 7f5661ffb700 1 -- 172.21.15.107:0/3678626054 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f56500cc710 con 0x7f5664101700 2023-07-21T22:28:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.843+0000 7f5661ffb700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533] conn(0x7f56540aa9b0 0x7f56540acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.843+0000 7f5661ffb700 1 -- 172.21.15.107:0/3678626054 --> [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56540ad470 con 0x7f56540aa9b0 2023-07-21T22:28:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.843+0000 7f5661ffb700 1 -- 172.21.15.107:0/3678626054 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56500cee20 con 0x7f5664101700 2023-07-21T22:28:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.843+0000 7f5668d41700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56540a11e0 0x7f56540a3690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5658005fd0 tx=0x7f5658005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.843+0000 7f5669542700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533] conn(0x7f56540aa9b0 0x7f56540acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.843+0000 7f5669542700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533] conn(0x7f56540aa9b0 0x7f56540acdc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.844+0000 7f5661ffb700 1 -- 172.21.15.107:0/3678626054 <== osd.5 v2:172.21.15.190:6808/1950039533 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f56540ad470 con 0x7f56540aa9b0 2023-07-21T22:28:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.860+0000 7f566afa5700 1 -- 172.21.15.107:0/3678626054 --> [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f566404e210 con 0x7f56540aa9b0 2023-07-21T22:28:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.860+0000 7f5661ffb700 1 -- 172.21.15.107:0/3678626054 <== osd.5 v2:172.21.15.190:6808/1950039533 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f566404e210 con 0x7f56540aa9b0 2023-07-21T22:28:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.861+0000 7f566afa5700 1 -- 172.21.15.107:0/3678626054 >> [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533] conn(0x7f56540aa9b0 msgr2=0x7f56540acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.861+0000 7f566afa5700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533] conn(0x7f56540aa9b0 0x7f56540acdc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.861+0000 7f566afa5700 1 -- 172.21.15.107:0/3678626054 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56540a11e0 msgr2=0x7f56540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.861+0000 7f566afa5700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56540a11e0 0x7f56540a3690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5658005fd0 tx=0x7f5658005e30 comp rx=0 tx=0).stop 2023-07-21T22:28:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.861+0000 7f566afa5700 1 -- 172.21.15.107:0/3678626054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5664101700 msgr2=0x7f56641961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.861+0000 7f566afa5700 1 --2- 172.21.15.107:0/3678626054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5664101700 0x7f56641961e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f565000d950 tx=0x7f565000dc60 comp rx=0 tx=0).stop 2023-07-21T22:28:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.861+0000 7f5669542700 1 -- 172.21.15.107:0/3678626054 reap_dead start 2023-07-21T22:28:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.861+0000 7f566afa5700 1 -- 172.21.15.107:0/3678626054 shutdown_connections 2023-07-21T22:28:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.861+0000 7f566afa5700 1 -- 172.21.15.107:0/3678626054 >> 172.21.15.107:0/3678626054 conn(0x7f5664075b70 msgr2=0x7f56640fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.861+0000 7f566afa5700 1 -- 172.21.15.107:0/3678626054 shutdown_connections 2023-07-21T22:28:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:03.861+0000 7f566afa5700 1 -- 172.21.15.107:0/3678626054 wait complete. 2023-07-21T22:28:04.006 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:04.007 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:04.227 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:04.516 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:05.291 INFO:teuthology.orchestra.run.smithi107.stdout:163208757289 2023-07-21T22:28:05.291 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-21T22:28:05.318 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:05 smithi107 ceph-mon[96543]: pgmap v261: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-07-21T22:28:05.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:05 smithi190 ceph-mon[98334]: pgmap v261: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-07-21T22:28:05.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:05 smithi132 ceph-mon[97863]: pgmap v261: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-07-21T22:28:05.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.410+0000 7f4ec016d700 1 -- 172.21.15.107:0/3168048177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb80ffe30 msgr2=0x7f4eb8100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:05.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.410+0000 7f4ec016d700 1 --2- 172.21.15.107:0/3168048177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb80ffe30 0x7f4eb8100200 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40099c0 tx=0x7f4eb4009cd0 comp rx=0 tx=0).stop 2023-07-21T22:28:05.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.410+0000 7f4ec016d700 1 -- 172.21.15.107:0/3168048177 shutdown_connections 2023-07-21T22:28:05.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.410+0000 7f4ec016d700 1 --2- 172.21.15.107:0/3168048177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4eb8104ec0 0x7f4eb810ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:05.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.410+0000 7f4ec016d700 1 --2- 172.21.15.107:0/3168048177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb80ffe30 0x7f4eb8100200 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:05.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.410+0000 7f4ec016d700 1 --2- 172.21.15.107:0/3168048177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eb81007d0 0x7f4eb8104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:05.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.410+0000 7f4ec016d700 1 -- 172.21.15.107:0/3168048177 >> 172.21.15.107:0/3168048177 conn(0x7f4eb8077b70 msgr2=0x7f4eb8077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:05.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.410+0000 7f4ec016d700 1 -- 172.21.15.107:0/3168048177 shutdown_connections 2023-07-21T22:28:05.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.410+0000 7f4ec016d700 1 -- 172.21.15.107:0/3168048177 wait complete. 2023-07-21T22:28:05.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.411+0000 7f4ec016d700 1 Processor -- start 2023-07-21T22:28:05.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.411+0000 7f4ec016d700 1 -- start start 2023-07-21T22:28:05.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.411+0000 7f4ec016d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eb80ffe30 0x7f4eb8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:05.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.411+0000 7f4ec016d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4eb81007d0 0x7f4eb8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:05.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.411+0000 7f4ec016d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb8104ec0 0x7f4eb80732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:05.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.411+0000 7f4ec016d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81a40f0 con 0x7f4eb8104ec0 2023-07-21T22:28:05.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.411+0000 7f4ec016d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81a4260 con 0x7f4eb80ffe30 2023-07-21T22:28:05.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.411+0000 7f4ec016d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81a4420 con 0x7f4eb81007d0 2023-07-21T22:28:05.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.411+0000 7f4ebe70a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb8104ec0 0x7f4eb80732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:05.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.411+0000 7f4ebe70a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb8104ec0 0x7f4eb80732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46734/0 (socket says 172.21.15.107:46734) 2023-07-21T22:28:05.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.411+0000 7f4ebe70a700 1 -- 172.21.15.107:0/2528431844 learned_addr learned my addr 172.21.15.107:0/2528431844 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:05.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.412+0000 7f4ebd708700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4eb81007d0 0x7f4eb8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:05.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.412+0000 7f4ebdf09700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eb80ffe30 0x7f4eb8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:05.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.412+0000 7f4ebe70a700 1 -- 172.21.15.107:0/2528431844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4eb81007d0 msgr2=0x7f4eb8072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:05.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.412+0000 7f4ebe70a700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4eb81007d0 0x7f4eb8072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:05.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.412+0000 7f4ebe70a700 1 -- 172.21.15.107:0/2528431844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eb80ffe30 msgr2=0x7f4eb8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:05.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.412+0000 7f4ebe70a700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eb80ffe30 0x7f4eb8072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:05.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.412+0000 7f4ebe70a700 1 -- 172.21.15.107:0/2528431844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb40095f0 con 0x7f4eb8104ec0 2023-07-21T22:28:05.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.412+0000 7f4ebe70a700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb8104ec0 0x7f4eb80732d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0007ee0 tx=0x7f4eb000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:05.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.412+0000 7f4eaeffd700 1 -- 172.21.15.107:0/2528431844 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb000ddd0 con 0x7f4eb8104ec0 2023-07-21T22:28:05.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.412+0000 7f4eaeffd700 1 -- 172.21.15.107:0/2528431844 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4eb000f040 con 0x7f4eb8104ec0 2023-07-21T22:28:05.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.412+0000 7f4ec016d700 1 -- 172.21.15.107:0/2528431844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb81a4700 con 0x7f4eb8104ec0 2023-07-21T22:28:05.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.412+0000 7f4eaeffd700 1 -- 172.21.15.107:0/2528431844 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0015400 con 0x7f4eb8104ec0 2023-07-21T22:28:05.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.412+0000 7f4ec016d700 1 -- 172.21.15.107:0/2528431844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb81a4bb0 con 0x7f4eb8104ec0 2023-07-21T22:28:05.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.413+0000 7f4ec016d700 1 -- 172.21.15.107:0/2528431844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4eb804e210 con 0x7f4eb8104ec0 2023-07-21T22:28:05.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.415+0000 7f4eaeffd700 1 -- 172.21.15.107:0/2528431844 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f4eb0004020 con 0x7f4eb8104ec0 2023-07-21T22:28:05.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.415+0000 7f4eaeffd700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4ea40a11e0 0x7f4ea40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:05.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.415+0000 7f4eaeffd700 1 -- 172.21.15.107:0/2528431844 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4eb00ccd60 con 0x7f4eb8104ec0 2023-07-21T22:28:05.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.415+0000 7f4eaeffd700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216] conn(0x7f4ea40aa9b0 0x7f4ea40acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:05.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.415+0000 7f4eaeffd700 1 -- 172.21.15.107:0/2528431844 --> [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ea40ad470 con 0x7f4ea40aa9b0 2023-07-21T22:28:05.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.416+0000 7f4eaeffd700 1 -- 172.21.15.107:0/2528431844 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4eb00cd200 con 0x7f4eb8104ec0 2023-07-21T22:28:05.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.416+0000 7f4ebdf09700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4ea40a11e0 0x7f4ea40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:05.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.416+0000 7f4ebdf09700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4ea40a11e0 0x7f4ea40a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009fd0 tx=0x7f4eb4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:05.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.417+0000 7f4ebd708700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216] conn(0x7f4ea40aa9b0 0x7f4ea40acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:05.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.417+0000 7f4ebd708700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216] conn(0x7f4ea40aa9b0 0x7f4ea40acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:05.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.418+0000 7f4eaeffd700 1 -- 172.21.15.107:0/2528431844 <== osd.4 v2:172.21.15.132:6808/2021101216 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4ea40ad470 con 0x7f4ea40aa9b0 2023-07-21T22:28:05.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.435+0000 7f4ec016d700 1 -- 172.21.15.107:0/2528431844 --> [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4eb810fd70 con 0x7f4ea40aa9b0 2023-07-21T22:28:05.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.436+0000 7f4eaeffd700 1 -- 172.21.15.107:0/2528431844 <== osd.4 v2:172.21.15.132:6808/2021101216 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4ea800a7c0 con 0x7f4ea40aa9b0 2023-07-21T22:28:05.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.436+0000 7f4ec016d700 1 -- 172.21.15.107:0/2528431844 >> [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216] conn(0x7f4ea40aa9b0 msgr2=0x7f4ea40acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:05.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.436+0000 7f4ec016d700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216] conn(0x7f4ea40aa9b0 0x7f4ea40acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:05.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.436+0000 7f4ec016d700 1 -- 172.21.15.107:0/2528431844 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4ea40a11e0 msgr2=0x7f4ea40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:05.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.436+0000 7f4ec016d700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4ea40a11e0 0x7f4ea40a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009fd0 tx=0x7f4eb4005fb0 comp rx=0 tx=0).stop 2023-07-21T22:28:05.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.436+0000 7f4ec016d700 1 -- 172.21.15.107:0/2528431844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb8104ec0 msgr2=0x7f4eb80732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:05.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.436+0000 7f4ec016d700 1 --2- 172.21.15.107:0/2528431844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4eb8104ec0 0x7f4eb80732d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0007ee0 tx=0x7f4eb000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:05.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.436+0000 7f4ebe70a700 1 -- 172.21.15.107:0/2528431844 reap_dead start 2023-07-21T22:28:05.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.436+0000 7f4ec016d700 1 -- 172.21.15.107:0/2528431844 shutdown_connections 2023-07-21T22:28:05.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.436+0000 7f4ec016d700 1 -- 172.21.15.107:0/2528431844 >> 172.21.15.107:0/2528431844 conn(0x7f4eb8077b70 msgr2=0x7f4eb80fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:05.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.436+0000 7f4ec016d700 1 -- 172.21.15.107:0/2528431844 shutdown_connections 2023-07-21T22:28:05.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:05.436+0000 7f4ec016d700 1 -- 172.21.15.107:0/2528431844 wait complete. 2023-07-21T22:28:07.352 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:07 smithi107 ceph-mon[96543]: pgmap v262: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2023-07-21T22:28:07.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:07 smithi190 ceph-mon[98334]: pgmap v262: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2023-07-21T22:28:07.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:07 smithi132 ceph-mon[97863]: pgmap v262: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2023-07-21T22:28:07.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.750+0000 7fb90c906700 1 -- 172.21.15.107:0/2229684308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9081007a0 msgr2=0x7fb908104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.750+0000 7fb90c906700 1 --2- 172.21.15.107:0/2229684308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9081007a0 0x7fb908104770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00099c0 tx=0x7fb8f0009cd0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.751+0000 7fb90c906700 1 -- 172.21.15.107:0/2229684308 shutdown_connections 2023-07-21T22:28:07.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.751+0000 7fb90c906700 1 --2- 172.21.15.107:0/2229684308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb908104ea0 0x7fb90810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.751+0000 7fb90c906700 1 --2- 172.21.15.107:0/2229684308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9080ffe00 0x7fb9081001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.751+0000 7fb90c906700 1 --2- 172.21.15.107:0/2229684308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9081007a0 0x7fb908104770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.751+0000 7fb90c906700 1 -- 172.21.15.107:0/2229684308 >> 172.21.15.107:0/2229684308 conn(0x7fb9080fb660 msgr2=0x7fb9080fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:07.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.751+0000 7fb90c906700 1 -- 172.21.15.107:0/2229684308 shutdown_connections 2023-07-21T22:28:07.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.751+0000 7fb90c906700 1 -- 172.21.15.107:0/2229684308 wait complete. 2023-07-21T22:28:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.751+0000 7fb90c906700 1 Processor -- start 2023-07-21T22:28:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.751+0000 7fb90c906700 1 -- start start 2023-07-21T22:28:07.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb90c906700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9080ffe00 0x7fb90819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb90c906700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9081007a0 0x7fb90819bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb90c906700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb908104ea0 0x7fb908195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb90c906700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb908196160 con 0x7fb9081007a0 2023-07-21T22:28:07.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb90c906700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9081962d0 con 0x7fb9080ffe00 2023-07-21T22:28:07.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb90c906700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb908196490 con 0x7fb908104ea0 2023-07-21T22:28:07.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb905d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9081007a0 0x7fb90819bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb905d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9081007a0 0x7fb90819bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46746/0 (socket says 172.21.15.107:46746) 2023-07-21T22:28:07.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb905d9b700 1 -- 172.21.15.107:0/977605100 learned_addr learned my addr 172.21.15.107:0/977605100 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:07.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb906d9d700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb908104ea0 0x7fb908195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb90659c700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9080ffe00 0x7fb90819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb905d9b700 1 -- 172.21.15.107:0/977605100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb908104ea0 msgr2=0x7fb908195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb905d9b700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb908104ea0 0x7fb908195af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.804 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb905d9b700 1 -- 172.21.15.107:0/977605100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9080ffe00 msgr2=0x7fb90819ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.804 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb905d9b700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9080ffe00 0x7fb90819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.804 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb905d9b700 1 -- 172.21.15.107:0/977605100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f00095f0 con 0x7fb9081007a0 2023-07-21T22:28:07.804 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb906d9d700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb908104ea0 0x7fb908195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:07.805 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb905d9b700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9081007a0 0x7fb90819bf60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009990 tx=0x7fb8f0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:07.805 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb8ff7fe700 1 -- 172.21.15.107:0/977605100 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f001d070 con 0x7fb9081007a0 2023-07-21T22:28:07.805 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb8ff7fe700 1 -- 172.21.15.107:0/977605100 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8f0004090 con 0x7fb9081007a0 2023-07-21T22:28:07.806 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb8ff7fe700 1 -- 172.21.15.107:0/977605100 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0022d20 con 0x7fb9081007a0 2023-07-21T22:28:07.806 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb90c906700 1 -- 172.21.15.107:0/977605100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb908196710 con 0x7fb9081007a0 2023-07-21T22:28:07.806 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.752+0000 7fb90c906700 1 -- 172.21.15.107:0/977605100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9081a2a10 con 0x7fb9081007a0 2023-07-21T22:28:07.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.753+0000 7fb90c906700 1 -- 172.21.15.107:0/977605100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb90804e210 con 0x7fb9081007a0 2023-07-21T22:28:07.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.754+0000 7fb8ff7fe700 1 -- 172.21.15.107:0/977605100 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb8f0022470 con 0x7fb9081007a0 2023-07-21T22:28:07.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.755+0000 7fb8ff7fe700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb8f40a1260 0x7fb8f40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.755+0000 7fb90659c700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb8f40a1260 0x7fb8f40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.755+0000 7fb8ff7fe700 1 -- 172.21.15.107:0/977605100 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb8f00ce230 con 0x7fb9081007a0 2023-07-21T22:28:07.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.755+0000 7fb8ff7fe700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995] conn(0x7fb8f40aa900 0x7fb8f40acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:07.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.755+0000 7fb8ff7fe700 1 -- 172.21.15.107:0/977605100 --> [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8f40ad3c0 con 0x7fb8f40aa900 2023-07-21T22:28:07.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.755+0000 7fb8ff7fe700 1 -- 172.21.15.107:0/977605100 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8f00ce5e0 con 0x7fb9081007a0 2023-07-21T22:28:07.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.755+0000 7fb90659c700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb8f40a1260 0x7fb8f40a3710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8005950 tx=0x7fb8f80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:07.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.756+0000 7fb906d9d700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995] conn(0x7fb8f40aa900 0x7fb8f40acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:07.811 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.756+0000 7fb906d9d700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995] conn(0x7fb8f40aa900 0x7fb8f40acd10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:07.811 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.757+0000 7fb8ff7fe700 1 -- 172.21.15.107:0/977605100 <== osd.2 v2:172.21.15.190:6800/1160922995 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb8f40ad3c0 con 0x7fb8f40aa900 2023-07-21T22:28:07.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.772+0000 7fb90c906700 1 -- 172.21.15.107:0/977605100 --> [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9080fca50 con 0x7fb8f40aa900 2023-07-21T22:28:07.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.772+0000 7fb8ff7fe700 1 -- 172.21.15.107:0/977605100 <== osd.2 v2:172.21.15.190:6800/1160922995 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb9080fca50 con 0x7fb8f40aa900 2023-07-21T22:28:07.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.773+0000 7fb90c906700 1 -- 172.21.15.107:0/977605100 >> [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995] conn(0x7fb8f40aa900 msgr2=0x7fb8f40acd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.773+0000 7fb90c906700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995] conn(0x7fb8f40aa900 0x7fb8f40acd10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.773+0000 7fb90c906700 1 -- 172.21.15.107:0/977605100 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb8f40a1260 msgr2=0x7fb8f40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.773+0000 7fb90c906700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb8f40a1260 0x7fb8f40a3710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8005950 tx=0x7fb8f80058e0 comp rx=0 tx=0).stop 2023-07-21T22:28:07.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.773+0000 7fb90c906700 1 -- 172.21.15.107:0/977605100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9081007a0 msgr2=0x7fb90819bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:07.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.773+0000 7fb90c906700 1 --2- 172.21.15.107:0/977605100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9081007a0 0x7fb90819bf60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009990 tx=0x7fb8f0005e80 comp rx=0 tx=0).stop 2023-07-21T22:28:07.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.773+0000 7fb906d9d700 1 -- 172.21.15.107:0/977605100 reap_dead start 2023-07-21T22:28:07.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.773+0000 7fb90c906700 1 -- 172.21.15.107:0/977605100 shutdown_connections 2023-07-21T22:28:07.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.773+0000 7fb90c906700 1 -- 172.21.15.107:0/977605100 >> 172.21.15.107:0/977605100 conn(0x7fb9080fb660 msgr2=0x7fb9080fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:07.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.773+0000 7fb90c906700 1 -- 172.21.15.107:0/977605100 shutdown_connections 2023-07-21T22:28:07.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:07.773+0000 7fb90c906700 1 -- 172.21.15.107:0/977605100 wait complete. 2023-07-21T22:28:08.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.121+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/123629232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e78104ab0 msgr2=0x7f7e78104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:08.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.121+0000 7f7e7ffdf700 1 --2- 172.21.15.107:0/123629232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e78104ab0 0x7f7e78104e80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009990 tx=0x7f7e74009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.121+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/123629232 shutdown_connections 2023-07-21T22:28:08.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.121+0000 7f7e7ffdf700 1 --2- 172.21.15.107:0/123629232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e780ff460 0x7f7e7810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.121+0000 7f7e7ffdf700 1 --2- 172.21.15.107:0/123629232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e78104ab0 0x7f7e78104e80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.121+0000 7f7e7ffdf700 1 --2- 172.21.15.107:0/123629232 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e780feab0 0x7f7e780fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.121+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/123629232 >> 172.21.15.107:0/123629232 conn(0x7f7e78075b70 msgr2=0x7f7e78075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:08.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.121+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/123629232 shutdown_connections 2023-07-21T22:28:08.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.121+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/123629232 wait complete. 2023-07-21T22:28:08.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7ffdf700 1 Processor -- start 2023-07-21T22:28:08.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7ffdf700 1 -- start start 2023-07-21T22:28:08.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7ffdf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e780feab0 0x7f7e7819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:08.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7ffdf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e780ff460 0x7f7e7819bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:08.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7ffdf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e78104ab0 0x7f7e78195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:08.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7ffdf700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781960c0 con 0x7f7e78104ab0 2023-07-21T22:28:08.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7ffdf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e78196230 con 0x7f7e780feab0 2023-07-21T22:28:08.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7ffdf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781963f0 con 0x7f7e780ff460 2023-07-21T22:28:08.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7e57c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e78104ab0 0x7f7e78195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:08.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7e57c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e78104ab0 0x7f7e78195a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46764/0 (socket says 172.21.15.107:46764) 2023-07-21T22:28:08.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7e57c700 1 -- 172.21.15.107:0/3156917070 learned_addr learned my addr 172.21.15.107:0/3156917070 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:08.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7d57a700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e780ff460 0x7f7e7819bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:08.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7dd7b700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e780feab0 0x7f7e7819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:08.172 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7e57c700 1 -- 172.21.15.107:0/3156917070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e780ff460 msgr2=0x7f7e7819bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:08.172 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7e57c700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e780ff460 0x7f7e7819bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.172 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7e57c700 1 -- 172.21.15.107:0/3156917070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e780feab0 msgr2=0x7f7e7819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:08.172 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7e57c700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e780feab0 0x7f7e7819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.172 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.122+0000 7f7e7e57c700 1 -- 172.21.15.107:0/3156917070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e740095f0 con 0x7f7e78104ab0 2023-07-21T22:28:08.172 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.123+0000 7f7e7e57c700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e78104ab0 0x7f7e78195a50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7e70007f10 tx=0x7f7e7000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:08.172 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.123+0000 7f7e6effd700 1 -- 172.21.15.107:0/3156917070 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70017070 con 0x7f7e78104ab0 2023-07-21T22:28:08.173 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.123+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/3156917070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e781966d0 con 0x7f7e78104ab0 2023-07-21T22:28:08.173 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.123+0000 7f7e6effd700 1 -- 172.21.15.107:0/3156917070 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e7000f040 con 0x7f7e78104ab0 2023-07-21T22:28:08.173 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.123+0000 7f7e6effd700 1 -- 172.21.15.107:0/3156917070 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e700054c0 con 0x7f7e78104ab0 2023-07-21T22:28:08.173 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.123+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/3156917070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e781058c0 con 0x7f7e78104ab0 2023-07-21T22:28:08.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.124+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/3156917070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7e7804e210 con 0x7f7e78104ab0 2023-07-21T22:28:08.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.126+0000 7f7e6effd700 1 -- 172.21.15.107:0/3156917070 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f7e70004ad0 con 0x7f7e78104ab0 2023-07-21T22:28:08.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.126+0000 7f7e6effd700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7e640a55f0 0x7f7e640a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:08.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.126+0000 7f7e7dd7b700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7e640a55f0 0x7f7e640a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:08.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.126+0000 7f7e6effd700 1 -- 172.21.15.107:0/3156917070 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7e700cc5d0 con 0x7f7e78104ab0 2023-07-21T22:28:08.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.126+0000 7f7e6effd700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004] conn(0x7f7e640aedc0 0x7f7e640b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:08.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.126+0000 7f7e6effd700 1 -- 172.21.15.107:0/3156917070 --> [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e640b1880 con 0x7f7e640aedc0 2023-07-21T22:28:08.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.126+0000 7f7e6effd700 1 -- 172.21.15.107:0/3156917070 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e700cca70 con 0x7f7e78104ab0 2023-07-21T22:28:08.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.126+0000 7f7e7d57a700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004] conn(0x7f7e640aedc0 0x7f7e640b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:08.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.127+0000 7f7e7dd7b700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7e640a55f0 0x7f7e640a7aa0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400b2e0 tx=0x7f7e74005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:08.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.127+0000 7f7e7d57a700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004] conn(0x7f7e640aedc0 0x7f7e640b11d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:08.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.127+0000 7f7e6effd700 1 -- 172.21.15.107:0/3156917070 <== osd.3 v2:172.21.15.107:6810/1375259004 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7e640b1880 con 0x7f7e640aedc0 2023-07-21T22:28:08.194 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.147+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/3156917070 --> [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7e780fd5a0 con 0x7f7e640aedc0 2023-07-21T22:28:08.194 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.148+0000 7f7e6effd700 1 -- 172.21.15.107:0/3156917070 <== osd.3 v2:172.21.15.107:6810/1375259004 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7e780fd5a0 con 0x7f7e640aedc0 2023-07-21T22:28:08.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.148+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/3156917070 >> [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004] conn(0x7f7e640aedc0 msgr2=0x7f7e640b11d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:08.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.148+0000 7f7e7ffdf700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004] conn(0x7f7e640aedc0 0x7f7e640b11d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.148+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/3156917070 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7e640a55f0 msgr2=0x7f7e640a7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:08.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.148+0000 7f7e7ffdf700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7e640a55f0 0x7f7e640a7aa0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400b2e0 tx=0x7f7e74005fd0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.148+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/3156917070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e78104ab0 msgr2=0x7f7e78195a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:08.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.148+0000 7f7e7ffdf700 1 --2- 172.21.15.107:0/3156917070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e78104ab0 0x7f7e78195a50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7e70007f10 tx=0x7f7e7000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.148+0000 7f7e7e57c700 1 -- 172.21.15.107:0/3156917070 reap_dead start 2023-07-21T22:28:08.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.148+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/3156917070 shutdown_connections 2023-07-21T22:28:08.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.148+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/3156917070 >> 172.21.15.107:0/3156917070 conn(0x7f7e78075b70 msgr2=0x7f7e780fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:08.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.148+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/3156917070 shutdown_connections 2023-07-21T22:28:08.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.148+0000 7f7e7ffdf700 1 -- 172.21.15.107:0/3156917070 wait complete. 2023-07-21T22:28:08.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.754+0000 7fe775211700 1 -- 172.21.15.107:0/4117088116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe770104ab0 msgr2=0x7fe770104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:08.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.754+0000 7fe775211700 1 --2- 172.21.15.107:0/4117088116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe770104ab0 0x7fe770104e80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe760009990 tx=0x7fe760009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.754+0000 7fe775211700 1 -- 172.21.15.107:0/4117088116 shutdown_connections 2023-07-21T22:28:08.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.754+0000 7fe775211700 1 --2- 172.21.15.107:0/4117088116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700ff460 0x7fe77010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.754+0000 7fe775211700 1 --2- 172.21.15.107:0/4117088116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe770104ab0 0x7fe770104e80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.754+0000 7fe775211700 1 --2- 172.21.15.107:0/4117088116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7700feab0 0x7fe7700fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.754+0000 7fe775211700 1 -- 172.21.15.107:0/4117088116 >> 172.21.15.107:0/4117088116 conn(0x7fe770075b70 msgr2=0x7fe770075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:08.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.754+0000 7fe775211700 1 -- 172.21.15.107:0/4117088116 shutdown_connections 2023-07-21T22:28:08.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.754+0000 7fe775211700 1 -- 172.21.15.107:0/4117088116 wait complete. 2023-07-21T22:28:08.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.755+0000 7fe775211700 1 Processor -- start 2023-07-21T22:28:08.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.755+0000 7fe775211700 1 -- start start 2023-07-21T22:28:08.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.755+0000 7fe775211700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700feab0 0x7fe770196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:08.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.755+0000 7fe775211700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700ff460 0x7fe770196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:08.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.755+0000 7fe775211700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe770104ab0 0x7fe7701a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:08.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.755+0000 7fe775211700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7701978a0 con 0x7fe7700feab0 2023-07-21T22:28:08.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.755+0000 7fe775211700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7701a1fe0 con 0x7fe770104ab0 2023-07-21T22:28:08.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.755+0000 7fe775211700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7701a21a0 con 0x7fe7700ff460 2023-07-21T22:28:08.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.755+0000 7fe76ed9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700feab0 0x7fe770196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:08.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.755+0000 7fe76ed9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700feab0 0x7fe770196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46774/0 (socket says 172.21.15.107:46774) 2023-07-21T22:28:08.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.755+0000 7fe76ed9d700 1 -- 172.21.15.107:0/801228406 learned_addr learned my addr 172.21.15.107:0/801228406 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:08.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.755+0000 7fe76f59e700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe770104ab0 0x7fe7701a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:08.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.756+0000 7fe76ed9d700 1 -- 172.21.15.107:0/801228406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700ff460 msgr2=0x7fe770196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:08.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.756+0000 7fe76e59c700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700ff460 0x7fe770196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:08.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.756+0000 7fe76ed9d700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7700ff460 0x7fe770196f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.756+0000 7fe76ed9d700 1 -- 172.21.15.107:0/801228406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe770104ab0 msgr2=0x7fe7701a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:08.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.756+0000 7fe76ed9d700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe770104ab0 0x7fe7701a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.756+0000 7fe76ed9d700 1 -- 172.21.15.107:0/801228406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7600095f0 con 0x7fe7700feab0 2023-07-21T22:28:08.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.756+0000 7fe76f59e700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe770104ab0 0x7fe7701a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:08.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.756+0000 7fe76ed9d700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700feab0 0x7fe770196a20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe760000c00 tx=0x7fe76000f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:08.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.756+0000 7fe757fff700 1 -- 172.21.15.107:0/801228406 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76001d070 con 0x7fe7700feab0 2023-07-21T22:28:08.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.756+0000 7fe775211700 1 -- 172.21.15.107:0/801228406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7701a2420 con 0x7fe7700feab0 2023-07-21T22:28:08.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.756+0000 7fe757fff700 1 -- 172.21.15.107:0/801228406 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe76000fca0 con 0x7fe7700feab0 2023-07-21T22:28:08.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.756+0000 7fe775211700 1 -- 172.21.15.107:0/801228406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7701a2890 con 0x7fe7700feab0 2023-07-21T22:28:08.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.756+0000 7fe757fff700 1 -- 172.21.15.107:0/801228406 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760017760 con 0x7fe7700feab0 2023-07-21T22:28:08.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.757+0000 7fe775211700 1 -- 172.21.15.107:0/801228406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe77004e210 con 0x7fe7700feab0 2023-07-21T22:28:08.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.758+0000 7fe757fff700 1 -- 172.21.15.107:0/801228406 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe7600178c0 con 0x7fe7700feab0 2023-07-21T22:28:08.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.758+0000 7fe757fff700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe7580a1260 0x7fe7580a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:08.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.759+0000 7fe76e59c700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe7580a1260 0x7fe7580a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:08.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.759+0000 7fe757fff700 1 -- 172.21.15.107:0/801228406 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe7600cea00 con 0x7fe7700feab0 2023-07-21T22:28:08.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.759+0000 7fe757fff700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159] conn(0x7fe7580aaa30 0x7fe7580ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:08.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.759+0000 7fe757fff700 1 -- 172.21.15.107:0/801228406 --> [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7580ad4f0 con 0x7fe7580aaa30 2023-07-21T22:28:08.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.759+0000 7fe757fff700 1 -- 172.21.15.107:0/801228406 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7600ced70 con 0x7fe7700feab0 2023-07-21T22:28:08.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.759+0000 7fe76e59c700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe7580a1260 0x7fe7580a3710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe7700688a0 tx=0x7fe764005e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:08.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.759+0000 7fe76f59e700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159] conn(0x7fe7580aaa30 0x7fe7580ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:08.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.759+0000 7fe76f59e700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159] conn(0x7fe7580aaa30 0x7fe7580ace40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:08.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.760+0000 7fe757fff700 1 -- 172.21.15.107:0/801228406 <== osd.8 v2:172.21.15.190:6816/3597513159 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe7580ad4f0 con 0x7fe7580aaa30 2023-07-21T22:28:08.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.776+0000 7fe775211700 1 -- 172.21.15.107:0/801228406 --> [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe77010ec50 con 0x7fe7580aaa30 2023-07-21T22:28:08.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.776+0000 7fe757fff700 1 -- 172.21.15.107:0/801228406 <== osd.8 v2:172.21.15.190:6816/3597513159 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe768010040 con 0x7fe7580aaa30 2023-07-21T22:28:08.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.777+0000 7fe775211700 1 -- 172.21.15.107:0/801228406 >> [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159] conn(0x7fe7580aaa30 msgr2=0x7fe7580ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:08.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.777+0000 7fe775211700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159] conn(0x7fe7580aaa30 0x7fe7580ace40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.777+0000 7fe775211700 1 -- 172.21.15.107:0/801228406 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe7580a1260 msgr2=0x7fe7580a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:08.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.777+0000 7fe775211700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe7580a1260 0x7fe7580a3710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe7700688a0 tx=0x7fe764005e10 comp rx=0 tx=0).stop 2023-07-21T22:28:08.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.777+0000 7fe775211700 1 -- 172.21.15.107:0/801228406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700feab0 msgr2=0x7fe770196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:08.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.777+0000 7fe775211700 1 --2- 172.21.15.107:0/801228406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7700feab0 0x7fe770196a20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe760000c00 tx=0x7fe76000f7b0 comp rx=0 tx=0).stop 2023-07-21T22:28:08.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.777+0000 7fe76f59e700 1 -- 172.21.15.107:0/801228406 reap_dead start 2023-07-21T22:28:08.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.777+0000 7fe775211700 1 -- 172.21.15.107:0/801228406 shutdown_connections 2023-07-21T22:28:08.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.777+0000 7fe775211700 1 -- 172.21.15.107:0/801228406 >> 172.21.15.107:0/801228406 conn(0x7fe770075b70 msgr2=0x7fe7700fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:08.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.777+0000 7fe775211700 1 -- 172.21.15.107:0/801228406 shutdown_connections 2023-07-21T22:28:08.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:08.777+0000 7fe775211700 1 -- 172.21.15.107:0/801228406 wait complete. 2023-07-21T22:28:09.128 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.081+0000 7fcff558c700 1 -- 172.21.15.107:0/1850778703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff00ff2d0 msgr2=0x7fcff00ff740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:09.128 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.081+0000 7fcff558c700 1 --2- 172.21.15.107:0/1850778703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff00ff2d0 0x7fcff00ff740 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00099c0 tx=0x7fcfe0009cd0 comp rx=0 tx=0).stop 2023-07-21T22:28:09.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.082+0000 7fcff558c700 1 -- 172.21.15.107:0/1850778703 shutdown_connections 2023-07-21T22:28:09.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.082+0000 7fcff558c700 1 --2- 172.21.15.107:0/1850778703 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff00ffc80 0x7fcff010b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:09.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.082+0000 7fcff558c700 1 --2- 172.21.15.107:0/1850778703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff0103ab0 0x7fcff0103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:09.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.082+0000 7fcff558c700 1 --2- 172.21.15.107:0/1850778703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff00ff2d0 0x7fcff00ff740 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:09.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.082+0000 7fcff558c700 1 -- 172.21.15.107:0/1850778703 >> 172.21.15.107:0/1850778703 conn(0x7fcff00750b0 msgr2=0x7fcff00754b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:09.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.082+0000 7fcff558c700 1 -- 172.21.15.107:0/1850778703 shutdown_connections 2023-07-21T22:28:09.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.082+0000 7fcff558c700 1 -- 172.21.15.107:0/1850778703 wait complete. 2023-07-21T22:28:09.131 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.082+0000 7fcff558c700 1 Processor -- start 2023-07-21T22:28:09.131 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.082+0000 7fcff558c700 1 -- start start 2023-07-21T22:28:09.131 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcff558c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff00ff2d0 0x7fcff0196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:09.131 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcff558c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff00ffc80 0x7fcff0196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:09.131 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcff558c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff0103ab0 0x7fcff01a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:09.131 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcff558c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff01978c0 con 0x7fcff0103ab0 2023-07-21T22:28:09.131 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcff558c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff01a2030 con 0x7fcff00ff2d0 2023-07-21T22:28:09.132 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcff558c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff01a21a0 con 0x7fcff00ffc80 2023-07-21T22:28:09.132 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff0103ab0 0x7fcff01a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:09.132 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff0103ab0 0x7fcff01a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46792/0 (socket says 172.21.15.107:46792) 2023-07-21T22:28:09.132 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfef7fe700 1 -- 172.21.15.107:0/1890563363 learned_addr learned my addr 172.21.15.107:0/1890563363 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:09.132 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfee7fc700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff00ffc80 0x7fcff0196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:09.132 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfeeffd700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff00ff2d0 0x7fcff0196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:09.132 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfef7fe700 1 -- 172.21.15.107:0/1890563363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff00ffc80 msgr2=0x7fcff0196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:09.133 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfef7fe700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcff00ffc80 0x7fcff0196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:09.133 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfef7fe700 1 -- 172.21.15.107:0/1890563363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff00ff2d0 msgr2=0x7fcff0196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:09.133 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfef7fe700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff00ff2d0 0x7fcff0196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:09.133 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfef7fe700 1 -- 172.21.15.107:0/1890563363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe00095f0 con 0x7fcff0103ab0 2023-07-21T22:28:09.133 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfef7fe700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff0103ab0 0x7fcff01a1af0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800ef30 tx=0x7fcfe800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:09.133 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfd7fff700 1 -- 172.21.15.107:0/1890563363 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8018070 con 0x7fcff0103ab0 2023-07-21T22:28:09.133 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfd7fff700 1 -- 172.21.15.107:0/1890563363 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfe800f040 con 0x7fcff0103ab0 2023-07-21T22:28:09.133 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcff558c700 1 -- 172.21.15.107:0/1890563363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff01a2420 con 0x7fcff0103ab0 2023-07-21T22:28:09.134 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcfd7fff700 1 -- 172.21.15.107:0/1890563363 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800cd00 con 0x7fcff0103ab0 2023-07-21T22:28:09.134 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.083+0000 7fcff558c700 1 -- 172.21.15.107:0/1890563363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff01a28a0 con 0x7fcff0103ab0 2023-07-21T22:28:09.134 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.084+0000 7fcff558c700 1 -- 172.21.15.107:0/1890563363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcff004e210 con 0x7fcff0103ab0 2023-07-21T22:28:09.135 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.085+0000 7fcfd7fff700 1 -- 172.21.15.107:0/1890563363 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fcfe8003a00 con 0x7fcff0103ab0 2023-07-21T22:28:09.135 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.086+0000 7fcfd7fff700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcfd80a11e0 0x7fcfd80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:09.135 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.086+0000 7fcfeeffd700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcfd80a11e0 0x7fcfd80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:09.135 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.086+0000 7fcfd7fff700 1 -- 172.21.15.107:0/1890563363 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcfe80ccdb0 con 0x7fcff0103ab0 2023-07-21T22:28:09.135 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.086+0000 7fcfd7fff700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199] conn(0x7fcfd80aa9b0 0x7fcfd80acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:09.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.086+0000 7fcfd7fff700 1 -- 172.21.15.107:0/1890563363 --> [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcfd80ad470 con 0x7fcfd80aa9b0 2023-07-21T22:28:09.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.086+0000 7fcfd7fff700 1 -- 172.21.15.107:0/1890563363 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfe80cd250 con 0x7fcff0103ab0 2023-07-21T22:28:09.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.086+0000 7fcfeeffd700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcfd80a11e0 0x7fcfd80a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4007b60 tx=0x7fcfe40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:09.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.087+0000 7fcfee7fc700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199] conn(0x7fcfd80aa9b0 0x7fcfd80acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:09.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.087+0000 7fcfee7fc700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199] conn(0x7fcfd80aa9b0 0x7fcfd80acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:09.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.088+0000 7fcfd7fff700 1 -- 172.21.15.107:0/1890563363 <== osd.1 v2:172.21.15.132:6800/1473726199 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcfd80ad470 con 0x7fcfd80aa9b0 2023-07-21T22:28:09.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.104+0000 7fcff558c700 1 -- 172.21.15.107:0/1890563363 --> [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcff010ed00 con 0x7fcfd80aa9b0 2023-07-21T22:28:09.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.104+0000 7fcfd7fff700 1 -- 172.21.15.107:0/1890563363 <== osd.1 v2:172.21.15.132:6800/1473726199 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcfe001e040 con 0x7fcfd80aa9b0 2023-07-21T22:28:09.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.104+0000 7fcff558c700 1 -- 172.21.15.107:0/1890563363 >> [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199] conn(0x7fcfd80aa9b0 msgr2=0x7fcfd80acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:09.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.104+0000 7fcff558c700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199] conn(0x7fcfd80aa9b0 0x7fcfd80acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:09.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.105+0000 7fcff558c700 1 -- 172.21.15.107:0/1890563363 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcfd80a11e0 msgr2=0x7fcfd80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:09.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.105+0000 7fcff558c700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcfd80a11e0 0x7fcfd80a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4007b60 tx=0x7fcfe40073d0 comp rx=0 tx=0).stop 2023-07-21T22:28:09.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.105+0000 7fcff558c700 1 -- 172.21.15.107:0/1890563363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff0103ab0 msgr2=0x7fcff01a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:09.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.105+0000 7fcff558c700 1 --2- 172.21.15.107:0/1890563363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcff0103ab0 0x7fcff01a1af0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800ef30 tx=0x7fcfe800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:28:09.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.105+0000 7fcfef7fe700 1 -- 172.21.15.107:0/1890563363 reap_dead start 2023-07-21T22:28:09.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.105+0000 7fcff558c700 1 -- 172.21.15.107:0/1890563363 shutdown_connections 2023-07-21T22:28:09.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.105+0000 7fcff558c700 1 -- 172.21.15.107:0/1890563363 >> 172.21.15.107:0/1890563363 conn(0x7fcff00750b0 msgr2=0x7fcff00fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:09.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.105+0000 7fcff558c700 1 -- 172.21.15.107:0/1890563363 shutdown_connections 2023-07-21T22:28:09.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:09.105+0000 7fcff558c700 1 -- 172.21.15.107:0/1890563363 wait complete. 2023-07-21T22:28:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:09 smithi190 ceph-mon[98334]: pgmap v263: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-07-21T22:28:09.403 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:09 smithi107 ceph-mon[96543]: pgmap v263: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-07-21T22:28:09.404 INFO:teuthology.orchestra.run.smithi107.stdout:141733920813 2023-07-21T22:28:09.405 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-21T22:28:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:09 smithi132 ceph-mon[97863]: pgmap v263: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-07-21T22:28:10.374 INFO:teuthology.orchestra.run.smithi107.stdout:55834574906 2023-07-21T22:28:10.374 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-21T22:28:10.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.712+0000 7f57382e7700 1 -- 172.21.15.107:0/1942525731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57300feab0 msgr2=0x7f57300fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:10.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.712+0000 7f57382e7700 1 --2- 172.21.15.107:0/1942525731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57300feab0 0x7f57300fef20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5724009990 tx=0x7f5724009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.712+0000 7f57382e7700 1 -- 172.21.15.107:0/1942525731 shutdown_connections 2023-07-21T22:28:10.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.712+0000 7f57382e7700 1 --2- 172.21.15.107:0/1942525731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57300ff460 0x7f573010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.712+0000 7f57382e7700 1 --2- 172.21.15.107:0/1942525731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5730104ab0 0x7f5730104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.712+0000 7f57382e7700 1 --2- 172.21.15.107:0/1942525731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57300feab0 0x7f57300fef20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.712+0000 7f57382e7700 1 -- 172.21.15.107:0/1942525731 >> 172.21.15.107:0/1942525731 conn(0x7f5730075b70 msgr2=0x7f5730075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:10.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.712+0000 7f57382e7700 1 -- 172.21.15.107:0/1942525731 shutdown_connections 2023-07-21T22:28:10.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.712+0000 7f57382e7700 1 -- 172.21.15.107:0/1942525731 wait complete. 2023-07-21T22:28:10.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.713+0000 7f57382e7700 1 Processor -- start 2023-07-21T22:28:10.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.713+0000 7f57382e7700 1 -- start start 2023-07-21T22:28:10.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.713+0000 7f57382e7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57300feab0 0x7f573019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:10.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.713+0000 7f57382e7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57300ff460 0x7f573019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:10.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.713+0000 7f57382e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5730104ab0 0x7f5730194930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:10.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.713+0000 7f57382e7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5730194fa0 con 0x7f5730104ab0 2023-07-21T22:28:10.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.713+0000 7f57382e7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5730195110 con 0x7f57300feab0 2023-07-21T22:28:10.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.713+0000 7f57382e7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57301952d0 con 0x7f57300ff460 2023-07-21T22:28:10.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.713+0000 7f5736884700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5730104ab0 0x7f5730194930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:10.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.713+0000 7f5736884700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5730104ab0 0x7f5730194930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46806/0 (socket says 172.21.15.107:46806) 2023-07-21T22:28:10.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.713+0000 7f5736884700 1 -- 172.21.15.107:0/2814847515 learned_addr learned my addr 172.21.15.107:0/2814847515 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:10.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.713+0000 7f5736083700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57300feab0 0x7f573019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:10.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.713+0000 7f5735882700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57300ff460 0x7f573019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:10.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.714+0000 7f5736884700 1 -- 172.21.15.107:0/2814847515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57300ff460 msgr2=0x7f573019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:10.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.714+0000 7f5736884700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57300ff460 0x7f573019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.714+0000 7f5736884700 1 -- 172.21.15.107:0/2814847515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57300feab0 msgr2=0x7f573019a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:10.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.714+0000 7f5736884700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57300feab0 0x7f573019a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.714+0000 7f5736884700 1 -- 172.21.15.107:0/2814847515 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57240095f0 con 0x7f5730104ab0 2023-07-21T22:28:10.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.714+0000 7f5736884700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5730104ab0 0x7f5730194930 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f572800d6d0 tx=0x7f572800d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.714+0000 7f572f7fe700 1 -- 172.21.15.107:0/2814847515 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728004a30 con 0x7f5730104ab0 2023-07-21T22:28:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.714+0000 7f572f7fe700 1 -- 172.21.15.107:0/2814847515 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57280054e0 con 0x7f5730104ab0 2023-07-21T22:28:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.714+0000 7f57382e7700 1 -- 172.21.15.107:0/2814847515 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57301955b0 con 0x7f5730104ab0 2023-07-21T22:28:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.714+0000 7f57382e7700 1 -- 172.21.15.107:0/2814847515 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57300681a0 con 0x7f5730104ab0 2023-07-21T22:28:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.714+0000 7f572f7fe700 1 -- 172.21.15.107:0/2814847515 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572800fbf0 con 0x7f5730104ab0 2023-07-21T22:28:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.715+0000 7f57382e7700 1 -- 172.21.15.107:0/2814847515 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f571c000ff0 con 0x7f5730104ab0 2023-07-21T22:28:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.716+0000 7f572f7fe700 1 -- 172.21.15.107:0/2814847515 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f572800fd50 con 0x7f5730104ab0 2023-07-21T22:28:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.717+0000 7f572f7fe700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57180a11e0 0x7f57180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.717+0000 7f5736083700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57180a11e0 0x7f57180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.717+0000 7f572f7fe700 1 -- 172.21.15.107:0/2814847515 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f57280cccf0 con 0x7f5730104ab0 2023-07-21T22:28:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.718+0000 7f572f7fe700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252] conn(0x7f57180aa9b0 0x7f57180acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.718+0000 7f572f7fe700 1 -- 172.21.15.107:0/2814847515 --> [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57180ad470 con 0x7f57180aa9b0 2023-07-21T22:28:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.718+0000 7f572f7fe700 1 -- 172.21.15.107:0/2814847515 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57280cd190 con 0x7f5730104ab0 2023-07-21T22:28:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.718+0000 7f5735882700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252] conn(0x7f57180aa9b0 0x7f57180acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:10.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.718+0000 7f5736083700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57180a11e0 0x7f57180a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f57300ffbf0 tx=0x7f57200091c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:10.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.718+0000 7f5735882700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252] conn(0x7f57180aa9b0 0x7f57180acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:10.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.719+0000 7f572f7fe700 1 -- 172.21.15.107:0/2814847515 <== osd.9 v2:172.21.15.107:6826/458109252 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f57180ad470 con 0x7f57180aa9b0 2023-07-21T22:28:10.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.734+0000 7f57382e7700 1 -- 172.21.15.107:0/2814847515 --> [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f571c002d70 con 0x7f57180aa9b0 2023-07-21T22:28:10.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.735+0000 7f572f7fe700 1 -- 172.21.15.107:0/2814847515 <== osd.9 v2:172.21.15.107:6826/458109252 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f571c002d70 con 0x7f57180aa9b0 2023-07-21T22:28:10.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.735+0000 7f57382e7700 1 -- 172.21.15.107:0/2814847515 >> [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252] conn(0x7f57180aa9b0 msgr2=0x7f57180acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:10.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.735+0000 7f57382e7700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252] conn(0x7f57180aa9b0 0x7f57180acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.735+0000 7f57382e7700 1 -- 172.21.15.107:0/2814847515 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57180a11e0 msgr2=0x7f57180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:10.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.735+0000 7f57382e7700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f57180a11e0 0x7f57180a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f57300ffbf0 tx=0x7f57200091c0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.735+0000 7f57382e7700 1 -- 172.21.15.107:0/2814847515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5730104ab0 msgr2=0x7f5730194930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:10.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.735+0000 7f57382e7700 1 --2- 172.21.15.107:0/2814847515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5730104ab0 0x7f5730194930 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f572800d6d0 tx=0x7f572800d9e0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.735+0000 7f5736884700 1 -- 172.21.15.107:0/2814847515 reap_dead start 2023-07-21T22:28:10.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.736+0000 7f57382e7700 1 -- 172.21.15.107:0/2814847515 shutdown_connections 2023-07-21T22:28:10.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.736+0000 7f57382e7700 1 -- 172.21.15.107:0/2814847515 >> 172.21.15.107:0/2814847515 conn(0x7f5730075b70 msgr2=0x7f57300fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:10.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.736+0000 7f57382e7700 1 -- 172.21.15.107:0/2814847515 shutdown_connections 2023-07-21T22:28:10.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.736+0000 7f57382e7700 1 -- 172.21.15.107:0/2814847515 wait complete. 2023-07-21T22:28:10.863 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.816+0000 7f9aa4be1700 1 -- 172.21.15.107:0/3102797255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa00feb70 msgr2=0x7f9aa010cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:10.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.816+0000 7f9aa4be1700 1 --2- 172.21.15.107:0/3102797255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa00feb70 0x7f9aa010cab0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800b0a0 tx=0x7f9a9800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f9aa4be1700 1 -- 172.21.15.107:0/3102797255 shutdown_connections 2023-07-21T22:28:10.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f9aa4be1700 1 --2- 172.21.15.107:0/3102797255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa00feb70 0x7f9aa010cab0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f9aa4be1700 1 --2- 172.21.15.107:0/3102797255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9aa0105d70 0x7f9aa0106140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f9aa4be1700 1 --2- 172.21.15.107:0/3102797255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa00fe1c0 0x7f9aa00fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f9aa4be1700 1 -- 172.21.15.107:0/3102797255 >> 172.21.15.107:0/3102797255 conn(0x7f9aa0075150 msgr2=0x7f9aa0075550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:10.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f9aa4be1700 1 -- 172.21.15.107:0/3102797255 shutdown_connections 2023-07-21T22:28:10.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f9aa4be1700 1 -- 172.21.15.107:0/3102797255 wait complete. 2023-07-21T22:28:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9aa4be1700 1 Processor -- start 2023-07-21T22:28:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9aa4be1700 1 -- start start 2023-07-21T22:28:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9aa4be1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9aa00fe1c0 0x7f9aa010b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9aa4be1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa00feb70 0x7f9aa010bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9aa4be1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa0105d70 0x7f9aa0199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9aa4be1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa0199f00 con 0x7f9aa00feb70 2023-07-21T22:28:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9aa4be1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa019a070 con 0x7f9aa00fe1c0 2023-07-21T22:28:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9aa4be1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa019a1e0 con 0x7f9aa0105d70 2023-07-21T22:28:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9a9dd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa00feb70 0x7f9aa010bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9a9dd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa00feb70 0x7f9aa010bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46836/0 (socket says 172.21.15.107:46836) 2023-07-21T22:28:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9a9dd9b700 1 -- 172.21.15.107:0/2702567653 learned_addr learned my addr 172.21.15.107:0/2702567653 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9a9ed9d700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa0105d70 0x7f9aa0199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9a9e59c700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9aa00fe1c0 0x7f9aa010b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9a9ed9d700 1 -- 172.21.15.107:0/2702567653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9aa00fe1c0 msgr2=0x7f9aa010b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9a9ed9d700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9aa00fe1c0 0x7f9aa010b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9a9ed9d700 1 -- 172.21.15.107:0/2702567653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa00feb70 msgr2=0x7f9aa010bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9a9ed9d700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa00feb70 0x7f9aa010bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.818+0000 7f9a9ed9d700 1 -- 172.21.15.107:0/2702567653 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a98009d00 con 0x7f9aa0105d70 2023-07-21T22:28:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.819+0000 7f9a9dd9b700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa00feb70 0x7f9aa010bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:28:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.819+0000 7f9a9ed9d700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa0105d70 0x7f9aa0199900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009fd0 tx=0x7f9a9800bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.819+0000 7f9a8f7fe700 1 -- 172.21.15.107:0/2702567653 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9800e040 con 0x7f9aa0105d70 2023-07-21T22:28:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.819+0000 7f9a8f7fe700 1 -- 172.21.15.107:0/2702567653 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a980094c0 con 0x7f9aa0105d70 2023-07-21T22:28:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.819+0000 7f9aa4be1700 1 -- 172.21.15.107:0/2702567653 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa019a400 con 0x7f9aa0105d70 2023-07-21T22:28:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.819+0000 7f9a8f7fe700 1 -- 172.21.15.107:0/2702567653 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98005040 con 0x7f9aa0105d70 2023-07-21T22:28:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.819+0000 7f9aa4be1700 1 -- 172.21.15.107:0/2702567653 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa0196830 con 0x7f9aa0105d70 2023-07-21T22:28:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.820+0000 7f9aa4be1700 1 -- 172.21.15.107:0/2702567653 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9aa010fe30 con 0x7f9aa0105d70 2023-07-21T22:28:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.822+0000 7f9a8f7fe700 1 -- 172.21.15.107:0/2702567653 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f9a9801d470 con 0x7f9aa0105d70 2023-07-21T22:28:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.822+0000 7f9a8f7fe700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a880a11e0 0x7f9a880a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.823+0000 7f9a9e59c700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a880a11e0 0x7f9a880a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.823+0000 7f9a8f7fe700 1 -- 172.21.15.107:0/2702567653 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9a980ce680 con 0x7f9aa0105d70 2023-07-21T22:28:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.823+0000 7f9a8f7fe700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367] conn(0x7f9a880aa9b0 0x7f9a880acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.823+0000 7f9a8f7fe700 1 -- 172.21.15.107:0/2702567653 --> [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a880ad470 con 0x7f9a880aa9b0 2023-07-21T22:28:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.823+0000 7f9a8f7fe700 1 -- 172.21.15.107:0/2702567653 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a980ce9f0 con 0x7f9aa0105d70 2023-07-21T22:28:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.823+0000 7f9a9e59c700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a880a11e0 0x7f9a880a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9a90005950 tx=0x7f9a9000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.824+0000 7f9a9dd9b700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367] conn(0x7f9a880aa9b0 0x7f9a880acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.825+0000 7f9a9dd9b700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367] conn(0x7f9a880aa9b0 0x7f9a880acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.826+0000 7f9a8f7fe700 1 -- 172.21.15.107:0/2702567653 <== osd.10 v2:172.21.15.132:6824/2698708367 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9a880ad470 con 0x7f9a880aa9b0 2023-07-21T22:28:10.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.843+0000 7f9aa4be1700 1 -- 172.21.15.107:0/2702567653 --> [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9aa004e210 con 0x7f9a880aa9b0 2023-07-21T22:28:10.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.844+0000 7f9a8f7fe700 1 -- 172.21.15.107:0/2702567653 <== osd.10 v2:172.21.15.132:6824/2698708367 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9aa004e210 con 0x7f9a880aa9b0 2023-07-21T22:28:10.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.844+0000 7f9aa4be1700 1 -- 172.21.15.107:0/2702567653 >> [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367] conn(0x7f9a880aa9b0 msgr2=0x7f9a880acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:10.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.844+0000 7f9aa4be1700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367] conn(0x7f9a880aa9b0 0x7f9a880acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.845+0000 7f9aa4be1700 1 -- 172.21.15.107:0/2702567653 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a880a11e0 msgr2=0x7f9a880a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:10.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.845+0000 7f9aa4be1700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a880a11e0 0x7f9a880a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9a90005950 tx=0x7f9a9000b410 comp rx=0 tx=0).stop 2023-07-21T22:28:10.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.845+0000 7f9aa4be1700 1 -- 172.21.15.107:0/2702567653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa0105d70 msgr2=0x7f9aa0199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:10.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.845+0000 7f9aa4be1700 1 --2- 172.21.15.107:0/2702567653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa0105d70 0x7f9aa0199900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009fd0 tx=0x7f9a9800bce0 comp rx=0 tx=0).stop 2023-07-21T22:28:10.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.845+0000 7f9a9ed9d700 1 -- 172.21.15.107:0/2702567653 reap_dead start 2023-07-21T22:28:10.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.845+0000 7f9aa4be1700 1 -- 172.21.15.107:0/2702567653 shutdown_connections 2023-07-21T22:28:10.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.845+0000 7f9aa4be1700 1 -- 172.21.15.107:0/2702567653 >> 172.21.15.107:0/2702567653 conn(0x7f9aa0075150 msgr2=0x7f9aa00fbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:10.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.845+0000 7f9aa4be1700 1 -- 172.21.15.107:0/2702567653 shutdown_connections 2023-07-21T22:28:10.893 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.845+0000 7f9aa4be1700 1 -- 172.21.15.107:0/2702567653 wait complete. 2023-07-21T22:28:11.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.024+0000 7f7f81db8700 1 -- 172.21.15.107:0/1480013174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f7c0ffe30 msgr2=0x7f7f7c100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:11.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.024+0000 7f7f81db8700 1 --2- 172.21.15.107:0/1480013174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f7c0ffe30 0x7f7f7c100200 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0099c0 tx=0x7f7f6c009cd0 comp rx=0 tx=0).stop 2023-07-21T22:28:11.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.025+0000 7f7f81db8700 1 -- 172.21.15.107:0/1480013174 shutdown_connections 2023-07-21T22:28:11.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.025+0000 7f7f81db8700 1 --2- 172.21.15.107:0/1480013174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f7c104ed0 0x7f7f7c10ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:11.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.025+0000 7f7f81db8700 1 --2- 172.21.15.107:0/1480013174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f7c0ffe30 0x7f7f7c100200 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:11.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.025+0000 7f7f81db8700 1 --2- 172.21.15.107:0/1480013174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c1007d0 0x7f7f7c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:11.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.025+0000 7f7f81db8700 1 -- 172.21.15.107:0/1480013174 >> 172.21.15.107:0/1480013174 conn(0x7f7f7c0fb710 msgr2=0x7f7f7c0fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:11.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.025+0000 7f7f81db8700 1 -- 172.21.15.107:0/1480013174 shutdown_connections 2023-07-21T22:28:11.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.025+0000 7f7f81db8700 1 -- 172.21.15.107:0/1480013174 wait complete. 2023-07-21T22:28:11.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.025+0000 7f7f81db8700 1 Processor -- start 2023-07-21T22:28:11.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f81db8700 1 -- start start 2023-07-21T22:28:11.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f81db8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0ffe30 0x7f7f7c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:11.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f81db8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f7c1007d0 0x7f7f7c19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:11.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f81db8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f7c104ed0 0x7f7f7c195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:11.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f81db8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c196150 con 0x7f7f7c104ed0 2023-07-21T22:28:11.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f81db8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c1962c0 con 0x7f7f7c1007d0 2023-07-21T22:28:11.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f81db8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c196480 con 0x7f7f7c0ffe30 2023-07-21T22:28:11.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f7bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f7c104ed0 0x7f7f7c195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:11.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f7bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f7c104ed0 0x7f7f7c195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46854/0 (socket says 172.21.15.107:46854) 2023-07-21T22:28:11.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f7bfff700 1 -- 172.21.15.107:0/3992352368 learned_addr learned my addr 172.21.15.107:0/3992352368 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:11.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f7b7fe700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0ffe30 0x7f7f7c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:11.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f7affd700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f7c1007d0 0x7f7f7c19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:11.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f7bfff700 1 -- 172.21.15.107:0/3992352368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0ffe30 msgr2=0x7f7f7c19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:11.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f7bfff700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f7c0ffe30 0x7f7f7c19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:11.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f7bfff700 1 -- 172.21.15.107:0/3992352368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f7c1007d0 msgr2=0x7f7f7c19bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:11.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f7bfff700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f7c1007d0 0x7f7f7c19bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:11.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f7bfff700 1 -- 172.21.15.107:0/3992352368 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0095f0 con 0x7f7f7c104ed0 2023-07-21T22:28:11.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.026+0000 7f7f7bfff700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f7c104ed0 0x7f7f7c195ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7f74007ee0 tx=0x7f7f7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:11.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.027+0000 7f7f78ff9700 1 -- 172.21.15.107:0/3992352368 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74017070 con 0x7f7f7c104ed0 2023-07-21T22:28:11.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.027+0000 7f7f78ff9700 1 -- 172.21.15.107:0/3992352368 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f7400f040 con 0x7f7f7c104ed0 2023-07-21T22:28:11.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.027+0000 7f7f81db8700 1 -- 172.21.15.107:0/3992352368 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c196760 con 0x7f7f7c104ed0 2023-07-21T22:28:11.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.027+0000 7f7f78ff9700 1 -- 172.21.15.107:0/3992352368 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74005580 con 0x7f7f7c104ed0 2023-07-21T22:28:11.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.027+0000 7f7f81db8700 1 -- 172.21.15.107:0/3992352368 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c1a2a50 con 0x7f7f7c104ed0 2023-07-21T22:28:11.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.027+0000 7f7f81db8700 1 -- 172.21.15.107:0/3992352368 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f7c04e210 con 0x7f7f7c104ed0 2023-07-21T22:28:11.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.029+0000 7f7f78ff9700 1 -- 172.21.15.107:0/3992352368 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f7f7400db20 con 0x7f7f7c104ed0 2023-07-21T22:28:11.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.029+0000 7f7f78ff9700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7f640a1260 0x7f7f640a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:11.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.029+0000 7f7f7b7fe700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7f640a1260 0x7f7f640a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:11.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.029+0000 7f7f78ff9700 1 -- 172.21.15.107:0/3992352368 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7f740cc5e0 con 0x7f7f7c104ed0 2023-07-21T22:28:11.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.029+0000 7f7f78ff9700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368] conn(0x7f7f640aaa30 0x7f7f640ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:11.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.029+0000 7f7f78ff9700 1 -- 172.21.15.107:0/3992352368 --> [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f640ad4f0 con 0x7f7f640aaa30 2023-07-21T22:28:11.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.029+0000 7f7f78ff9700 1 -- 172.21.15.107:0/3992352368 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f740cca80 con 0x7f7f7c104ed0 2023-07-21T22:28:11.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.030+0000 7f7f7affd700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368] conn(0x7f7f640aaa30 0x7f7f640ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:11.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.030+0000 7f7f7b7fe700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7f640a1260 0x7f7f640a3710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c000c00 tx=0x7f7f6c003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:11.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.030+0000 7f7f7affd700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368] conn(0x7f7f640aaa30 0x7f7f640ace40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:11.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.030+0000 7f7f78ff9700 1 -- 172.21.15.107:0/3992352368 <== osd.6 v2:172.21.15.107:6818/3497265368 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7f640ad4f0 con 0x7f7f640aaa30 2023-07-21T22:28:11.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.046+0000 7f7f81db8700 1 -- 172.21.15.107:0/3992352368 --> [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f7c0fcbe0 con 0x7f7f640aaa30 2023-07-21T22:28:11.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.047+0000 7f7f78ff9700 1 -- 172.21.15.107:0/3992352368 <== osd.6 v2:172.21.15.107:6818/3497265368 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7f7c0fcbe0 con 0x7f7f640aaa30 2023-07-21T22:28:11.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.047+0000 7f7f81db8700 1 -- 172.21.15.107:0/3992352368 >> [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368] conn(0x7f7f640aaa30 msgr2=0x7f7f640ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:11.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.047+0000 7f7f81db8700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368] conn(0x7f7f640aaa30 0x7f7f640ace40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:11.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.047+0000 7f7f81db8700 1 -- 172.21.15.107:0/3992352368 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7f640a1260 msgr2=0x7f7f640a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:11.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.047+0000 7f7f81db8700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7f640a1260 0x7f7f640a3710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c000c00 tx=0x7f7f6c003680 comp rx=0 tx=0).stop 2023-07-21T22:28:11.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.047+0000 7f7f81db8700 1 -- 172.21.15.107:0/3992352368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f7c104ed0 msgr2=0x7f7f7c195ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:11.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.047+0000 7f7f81db8700 1 --2- 172.21.15.107:0/3992352368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f7c104ed0 0x7f7f7c195ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7f74007ee0 tx=0x7f7f7400d3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:11.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.047+0000 7f7f7bfff700 1 -- 172.21.15.107:0/3992352368 reap_dead start 2023-07-21T22:28:11.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.047+0000 7f7f81db8700 1 -- 172.21.15.107:0/3992352368 shutdown_connections 2023-07-21T22:28:11.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.047+0000 7f7f81db8700 1 -- 172.21.15.107:0/3992352368 >> 172.21.15.107:0/3992352368 conn(0x7f7f7c0fb710 msgr2=0x7f7f7c0fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:11.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.047+0000 7f7f81db8700 1 -- 172.21.15.107:0/3992352368 shutdown_connections 2023-07-21T22:28:11.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:11.047+0000 7f7f81db8700 1 -- 172.21.15.107:0/3992352368 wait complete. 2023-07-21T22:28:11.180 INFO:teuthology.orchestra.run.smithi107.stdout:107374182450 2023-07-21T22:28:11.181 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-21T22:28:11.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:11 smithi190 ceph-mon[98334]: pgmap v264: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:28:11.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:11 smithi132 ceph-mon[97863]: pgmap v264: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:28:11.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:11 smithi107 ceph-mon[96543]: pgmap v264: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:28:11.986 INFO:teuthology.orchestra.run.smithi107.stdout:300647710741 2023-07-21T22:28:11.986 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-21T22:28:11.987 INFO:teuthology.orchestra.run.smithi107.stdout:193273528357 2023-07-21T22:28:11.988 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-21T22:28:12.062 INFO:teuthology.orchestra.run.smithi107.stdout:244813135901 2023-07-21T22:28:12.063 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-21T22:28:12.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.792+0000 7f1b55ecb700 1 -- 172.21.15.107:0/3845374167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b500feab0 msgr2=0x7f1b500fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:12.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.792+0000 7f1b55ecb700 1 --2- 172.21.15.107:0/3845374167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b500feab0 0x7f1b500fef20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009990 tx=0x7f1b38009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:12.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.792+0000 7f1b55ecb700 1 -- 172.21.15.107:0/3845374167 shutdown_connections 2023-07-21T22:28:12.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.792+0000 7f1b55ecb700 1 --2- 172.21.15.107:0/3845374167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b500ff460 0x7f1b5010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:12.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.792+0000 7f1b55ecb700 1 --2- 172.21.15.107:0/3845374167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b50104ab0 0x7f1b50104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:12.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.792+0000 7f1b55ecb700 1 --2- 172.21.15.107:0/3845374167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b500feab0 0x7f1b500fef20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:12.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.792+0000 7f1b55ecb700 1 -- 172.21.15.107:0/3845374167 >> 172.21.15.107:0/3845374167 conn(0x7f1b50075b70 msgr2=0x7f1b50075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:12.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.792+0000 7f1b55ecb700 1 -- 172.21.15.107:0/3845374167 shutdown_connections 2023-07-21T22:28:12.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.792+0000 7f1b55ecb700 1 -- 172.21.15.107:0/3845374167 wait complete. 2023-07-21T22:28:12.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.793+0000 7f1b55ecb700 1 Processor -- start 2023-07-21T22:28:12.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.793+0000 7f1b55ecb700 1 -- start start 2023-07-21T22:28:12.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b55ecb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b500feab0 0x7f1b5019a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:12.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b55ecb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b500ff460 0x7f1b5019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:12.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b55ecb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b50104ab0 0x7f1b50194900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:12.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b55ecb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b50194f70 con 0x7f1b500feab0 2023-07-21T22:28:12.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b55ecb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b501950e0 con 0x7f1b500ff460 2023-07-21T22:28:12.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b55ecb700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b501952a0 con 0x7f1b50104ab0 2023-07-21T22:28:12.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b4f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b500feab0 0x7f1b5019a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:12.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b4f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b500feab0 0x7f1b5019a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46820/0 (socket says 172.21.15.107:46820) 2023-07-21T22:28:12.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b4f7fe700 1 -- 172.21.15.107:0/749962805 learned_addr learned my addr 172.21.15.107:0/749962805 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:12.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b4ffff700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b50104ab0 0x7f1b50194900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:12.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b4effd700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b500ff460 0x7f1b5019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:12.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b4f7fe700 1 -- 172.21.15.107:0/749962805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b50104ab0 msgr2=0x7f1b50194900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:12.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b4f7fe700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b50104ab0 0x7f1b50194900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:12.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b4f7fe700 1 -- 172.21.15.107:0/749962805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b500ff460 msgr2=0x7f1b5019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:12.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b4f7fe700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b500ff460 0x7f1b5019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:12.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b4f7fe700 1 -- 172.21.15.107:0/749962805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b380095f0 con 0x7f1b500feab0 2023-07-21T22:28:12.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b4f7fe700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b500feab0 0x7f1b5019a880 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1b48007ee0 tx=0x7f1b4800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:12.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.794+0000 7f1b4cff9700 1 -- 172.21.15.107:0/749962805 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4800dd20 con 0x7f1b500feab0 2023-07-21T22:28:12.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.795+0000 7f1b55ecb700 1 -- 172.21.15.107:0/749962805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b50195580 con 0x7f1b500feab0 2023-07-21T22:28:12.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.795+0000 7f1b4cff9700 1 -- 172.21.15.107:0/749962805 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b4800f040 con 0x7f1b500feab0 2023-07-21T22:28:12.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.795+0000 7f1b4cff9700 1 -- 172.21.15.107:0/749962805 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48014600 con 0x7f1b500feab0 2023-07-21T22:28:12.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.795+0000 7f1b55ecb700 1 -- 172.21.15.107:0/749962805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b501a1870 con 0x7f1b500feab0 2023-07-21T22:28:12.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.795+0000 7f1b55ecb700 1 -- 172.21.15.107:0/749962805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1b5004e210 con 0x7f1b500feab0 2023-07-21T22:28:12.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.797+0000 7f1b4cff9700 1 -- 172.21.15.107:0/749962805 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f1b48014820 con 0x7f1b500feab0 2023-07-21T22:28:12.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.797+0000 7f1b4cff9700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1b3c0a11e0 0x7f1b3c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:12.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.797+0000 7f1b4effd700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1b3c0a11e0 0x7f1b3c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:12.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.797+0000 7f1b4cff9700 1 -- 172.21.15.107:0/749962805 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1b480cca80 con 0x7f1b500feab0 2023-07-21T22:28:12.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.797+0000 7f1b4cff9700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112] conn(0x7f1b3c0aa9b0 0x7f1b3c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:12.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.797+0000 7f1b4cff9700 1 -- 172.21.15.107:0/749962805 --> [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b3c0ad470 con 0x7f1b3c0aa9b0 2023-07-21T22:28:12.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.797+0000 7f1b4cff9700 1 -- 172.21.15.107:0/749962805 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b480ccf20 con 0x7f1b500feab0 2023-07-21T22:28:12.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.798+0000 7f1b4effd700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1b3c0a11e0 0x7f1b3c0a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800b5c0 tx=0x7f1b38005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:12.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.798+0000 7f1b4ffff700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112] conn(0x7f1b3c0aa9b0 0x7f1b3c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:12.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.798+0000 7f1b4ffff700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112] conn(0x7f1b3c0aa9b0 0x7f1b3c0acdc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:12.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.799+0000 7f1b4cff9700 1 -- 172.21.15.107:0/749962805 <== osd.11 v2:172.21.15.190:6824/1655561112 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1b3c0ad470 con 0x7f1b3c0aa9b0 2023-07-21T22:28:12.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.816+0000 7f1b55ecb700 1 -- 172.21.15.107:0/749962805 --> [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1b500fd780 con 0x7f1b3c0aa9b0 2023-07-21T22:28:12.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.816+0000 7f1b4cff9700 1 -- 172.21.15.107:0/749962805 <== osd.11 v2:172.21.15.190:6824/1655561112 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1b500fd780 con 0x7f1b3c0aa9b0 2023-07-21T22:28:12.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f1b55ecb700 1 -- 172.21.15.107:0/749962805 >> [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112] conn(0x7f1b3c0aa9b0 msgr2=0x7f1b3c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:12.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f1b55ecb700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112] conn(0x7f1b3c0aa9b0 0x7f1b3c0acdc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:12.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f1b55ecb700 1 -- 172.21.15.107:0/749962805 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1b3c0a11e0 msgr2=0x7f1b3c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:12.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f1b55ecb700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1b3c0a11e0 0x7f1b3c0a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800b5c0 tx=0x7f1b38005960 comp rx=0 tx=0).stop 2023-07-21T22:28:12.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f1b55ecb700 1 -- 172.21.15.107:0/749962805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b500feab0 msgr2=0x7f1b5019a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:12.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f1b55ecb700 1 --2- 172.21.15.107:0/749962805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b500feab0 0x7f1b5019a880 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1b48007ee0 tx=0x7f1b4800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:12.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f1b4ffff700 1 -- 172.21.15.107:0/749962805 reap_dead start 2023-07-21T22:28:12.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f1b55ecb700 1 -- 172.21.15.107:0/749962805 shutdown_connections 2023-07-21T22:28:12.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f1b55ecb700 1 -- 172.21.15.107:0/749962805 >> 172.21.15.107:0/749962805 conn(0x7f1b50075b70 msgr2=0x7f1b500fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:12.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f1b55ecb700 1 -- 172.21.15.107:0/749962805 shutdown_connections 2023-07-21T22:28:12.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:10.817+0000 7f1b55ecb700 1 -- 172.21.15.107:0/749962805 wait complete. 2023-07-21T22:28:12.088 INFO:teuthology.orchestra.run.smithi107.stdout:77309411383 2023-07-21T22:28:12.088 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-21T22:28:12.339 INFO:teuthology.orchestra.run.smithi107.stdout:279172874266 2023-07-21T22:28:12.340 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-21T22:28:12.626 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:12.682 INFO:teuthology.orchestra.run.smithi107.stdout:330712481809 2023-07-21T22:28:12.682 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-21T22:28:12.749 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:13.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.041+0000 7ff42611a700 1 -- 172.21.15.107:0/2312749289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4200ff460 msgr2=0x7ff42010b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:13.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.041+0000 7ff42611a700 1 --2- 172.21.15.107:0/2312749289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4200ff460 0x7ff42010b8f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff41800b0a0 tx=0x7ff41800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.041+0000 7ff42611a700 1 -- 172.21.15.107:0/2312749289 shutdown_connections 2023-07-21T22:28:13.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.041+0000 7ff42611a700 1 --2- 172.21.15.107:0/2312749289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4200ff460 0x7ff42010b8f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.041+0000 7ff42611a700 1 --2- 172.21.15.107:0/2312749289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff420104ab0 0x7ff420104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.041+0000 7ff42611a700 1 --2- 172.21.15.107:0/2312749289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4200feab0 0x7ff4200fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.041+0000 7ff42611a700 1 -- 172.21.15.107:0/2312749289 >> 172.21.15.107:0/2312749289 conn(0x7ff420075b70 msgr2=0x7ff420075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:13.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.041+0000 7ff42611a700 1 -- 172.21.15.107:0/2312749289 shutdown_connections 2023-07-21T22:28:13.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.041+0000 7ff42611a700 1 -- 172.21.15.107:0/2312749289 wait complete. 2023-07-21T22:28:13.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.041+0000 7ff42611a700 1 Processor -- start 2023-07-21T22:28:13.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.042+0000 7ff42611a700 1 -- start start 2023-07-21T22:28:13.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.042+0000 7ff42611a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4200feab0 0x7ff420196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:13.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.042+0000 7ff42611a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4200ff460 0x7ff420196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:13.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.042+0000 7ff42611a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff420104ab0 0x7ff4201a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:13.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.042+0000 7ff42611a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4201978e0 con 0x7ff4200ff460 2023-07-21T22:28:13.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.042+0000 7ff42611a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4201a2020 con 0x7ff4200feab0 2023-07-21T22:28:13.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.042+0000 7ff42611a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4201a21e0 con 0x7ff420104ab0 2023-07-21T22:28:13.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.042+0000 7ff41effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4200ff460 0x7ff420196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:13.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.042+0000 7ff41effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4200ff460 0x7ff420196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46866/0 (socket says 172.21.15.107:46866) 2023-07-21T22:28:13.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.042+0000 7ff41effd700 1 -- 172.21.15.107:0/868363045 learned_addr learned my addr 172.21.15.107:0/868363045 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:13.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.042+0000 7ff41ffff700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff420104ab0 0x7ff4201a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:13.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.043+0000 7ff41effd700 1 -- 172.21.15.107:0/868363045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff420104ab0 msgr2=0x7ff4201a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:13.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.043+0000 7ff41effd700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff420104ab0 0x7ff4201a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.043+0000 7ff41effd700 1 -- 172.21.15.107:0/868363045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4200feab0 msgr2=0x7ff420196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:13.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.043+0000 7ff41f7fe700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4200feab0 0x7ff420196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:13.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.043+0000 7ff41effd700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff4200feab0 0x7ff420196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.043+0000 7ff41effd700 1 -- 172.21.15.107:0/868363045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff418009d00 con 0x7ff4200ff460 2023-07-21T22:28:13.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.043+0000 7ff41effd700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4200ff460 0x7ff420196fa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff41400d950 tx=0x7ff41400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:13.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.043+0000 7ff41cff9700 1 -- 172.21.15.107:0/868363045 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414009940 con 0x7ff4200ff460 2023-07-21T22:28:13.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.043+0000 7ff42611a700 1 -- 172.21.15.107:0/868363045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4201a24c0 con 0x7ff4200ff460 2023-07-21T22:28:13.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.043+0000 7ff41cff9700 1 -- 172.21.15.107:0/868363045 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff414010460 con 0x7ff4200ff460 2023-07-21T22:28:13.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.043+0000 7ff41cff9700 1 -- 172.21.15.107:0/868363045 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41400f620 con 0x7ff4200ff460 2023-07-21T22:28:13.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.043+0000 7ff42611a700 1 -- 172.21.15.107:0/868363045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4201a29c0 con 0x7ff4200ff460 2023-07-21T22:28:13.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.044+0000 7ff42611a700 1 -- 172.21.15.107:0/868363045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff42004e210 con 0x7ff4200ff460 2023-07-21T22:28:13.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.045+0000 7ff41cff9700 1 -- 172.21.15.107:0/868363045 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff4140108d0 con 0x7ff4200ff460 2023-07-21T22:28:13.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.046+0000 7ff41cff9700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff4080a11e0 0x7ff4080a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:13.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.046+0000 7ff41f7fe700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff4080a11e0 0x7ff4080a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:13.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.046+0000 7ff41cff9700 1 -- 172.21.15.107:0/868363045 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff4140ccd30 con 0x7ff4200ff460 2023-07-21T22:28:13.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.046+0000 7ff41cff9700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834] conn(0x7ff4080aa9b0 0x7ff4080acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:13.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.046+0000 7ff41cff9700 1 -- 172.21.15.107:0/868363045 --> [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4080ad470 con 0x7ff4080aa9b0 2023-07-21T22:28:13.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.046+0000 7ff41cff9700 1 -- 172.21.15.107:0/868363045 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4140cd1d0 con 0x7ff4200ff460 2023-07-21T22:28:13.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.046+0000 7ff41f7fe700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff4080a11e0 0x7ff4080a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff4200ffbf0 tx=0x7ff410005d70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:13.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.046+0000 7ff41ffff700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834] conn(0x7ff4080aa9b0 0x7ff4080acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:13.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.047+0000 7ff41ffff700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834] conn(0x7ff4080aa9b0 0x7ff4080acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:13.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.048+0000 7ff41cff9700 1 -- 172.21.15.107:0/868363045 <== osd.7 v2:172.21.15.132:6816/2500130834 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff4080ad470 con 0x7ff4080aa9b0 2023-07-21T22:28:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.063+0000 7ff42611a700 1 -- 172.21.15.107:0/868363045 --> [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff42010ec50 con 0x7ff4080aa9b0 2023-07-21T22:28:13.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.064+0000 7ff41cff9700 1 -- 172.21.15.107:0/868363045 <== osd.7 v2:172.21.15.132:6816/2500130834 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff418009b80 con 0x7ff4080aa9b0 2023-07-21T22:28:13.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.064+0000 7ff42611a700 1 -- 172.21.15.107:0/868363045 >> [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834] conn(0x7ff4080aa9b0 msgr2=0x7ff4080acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:13.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.064+0000 7ff42611a700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834] conn(0x7ff4080aa9b0 0x7ff4080acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.064+0000 7ff42611a700 1 -- 172.21.15.107:0/868363045 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff4080a11e0 msgr2=0x7ff4080a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:13.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.064+0000 7ff42611a700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff4080a11e0 0x7ff4080a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff4200ffbf0 tx=0x7ff410005d70 comp rx=0 tx=0).stop 2023-07-21T22:28:13.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.064+0000 7ff42611a700 1 -- 172.21.15.107:0/868363045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4200ff460 msgr2=0x7ff420196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.064+0000 7ff42611a700 1 --2- 172.21.15.107:0/868363045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4200ff460 0x7ff420196fa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff41400d950 tx=0x7ff41400dc60 comp rx=0 tx=0).stop 2023-07-21T22:28:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.064+0000 7ff41ffff700 1 -- 172.21.15.107:0/868363045 reap_dead start 2023-07-21T22:28:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.065+0000 7ff42611a700 1 -- 172.21.15.107:0/868363045 shutdown_connections 2023-07-21T22:28:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.065+0000 7ff42611a700 1 -- 172.21.15.107:0/868363045 >> 172.21.15.107:0/868363045 conn(0x7ff420075b70 msgr2=0x7ff420102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.065+0000 7ff42611a700 1 -- 172.21.15.107:0/868363045 shutdown_connections 2023-07-21T22:28:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.065+0000 7ff42611a700 1 -- 172.21.15.107:0/868363045 wait complete. 2023-07-21T22:28:13.154 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:13.155 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:13.197 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:13.228 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:13.408 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:13 smithi107 ceph-mon[96543]: pgmap v265: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-07-21T22:28:13.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:13 smithi132 ceph-mon[97863]: pgmap v265: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-07-21T22:28:13.626 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:13.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:13 smithi190 ceph-mon[98334]: pgmap v265: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2023-07-21T22:28:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.721+0000 7fc8095b3700 1 -- 172.21.15.107:0/2505121591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8040fee80 msgr2=0x7fc8040ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.721+0000 7fc8095b3700 1 --2- 172.21.15.107:0/2505121591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8040fee80 0x7fc8040ff250 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40099c0 tx=0x7fc7f4009cd0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.721+0000 7fc8095b3700 1 -- 172.21.15.107:0/2505121591 shutdown_connections 2023-07-21T22:28:13.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.722+0000 7fc8095b3700 1 --2- 172.21.15.107:0/2505121591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc804100f80 0x7fc80410ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.722+0000 7fc8095b3700 1 --2- 172.21.15.107:0/2505121591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8040fee80 0x7fc8040ff250 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.722+0000 7fc8095b3700 1 --2- 172.21.15.107:0/2505121591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8041005d0 0x7fc804100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.722+0000 7fc8095b3700 1 -- 172.21.15.107:0/2505121591 >> 172.21.15.107:0/2505121591 conn(0x7fc804075b60 msgr2=0x7fc804075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:13.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.722+0000 7fc8095b3700 1 -- 172.21.15.107:0/2505121591 shutdown_connections 2023-07-21T22:28:13.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.722+0000 7fc8095b3700 1 -- 172.21.15.107:0/2505121591 wait complete. 2023-07-21T22:28:13.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.722+0000 7fc8095b3700 1 Processor -- start 2023-07-21T22:28:13.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.722+0000 7fc8095b3700 1 -- start start 2023-07-21T22:28:13.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc8095b3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8040fee80 0x7fc804105ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:13.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc8095b3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8041005d0 0x7fc804105ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:13.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc8095b3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc804100f80 0x7fc804106570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:13.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc8095b3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8041a40d0 con 0x7fc8040fee80 2023-07-21T22:28:13.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc8095b3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8041a4240 con 0x7fc8041005d0 2023-07-21T22:28:13.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc8095b3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8041a4400 con 0x7fc804100f80 2023-07-21T22:28:13.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc802ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8040fee80 0x7fc804105ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:13.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc802ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8040fee80 0x7fc804105ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46880/0 (socket says 172.21.15.107:46880) 2023-07-21T22:28:13.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc802ffd700 1 -- 172.21.15.107:0/846277375 learned_addr learned my addr 172.21.15.107:0/846277375 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:13.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc802ffd700 1 -- 172.21.15.107:0/846277375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc804100f80 msgr2=0x7fc804106570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:13.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc802ffd700 1 --2- 172.21.15.107:0/846277375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc804100f80 0x7fc804106570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.773 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc802ffd700 1 -- 172.21.15.107:0/846277375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8041005d0 msgr2=0x7fc804105ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:13.773 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc802ffd700 1 --2- 172.21.15.107:0/846277375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8041005d0 0x7fc804105ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.773 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc802ffd700 1 -- 172.21.15.107:0/846277375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f40095f0 con 0x7fc8040fee80 2023-07-21T22:28:13.773 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc802ffd700 1 --2- 172.21.15.107:0/846277375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8040fee80 0x7fc804105ab0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400b580 tx=0x7fc7f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:13.773 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc7ebfff700 1 -- 172.21.15.107:0/846277375 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f401d070 con 0x7fc8040fee80 2023-07-21T22:28:13.773 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc8095b3700 1 -- 172.21.15.107:0/846277375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8041a4680 con 0x7fc8040fee80 2023-07-21T22:28:13.773 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc7ebfff700 1 -- 172.21.15.107:0/846277375 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7f4004010 con 0x7fc8040fee80 2023-07-21T22:28:13.773 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc7ebfff700 1 -- 172.21.15.107:0/846277375 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4022e40 con 0x7fc8040fee80 2023-07-21T22:28:13.774 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.723+0000 7fc8095b3700 1 -- 172.21.15.107:0/846277375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8041a4a90 con 0x7fc8040fee80 2023-07-21T22:28:13.774 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.724+0000 7fc8095b3700 1 -- 172.21.15.107:0/846277375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc80404e210 con 0x7fc8040fee80 2023-07-21T22:28:13.775 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.725+0000 7fc7ebfff700 1 -- 172.21.15.107:0/846277375 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fc7f4022570 con 0x7fc8040fee80 2023-07-21T22:28:13.775 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.726+0000 7fc7ebfff700 1 --2- 172.21.15.107:0/846277375 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc7ec0a11e0 0x7fc7ec0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:13.775 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.726+0000 7fc8027fc700 1 --2- 172.21.15.107:0/846277375 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc7ec0a11e0 0x7fc7ec0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:13.775 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.726+0000 7fc7ebfff700 1 -- 172.21.15.107:0/846277375 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc7f40ce370 con 0x7fc8040fee80 2023-07-21T22:28:13.775 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.726+0000 7fc7ebfff700 1 --2- 172.21.15.107:0/846277375 >> [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012] conn(0x7fc7ec0aa9b0 0x7fc7ec0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:13.775 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.726+0000 7fc7ebfff700 1 -- 172.21.15.107:0/846277375 --> [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7ec0ad470 con 0x7fc7ec0aa9b0 2023-07-21T22:28:13.776 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.726+0000 7fc7ebfff700 1 -- 172.21.15.107:0/846277375 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7f40ce810 con 0x7fc8040fee80 2023-07-21T22:28:13.776 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.726+0000 7fc8037fe700 1 --2- 172.21.15.107:0/846277375 >> [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012] conn(0x7fc7ec0aa9b0 0x7fc7ec0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:13.776 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.726+0000 7fc8027fc700 1 --2- 172.21.15.107:0/846277375 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc7ec0a11e0 0x7fc7ec0a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8005950 tx=0x7fc7f80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:13.776 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.727+0000 7fc8037fe700 1 --2- 172.21.15.107:0/846277375 >> [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012] conn(0x7fc7ec0aa9b0 0x7fc7ec0acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:13.776 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.727+0000 7fc7ebfff700 1 -- 172.21.15.107:0/846277375 <== osd.0 v2:172.21.15.107:6802/4235328012 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc7ec0ad470 con 0x7fc7ec0aa9b0 2023-07-21T22:28:13.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.743+0000 7fc8095b3700 1 -- 172.21.15.107:0/846277375 --> [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc80410ed50 con 0x7fc7ec0aa9b0 2023-07-21T22:28:13.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.743+0000 7fc7ebfff700 1 -- 172.21.15.107:0/846277375 <== osd.0 v2:172.21.15.107:6802/4235328012 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc7fc008b30 con 0x7fc7ec0aa9b0 2023-07-21T22:28:13.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.743+0000 7fc8095b3700 1 -- 172.21.15.107:0/846277375 >> [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012] conn(0x7fc7ec0aa9b0 msgr2=0x7fc7ec0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:13.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.743+0000 7fc8095b3700 1 --2- 172.21.15.107:0/846277375 >> [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012] conn(0x7fc7ec0aa9b0 0x7fc7ec0acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.743+0000 7fc8095b3700 1 -- 172.21.15.107:0/846277375 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc7ec0a11e0 msgr2=0x7fc7ec0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:13.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.743+0000 7fc8095b3700 1 --2- 172.21.15.107:0/846277375 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc7ec0a11e0 0x7fc7ec0a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8005950 tx=0x7fc7f80058e0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.743+0000 7fc8095b3700 1 -- 172.21.15.107:0/846277375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8040fee80 msgr2=0x7fc804105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:13.791 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.743+0000 7fc8095b3700 1 --2- 172.21.15.107:0/846277375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8040fee80 0x7fc804105ab0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400b580 tx=0x7fc7f40057f0 comp rx=0 tx=0).stop 2023-07-21T22:28:13.791 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.743+0000 7fc8037fe700 1 -- 172.21.15.107:0/846277375 reap_dead start 2023-07-21T22:28:13.791 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.743+0000 7fc8095b3700 1 -- 172.21.15.107:0/846277375 shutdown_connections 2023-07-21T22:28:13.791 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.743+0000 7fc8095b3700 1 -- 172.21.15.107:0/846277375 >> 172.21.15.107:0/846277375 conn(0x7fc804075b60 msgr2=0x7fc8040fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:13.791 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.743+0000 7fc8095b3700 1 -- 172.21.15.107:0/846277375 shutdown_connections 2023-07-21T22:28:13.791 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:13.744+0000 7fc8095b3700 1 -- 172.21.15.107:0/846277375 wait complete. 2023-07-21T22:28:15.118 INFO:teuthology.orchestra.run.smithi107.stdout:34359738432 2023-07-21T22:28:15.118 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-21T22:28:15.135 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:15.165 INFO:teuthology.orchestra.run.smithi107.stdout:214748364834 2023-07-21T22:28:15.166 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-21T22:28:15.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:15 smithi107 ceph-mon[96543]: pgmap v266: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:28:15.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:15 smithi132 ceph-mon[97863]: pgmap v266: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:28:15.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:15 smithi190 ceph-mon[98334]: pgmap v266: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:28:15.935 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:16.222 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:16.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.748+0000 7fd0fdcab700 1 -- 172.21.15.107:0/1828430594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0f80ff490 msgr2=0x7fd0f810b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:16.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.748+0000 7fd0fdcab700 1 --2- 172.21.15.107:0/1828430594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0f80ff490 0x7fd0f810b920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000b0a0 tx=0x7fd0f000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:16.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.748+0000 7fd0fdcab700 1 -- 172.21.15.107:0/1828430594 shutdown_connections 2023-07-21T22:28:16.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.748+0000 7fd0fdcab700 1 --2- 172.21.15.107:0/1828430594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0f80ff490 0x7fd0f810b920 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:16.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.748+0000 7fd0fdcab700 1 --2- 172.21.15.107:0/1828430594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0f8104ae0 0x7fd0f8104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.748+0000 7fd0fdcab700 1 --2- 172.21.15.107:0/1828430594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0f80feae0 0x7fd0f80fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.748+0000 7fd0fdcab700 1 -- 172.21.15.107:0/1828430594 >> 172.21.15.107:0/1828430594 conn(0x7fd0f80759b0 msgr2=0x7fd0f8075db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.748+0000 7fd0fdcab700 1 -- 172.21.15.107:0/1828430594 shutdown_connections 2023-07-21T22:28:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.748+0000 7fd0fdcab700 1 -- 172.21.15.107:0/1828430594 wait complete. 2023-07-21T22:28:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.748+0000 7fd0fdcab700 1 Processor -- start 2023-07-21T22:28:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0fdcab700 1 -- start start 2023-07-21T22:28:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0fdcab700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0f80feae0 0x7fd0f8196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0fdcab700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0f80ff490 0x7fd0f8197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0fdcab700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0f8104ae0 0x7fd0f81a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0fdcab700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8197a60 con 0x7fd0f8104ae0 2023-07-21T22:28:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0fdcab700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f81a21d0 con 0x7fd0f80ff490 2023-07-21T22:28:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0fdcab700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f81a2340 con 0x7fd0f80feae0 2023-07-21T22:28:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0f7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0f8104ae0 0x7fd0f81a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0f7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0f8104ae0 0x7fd0f81a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46896/0 (socket says 172.21.15.107:46896) 2023-07-21T22:28:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0f7fff700 1 -- 172.21.15.107:0/3343619280 learned_addr learned my addr 172.21.15.107:0/3343619280 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0f6ffd700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0f80ff490 0x7fd0f8197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0f77fe700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0f80feae0 0x7fd0f8196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0f7fff700 1 -- 172.21.15.107:0/3343619280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0f80feae0 msgr2=0x7fd0f8196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0f7fff700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0f80feae0 0x7fd0f8196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0f7fff700 1 -- 172.21.15.107:0/3343619280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0f80ff490 msgr2=0x7fd0f8197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0f7fff700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0f80ff490 0x7fd0f8197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0f7fff700 1 -- 172.21.15.107:0/3343619280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f0009d00 con 0x7fd0f8104ae0 2023-07-21T22:28:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.749+0000 7fd0f7fff700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0f8104ae0 0x7fd0f81a1c90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000b830 tx=0x7fd0f00079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.750+0000 7fd0f4ff9700 1 -- 172.21.15.107:0/3343619280 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f000e040 con 0x7fd0f8104ae0 2023-07-21T22:28:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.750+0000 7fd0f4ff9700 1 -- 172.21.15.107:0/3343619280 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0f0004d60 con 0x7fd0f8104ae0 2023-07-21T22:28:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.750+0000 7fd0fdcab700 1 -- 172.21.15.107:0/3343619280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f81a25c0 con 0x7fd0f8104ae0 2023-07-21T22:28:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.750+0000 7fd0f4ff9700 1 -- 172.21.15.107:0/3343619280 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f001d480 con 0x7fd0f8104ae0 2023-07-21T22:28:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.750+0000 7fd0fdcab700 1 -- 172.21.15.107:0/3343619280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f81a2a30 con 0x7fd0f8104ae0 2023-07-21T22:28:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.751+0000 7fd0fdcab700 1 -- 172.21.15.107:0/3343619280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f804e210 con 0x7fd0f8104ae0 2023-07-21T22:28:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.752+0000 7fd0f4ff9700 1 -- 172.21.15.107:0/3343619280 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd0f0003a00 con 0x7fd0f8104ae0 2023-07-21T22:28:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.752+0000 7fd0f4ff9700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd0e00a11e0 0x7fd0e00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.753+0000 7fd0f77fe700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd0e00a11e0 0x7fd0e00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.753+0000 7fd0f4ff9700 1 -- 172.21.15.107:0/3343619280 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd0f00ce040 con 0x7fd0f8104ae0 2023-07-21T22:28:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.753+0000 7fd0f77fe700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd0e00a11e0 0x7fd0e00a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec005950 tx=0x7fd0ec0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.757+0000 7fd0f4ff9700 1 -- 172.21.15.107:0/3343619280 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd0f0097870 con 0x7fd0f8104ae0 2023-07-21T22:28:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.959+0000 7fd0fdcab700 1 -- 172.21.15.107:0/3343619280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd0f81a2c50 con 0x7fd0f8104ae0 2023-07-21T22:28:17.006 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.959+0000 7fd0f4ff9700 1 -- 172.21.15.107:0/3343619280 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd0f009ae90 con 0x7fd0f8104ae0 2023-07-21T22:28:17.006 INFO:teuthology.orchestra.run.smithi107.stdout:163208757292 2023-07-21T22:28:17.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.961+0000 7fd0fdcab700 1 -- 172.21.15.107:0/3343619280 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd0e00a11e0 msgr2=0x7fd0e00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:17.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.961+0000 7fd0fdcab700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd0e00a11e0 0x7fd0e00a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec005950 tx=0x7fd0ec0058e0 comp rx=0 tx=0).stop 2023-07-21T22:28:17.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.961+0000 7fd0fdcab700 1 -- 172.21.15.107:0/3343619280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0f8104ae0 msgr2=0x7fd0f81a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:17.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.961+0000 7fd0fdcab700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0f8104ae0 0x7fd0f81a1c90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000b830 tx=0x7fd0f00079b0 comp rx=0 tx=0).stop 2023-07-21T22:28:17.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.961+0000 7fd0fdcab700 1 -- 172.21.15.107:0/3343619280 shutdown_connections 2023-07-21T22:28:17.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.961+0000 7fd0fdcab700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0f8104ae0 0x7fd0f81a1c90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:17.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.961+0000 7fd0fdcab700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd0e00a11e0 0x7fd0e00a3690 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:17.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.961+0000 7fd0fdcab700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0f80feae0 0x7fd0f8196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:17.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.962+0000 7fd0fdcab700 1 --2- 172.21.15.107:0/3343619280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0f80ff490 0x7fd0f8197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:17.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.962+0000 7fd0fdcab700 1 -- 172.21.15.107:0/3343619280 >> 172.21.15.107:0/3343619280 conn(0x7fd0f80759b0 msgr2=0x7fd0f80fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:17.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.962+0000 7fd0fdcab700 1 -- 172.21.15.107:0/3343619280 shutdown_connections 2023-07-21T22:28:17.010 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:16.962+0000 7fd0fdcab700 1 -- 172.21.15.107:0/3343619280 wait complete. 2023-07-21T22:28:17.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:17 smithi107 ceph-mon[96543]: pgmap v267: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:28:17.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:17 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3343619280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T22:28:17.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:17 smithi132 ceph-mon[97863]: pgmap v267: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:28:17.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:17 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3343619280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T22:28:17.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:17 smithi190 ceph-mon[98334]: pgmap v267: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-21T22:28:17.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:17 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3343619280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T22:28:17.775 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757289 got 163208757292 for osd.5 2023-07-21T22:28:17.776 DEBUG:teuthology.parallel:result is None 2023-07-21T22:28:18.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.190+0000 7f0fc51e8700 1 -- 172.21.15.107:0/1429513584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fc00ffe00 msgr2=0x7f0fc01001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.190+0000 7f0fc51e8700 1 --2- 172.21.15.107:0/1429513584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fc00ffe00 0x7f0fc01001d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00099c0 tx=0x7f0fb0009cd0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.191+0000 7f0fc51e8700 1 -- 172.21.15.107:0/1429513584 shutdown_connections 2023-07-21T22:28:18.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.191+0000 7f0fc51e8700 1 --2- 172.21.15.107:0/1429513584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fc0104ea0 0x7f0fc010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.191+0000 7f0fc51e8700 1 --2- 172.21.15.107:0/1429513584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fc00ffe00 0x7f0fc01001d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.191+0000 7f0fc51e8700 1 --2- 172.21.15.107:0/1429513584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fc01007a0 0x7f0fc0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.191+0000 7f0fc51e8700 1 -- 172.21.15.107:0/1429513584 >> 172.21.15.107:0/1429513584 conn(0x7f0fc00fb660 msgr2=0x7f0fc00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:18.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.191+0000 7f0fc51e8700 1 -- 172.21.15.107:0/1429513584 shutdown_connections 2023-07-21T22:28:18.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.191+0000 7f0fc51e8700 1 -- 172.21.15.107:0/1429513584 wait complete. 2023-07-21T22:28:18.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.191+0000 7f0fc51e8700 1 Processor -- start 2023-07-21T22:28:18.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.192+0000 7f0fc51e8700 1 -- start start 2023-07-21T22:28:18.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.192+0000 7f0fc51e8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fc00ffe00 0x7f0fc0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:18.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.192+0000 7f0fc51e8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fc01007a0 0x7f0fc0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:18.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.192+0000 7f0fc51e8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fc0104ea0 0x7f0fc00732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:18.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.192+0000 7f0fc51e8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc01a4150 con 0x7f0fc01007a0 2023-07-21T22:28:18.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.192+0000 7f0fc51e8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc01a42c0 con 0x7f0fc0104ea0 2023-07-21T22:28:18.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.192+0000 7f0fc51e8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc01a4480 con 0x7f0fc00ffe00 2023-07-21T22:28:18.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.192+0000 7f0fbe59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fc01007a0 0x7f0fc0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:18.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.192+0000 7f0fbe59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fc01007a0 0x7f0fc0072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46904/0 (socket says 172.21.15.107:46904) 2023-07-21T22:28:18.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.192+0000 7f0fbe59c700 1 -- 172.21.15.107:0/853898584 learned_addr learned my addr 172.21.15.107:0/853898584 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:18.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.192+0000 7f0fbf59e700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fc0104ea0 0x7f0fc00732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:18.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.192+0000 7f0fbed9d700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fc00ffe00 0x7f0fc0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:18.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.192+0000 7f0fbe59c700 1 -- 172.21.15.107:0/853898584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fc00ffe00 msgr2=0x7f0fc0072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.193+0000 7f0fbe59c700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fc00ffe00 0x7f0fc0072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.193+0000 7f0fbe59c700 1 -- 172.21.15.107:0/853898584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fc0104ea0 msgr2=0x7f0fc00732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.193+0000 7f0fbe59c700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fc0104ea0 0x7f0fc00732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.193+0000 7f0fbe59c700 1 -- 172.21.15.107:0/853898584 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb00095f0 con 0x7f0fc01007a0 2023-07-21T22:28:18.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.193+0000 7f0fbe59c700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fc01007a0 0x7f0fc0072d50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400d980 tx=0x7f0fb400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:18.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.193+0000 7f0fa7fff700 1 -- 172.21.15.107:0/853898584 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4009930 con 0x7f0fc01007a0 2023-07-21T22:28:18.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.193+0000 7f0fa7fff700 1 -- 172.21.15.107:0/853898584 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb4010460 con 0x7f0fc01007a0 2023-07-21T22:28:18.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.193+0000 7f0fc51e8700 1 -- 172.21.15.107:0/853898584 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc01a4760 con 0x7f0fc01007a0 2023-07-21T22:28:18.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.193+0000 7f0fa7fff700 1 -- 172.21.15.107:0/853898584 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb400f620 con 0x7f0fc01007a0 2023-07-21T22:28:18.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.193+0000 7f0fc51e8700 1 -- 172.21.15.107:0/853898584 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc01a4c60 con 0x7f0fc01007a0 2023-07-21T22:28:18.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.194+0000 7f0fc51e8700 1 -- 172.21.15.107:0/853898584 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc004e210 con 0x7f0fc01007a0 2023-07-21T22:28:18.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.196+0000 7f0fa7fff700 1 -- 172.21.15.107:0/853898584 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f0fb4010890 con 0x7f0fc01007a0 2023-07-21T22:28:18.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.196+0000 7f0fa7fff700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0fa80a11e0 0x7f0fa80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:18.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.196+0000 7f0fbed9d700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0fa80a11e0 0x7f0fa80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:18.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.196+0000 7f0fa7fff700 1 -- 172.21.15.107:0/853898584 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0fb40cc390 con 0x7f0fc01007a0 2023-07-21T22:28:18.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.197+0000 7f0fbed9d700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0fa80a11e0 0x7f0fa80a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0000c00 tx=0x7f0fb0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:18.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.199+0000 7f0fa7fff700 1 -- 172.21.15.107:0/853898584 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0fb4095b30 con 0x7f0fc01007a0 2023-07-21T22:28:18.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.203+0000 7f6f32e3a700 1 -- 172.21.15.107:0/2679794758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c104ab0 msgr2=0x7f6f2c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.203+0000 7f6f32e3a700 1 --2- 172.21.15.107:0/2679794758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c104ab0 0x7f6f2c104e80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009990 tx=0x7f6f20009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.203+0000 7f6f32e3a700 1 -- 172.21.15.107:0/2679794758 shutdown_connections 2023-07-21T22:28:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.203+0000 7f6f32e3a700 1 --2- 172.21.15.107:0/2679794758 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f2c0ff460 0x7f6f2c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.203+0000 7f6f32e3a700 1 --2- 172.21.15.107:0/2679794758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c104ab0 0x7f6f2c104e80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.203+0000 7f6f32e3a700 1 --2- 172.21.15.107:0/2679794758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f2c0feab0 0x7f6f2c0fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.203+0000 7f6f32e3a700 1 -- 172.21.15.107:0/2679794758 >> 172.21.15.107:0/2679794758 conn(0x7f6f2c075b70 msgr2=0x7f6f2c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.203+0000 7f6f32e3a700 1 -- 172.21.15.107:0/2679794758 shutdown_connections 2023-07-21T22:28:18.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.203+0000 7f6f32e3a700 1 -- 172.21.15.107:0/2679794758 wait complete. 2023-07-21T22:28:18.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.204+0000 7f6f32e3a700 1 Processor -- start 2023-07-21T22:28:18.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.204+0000 7f6f32e3a700 1 -- start start 2023-07-21T22:28:18.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.204+0000 7f6f32e3a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f2c0feab0 0x7f6f2c19b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:18.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.204+0000 7f6f32e3a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f2c0ff460 0x7f6f2c19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:18.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.204+0000 7f6f32e3a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c104ab0 0x7f6f2c195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:18.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.204+0000 7f6f32e3a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c1960c0 con 0x7f6f2c104ab0 2023-07-21T22:28:18.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.204+0000 7f6f32e3a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c196230 con 0x7f6f2c0ff460 2023-07-21T22:28:18.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.204+0000 7f6f32e3a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c1963f0 con 0x7f6f2c0feab0 2023-07-21T22:28:18.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.204+0000 7f6f313d7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c104ab0 0x7f6f2c195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:18.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.204+0000 7f6f313d7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c104ab0 0x7f6f2c195a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46922/0 (socket says 172.21.15.107:46922) 2023-07-21T22:28:18.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.204+0000 7f6f313d7700 1 -- 172.21.15.107:0/3982871649 learned_addr learned my addr 172.21.15.107:0/3982871649 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:18.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.204+0000 7f6f2bfff700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f2c0ff460 0x7f6f2c19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:18.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.204+0000 7f6f30bd6700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f2c0feab0 0x7f6f2c19b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:18.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.205+0000 7f6f313d7700 1 -- 172.21.15.107:0/3982871649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f2c0feab0 msgr2=0x7f6f2c19b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.205+0000 7f6f313d7700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f2c0feab0 0x7f6f2c19b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.205+0000 7f6f313d7700 1 -- 172.21.15.107:0/3982871649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f2c0ff460 msgr2=0x7f6f2c19bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.205+0000 7f6f313d7700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f2c0ff460 0x7f6f2c19bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.205+0000 7f6f313d7700 1 -- 172.21.15.107:0/3982871649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f200095f0 con 0x7f6f2c104ab0 2023-07-21T22:28:18.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.205+0000 7f6f313d7700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c104ab0 0x7f6f2c195a50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400ab50 tx=0x7f6f2400ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:18.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.205+0000 7f6f29ffb700 1 -- 172.21.15.107:0/3982871649 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400f070 con 0x7f6f2c104ab0 2023-07-21T22:28:18.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.205+0000 7f6f29ffb700 1 -- 172.21.15.107:0/3982871649 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f24007a80 con 0x7f6f2c104ab0 2023-07-21T22:28:18.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.205+0000 7f6f32e3a700 1 -- 172.21.15.107:0/3982871649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2c1966d0 con 0x7f6f2c104ab0 2023-07-21T22:28:18.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.205+0000 7f6f29ffb700 1 -- 172.21.15.107:0/3982871649 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24005280 con 0x7f6f2c104ab0 2023-07-21T22:28:18.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.205+0000 7f6f32e3a700 1 -- 172.21.15.107:0/3982871649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c1a29c0 con 0x7f6f2c104ab0 2023-07-21T22:28:18.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.206+0000 7f6f32e3a700 1 -- 172.21.15.107:0/3982871649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2c04e210 con 0x7f6f2c104ab0 2023-07-21T22:28:18.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.207+0000 7f6f29ffb700 1 -- 172.21.15.107:0/3982871649 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6f2400c620 con 0x7f6f2c104ab0 2023-07-21T22:28:18.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.207+0000 7f6f29ffb700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f1c0a11e0 0x7f6f1c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:18.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.207+0000 7f6f30bd6700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f1c0a11e0 0x7f6f1c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:18.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.208+0000 7f6f29ffb700 1 -- 172.21.15.107:0/3982871649 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6f240cbdb0 con 0x7f6f2c104ab0 2023-07-21T22:28:18.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.208+0000 7f6f30bd6700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f1c0a11e0 0x7f6f1c0a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6f20006010 tx=0x7f6f2000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:18.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.212+0000 7f6f29ffb700 1 -- 172.21.15.107:0/3982871649 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6f240955e0 con 0x7f6f2c104ab0 2023-07-21T22:28:18.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.405+0000 7f0fc51e8700 1 -- 172.21.15.107:0/853898584 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0fc0073d40 con 0x7f0fc01007a0 2023-07-21T22:28:18.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.406+0000 7f0fa7fff700 1 -- 172.21.15.107:0/853898584 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0fb4020090 con 0x7f0fc01007a0 2023-07-21T22:28:18.452 INFO:teuthology.orchestra.run.smithi107.stdout:55834574908 2023-07-21T22:28:18.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.408+0000 7f0fc51e8700 1 -- 172.21.15.107:0/853898584 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0fa80a11e0 msgr2=0x7f0fa80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.408+0000 7f0fc51e8700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0fa80a11e0 0x7f0fa80a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0000c00 tx=0x7f0fb0005fb0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.408+0000 7f0fc51e8700 1 -- 172.21.15.107:0/853898584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fc01007a0 msgr2=0x7f0fc0072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.408+0000 7f0fc51e8700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fc01007a0 0x7f0fc0072d50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400d980 tx=0x7f0fb400dc90 comp rx=0 tx=0).stop 2023-07-21T22:28:18.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.408+0000 7f0fc51e8700 1 -- 172.21.15.107:0/853898584 shutdown_connections 2023-07-21T22:28:18.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.408+0000 7f0fc51e8700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fc0104ea0 0x7f0fc00732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.408+0000 7f0fc51e8700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0fa80a11e0 0x7f0fa80a3690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.408+0000 7f0fc51e8700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fc00ffe00 0x7f0fc0072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.408+0000 7f0fc51e8700 1 --2- 172.21.15.107:0/853898584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fc01007a0 0x7f0fc0072d50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.408+0000 7f0fc51e8700 1 -- 172.21.15.107:0/853898584 >> 172.21.15.107:0/853898584 conn(0x7f0fc00fb660 msgr2=0x7f0fc0103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:18.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.408+0000 7f0fc51e8700 1 -- 172.21.15.107:0/853898584 shutdown_connections 2023-07-21T22:28:18.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.408+0000 7f0fc51e8700 1 -- 172.21.15.107:0/853898584 wait complete. 2023-07-21T22:28:18.479 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.433+0000 7f6f32e3a700 1 -- 172.21.15.107:0/3982871649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6f2c1a2bf0 con 0x7f6f2c104ab0 2023-07-21T22:28:18.479 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.433+0000 7f6f29ffb700 1 -- 172.21.15.107:0/3982871649 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f24098c00 con 0x7f6f2c104ab0 2023-07-21T22:28:18.480 INFO:teuthology.orchestra.run.smithi107.stdout:141733920815 2023-07-21T22:28:18.482 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.435+0000 7f6f32e3a700 1 -- 172.21.15.107:0/3982871649 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f1c0a11e0 msgr2=0x7f6f1c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.482 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.435+0000 7f6f32e3a700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f1c0a11e0 0x7f6f1c0a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6f20006010 tx=0x7f6f2000b540 comp rx=0 tx=0).stop 2023-07-21T22:28:18.482 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.435+0000 7f6f32e3a700 1 -- 172.21.15.107:0/3982871649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c104ab0 msgr2=0x7f6f2c195a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.483 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.435+0000 7f6f32e3a700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c104ab0 0x7f6f2c195a50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400ab50 tx=0x7f6f2400ae60 comp rx=0 tx=0).stop 2023-07-21T22:28:18.483 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.435+0000 7f6f32e3a700 1 -- 172.21.15.107:0/3982871649 shutdown_connections 2023-07-21T22:28:18.483 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.435+0000 7f6f32e3a700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c104ab0 0x7f6f2c195a50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.483 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.435+0000 7f6f32e3a700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f1c0a11e0 0x7f6f1c0a3690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.483 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.435+0000 7f6f32e3a700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f2c0feab0 0x7f6f2c19b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.483 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.435+0000 7f6f32e3a700 1 --2- 172.21.15.107:0/3982871649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f2c0ff460 0x7f6f2c19bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.483 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.436+0000 7f6f32e3a700 1 -- 172.21.15.107:0/3982871649 >> 172.21.15.107:0/3982871649 conn(0x7f6f2c075b70 msgr2=0x7f6f2c1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:18.483 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.436+0000 7f6f32e3a700 1 -- 172.21.15.107:0/3982871649 shutdown_connections 2023-07-21T22:28:18.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.436+0000 7f6f32e3a700 1 -- 172.21.15.107:0/3982871649 wait complete. 2023-07-21T22:28:18.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.666+0000 7f5d06178700 1 -- 172.21.15.107:0/3383736193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d000ff460 msgr2=0x7f5d0010b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.666+0000 7f5d06178700 1 --2- 172.21.15.107:0/3383736193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d000ff460 0x7f5d0010b8f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800b0a0 tx=0x7f5cf800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.666+0000 7f5d06178700 1 -- 172.21.15.107:0/3383736193 shutdown_connections 2023-07-21T22:28:18.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.666+0000 7f5d06178700 1 --2- 172.21.15.107:0/3383736193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d000ff460 0x7f5d0010b8f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.666+0000 7f5d06178700 1 --2- 172.21.15.107:0/3383736193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d00104ab0 0x7f5d00104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.666+0000 7f5d06178700 1 --2- 172.21.15.107:0/3383736193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d000feab0 0x7f5d000fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.666+0000 7f5d06178700 1 -- 172.21.15.107:0/3383736193 >> 172.21.15.107:0/3383736193 conn(0x7f5d00075b70 msgr2=0x7f5d00075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:18.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.666+0000 7f5d06178700 1 -- 172.21.15.107:0/3383736193 shutdown_connections 2023-07-21T22:28:18.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.666+0000 7f5d06178700 1 -- 172.21.15.107:0/3383736193 wait complete. 2023-07-21T22:28:18.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.666+0000 7f5d06178700 1 Processor -- start 2023-07-21T22:28:18.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5d06178700 1 -- start start 2023-07-21T22:28:18.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5d06178700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d000feab0 0x7f5d00196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:18.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5d06178700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d000ff460 0x7f5d00196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:18.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5d06178700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d00104ab0 0x7f5d001a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:18.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5d06178700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d001978e0 con 0x7f5d000ff460 2023-07-21T22:28:18.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5d06178700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d001a2020 con 0x7f5d000feab0 2023-07-21T22:28:18.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5d06178700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d001a21e0 con 0x7f5d00104ab0 2023-07-21T22:28:18.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cfeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d000ff460 0x7f5d00196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:18.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cfeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d000ff460 0x7f5d00196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46934/0 (socket says 172.21.15.107:46934) 2023-07-21T22:28:18.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cfeffd700 1 -- 172.21.15.107:0/3617805769 learned_addr learned my addr 172.21.15.107:0/3617805769 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:18.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cfffff700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d00104ab0 0x7f5d001a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:18.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cff7fe700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d000feab0 0x7f5d00196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:18.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cfeffd700 1 -- 172.21.15.107:0/3617805769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d00104ab0 msgr2=0x7f5d001a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cfeffd700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d00104ab0 0x7f5d001a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cfeffd700 1 -- 172.21.15.107:0/3617805769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d000feab0 msgr2=0x7f5d00196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cfeffd700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d000feab0 0x7f5d00196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cfeffd700 1 -- 172.21.15.107:0/3617805769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf8009d00 con 0x7f5d000ff460 2023-07-21T22:28:18.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cfeffd700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d000ff460 0x7f5d00196fa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400e9a0 tx=0x7f5cf400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:18.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cfcff9700 1 -- 172.21.15.107:0/3617805769 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4009790 con 0x7f5d000ff460 2023-07-21T22:28:18.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cfcff9700 1 -- 172.21.15.107:0/3617805769 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cf400cbd0 con 0x7f5d000ff460 2023-07-21T22:28:18.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5cfcff9700 1 -- 172.21.15.107:0/3617805769 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4018680 con 0x7f5d000ff460 2023-07-21T22:28:18.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.667+0000 7f5d06178700 1 -- 172.21.15.107:0/3617805769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d001a24c0 con 0x7f5d000ff460 2023-07-21T22:28:18.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.668+0000 7f5d06178700 1 -- 172.21.15.107:0/3617805769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d001a2970 con 0x7f5d000ff460 2023-07-21T22:28:18.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.668+0000 7f5d06178700 1 -- 172.21.15.107:0/3617805769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0004e210 con 0x7f5d000ff460 2023-07-21T22:28:18.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.669+0000 7f5cfcff9700 1 -- 172.21.15.107:0/3617805769 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f5cf400cd40 con 0x7f5d000ff460 2023-07-21T22:28:18.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.670+0000 7f5cfcff9700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5ce80a11e0 0x7f5ce80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:18.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.670+0000 7f5cff7fe700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5ce80a11e0 0x7f5ce80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:18.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.670+0000 7f5cfcff9700 1 -- 172.21.15.107:0/3617805769 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5cf4014070 con 0x7f5d000ff460 2023-07-21T22:28:18.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.670+0000 7f5cff7fe700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5ce80a11e0 0x7f5ce80a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d000ffbf0 tx=0x7f5cf0005d70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:18.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.674+0000 7f5cfcff9700 1 -- 172.21.15.107:0/3617805769 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5cf4051770 con 0x7f5d000ff460 2023-07-21T22:28:18.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.881+0000 7f5d06178700 1 -- 172.21.15.107:0/3617805769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f5d000685e0 con 0x7f5d000ff460 2023-07-21T22:28:18.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.881+0000 7f5cfcff9700 1 -- 172.21.15.107:0/3617805769 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5cf40962e0 con 0x7f5d000ff460 2023-07-21T22:28:18.928 INFO:teuthology.orchestra.run.smithi107.stdout:244813135903 2023-07-21T22:28:18.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.884+0000 7f5d06178700 1 -- 172.21.15.107:0/3617805769 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5ce80a11e0 msgr2=0x7f5ce80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.884+0000 7f5d06178700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5ce80a11e0 0x7f5ce80a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d000ffbf0 tx=0x7f5cf0005d70 comp rx=0 tx=0).stop 2023-07-21T22:28:18.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.884+0000 7f5d06178700 1 -- 172.21.15.107:0/3617805769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d000ff460 msgr2=0x7f5d00196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:18.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.884+0000 7f5d06178700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d000ff460 0x7f5d00196fa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400e9a0 tx=0x7f5cf400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.884+0000 7f5d06178700 1 -- 172.21.15.107:0/3617805769 shutdown_connections 2023-07-21T22:28:18.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.884+0000 7f5d06178700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5ce80a11e0 0x7f5ce80a3690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.884+0000 7f5d06178700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5d00104ab0 0x7f5d001a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.884+0000 7f5d06178700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d000feab0 0x7f5d00196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.884+0000 7f5d06178700 1 --2- 172.21.15.107:0/3617805769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d000ff460 0x7f5d00196fa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:18.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.884+0000 7f5d06178700 1 -- 172.21.15.107:0/3617805769 >> 172.21.15.107:0/3617805769 conn(0x7f5d00075b70 msgr2=0x7f5d00102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:18.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.884+0000 7f5d06178700 1 -- 172.21.15.107:0/3617805769 shutdown_connections 2023-07-21T22:28:18.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:18.884+0000 7f5d06178700 1 -- 172.21.15.107:0/3617805769 wait complete. 2023-07-21T22:28:19.391 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:19 smithi107 ceph-mon[96543]: pgmap v268: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-07-21T22:28:19.392 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:19 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/853898584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T22:28:19.392 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:19 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3982871649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T22:28:19.392 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:19 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3617805769' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T22:28:19.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:19 smithi132 ceph-mon[97863]: pgmap v268: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-07-21T22:28:19.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:19 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/853898584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T22:28:19.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:19 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3982871649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T22:28:19.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:19 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3617805769' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T22:28:19.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:19 smithi190 ceph-mon[98334]: pgmap v268: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-07-21T22:28:19.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:19 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/853898584' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T22:28:19.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:19 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3982871649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T22:28:19.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:19 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3617805769' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T22:28:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.631+0000 7f542822b700 1 -- 172.21.15.107:0/1837383247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54200fee90 msgr2=0x7f5420102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.631+0000 7f542822b700 1 --2- 172.21.15.107:0/1837383247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54200fee90 0x7f5420102e60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f541000aad0 tx=0x7f541000ade0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.631+0000 7f542822b700 1 -- 172.21.15.107:0/1837383247 shutdown_connections 2023-07-21T22:28:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.631+0000 7f542822b700 1 --2- 172.21.15.107:0/1837383247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5420103590 0x7f542010b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.631+0000 7f542822b700 1 --2- 172.21.15.107:0/1837383247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54200fe4f0 0x7f54200fe8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.631+0000 7f542822b700 1 --2- 172.21.15.107:0/1837383247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54200fee90 0x7f5420102e60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.631+0000 7f542822b700 1 -- 172.21.15.107:0/1837383247 >> 172.21.15.107:0/1837383247 conn(0x7f54200750b0 msgr2=0x7f54200754b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.631+0000 7f542822b700 1 -- 172.21.15.107:0/1837383247 shutdown_connections 2023-07-21T22:28:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.631+0000 7f542822b700 1 -- 172.21.15.107:0/1837383247 wait complete. 2023-07-21T22:28:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f542822b700 1 Processor -- start 2023-07-21T22:28:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f542822b700 1 -- start start 2023-07-21T22:28:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f542822b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54200fe4f0 0x7f5420195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f542822b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54200fee90 0x7f5420196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f542822b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5420103590 0x7f542019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f542822b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019ad00 con 0x7f54200fee90 2023-07-21T22:28:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f542822b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019ae70 con 0x7f5420103590 2023-07-21T22:28:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f542822b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019b030 con 0x7f54200fe4f0 2023-07-21T22:28:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f54257c6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54200fee90 0x7f5420196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f54257c6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54200fee90 0x7f5420196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46960/0 (socket says 172.21.15.107:46960) 2023-07-21T22:28:19.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f54257c6700 1 -- 172.21.15.107:0/168309792 learned_addr learned my addr 172.21.15.107:0/168309792 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:19.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f5425fc7700 1 --2- 172.21.15.107:0/168309792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54200fe4f0 0x7f5420195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:19.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f54257c6700 1 -- 172.21.15.107:0/168309792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54200fe4f0 msgr2=0x7f5420195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:19.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f54257c6700 1 --2- 172.21.15.107:0/168309792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54200fe4f0 0x7f5420195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f54257c6700 1 -- 172.21.15.107:0/168309792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5420103590 msgr2=0x7f542019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:19.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f54257c6700 1 --2- 172.21.15.107:0/168309792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5420103590 0x7f542019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f54257c6700 1 -- 172.21.15.107:0/168309792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541000a700 con 0x7f54200fee90 2023-07-21T22:28:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f54257c6700 1 --2- 172.21.15.107:0/168309792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54200fee90 0x7f5420196280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f541000a9f0 tx=0x7f5410004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.632+0000 7f5416ffd700 1 -- 172.21.15.107:0/168309792 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410003680 con 0x7f54200fee90 2023-07-21T22:28:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.633+0000 7f5416ffd700 1 -- 172.21.15.107:0/168309792 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54100037e0 con 0x7f54200fee90 2023-07-21T22:28:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.633+0000 7f542822b700 1 -- 172.21.15.107:0/168309792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542019b2b0 con 0x7f54200fee90 2023-07-21T22:28:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.633+0000 7f5416ffd700 1 -- 172.21.15.107:0/168309792 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410016680 con 0x7f54200fee90 2023-07-21T22:28:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.633+0000 7f542822b700 1 -- 172.21.15.107:0/168309792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54201a2b70 con 0x7f54200fee90 2023-07-21T22:28:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.633+0000 7f542822b700 1 -- 172.21.15.107:0/168309792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5420108870 con 0x7f54200fee90 2023-07-21T22:28:19.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.635+0000 7f5416ffd700 1 -- 172.21.15.107:0/168309792 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f541000e450 con 0x7f54200fee90 2023-07-21T22:28:19.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.635+0000 7f5416ffd700 1 --2- 172.21.15.107:0/168309792 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f540c0a11e0 0x7f540c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:19.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.635+0000 7f5425fc7700 1 --2- 172.21.15.107:0/168309792 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f540c0a11e0 0x7f540c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:19.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.635+0000 7f5416ffd700 1 -- 172.21.15.107:0/168309792 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5410012070 con 0x7f54200fee90 2023-07-21T22:28:19.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.636+0000 7f5425fc7700 1 --2- 172.21.15.107:0/168309792 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f540c0a11e0 0x7f540c0a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f541c005950 tx=0x7f541c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:19.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.638+0000 7f5416ffd700 1 -- 172.21.15.107:0/168309792 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f54100982a0 con 0x7f54200fee90 2023-07-21T22:28:19.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.380+0000 7f5570289700 1 -- 172.21.15.107:0/3287750896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55680ffdf0 msgr2=0x7f55681001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:19.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.380+0000 7f5570289700 1 --2- 172.21.15.107:0/3287750896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55680ffdf0 0x7f55681001c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f55580099c0 tx=0x7f5558009cd0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.380+0000 7f5570289700 1 -- 172.21.15.107:0/3287750896 shutdown_connections 2023-07-21T22:28:19.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.380+0000 7f5570289700 1 --2- 172.21.15.107:0/3287750896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5568104e90 0x7f556810ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.380+0000 7f5570289700 1 --2- 172.21.15.107:0/3287750896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55680ffdf0 0x7f55681001c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.380+0000 7f5570289700 1 --2- 172.21.15.107:0/3287750896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5568100790 0x7f5568104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.380+0000 7f5570289700 1 -- 172.21.15.107:0/3287750896 >> 172.21.15.107:0/3287750896 conn(0x7f55680fb670 msgr2=0x7f55680fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:19.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.380+0000 7f5570289700 1 -- 172.21.15.107:0/3287750896 shutdown_connections 2023-07-21T22:28:19.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.380+0000 7f5570289700 1 -- 172.21.15.107:0/3287750896 wait complete. 2023-07-21T22:28:19.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f5570289700 1 Processor -- start 2023-07-21T22:28:19.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f5570289700 1 -- start start 2023-07-21T22:28:19.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f5570289700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55680ffdf0 0x7f556819ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:19.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f5570289700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5568100790 0x7f556819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:19.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f5570289700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5568104e90 0x7f5568195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:19.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f5570289700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5568196220 con 0x7f55680ffdf0 2023-07-21T22:28:19.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f5570289700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5568196390 con 0x7f5568104e90 2023-07-21T22:28:19.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f5570289700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5568196550 con 0x7f5568100790 2023-07-21T22:28:19.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f556e025700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55680ffdf0 0x7f556819ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:19.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f556e025700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55680ffdf0 0x7f556819ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46946/0 (socket says 172.21.15.107:46946) 2023-07-21T22:28:19.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f556e025700 1 -- 172.21.15.107:0/3774053557 learned_addr learned my addr 172.21.15.107:0/3774053557 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:19.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f556d824700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5568100790 0x7f556819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:19.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f556e025700 1 -- 172.21.15.107:0/3774053557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5568100790 msgr2=0x7f556819bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:19.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f556e025700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5568100790 0x7f556819bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f556e025700 1 -- 172.21.15.107:0/3774053557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5568104e90 msgr2=0x7f5568195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:19.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f556e826700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5568104e90 0x7f5568195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:19.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f556e025700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5568104e90 0x7f5568195a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f556e025700 1 -- 172.21.15.107:0/3774053557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55580095f0 con 0x7f55680ffdf0 2023-07-21T22:28:19.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f556d824700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5568100790 0x7f556819bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:19.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.381+0000 7f556e025700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55680ffdf0 0x7f556819ba80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5558009990 tx=0x7f5558009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:19.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.382+0000 7f555f7fe700 1 -- 172.21.15.107:0/3774053557 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555801d070 con 0x7f55680ffdf0 2023-07-21T22:28:19.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.382+0000 7f555f7fe700 1 -- 172.21.15.107:0/3774053557 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5558004010 con 0x7f55680ffdf0 2023-07-21T22:28:19.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.382+0000 7f5570289700 1 -- 172.21.15.107:0/3774053557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55681967d0 con 0x7f55680ffdf0 2023-07-21T22:28:19.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.382+0000 7f555f7fe700 1 -- 172.21.15.107:0/3774053557 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5558022d40 con 0x7f55680ffdf0 2023-07-21T22:28:19.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.382+0000 7f5570289700 1 -- 172.21.15.107:0/3774053557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55681a2a00 con 0x7f55680ffdf0 2023-07-21T22:28:19.703 INFO:teuthology.orchestra.run.smithi107.stdout:107374182452 2023-07-21T22:28:19.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.383+0000 7f5570289700 1 -- 172.21.15.107:0/3774053557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556804e210 con 0x7f55680ffdf0 2023-07-21T22:28:19.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.384+0000 7f555f7fe700 1 -- 172.21.15.107:0/3774053557 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f5558022470 con 0x7f55680ffdf0 2023-07-21T22:28:19.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.384+0000 7f555f7fe700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f55540a11e0 0x7f55540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:19.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.384+0000 7f556d824700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f55540a11e0 0x7f55540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:19.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.384+0000 7f555f7fe700 1 -- 172.21.15.107:0/3774053557 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f55580ce0e0 con 0x7f55680ffdf0 2023-07-21T22:28:19.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.385+0000 7f556d824700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f55540a11e0 0x7f55540a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5564005fd0 tx=0x7f5564005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:19.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.388+0000 7f555f7fe700 1 -- 172.21.15.107:0/3774053557 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5558097910 con 0x7f55680ffdf0 2023-07-21T22:28:19.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.592+0000 7f5570289700 1 -- 172.21.15.107:0/3774053557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f55681a2b90 con 0x7f55680ffdf0 2023-07-21T22:28:19.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.592+0000 7f555f7fe700 1 -- 172.21.15.107:0/3774053557 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f555809af30 con 0x7f55680ffdf0 2023-07-21T22:28:19.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.594+0000 7f5570289700 1 -- 172.21.15.107:0/3774053557 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f55540a11e0 msgr2=0x7f55540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:19.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.594+0000 7f5570289700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f55540a11e0 0x7f55540a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5564005fd0 tx=0x7f5564005e30 comp rx=0 tx=0).stop 2023-07-21T22:28:19.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.594+0000 7f5570289700 1 -- 172.21.15.107:0/3774053557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55680ffdf0 msgr2=0x7f556819ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:19.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.594+0000 7f5570289700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55680ffdf0 0x7f556819ba80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5558009990 tx=0x7f5558009580 comp rx=0 tx=0).stop 2023-07-21T22:28:19.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.594+0000 7f5570289700 1 -- 172.21.15.107:0/3774053557 shutdown_connections 2023-07-21T22:28:19.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.594+0000 7f5570289700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5568104e90 0x7f5568195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.594+0000 7f5570289700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55680ffdf0 0x7f556819ba80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.594+0000 7f5570289700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f55540a11e0 0x7f55540a3690 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.594+0000 7f5570289700 1 --2- 172.21.15.107:0/3774053557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5568100790 0x7f556819bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.594+0000 7f5570289700 1 -- 172.21.15.107:0/3774053557 >> 172.21.15.107:0/3774053557 conn(0x7f55680fb670 msgr2=0x7f55680fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:19.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.594+0000 7f5570289700 1 -- 172.21.15.107:0/3774053557 shutdown_connections 2023-07-21T22:28:19.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.594+0000 7f5570289700 1 -- 172.21.15.107:0/3774053557 wait complete. 2023-07-21T22:28:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.840+0000 7f542822b700 1 -- 172.21.15.107:0/168309792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f5420197140 con 0x7f54200fee90 2023-07-21T22:28:19.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.840+0000 7f5416ffd700 1 -- 172.21.15.107:0/168309792 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f541009b8c0 con 0x7f54200fee90 2023-07-21T22:28:19.887 INFO:teuthology.orchestra.run.smithi107.stdout:330712481811 2023-07-21T22:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.842+0000 7f542822b700 1 -- 172.21.15.107:0/168309792 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f540c0a11e0 msgr2=0x7f540c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.842+0000 7f542822b700 1 --2- 172.21.15.107:0/168309792 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f540c0a11e0 0x7f540c0a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f541c005950 tx=0x7f541c00b410 comp rx=0 tx=0).stop 2023-07-21T22:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.842+0000 7f542822b700 1 -- 172.21.15.107:0/168309792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54200fee90 msgr2=0x7f5420196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.842+0000 7f542822b700 1 --2- 172.21.15.107:0/168309792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54200fee90 0x7f5420196280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f541000a9f0 tx=0x7f5410004750 comp rx=0 tx=0).stop 2023-07-21T22:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.842+0000 7f542822b700 1 -- 172.21.15.107:0/168309792 shutdown_connections 2023-07-21T22:28:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.842+0000 7f542822b700 1 --2- 172.21.15.107:0/168309792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5420103590 0x7f542019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.842+0000 7f542822b700 1 --2- 172.21.15.107:0/168309792 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f540c0a11e0 0x7f540c0a3690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.842+0000 7f542822b700 1 --2- 172.21.15.107:0/168309792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54200fe4f0 0x7f5420195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.842+0000 7f542822b700 1 --2- 172.21.15.107:0/168309792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54200fee90 0x7f5420196280 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.842+0000 7f542822b700 1 -- 172.21.15.107:0/168309792 >> 172.21.15.107:0/168309792 conn(0x7f54200750b0 msgr2=0x7f5420102180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.842+0000 7f542822b700 1 -- 172.21.15.107:0/168309792 shutdown_connections 2023-07-21T22:28:19.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:19.842+0000 7f542822b700 1 -- 172.21.15.107:0/168309792 wait complete. 2023-07-21T22:28:19.919 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135901 got 244813135903 for osd.8 2023-07-21T22:28:19.919 DEBUG:teuthology.parallel:result is None 2023-07-21T22:28:20.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:20 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3774053557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T22:28:20.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:20 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/168309792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T22:28:20.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:20 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3774053557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T22:28:20.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:20 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/168309792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T22:28:20.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:20 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3774053557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T22:28:20.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:20 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/168309792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T22:28:20.829 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182450 got 107374182452 for osd.3 2023-07-21T22:28:20.829 DEBUG:teuthology.parallel:result is None 2023-07-21T22:28:20.860 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574906 got 55834574908 for osd.1 2023-07-21T22:28:20.860 DEBUG:teuthology.parallel:result is None 2023-07-21T22:28:20.980 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920813 got 141733920815 for osd.4 2023-07-21T22:28:20.980 DEBUG:teuthology.parallel:result is None 2023-07-21T22:28:21.012 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.965+0000 7fab63eed700 1 -- 172.21.15.107:0/3167597792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab5c0ffe00 msgr2=0x7fab5c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:21.012 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.965+0000 7fab63eed700 1 --2- 172.21.15.107:0/3167597792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab5c0ffe00 0x7fab5c1001d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00b0d0 tx=0x7fab4c00b3e0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.012 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.965+0000 7fab63eed700 1 -- 172.21.15.107:0/3167597792 shutdown_connections 2023-07-21T22:28:21.012 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.965+0000 7fab63eed700 1 --2- 172.21.15.107:0/3167597792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab5c104ea0 0x7fab5c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.012 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.965+0000 7fab63eed700 1 --2- 172.21.15.107:0/3167597792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab5c0ffe00 0x7fab5c1001d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.965+0000 7fab63eed700 1 --2- 172.21.15.107:0/3167597792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab5c1007a0 0x7fab5c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.965+0000 7fab63eed700 1 -- 172.21.15.107:0/3167597792 >> 172.21.15.107:0/3167597792 conn(0x7fab5c0fb660 msgr2=0x7fab5c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:21.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.965+0000 7fab63eed700 1 -- 172.21.15.107:0/3167597792 shutdown_connections 2023-07-21T22:28:21.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.965+0000 7fab63eed700 1 -- 172.21.15.107:0/3167597792 wait complete. 2023-07-21T22:28:21.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab63eed700 1 Processor -- start 2023-07-21T22:28:21.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab63eed700 1 -- start start 2023-07-21T22:28:21.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab63eed700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab5c0ffe00 0x7fab5c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:21.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab63eed700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab5c1007a0 0x7fab5c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:21.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab63eed700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab5c104ea0 0x7fab5c0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:21.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab63eed700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c1a4240 con 0x7fab5c104ea0 2023-07-21T22:28:21.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab63eed700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c1a43b0 con 0x7fab5c0ffe00 2023-07-21T22:28:21.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab63eed700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c1a4570 con 0x7fab5c1007a0 2023-07-21T22:28:21.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab6248a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab5c104ea0 0x7fab5c0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:21.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab61c89700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab5c0ffe00 0x7fab5c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab61488700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab5c1007a0 0x7fab5c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab6248a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab5c104ea0 0x7fab5c0732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46976/0 (socket says 172.21.15.107:46976) 2023-07-21T22:28:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab6248a700 1 -- 172.21.15.107:0/1352831671 learned_addr learned my addr 172.21.15.107:0/1352831671 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.966+0000 7fab61c89700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab5c0ffe00 0x7fab5c072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:37800/0 (socket says 172.21.15.107:37800) 2023-07-21T22:28:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.967+0000 7fab6248a700 1 -- 172.21.15.107:0/1352831671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab5c1007a0 msgr2=0x7fab5c072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.967+0000 7fab6248a700 1 --2- 172.21.15.107:0/1352831671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab5c1007a0 0x7fab5c072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.967+0000 7fab6248a700 1 -- 172.21.15.107:0/1352831671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab5c0ffe00 msgr2=0x7fab5c072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:21.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.967+0000 7fab6248a700 1 --2- 172.21.15.107:0/1352831671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab5c0ffe00 0x7fab5c072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.967+0000 7fab6248a700 1 -- 172.21.15.107:0/1352831671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c009d00 con 0x7fab5c104ea0 2023-07-21T22:28:21.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.967+0000 7fab61c89700 1 --2- 172.21.15.107:0/1352831671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab5c0ffe00 0x7fab5c072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:28:21.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.967+0000 7fab6248a700 1 --2- 172.21.15.107:0/1352831671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab5c104ea0 0x7fab5c0732d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fab5400c970 tx=0x7fab5400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:21.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.967+0000 7fab5affd700 1 -- 172.21.15.107:0/1352831671 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54004a30 con 0x7fab5c104ea0 2023-07-21T22:28:21.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.967+0000 7fab5affd700 1 -- 172.21.15.107:0/1352831671 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab540054e0 con 0x7fab5c104ea0 2023-07-21T22:28:21.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.967+0000 7fab63eed700 1 -- 172.21.15.107:0/1352831671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c1a4850 con 0x7fab5c104ea0 2023-07-21T22:28:21.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.967+0000 7fab5affd700 1 -- 172.21.15.107:0/1352831671 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5400eb20 con 0x7fab5c104ea0 2023-07-21T22:28:21.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.967+0000 7fab63eed700 1 -- 172.21.15.107:0/1352831671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c1a4d00 con 0x7fab5c104ea0 2023-07-21T22:28:21.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.968+0000 7fab63eed700 1 -- 172.21.15.107:0/1352831671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c04e210 con 0x7fab5c104ea0 2023-07-21T22:28:21.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.970+0000 7fab5affd700 1 -- 172.21.15.107:0/1352831671 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fab54004b90 con 0x7fab5c104ea0 2023-07-21T22:28:21.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.970+0000 7fab5affd700 1 --2- 172.21.15.107:0/1352831671 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fab440a11e0 0x7fab440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:21.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.971+0000 7fab61c89700 1 --2- 172.21.15.107:0/1352831671 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fab440a11e0 0x7fab440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:21.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.971+0000 7fab5affd700 1 -- 172.21.15.107:0/1352831671 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fab540cc820 con 0x7fab5c104ea0 2023-07-21T22:28:21.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.971+0000 7fab61c89700 1 --2- 172.21.15.107:0/1352831671 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fab440a11e0 0x7fab440a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009fd0 tx=0x7fab4c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:21.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:20.974+0000 7fab5affd700 1 -- 172.21.15.107:0/1352831671 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fab54095ec0 con 0x7fab5c104ea0 2023-07-21T22:28:21.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.115+0000 7f89a64a7700 1 -- 172.21.15.107:0/3565142640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89a0104ab0 msgr2=0x7f89a0104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:21.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.115+0000 7f89a64a7700 1 --2- 172.21.15.107:0/3565142640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89a0104ab0 0x7f89a0104e80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8990009990 tx=0x7f8990009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.115+0000 7f89a64a7700 1 -- 172.21.15.107:0/3565142640 shutdown_connections 2023-07-21T22:28:21.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.115+0000 7f89a64a7700 1 --2- 172.21.15.107:0/3565142640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89a00ff460 0x7f89a010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.115+0000 7f89a64a7700 1 --2- 172.21.15.107:0/3565142640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89a0104ab0 0x7f89a0104e80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.115+0000 7f89a64a7700 1 --2- 172.21.15.107:0/3565142640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89a00feab0 0x7f89a00fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.115+0000 7f89a64a7700 1 -- 172.21.15.107:0/3565142640 >> 172.21.15.107:0/3565142640 conn(0x7f89a0075b70 msgr2=0x7f89a0075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:21.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.115+0000 7f89a64a7700 1 -- 172.21.15.107:0/3565142640 shutdown_connections 2023-07-21T22:28:21.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.115+0000 7f89a64a7700 1 -- 172.21.15.107:0/3565142640 wait complete. 2023-07-21T22:28:21.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.116+0000 7f89a64a7700 1 Processor -- start 2023-07-21T22:28:21.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.116+0000 7f89a64a7700 1 -- start start 2023-07-21T22:28:21.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f89a64a7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89a00feab0 0x7f89a0196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:21.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f89a64a7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89a00ff460 0x7f89a0196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:21.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f89a64a7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89a0104ab0 0x7f89a01a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f89a64a7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a0197920 con 0x7f89a0104ab0 2023-07-21T22:28:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f89a64a7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a01a2060 con 0x7f89a00ff460 2023-07-21T22:28:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f89a64a7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a01a2220 con 0x7f89a00feab0 2023-07-21T22:28:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f899ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89a00feab0 0x7f89a0196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f89a4a44700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89a0104ab0 0x7f89a01a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f899ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89a00feab0 0x7f89a0196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:35544/0 (socket says 172.21.15.107:35544) 2023-07-21T22:28:21.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f899ffff700 1 -- 172.21.15.107:0/2316632775 learned_addr learned my addr 172.21.15.107:0/2316632775 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f899f7fe700 1 --2- 172.21.15.107:0/2316632775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89a00ff460 0x7f89a0196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f89a4a44700 1 -- 172.21.15.107:0/2316632775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89a00feab0 msgr2=0x7f89a0196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f89a4a44700 1 --2- 172.21.15.107:0/2316632775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89a00feab0 0x7f89a0196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f89a4a44700 1 -- 172.21.15.107:0/2316632775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89a00ff460 msgr2=0x7f89a0196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f89a4a44700 1 --2- 172.21.15.107:0/2316632775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89a00ff460 0x7f89a0196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f89a4a44700 1 -- 172.21.15.107:0/2316632775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89900095f0 con 0x7f89a0104ab0 2023-07-21T22:28:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.117+0000 7f89a4a44700 1 --2- 172.21.15.107:0/2316632775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89a0104ab0 0x7f89a01a1b20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f899800efc0 tx=0x7f899800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:21.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.118+0000 7f899d7fa700 1 -- 172.21.15.107:0/2316632775 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998018070 con 0x7f89a0104ab0 2023-07-21T22:28:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.118+0000 7f89a64a7700 1 -- 172.21.15.107:0/2316632775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a01a24a0 con 0x7f89a0104ab0 2023-07-21T22:28:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.118+0000 7f899d7fa700 1 -- 172.21.15.107:0/2316632775 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f899800f040 con 0x7f89a0104ab0 2023-07-21T22:28:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.118+0000 7f899d7fa700 1 -- 172.21.15.107:0/2316632775 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899800cc80 con 0x7f89a0104ab0 2023-07-21T22:28:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.118+0000 7f89a64a7700 1 -- 172.21.15.107:0/2316632775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a01a29a0 con 0x7f89a0104ab0 2023-07-21T22:28:21.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.119+0000 7f89a64a7700 1 -- 172.21.15.107:0/2316632775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a004e210 con 0x7f89a0104ab0 2023-07-21T22:28:21.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.120+0000 7f899d7fa700 1 -- 172.21.15.107:0/2316632775 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8998003a00 con 0x7f89a0104ab0 2023-07-21T22:28:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.121+0000 7f899d7fa700 1 --2- 172.21.15.107:0/2316632775 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f89880a1260 0x7f89880a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.121+0000 7f899ffff700 1 --2- 172.21.15.107:0/2316632775 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f89880a1260 0x7f89880a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.121+0000 7f899d7fa700 1 -- 172.21.15.107:0/2316632775 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f89980cc3a0 con 0x7f89a0104ab0 2023-07-21T22:28:21.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.121+0000 7f899ffff700 1 --2- 172.21.15.107:0/2316632775 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f89880a1260 0x7f89880a3710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f899000b2e0 tx=0x7f8990005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:21.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.125+0000 7f899d7fa700 1 -- 172.21.15.107:0/2316632775 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8998095bd0 con 0x7f89a0104ab0 2023-07-21T22:28:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.193+0000 7fab63eed700 1 -- 172.21.15.107:0/1352831671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fab5c073d40 con 0x7fab5c104ea0 2023-07-21T22:28:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.194+0000 7fab5affd700 1 -- 172.21.15.107:0/1352831671 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fab5401d020 con 0x7fab5c104ea0 2023-07-21T22:28:21.241 INFO:teuthology.orchestra.run.smithi107.stdout:279172874267 2023-07-21T22:28:21.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.198+0000 7fab63eed700 1 -- 172.21.15.107:0/1352831671 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fab440a11e0 msgr2=0x7fab440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:21.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.198+0000 7fab63eed700 1 --2- 172.21.15.107:0/1352831671 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fab440a11e0 0x7fab440a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009fd0 tx=0x7fab4c00b040 comp rx=0 tx=0).stop 2023-07-21T22:28:21.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.198+0000 7fab63eed700 1 -- 172.21.15.107:0/1352831671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab5c104ea0 msgr2=0x7fab5c0732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.198+0000 7fab63eed700 1 --2- 172.21.15.107:0/1352831671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab5c104ea0 0x7fab5c0732d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fab5400c970 tx=0x7fab5400cc80 comp rx=0 tx=0).stop 2023-07-21T22:28:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.198+0000 7fab63eed700 1 -- 172.21.15.107:0/1352831671 shutdown_connections 2023-07-21T22:28:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.198+0000 7fab63eed700 1 --2- 172.21.15.107:0/1352831671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab5c104ea0 0x7fab5c0732d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.198+0000 7fab63eed700 1 --2- 172.21.15.107:0/1352831671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab5c0ffe00 0x7fab5c072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.198+0000 7fab63eed700 1 --2- 172.21.15.107:0/1352831671 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fab440a11e0 0x7fab440a3690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.198+0000 7fab63eed700 1 --2- 172.21.15.107:0/1352831671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab5c1007a0 0x7fab5c072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.198+0000 7fab63eed700 1 -- 172.21.15.107:0/1352831671 >> 172.21.15.107:0/1352831671 conn(0x7fab5c0fb660 msgr2=0x7fab5c0fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.198+0000 7fab63eed700 1 -- 172.21.15.107:0/1352831671 shutdown_connections 2023-07-21T22:28:21.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.198+0000 7fab63eed700 1 -- 172.21.15.107:0/1352831671 wait complete. 2023-07-21T22:28:21.266 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481809 got 330712481811 for osd.11 2023-07-21T22:28:21.266 DEBUG:teuthology.parallel:result is None 2023-07-21T22:28:21.381 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.335+0000 7f89a64a7700 1 -- 172.21.15.107:0/2316632775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f89a01a2bc0 con 0x7f89a0104ab0 2023-07-21T22:28:21.381 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.335+0000 7f899d7fa700 1 -- 172.21.15.107:0/2316632775 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f89980991f0 con 0x7f89a0104ab0 2023-07-21T22:28:21.382 INFO:teuthology.orchestra.run.smithi107.stdout:193273528359 2023-07-21T22:28:21.384 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.338+0000 7f89a64a7700 1 -- 172.21.15.107:0/2316632775 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f89880a1260 msgr2=0x7f89880a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:21.384 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.338+0000 7f89a64a7700 1 --2- 172.21.15.107:0/2316632775 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f89880a1260 0x7f89880a3710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f899000b2e0 tx=0x7f8990005fb0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.338+0000 7f89a64a7700 1 -- 172.21.15.107:0/2316632775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89a0104ab0 msgr2=0x7f89a01a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:21.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.338+0000 7f89a64a7700 1 --2- 172.21.15.107:0/2316632775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89a0104ab0 0x7f89a01a1b20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f899800efc0 tx=0x7f899800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.338+0000 7f89a64a7700 1 -- 172.21.15.107:0/2316632775 shutdown_connections 2023-07-21T22:28:21.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.338+0000 7f89a64a7700 1 --2- 172.21.15.107:0/2316632775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89a0104ab0 0x7f89a01a1b20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.338+0000 7f89a64a7700 1 --2- 172.21.15.107:0/2316632775 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f89880a1260 0x7f89880a3710 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.338+0000 7f89a64a7700 1 --2- 172.21.15.107:0/2316632775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89a00feab0 0x7f89a0196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.338+0000 7f89a64a7700 1 --2- 172.21.15.107:0/2316632775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89a00ff460 0x7f89a0196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:21.386 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.338+0000 7f89a64a7700 1 -- 172.21.15.107:0/2316632775 >> 172.21.15.107:0/2316632775 conn(0x7f89a0075b70 msgr2=0x7f89a00fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:21.386 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.338+0000 7f89a64a7700 1 -- 172.21.15.107:0/2316632775 shutdown_connections 2023-07-21T22:28:21.386 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:21.338+0000 7f89a64a7700 1 -- 172.21.15.107:0/2316632775 wait complete. 2023-07-21T22:28:21.412 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:21 smithi107 ceph-mon[96543]: pgmap v269: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-07-21T22:28:21.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:21 smithi132 ceph-mon[97863]: pgmap v269: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-07-21T22:28:21.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:21 smithi190 ceph-mon[98334]: pgmap v269: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-07-21T22:28:21.909 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:21.959 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874266 got 279172874267 for osd.9 2023-07-21T22:28:21.960 DEBUG:teuthology.parallel:result is None 2023-07-21T22:28:22.119 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:22.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.115+0000 7fa1ec958700 1 -- 172.21.15.107:0/2235332105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e8100f80 msgr2=0x7fa1e810ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:22.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.115+0000 7fa1ec958700 1 --2- 172.21.15.107:0/2235332105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e8100f80 0x7fa1e810ba10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000b0d0 tx=0x7fa1e000b3e0 comp rx=0 tx=0).stop 2023-07-21T22:28:22.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.115+0000 7fa1ec958700 1 -- 172.21.15.107:0/2235332105 shutdown_connections 2023-07-21T22:28:22.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.115+0000 7fa1ec958700 1 --2- 172.21.15.107:0/2235332105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e8100f80 0x7fa1e810ba10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:22.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.115+0000 7fa1ec958700 1 --2- 172.21.15.107:0/2235332105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1e80fee80 0x7fa1e80ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:22.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.115+0000 7fa1ec958700 1 --2- 172.21.15.107:0/2235332105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e81005d0 0x7fa1e8100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:22.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.115+0000 7fa1ec958700 1 -- 172.21.15.107:0/2235332105 >> 172.21.15.107:0/2235332105 conn(0x7fa1e8075b60 msgr2=0x7fa1e8075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:22.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.116+0000 7fa1ec958700 1 -- 172.21.15.107:0/2235332105 shutdown_connections 2023-07-21T22:28:22.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.116+0000 7fa1ec958700 1 -- 172.21.15.107:0/2235332105 wait complete. 2023-07-21T22:28:22.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.116+0000 7fa1ec958700 1 Processor -- start 2023-07-21T22:28:22.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1ec958700 1 -- start start 2023-07-21T22:28:22.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1ec958700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e80fee80 0x7fa1e8105ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:22.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1ec958700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1e81005d0 0x7fa1e8105ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:22.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1ec958700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e8100f80 0x7fa1e8106570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:22.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1ec958700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e81a4240 con 0x7fa1e80fee80 2023-07-21T22:28:22.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1ec958700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e81a43b0 con 0x7fa1e8100f80 2023-07-21T22:28:22.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1ec958700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e81a4570 con 0x7fa1e81005d0 2023-07-21T22:28:22.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1e659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e80fee80 0x7fa1e8105ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:22.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1e659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e80fee80 0x7fa1e8105ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47000/0 (socket says 172.21.15.107:47000) 2023-07-21T22:28:22.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1e659c700 1 -- 172.21.15.107:0/3936235170 learned_addr learned my addr 172.21.15.107:0/3936235170 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:22.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1ddd9b700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1e81005d0 0x7fa1e8105ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:22.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1e6d9d700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e8100f80 0x7fa1e8106570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:22.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1e659c700 1 -- 172.21.15.107:0/3936235170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1e81005d0 msgr2=0x7fa1e8105ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:22.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1e659c700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1e81005d0 0x7fa1e8105ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:22.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1e659c700 1 -- 172.21.15.107:0/3936235170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e8100f80 msgr2=0x7fa1e8106570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:22.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1e659c700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e8100f80 0x7fa1e8106570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:22.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1e659c700 1 -- 172.21.15.107:0/3936235170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e0009d00 con 0x7fa1e80fee80 2023-07-21T22:28:22.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1e6d9d700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e8100f80 0x7fa1e8106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:22.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.117+0000 7fa1e659c700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e80fee80 0x7fa1e8105ab0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800d980 tx=0x7fa1d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:22.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.119+0000 7fa1dffff700 1 -- 172.21.15.107:0/3936235170 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8004220 con 0x7fa1e80fee80 2023-07-21T22:28:22.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.119+0000 7fa1ec958700 1 -- 172.21.15.107:0/3936235170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e81a4850 con 0x7fa1e80fee80 2023-07-21T22:28:22.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.119+0000 7fa1ec958700 1 -- 172.21.15.107:0/3936235170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e81a4da0 con 0x7fa1e80fee80 2023-07-21T22:28:22.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.119+0000 7fa1dffff700 1 -- 172.21.15.107:0/3936235170 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1d8007640 con 0x7fa1e80fee80 2023-07-21T22:28:22.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.119+0000 7fa1dffff700 1 -- 172.21.15.107:0/3936235170 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8005610 con 0x7fa1e80fee80 2023-07-21T22:28:22.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.120+0000 7fa1ec958700 1 -- 172.21.15.107:0/3936235170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c8005320 con 0x7fa1e80fee80 2023-07-21T22:28:22.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.121+0000 7fa1dffff700 1 -- 172.21.15.107:0/3936235170 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fa1d8004830 con 0x7fa1e80fee80 2023-07-21T22:28:22.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.122+0000 7fa1dffff700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa1d40a11e0 0x7fa1d40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:22.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.122+0000 7fa1ddd9b700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa1d40a11e0 0x7fa1d40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:22.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.122+0000 7fa1dffff700 1 -- 172.21.15.107:0/3936235170 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa1d80cc730 con 0x7fa1e80fee80 2023-07-21T22:28:22.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.122+0000 7fa1ddd9b700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa1d40a11e0 0x7fa1d40a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa1e81072a0 tx=0x7fa1d00073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:22.172 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.126+0000 7fa1dffff700 1 -- 172.21.15.107:0/3936235170 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa1d8095ed0 con 0x7fa1e80fee80 2023-07-21T22:28:22.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.326+0000 7fa1ec958700 1 -- 172.21.15.107:0/3936235170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa1c8005cc0 con 0x7fa1e80fee80 2023-07-21T22:28:22.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.327+0000 7fa1dffff700 1 -- 172.21.15.107:0/3936235170 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa1d8016090 con 0x7fa1e80fee80 2023-07-21T22:28:22.374 INFO:teuthology.orchestra.run.smithi107.stdout:77309411385 2023-07-21T22:28:22.375 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.329+0000 7fa1ec958700 1 -- 172.21.15.107:0/3936235170 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa1d40a11e0 msgr2=0x7fa1d40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:22.375 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.329+0000 7fa1ec958700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa1d40a11e0 0x7fa1d40a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa1e81072a0 tx=0x7fa1d00073d0 comp rx=0 tx=0).stop 2023-07-21T22:28:22.375 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.329+0000 7fa1ec958700 1 -- 172.21.15.107:0/3936235170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e80fee80 msgr2=0x7fa1e8105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:22.375 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.329+0000 7fa1ec958700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e80fee80 0x7fa1e8105ab0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800d980 tx=0x7fa1d800dc90 comp rx=0 tx=0).stop 2023-07-21T22:28:22.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.329+0000 7fa1ec958700 1 -- 172.21.15.107:0/3936235170 shutdown_connections 2023-07-21T22:28:22.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.329+0000 7fa1ec958700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e8100f80 0x7fa1e8106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:22.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.329+0000 7fa1ec958700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1e80fee80 0x7fa1e8105ab0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:22.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.329+0000 7fa1ec958700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa1d40a11e0 0x7fa1d40a3690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:22.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.329+0000 7fa1ec958700 1 --2- 172.21.15.107:0/3936235170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1e81005d0 0x7fa1e8105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:22.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.329+0000 7fa1ec958700 1 -- 172.21.15.107:0/3936235170 >> 172.21.15.107:0/3936235170 conn(0x7fa1e8075b60 msgr2=0x7fa1e80fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:22.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.329+0000 7fa1ec958700 1 -- 172.21.15.107:0/3936235170 shutdown_connections 2023-07-21T22:28:22.377 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:22.329+0000 7fa1ec958700 1 -- 172.21.15.107:0/3936235170 wait complete. 2023-07-21T22:28:22.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:22 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1352831671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T22:28:22.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:22 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2316632775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T22:28:22.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:22 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1352831671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T22:28:22.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:22 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2316632775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T22:28:22.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:22 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1352831671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T22:28:22.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:22 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2316632775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T22:28:22.662 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528357 got 193273528359 for osd.6 2023-07-21T22:28:22.663 DEBUG:teuthology.parallel:result is None 2023-07-21T22:28:23.182 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.132+0000 7f2ea1218700 1 -- 172.21.15.107:0/3980134868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e9c104b70 msgr2=0x7f2e9c104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:23.182 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.132+0000 7f2ea1218700 1 --2- 172.21.15.107:0/3980134868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e9c104b70 0x7f2e9c104f40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009990 tx=0x7f2e8c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:23.182 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.132+0000 7f2ea1218700 1 -- 172.21.15.107:0/3980134868 shutdown_connections 2023-07-21T22:28:23.183 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.132+0000 7f2ea1218700 1 --2- 172.21.15.107:0/3980134868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e9c0ff500 0x7f2e9c10b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:23.183 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.132+0000 7f2ea1218700 1 --2- 172.21.15.107:0/3980134868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e9c104b70 0x7f2e9c104f40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:23.183 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.132+0000 7f2ea1218700 1 --2- 172.21.15.107:0/3980134868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e9c0feb50 0x7f2e9c0fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:23.183 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.132+0000 7f2ea1218700 1 -- 172.21.15.107:0/3980134868 >> 172.21.15.107:0/3980134868 conn(0x7f2e9c0fa5f0 msgr2=0x7f2e9c0fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:23.184 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.132+0000 7f2ea1218700 1 -- 172.21.15.107:0/3980134868 shutdown_connections 2023-07-21T22:28:23.184 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.132+0000 7f2ea1218700 1 -- 172.21.15.107:0/3980134868 wait complete. 2023-07-21T22:28:23.184 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.133+0000 7f2ea1218700 1 Processor -- start 2023-07-21T22:28:23.184 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.133+0000 7f2ea1218700 1 -- start start 2023-07-21T22:28:23.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.133+0000 7f2ea1218700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e9c0feb50 0x7f2e9c196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:23.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2ea1218700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e9c0ff500 0x7f2e9c196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:23.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2ea1218700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e9c104b70 0x7f2e9c1a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:23.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2ea1218700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c1a2050 con 0x7f2e9c104b70 2023-07-21T22:28:23.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2ea1218700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c1a2190 con 0x7f2e9c0ff500 2023-07-21T22:28:23.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2ea1218700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c1a22d0 con 0x7f2e9c0feb50 2023-07-21T22:28:23.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e9ad9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e9c0feb50 0x7f2e9c196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:23.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e9b59e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e9c104b70 0x7f2e9c1a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:23.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e9ad9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e9c0feb50 0x7f2e9c196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:35570/0 (socket says 172.21.15.107:35570) 2023-07-21T22:28:23.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e9b59e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e9c104b70 0x7f2e9c1a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47016/0 (socket says 172.21.15.107:47016) 2023-07-21T22:28:23.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e9ad9d700 1 -- 172.21.15.107:0/2202435163 learned_addr learned my addr 172.21.15.107:0/2202435163 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:23.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e9a59c700 1 --2- 172.21.15.107:0/2202435163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e9c0ff500 0x7f2e9c196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:23.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e9b59e700 1 -- 172.21.15.107:0/2202435163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e9c0feb50 msgr2=0x7f2e9c196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:23.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e9b59e700 1 --2- 172.21.15.107:0/2202435163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e9c0feb50 0x7f2e9c196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:23.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e9b59e700 1 -- 172.21.15.107:0/2202435163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e9c0ff500 msgr2=0x7f2e9c196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:23.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e9b59e700 1 --2- 172.21.15.107:0/2202435163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e9c0ff500 0x7f2e9c196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:23.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e9b59e700 1 -- 172.21.15.107:0/2202435163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c0095f0 con 0x7f2e9c104b70 2023-07-21T22:28:23.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e9b59e700 1 --2- 172.21.15.107:0/2202435163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e9c104b70 0x7f2e9c1a1b10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2e94007f40 tx=0x7f2e9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:23.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e83fff700 1 -- 172.21.15.107:0/2202435163 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94004db0 con 0x7f2e9c104b70 2023-07-21T22:28:23.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2ea1218700 1 -- 172.21.15.107:0/2202435163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c1a2530 con 0x7f2e9c104b70 2023-07-21T22:28:23.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e83fff700 1 -- 172.21.15.107:0/2202435163 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e9400f040 con 0x7f2e9c104b70 2023-07-21T22:28:23.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2e83fff700 1 -- 172.21.15.107:0/2202435163 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94005a20 con 0x7f2e9c104b70 2023-07-21T22:28:23.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.134+0000 7f2ea1218700 1 -- 172.21.15.107:0/2202435163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c1a29e0 con 0x7f2e9c104b70 2023-07-21T22:28:23.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.135+0000 7f2ea1218700 1 -- 172.21.15.107:0/2202435163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c04e210 con 0x7f2e9c104b70 2023-07-21T22:28:23.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.136+0000 7f2e83fff700 1 -- 172.21.15.107:0/2202435163 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f2e94005000 con 0x7f2e9c104b70 2023-07-21T22:28:23.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.137+0000 7f2e83fff700 1 --2- 172.21.15.107:0/2202435163 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2e840a11e0 0x7f2e840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:23.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.137+0000 7f2e9ad9d700 1 --2- 172.21.15.107:0/2202435163 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2e840a11e0 0x7f2e840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:23.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.137+0000 7f2e83fff700 1 -- 172.21.15.107:0/2202435163 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2e940cc1e0 con 0x7f2e9c104b70 2023-07-21T22:28:23.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.137+0000 7f2e9ad9d700 1 --2- 172.21.15.107:0/2202435163 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2e840a11e0 0x7f2e840a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00b5c0 tx=0x7f2e8c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:23.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.141+0000 7f2e83fff700 1 -- 172.21.15.107:0/2202435163 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2e9404e770 con 0x7f2e9c104b70 2023-07-21T22:28:23.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.338+0000 7f2ea1218700 1 -- 172.21.15.107:0/2202435163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f2e9c068f00 con 0x7f2e9c104b70 2023-07-21T22:28:23.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.339+0000 7f2e83fff700 1 -- 172.21.15.107:0/2202435163 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f2e94095980 con 0x7f2e9c104b70 2023-07-21T22:28:23.385 INFO:teuthology.orchestra.run.smithi107.stdout:300647710744 2023-07-21T22:28:23.387 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.341+0000 7f2ea1218700 1 -- 172.21.15.107:0/2202435163 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2e840a11e0 msgr2=0x7f2e840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:23.387 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.341+0000 7f2ea1218700 1 --2- 172.21.15.107:0/2202435163 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2e840a11e0 0x7f2e840a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00b5c0 tx=0x7f2e8c005960 comp rx=0 tx=0).stop 2023-07-21T22:28:23.387 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.341+0000 7f2ea1218700 1 -- 172.21.15.107:0/2202435163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e9c104b70 msgr2=0x7f2e9c1a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:23.387 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.341+0000 7f2ea1218700 1 --2- 172.21.15.107:0/2202435163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e9c104b70 0x7f2e9c1a1b10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2e94007f40 tx=0x7f2e9400d3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.341+0000 7f2ea1218700 1 -- 172.21.15.107:0/2202435163 shutdown_connections 2023-07-21T22:28:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.341+0000 7f2ea1218700 1 --2- 172.21.15.107:0/2202435163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e9c104b70 0x7f2e9c1a1b10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.341+0000 7f2ea1218700 1 --2- 172.21.15.107:0/2202435163 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2e840a11e0 0x7f2e840a3690 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.341+0000 7f2ea1218700 1 --2- 172.21.15.107:0/2202435163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2e9c0feb50 0x7f2e9c196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.341+0000 7f2ea1218700 1 --2- 172.21.15.107:0/2202435163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e9c0ff500 0x7f2e9c196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.341+0000 7f2ea1218700 1 -- 172.21.15.107:0/2202435163 >> 172.21.15.107:0/2202435163 conn(0x7f2e9c0fa5f0 msgr2=0x7f2e9c1026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:23.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.341+0000 7f2ea1218700 1 -- 172.21.15.107:0/2202435163 shutdown_connections 2023-07-21T22:28:23.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:23.341+0000 7f2ea1218700 1 -- 172.21.15.107:0/2202435163 wait complete. 2023-07-21T22:28:23.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:23 smithi107 ceph-mon[96543]: pgmap v270: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T22:28:23.406 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:23 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3936235170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T22:28:23.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:23 smithi132 ceph-mon[97863]: pgmap v270: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T22:28:23.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:23 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3936235170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T22:28:23.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:23 smithi190 ceph-mon[98334]: pgmap v270: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T22:28:23.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:23 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3936235170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T22:28:23.847 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411383 got 77309411385 for osd.2 2023-07-21T22:28:23.847 DEBUG:teuthology.parallel:result is None 2023-07-21T22:28:24.055 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710741 got 300647710744 for osd.10 2023-07-21T22:28:24.055 DEBUG:teuthology.parallel:result is None 2023-07-21T22:28:24.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:24 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2202435163' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T22:28:24.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.388+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2341338996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab80fea60 msgr2=0x7f8ab80feed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:24.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.388+0000 7f8abfbbf700 1 --2- 172.21.15.107:0/2341338996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab80fea60 0x7f8ab80feed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009990 tx=0x7f8aa8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.388+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2341338996 shutdown_connections 2023-07-21T22:28:24.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.388+0000 7f8abfbbf700 1 --2- 172.21.15.107:0/2341338996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ab80ff410 0x7f8ab810b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.388+0000 7f8abfbbf700 1 --2- 172.21.15.107:0/2341338996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab8104a60 0x7f8ab8104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.388+0000 7f8abfbbf700 1 --2- 172.21.15.107:0/2341338996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab80fea60 0x7f8ab80feed0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.389+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2341338996 >> 172.21.15.107:0/2341338996 conn(0x7f8ab8075bc0 msgr2=0x7f8ab8075fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:24.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.389+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2341338996 shutdown_connections 2023-07-21T22:28:24.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.389+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2341338996 wait complete. 2023-07-21T22:28:24.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.389+0000 7f8abfbbf700 1 Processor -- start 2023-07-21T22:28:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.389+0000 7f8abfbbf700 1 -- start start 2023-07-21T22:28:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.389+0000 7f8abfbbf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab80fea60 0x7f8ab8196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.389+0000 7f8abfbbf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab80ff410 0x7f8ab8196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.389+0000 7f8abfbbf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ab8104a60 0x7f8ab81a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.389+0000 7f8abfbbf700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab81977d0 con 0x7f8ab80fea60 2023-07-21T22:28:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.389+0000 7f8abfbbf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab81a1f40 con 0x7f8ab80ff410 2023-07-21T22:28:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.389+0000 7f8abfbbf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab81a20b0 con 0x7f8ab8104a60 2023-07-21T22:28:24.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abd95b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab80fea60 0x7f8ab8196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:24.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abd95b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab80fea60 0x7f8ab8196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47026/0 (socket says 172.21.15.107:47026) 2023-07-21T22:28:24.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abd95b700 1 -- 172.21.15.107:0/2472050811 learned_addr learned my addr 172.21.15.107:0/2472050811 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:24.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abe15c700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ab8104a60 0x7f8ab81a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:24.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abd15a700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab80ff410 0x7f8ab8196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:24.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abd95b700 1 -- 172.21.15.107:0/2472050811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ab8104a60 msgr2=0x7f8ab81a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:24.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abd95b700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ab8104a60 0x7f8ab81a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abd95b700 1 -- 172.21.15.107:0/2472050811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab80ff410 msgr2=0x7f8ab8196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:24.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abd95b700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab80ff410 0x7f8ab8196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abd95b700 1 -- 172.21.15.107:0/2472050811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa80095f0 con 0x7f8ab80fea60 2023-07-21T22:28:24.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abd15a700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab80ff410 0x7f8ab8196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:24.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abd95b700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab80fea60 0x7f8ab8196980 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400c990 tx=0x7f8ab400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:24.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8aaeffd700 1 -- 172.21.15.107:0/2472050811 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4007b00 con 0x7f8ab80fea60 2023-07-21T22:28:24.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2472050811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab81a2390 con 0x7f8ab80fea60 2023-07-21T22:28:24.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8aaeffd700 1 -- 172.21.15.107:0/2472050811 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ab4007c60 con 0x7f8ab80fea60 2023-07-21T22:28:24.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8aaeffd700 1 -- 172.21.15.107:0/2472050811 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4018660 con 0x7f8ab80fea60 2023-07-21T22:28:24.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.390+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2472050811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab81a2890 con 0x7f8ab80fea60 2023-07-21T22:28:24.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.391+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2472050811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab804e210 con 0x7f8ab80fea60 2023-07-21T22:28:24.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.392+0000 7f8aaeffd700 1 -- 172.21.15.107:0/2472050811 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8ab400f450 con 0x7f8ab80fea60 2023-07-21T22:28:24.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:24 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2202435163' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T22:28:24.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.393+0000 7f8aaeffd700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8aa40a11e0 0x7f8aa40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:24.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.393+0000 7f8abd15a700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8aa40a11e0 0x7f8aa40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:24.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.393+0000 7f8aaeffd700 1 -- 172.21.15.107:0/2472050811 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8ab40cca60 con 0x7f8ab80fea60 2023-07-21T22:28:24.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.393+0000 7f8abd15a700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8aa40a11e0 0x7f8aa40a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800b5c0 tx=0x7f8aa8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:24.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.396+0000 7f8aaeffd700 1 -- 172.21.15.107:0/2472050811 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8ab4096290 con 0x7f8ab80fea60 2023-07-21T22:28:24.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:24 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2202435163' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T22:28:24.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.598+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2472050811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8ab81a2c20 con 0x7f8ab80fea60 2023-07-21T22:28:24.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.598+0000 7f8aaeffd700 1 -- 172.21.15.107:0/2472050811 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8ab40998b0 con 0x7f8ab80fea60 2023-07-21T22:28:24.644 INFO:teuthology.orchestra.run.smithi107.stdout:34359738434 2023-07-21T22:28:24.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.600+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2472050811 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8aa40a11e0 msgr2=0x7f8aa40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:24.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.600+0000 7f8abfbbf700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8aa40a11e0 0x7f8aa40a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800b5c0 tx=0x7f8aa8005fb0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.600+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2472050811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab80fea60 msgr2=0x7f8ab8196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:24.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.600+0000 7f8abfbbf700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab80fea60 0x7f8ab8196980 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400c990 tx=0x7f8ab400cd50 comp rx=0 tx=0).stop 2023-07-21T22:28:24.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.600+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2472050811 shutdown_connections 2023-07-21T22:28:24.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.600+0000 7f8abfbbf700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8aa40a11e0 0x7f8aa40a3690 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.600+0000 7f8abfbbf700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ab8104a60 0x7f8ab81a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.600+0000 7f8abfbbf700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab80fea60 0x7f8ab8196980 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.600+0000 7f8abfbbf700 1 --2- 172.21.15.107:0/2472050811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab80ff410 0x7f8ab8196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.600+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2472050811 >> 172.21.15.107:0/2472050811 conn(0x7f8ab8075bc0 msgr2=0x7f8ab81025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:24.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.601+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2472050811 shutdown_connections 2023-07-21T22:28:24.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.601+0000 7f8abfbbf700 1 -- 172.21.15.107:0/2472050811 wait complete. 2023-07-21T22:28:24.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.646+0000 7f9e2851f700 1 -- 172.21.15.107:0/3613091495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e200ffdf0 msgr2=0x7f9e201001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:24.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.646+0000 7f9e2851f700 1 --2- 172.21.15.107:0/3613091495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e200ffdf0 0x7f9e201001c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0099c0 tx=0x7f9e1c009cd0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.646+0000 7f9e2851f700 1 -- 172.21.15.107:0/3613091495 shutdown_connections 2023-07-21T22:28:24.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.646+0000 7f9e2851f700 1 --2- 172.21.15.107:0/3613091495 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e20104e90 0x7f9e2010ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.646+0000 7f9e2851f700 1 --2- 172.21.15.107:0/3613091495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e200ffdf0 0x7f9e201001c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.646+0000 7f9e2851f700 1 --2- 172.21.15.107:0/3613091495 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e20100790 0x7f9e20104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.646+0000 7f9e2851f700 1 -- 172.21.15.107:0/3613091495 >> 172.21.15.107:0/3613091495 conn(0x7f9e200fb670 msgr2=0x7f9e200fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:24.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.647+0000 7f9e2851f700 1 -- 172.21.15.107:0/3613091495 shutdown_connections 2023-07-21T22:28:24.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.647+0000 7f9e2851f700 1 -- 172.21.15.107:0/3613091495 wait complete. 2023-07-21T22:28:24.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.647+0000 7f9e2851f700 1 Processor -- start 2023-07-21T22:28:24.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.647+0000 7f9e2851f700 1 -- start start 2023-07-21T22:28:24.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.647+0000 7f9e2851f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e200ffdf0 0x7f9e2019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:24.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.647+0000 7f9e2851f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e20100790 0x7f9e2019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:24.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.647+0000 7f9e2851f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20104e90 0x7f9e20195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:24.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.647+0000 7f9e2851f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e20196070 con 0x7f9e20104e90 2023-07-21T22:28:24.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.647+0000 7f9e2851f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e201961e0 con 0x7f9e20100790 2023-07-21T22:28:24.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.647+0000 7f9e2851f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e201963a0 con 0x7f9e200ffdf0 2023-07-21T22:28:24.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.647+0000 7f9e26abc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20104e90 0x7f9e20195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:24.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e26abc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20104e90 0x7f9e20195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47044/0 (socket says 172.21.15.107:47044) 2023-07-21T22:28:24.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e26abc700 1 -- 172.21.15.107:0/380508325 learned_addr learned my addr 172.21.15.107:0/380508325 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:24.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e25aba700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e20100790 0x7f9e2019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:24.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e262bb700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e200ffdf0 0x7f9e2019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:24.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e26abc700 1 -- 172.21.15.107:0/380508325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e200ffdf0 msgr2=0x7f9e2019b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:24.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e26abc700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e200ffdf0 0x7f9e2019b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e26abc700 1 -- 172.21.15.107:0/380508325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e20100790 msgr2=0x7f9e2019bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:24.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e26abc700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e20100790 0x7f9e2019bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e26abc700 1 -- 172.21.15.107:0/380508325 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c0095f0 con 0x7f9e20104e90 2023-07-21T22:28:24.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e25aba700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e20100790 0x7f9e2019bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:24.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e26abc700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20104e90 0x7f9e20195a00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9e18007d70 tx=0x7f9e1800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:24.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e177fe700 1 -- 172.21.15.107:0/380508325 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1800ad40 con 0x7f9e20104e90 2023-07-21T22:28:24.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e177fe700 1 -- 172.21.15.107:0/380508325 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e180128e0 con 0x7f9e20104e90 2023-07-21T22:28:24.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e2851f700 1 -- 172.21.15.107:0/380508325 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e20196680 con 0x7f9e20104e90 2023-07-21T22:28:24.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e177fe700 1 -- 172.21.15.107:0/380508325 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18011780 con 0x7f9e20104e90 2023-07-21T22:28:24.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.648+0000 7f9e2851f700 1 -- 172.21.15.107:0/380508325 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e201a2970 con 0x7f9e20104e90 2023-07-21T22:28:24.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.649+0000 7f9e2851f700 1 -- 172.21.15.107:0/380508325 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2004e210 con 0x7f9e20104e90 2023-07-21T22:28:24.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.650+0000 7f9e177fe700 1 -- 172.21.15.107:0/380508325 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f9e18012a50 con 0x7f9e20104e90 2023-07-21T22:28:24.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.651+0000 7f9e177fe700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9e0c0a11e0 0x7f9e0c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:24.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.651+0000 7f9e262bb700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9e0c0a11e0 0x7f9e0c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:24.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.651+0000 7f9e177fe700 1 -- 172.21.15.107:0/380508325 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9e180ccba0 con 0x7f9e20104e90 2023-07-21T22:28:24.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.652+0000 7f9e262bb700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9e0c0a11e0 0x7f9e0c0a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c000c00 tx=0x7f9e1c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:24.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.656+0000 7f9e177fe700 1 -- 172.21.15.107:0/380508325 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9e18096350 con 0x7f9e20104e90 2023-07-21T22:28:24.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.854+0000 7f9e2851f700 1 -- 172.21.15.107:0/380508325 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9e20196f10 con 0x7f9e20104e90 2023-07-21T22:28:24.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.854+0000 7f9e177fe700 1 -- 172.21.15.107:0/380508325 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e18016090 con 0x7f9e20104e90 2023-07-21T22:28:24.900 INFO:teuthology.orchestra.run.smithi107.stdout:214748364836 2023-07-21T22:28:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.856+0000 7f9e2851f700 1 -- 172.21.15.107:0/380508325 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9e0c0a11e0 msgr2=0x7f9e0c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.856+0000 7f9e2851f700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9e0c0a11e0 0x7f9e0c0a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c000c00 tx=0x7f9e1c005fb0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.856+0000 7f9e2851f700 1 -- 172.21.15.107:0/380508325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20104e90 msgr2=0x7f9e20195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:24.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.856+0000 7f9e2851f700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20104e90 0x7f9e20195a00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9e18007d70 tx=0x7f9e1800a340 comp rx=0 tx=0).stop 2023-07-21T22:28:24.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.856+0000 7f9e2851f700 1 -- 172.21.15.107:0/380508325 shutdown_connections 2023-07-21T22:28:24.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.856+0000 7f9e2851f700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e20104e90 0x7f9e20195a00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.856+0000 7f9e2851f700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9e0c0a11e0 0x7f9e0c0a3690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.856+0000 7f9e2851f700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e200ffdf0 0x7f9e2019b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.857+0000 7f9e2851f700 1 --2- 172.21.15.107:0/380508325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e20100790 0x7f9e2019bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:24.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.857+0000 7f9e2851f700 1 -- 172.21.15.107:0/380508325 >> 172.21.15.107:0/380508325 conn(0x7f9e200fb670 msgr2=0x7f9e200fc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:24.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.857+0000 7f9e2851f700 1 -- 172.21.15.107:0/380508325 shutdown_connections 2023-07-21T22:28:24.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:24.857+0000 7f9e2851f700 1 -- 172.21.15.107:0/380508325 wait complete. 2023-07-21T22:28:25.125 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738432 got 34359738434 for osd.0 2023-07-21T22:28:25.126 DEBUG:teuthology.parallel:result is None 2023-07-21T22:28:25.348 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:25 smithi107 ceph-mon[96543]: pgmap v271: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-07-21T22:28:25.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:25 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2472050811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T22:28:25.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:25 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/380508325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T22:28:25.350 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364834 got 214748364836 for osd.7 2023-07-21T22:28:25.350 DEBUG:teuthology.parallel:result is None 2023-07-21T22:28:25.350 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-21T22:28:25.350 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-21T22:28:25.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:25 smithi132 ceph-mon[97863]: pgmap v271: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-07-21T22:28:25.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:25 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2472050811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T22:28:25.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:25 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/380508325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T22:28:25.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:25 smithi190 ceph-mon[98334]: pgmap v271: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-07-21T22:28:25.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:25 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2472050811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T22:28:25.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:25 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/380508325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T22:28:25.905 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:27.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.200+0000 7f6306ac0700 1 -- 172.21.15.107:0/679707370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63000feab0 msgr2=0x7f63000fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:27.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.200+0000 7f6306ac0700 1 --2- 172.21.15.107:0/679707370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63000feab0 0x7f63000fef20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009990 tx=0x7f62ec009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:27.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.200+0000 7f6306ac0700 1 -- 172.21.15.107:0/679707370 shutdown_connections 2023-07-21T22:28:27.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.200+0000 7f6306ac0700 1 --2- 172.21.15.107:0/679707370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63000ff460 0x7f630010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:27.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.200+0000 7f6306ac0700 1 --2- 172.21.15.107:0/679707370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6300104ab0 0x7f6300104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:27.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.200+0000 7f6306ac0700 1 --2- 172.21.15.107:0/679707370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63000feab0 0x7f63000fef20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:27.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.200+0000 7f6306ac0700 1 -- 172.21.15.107:0/679707370 >> 172.21.15.107:0/679707370 conn(0x7f6300075b70 msgr2=0x7f6300075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:27.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.200+0000 7f6306ac0700 1 -- 172.21.15.107:0/679707370 shutdown_connections 2023-07-21T22:28:27.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.200+0000 7f6306ac0700 1 -- 172.21.15.107:0/679707370 wait complete. 2023-07-21T22:28:27.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.200+0000 7f6306ac0700 1 Processor -- start 2023-07-21T22:28:27.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.200+0000 7f6306ac0700 1 -- start start 2023-07-21T22:28:27.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f6306ac0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63000feab0 0x7f6300196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:27.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f6306ac0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63000ff460 0x7f6300197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:27.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f6306ac0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6300104ab0 0x7f63001a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:27.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f6306ac0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6300197970 con 0x7f6300104ab0 2023-07-21T22:28:27.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f6306ac0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001a20e0 con 0x7f63000feab0 2023-07-21T22:28:27.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f6306ac0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001a2250 con 0x7f63000ff460 2023-07-21T22:28:27.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f630505d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6300104ab0 0x7f63001a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f630505d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6300104ab0 0x7f63001a1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47056/0 (socket says 172.21.15.107:47056) 2023-07-21T22:28:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f630505d700 1 -- 172.21.15.107:0/4237218960 learned_addr learned my addr 172.21.15.107:0/4237218960 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f630485c700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63000feab0 0x7f6300196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f62fffff700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63000ff460 0x7f6300197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f630505d700 1 -- 172.21.15.107:0/4237218960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63000ff460 msgr2=0x7f6300197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f630505d700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63000ff460 0x7f6300197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:27.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f630505d700 1 -- 172.21.15.107:0/4237218960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63000feab0 msgr2=0x7f6300196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:27.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f630505d700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63000feab0 0x7f6300196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:27.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f630505d700 1 -- 172.21.15.107:0/4237218960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec0095f0 con 0x7f6300104ab0 2023-07-21T22:28:27.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f62fffff700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63000ff460 0x7f6300197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:27.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f630505d700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6300104ab0 0x7f63001a1ba0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f62f800b9e0 tx=0x7f62f800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:27.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f62fdffb700 1 -- 172.21.15.107:0/4237218960 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f800d8e0 con 0x7f6300104ab0 2023-07-21T22:28:27.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f62fdffb700 1 -- 172.21.15.107:0/4237218960 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62f8015420 con 0x7f6300104ab0 2023-07-21T22:28:27.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f62fdffb700 1 -- 172.21.15.107:0/4237218960 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8014630 con 0x7f6300104ab0 2023-07-21T22:28:27.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.201+0000 7f6306ac0700 1 -- 172.21.15.107:0/4237218960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63001a2530 con 0x7f6300104ab0 2023-07-21T22:28:27.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.202+0000 7f6306ac0700 1 -- 172.21.15.107:0/4237218960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63001a2a60 con 0x7f6300104ab0 2023-07-21T22:28:27.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.202+0000 7f6306ac0700 1 -- 172.21.15.107:0/4237218960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630004e210 con 0x7f6300104ab0 2023-07-21T22:28:27.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.203+0000 7f62fdffb700 1 -- 172.21.15.107:0/4237218960 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f62f8015590 con 0x7f6300104ab0 2023-07-21T22:28:27.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.204+0000 7f62fdffb700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62f00a11e0 0x7f62f00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:27.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.204+0000 7f630485c700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62f00a11e0 0x7f62f00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:27.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.204+0000 7f62fdffb700 1 -- 172.21.15.107:0/4237218960 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f62f8011070 con 0x7f6300104ab0 2023-07-21T22:28:27.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.204+0000 7f630485c700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62f00a11e0 0x7f62f00a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f63000ffbf0 tx=0x7f62f4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:27.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.208+0000 7f62fdffb700 1 -- 172.21.15.107:0/4237218960 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f62f804e770 con 0x7f6300104ab0 2023-07-21T22:28:27.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:27 smithi132 ceph-mon[97863]: pgmap v272: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-07-21T22:28:27.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.390+0000 7f6306ac0700 1 -- 172.21.15.107:0/4237218960 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6300106040 con 0x7f62f00a11e0 2023-07-21T22:28:27.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:27 smithi107 ceph-mon[96543]: pgmap v272: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-07-21T22:28:27.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.400+0000 7f62fdffb700 1 -- 172.21.15.107:0/4237218960 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340297 (secure 0 0 0) 0x7f6300106040 con 0x7f62f00a11e0 2023-07-21T22:28:27.449 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:27.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.404+0000 7f6306ac0700 1 -- 172.21.15.107:0/4237218960 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62f00a11e0 msgr2=0x7f62f00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:27.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.404+0000 7f6306ac0700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62f00a11e0 0x7f62f00a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f63000ffbf0 tx=0x7f62f4009190 comp rx=0 tx=0).stop 2023-07-21T22:28:27.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.404+0000 7f6306ac0700 1 -- 172.21.15.107:0/4237218960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6300104ab0 msgr2=0x7f63001a1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:27.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.404+0000 7f6306ac0700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6300104ab0 0x7f63001a1ba0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f62f800b9e0 tx=0x7f62f800bcf0 comp rx=0 tx=0).stop 2023-07-21T22:28:27.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.404+0000 7f6306ac0700 1 -- 172.21.15.107:0/4237218960 shutdown_connections 2023-07-21T22:28:27.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.404+0000 7f6306ac0700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6300104ab0 0x7f63001a1ba0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:27.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.404+0000 7f6306ac0700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63000feab0 0x7f6300196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:27.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.404+0000 7f6306ac0700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62f00a11e0 0x7f62f00a3690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:27.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.404+0000 7f6306ac0700 1 --2- 172.21.15.107:0/4237218960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63000ff460 0x7f6300197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:27.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.404+0000 7f6306ac0700 1 -- 172.21.15.107:0/4237218960 >> 172.21.15.107:0/4237218960 conn(0x7f6300075b70 msgr2=0x7f6300105930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:27.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.404+0000 7f6306ac0700 1 -- 172.21.15.107:0/4237218960 shutdown_connections 2023-07-21T22:28:27.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:27.404+0000 7f6306ac0700 1 -- 172.21.15.107:0/4237218960 wait complete. 2023-07-21T22:28:27.453 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2023-07-21T22:28:27.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:27 smithi190 ceph-mon[98334]: pgmap v272: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2023-07-21T22:28:27.847 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2023-07-21T22:28:25.799491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652284,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":512,"num_read_kb":369,"num_write":412,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335988,"kb_used_data":12564,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605836,"statfs":{"total":1151940427776,"available":1151596376064,"internally_reserved":0,"allocated":12865536,"data_stored":7008879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":53,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003116"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075449+0000","last_change":"2023-07-21T22:27:42.091397+0000","last_active":"2023-07-21T22:27:43.075449+0000","last_peered":"2023-07-21T22:27:43.075449+0000","last_clean":"2023-07-21T22:27:43.075449+0000","last_became_active":"2023-07-21T22:27:42.090836+0000","last_became_peered":"2023-07-21T22:27:42.090836+0000","last_unstale":"2023-07-21T22:27:43.075449+0000","last_undegraded":"2023-07-21T22:27:43.075449+0000","last_fullsized":"2023-07-21T22:27:43.075449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:32:29.476744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957881+0000","last_change":"2023-07-21T22:27:42.097003+0000","last_active":"2023-07-21T22:27:43.957881+0000","last_peered":"2023-07-21T22:27:43.957881+0000","last_clean":"2023-07-21T22:27:43.957881+0000","last_became_active":"2023-07-21T22:27:42.096652+0000","last_became_peered":"2023-07-21T22:27:42.096652+0000","last_unstale":"2023-07-21T22:27:43.957881+0000","last_undegraded":"2023-07-21T22:27:43.957881+0000","last_fullsized":"2023-07-21T22:27:43.957881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:46:04.474133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957863+0000","last_change":"2023-07-21T22:27:42.090090+0000","last_active":"2023-07-21T22:27:43.957863+0000","last_peered":"2023-07-21T22:27:43.957863+0000","last_clean":"2023-07-21T22:27:43.957863+0000","last_became_active":"2023-07-21T22:27:42.089802+0000","last_became_peered":"2023-07-21T22:27:42.089802+0000","last_unstale":"2023-07-21T22:27:43.957863+0000","last_undegraded":"2023-07-21T22:27:43.957863+0000","last_fullsized":"2023-07-21T22:27:43.957863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:48:12.562625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075565+0000","last_change":"2023-07-21T22:27:42.096291+0000","last_active":"2023-07-21T22:27:43.075565+0000","last_peered":"2023-07-21T22:27:43.075565+0000","last_clean":"2023-07-21T22:27:43.075565+0000","last_became_active":"2023-07-21T22:27:42.095982+0000","last_became_peered":"2023-07-21T22:27:42.095982+0000","last_unstale":"2023-07-21T22:27:43.075565+0000","last_undegraded":"2023-07-21T22:27:43.075565+0000","last_fullsized":"2023-07-21T22:27:43.075565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:26:52.394622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375575+0000","last_change":"2023-07-21T22:27:42.097495+0000","last_active":"2023-07-21T22:27:43.375575+0000","last_peered":"2023-07-21T22:27:43.375575+0000","last_clean":"2023-07-21T22:27:43.375575+0000","last_became_active":"2023-07-21T22:27:42.095112+0000","last_became_peered":"2023-07-21T22:27:42.095112+0000","last_unstale":"2023-07-21T22:27:43.375575+0000","last_undegraded":"2023-07-21T22:27:43.375575+0000","last_fullsized":"2023-07-21T22:27:43.375575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:10:05.357425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802495+0000","last_change":"2023-07-21T22:27:42.089880+0000","last_active":"2023-07-21T22:27:44.802495+0000","last_peered":"2023-07-21T22:27:44.802495+0000","last_clean":"2023-07-21T22:27:44.802495+0000","last_became_active":"2023-07-21T22:27:42.088614+0000","last_became_peered":"2023-07-21T22:27:42.088614+0000","last_unstale":"2023-07-21T22:27:44.802495+0000","last_undegraded":"2023-07-21T22:27:44.802495+0000","last_fullsized":"2023-07-21T22:27:44.802495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:11.392499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171618+0000","last_change":"2023-07-21T22:27:42.085457+0000","last_active":"2023-07-21T22:27:44.171618+0000","last_peered":"2023-07-21T22:27:44.171618+0000","last_clean":"2023-07-21T22:27:44.171618+0000","last_became_active":"2023-07-21T22:27:42.085146+0000","last_became_peered":"2023-07-21T22:27:42.085146+0000","last_unstale":"2023-07-21T22:27:44.171618+0000","last_undegraded":"2023-07-21T22:27:44.171618+0000","last_fullsized":"2023-07-21T22:27:44.171618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:33:12.389852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390533+0000","last_change":"2023-07-21T22:27:42.089414+0000","last_active":"2023-07-21T22:27:44.390533+0000","last_peered":"2023-07-21T22:27:44.390533+0000","last_clean":"2023-07-21T22:27:44.390533+0000","last_became_active":"2023-07-21T22:27:42.089099+0000","last_became_peered":"2023-07-21T22:27:42.089099+0000","last_unstale":"2023-07-21T22:27:44.390533+0000","last_undegraded":"2023-07-21T22:27:44.390533+0000","last_fullsized":"2023-07-21T22:27:44.390533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:22:36.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802605+0000","last_change":"2023-07-21T22:27:42.094223+0000","last_active":"2023-07-21T22:27:44.802605+0000","last_peered":"2023-07-21T22:27:44.802605+0000","last_clean":"2023-07-21T22:27:44.802605+0000","last_became_active":"2023-07-21T22:27:42.093875+0000","last_became_peered":"2023-07-21T22:27:42.093875+0000","last_unstale":"2023-07-21T22:27:44.802605+0000","last_undegraded":"2023-07-21T22:27:44.802605+0000","last_fullsized":"2023-07-21T22:27:44.802605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:42:27.835710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.803127+0000","last_change":"2023-07-21T22:27:42.090442+0000","last_active":"2023-07-21T22:27:44.803127+0000","last_peered":"2023-07-21T22:27:44.803127+0000","last_clean":"2023-07-21T22:27:44.803127+0000","last_became_active":"2023-07-21T22:27:42.090103+0000","last_became_peered":"2023-07-21T22:27:42.090103+0000","last_unstale":"2023-07-21T22:27:44.803127+0000","last_undegraded":"2023-07-21T22:27:44.803127+0000","last_fullsized":"2023-07-21T22:27:44.803127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:48:58.891154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171944+0000","last_change":"2023-07-21T22:27:42.098364+0000","last_active":"2023-07-21T22:27:44.171944+0000","last_peered":"2023-07-21T22:27:44.171944+0000","last_clean":"2023-07-21T22:27:44.171944+0000","last_became_active":"2023-07-21T22:27:42.097378+0000","last_became_peered":"2023-07-21T22:27:42.097378+0000","last_unstale":"2023-07-21T22:27:44.171944+0000","last_undegraded":"2023-07-21T22:27:44.171944+0000","last_fullsized":"2023-07-21T22:27:44.171944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:04:59.958741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075627+0000","last_change":"2023-07-21T22:27:42.103110+0000","last_active":"2023-07-21T22:27:43.075627+0000","last_peered":"2023-07-21T22:27:43.075627+0000","last_clean":"2023-07-21T22:27:43.075627+0000","last_became_active":"2023-07-21T22:27:42.102790+0000","last_became_peered":"2023-07-21T22:27:42.102790+0000","last_unstale":"2023-07-21T22:27:43.075627+0000","last_undegraded":"2023-07-21T22:27:43.075627+0000","last_fullsized":"2023-07-21T22:27:43.075627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:15:45.908285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074861+0000","last_change":"2023-07-21T22:27:42.085176+0000","last_active":"2023-07-21T22:27:43.074861+0000","last_peered":"2023-07-21T22:27:43.074861+0000","last_clean":"2023-07-21T22:27:43.074861+0000","last_became_active":"2023-07-21T22:27:42.084721+0000","last_became_peered":"2023-07-21T22:27:42.084721+0000","last_unstale":"2023-07-21T22:27:43.074861+0000","last_undegraded":"2023-07-21T22:27:43.074861+0000","last_fullsized":"2023-07-21T22:27:43.074861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:54:19.144955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957681+0000","last_change":"2023-07-21T22:27:42.084423+0000","last_active":"2023-07-21T22:27:43.957681+0000","last_peered":"2023-07-21T22:27:43.957681+0000","last_clean":"2023-07-21T22:27:43.957681+0000","last_became_active":"2023-07-21T22:27:42.084074+0000","last_became_peered":"2023-07-21T22:27:42.084074+0000","last_unstale":"2023-07-21T22:27:43.957681+0000","last_undegraded":"2023-07-21T22:27:43.957681+0000","last_fullsized":"2023-07-21T22:27:43.957681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:41:12.520325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957649+0000","last_change":"2023-07-21T22:27:42.091542+0000","last_active":"2023-07-21T22:27:43.957649+0000","last_peered":"2023-07-21T22:27:43.957649+0000","last_clean":"2023-07-21T22:27:43.957649+0000","last_became_active":"2023-07-21T22:27:42.091217+0000","last_became_peered":"2023-07-21T22:27:42.091217+0000","last_unstale":"2023-07-21T22:27:43.957649+0000","last_undegraded":"2023-07-21T22:27:43.957649+0000","last_fullsized":"2023-07-21T22:27:43.957649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:45:58.131728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802320+0000","last_change":"2023-07-21T22:27:42.087505+0000","last_active":"2023-07-21T22:27:44.802320+0000","last_peered":"2023-07-21T22:27:44.802320+0000","last_clean":"2023-07-21T22:27:44.802320+0000","last_became_active":"2023-07-21T22:27:42.087177+0000","last_became_peered":"2023-07-21T22:27:42.087177+0000","last_unstale":"2023-07-21T22:27:44.802320+0000","last_undegraded":"2023-07-21T22:27:44.802320+0000","last_fullsized":"2023-07-21T22:27:44.802320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:58:06.502960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075614+0000","last_change":"2023-07-21T22:27:42.091490+0000","last_active":"2023-07-21T22:27:43.075614+0000","last_peered":"2023-07-21T22:27:43.075614+0000","last_clean":"2023-07-21T22:27:43.075614+0000","last_became_active":"2023-07-21T22:27:42.090782+0000","last_became_peered":"2023-07-21T22:27:42.090782+0000","last_unstale":"2023-07-21T22:27:43.075614+0000","last_undegraded":"2023-07-21T22:27:43.075614+0000","last_fullsized":"2023-07-21T22:27:43.075614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:59:46.774654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.303235+0000","last_change":"2023-07-21T22:27:42.092362+0000","last_active":"2023-07-21T22:27:43.303235+0000","last_peered":"2023-07-21T22:27:43.303235+0000","last_clean":"2023-07-21T22:27:43.303235+0000","last_became_active":"2023-07-21T22:27:42.091915+0000","last_became_peered":"2023-07-21T22:27:42.091915+0000","last_unstale":"2023-07-21T22:27:43.303235+0000","last_undegraded":"2023-07-21T22:27:43.303235+0000","last_fullsized":"2023-07-21T22:27:43.303235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:24:03.180394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701430+0000","last_change":"2023-07-21T22:27:42.098959+0000","last_active":"2023-07-21T22:27:43.701430+0000","last_peered":"2023-07-21T22:27:43.701430+0000","last_clean":"2023-07-21T22:27:43.701430+0000","last_became_active":"2023-07-21T22:27:42.094463+0000","last_became_peered":"2023-07-21T22:27:42.094463+0000","last_unstale":"2023-07-21T22:27:43.701430+0000","last_undegraded":"2023-07-21T22:27:43.701430+0000","last_fullsized":"2023-07-21T22:27:43.701430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:51:18.646837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:23.089907+0000","last_change":"2023-07-21T22:27:41.074299+0000","last_active":"2023-07-21T22:28:23.089907+0000","last_peered":"2023-07-21T22:28:23.089907+0000","last_clean":"2023-07-21T22:28:23.089907+0000","last_became_active":"2023-07-21T22:27:41.072895+0000","last_became_peered":"2023-07-21T22:27:41.072895+0000","last_unstale":"2023-07-21T22:28:23.089907+0000","last_undegraded":"2023-07-21T22:28:23.089907+0000","last_fullsized":"2023-07-21T22:28:23.089907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:56:42.897768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.391035+0000","last_change":"2023-07-21T22:27:42.090477+0000","last_active":"2023-07-21T22:27:44.391035+0000","last_peered":"2023-07-21T22:27:44.391035+0000","last_clean":"2023-07-21T22:27:44.391035+0000","last_became_active":"2023-07-21T22:27:42.090144+0000","last_became_peered":"2023-07-21T22:27:42.090144+0000","last_unstale":"2023-07-21T22:27:44.391035+0000","last_undegraded":"2023-07-21T22:27:44.391035+0000","last_fullsized":"2023-07-21T22:27:44.391035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:33:32.362360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.391141+0000","last_change":"2023-07-21T22:27:41.075178+0000","last_active":"2023-07-21T22:27:44.391141+0000","last_peered":"2023-07-21T22:27:44.391141+0000","last_clean":"2023-07-21T22:27:44.391141+0000","last_became_active":"2023-07-21T22:27:41.074861+0000","last_became_peered":"2023-07-21T22:27:41.074861+0000","last_unstale":"2023-07-21T22:27:44.391141+0000","last_undegraded":"2023-07-21T22:27:44.391141+0000","last_fullsized":"2023-07-21T22:27:44.391141+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:24:04.625686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.958039+0000","last_change":"2023-07-21T22:27:42.093135+0000","last_active":"2023-07-21T22:27:43.958039+0000","last_peered":"2023-07-21T22:27:43.958039+0000","last_clean":"2023-07-21T22:27:43.958039+0000","last_became_active":"2023-07-21T22:27:42.092457+0000","last_became_peered":"2023-07-21T22:27:42.092457+0000","last_unstale":"2023-07-21T22:27:43.958039+0000","last_undegraded":"2023-07-21T22:27:43.958039+0000","last_fullsized":"2023-07-21T22:27:43.958039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:26:26.440753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:49.040802+0000","last_change":"2023-07-21T22:27:41.076759+0000","last_active":"2023-07-21T22:27:49.040802+0000","last_peered":"2023-07-21T22:27:49.040802+0000","last_clean":"2023-07-21T22:27:49.040802+0000","last_became_active":"2023-07-21T22:27:41.076376+0000","last_became_peered":"2023-07-21T22:27:41.076376+0000","last_unstale":"2023-07-21T22:27:49.040802+0000","last_undegraded":"2023-07-21T22:27:49.040802+0000","last_fullsized":"2023-07-21T22:27:49.040802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:39:40.849097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802286+0000","last_change":"2023-07-21T22:27:42.093643+0000","last_active":"2023-07-21T22:27:44.802286+0000","last_peered":"2023-07-21T22:27:44.802286+0000","last_clean":"2023-07-21T22:27:44.802286+0000","last_became_active":"2023-07-21T22:27:42.093227+0000","last_became_peered":"2023-07-21T22:27:42.093227+0000","last_unstale":"2023-07-21T22:27:44.802286+0000","last_undegraded":"2023-07-21T22:27:44.802286+0000","last_fullsized":"2023-07-21T22:27:44.802286+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:13:02.515776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.043088+0000","last_change":"2023-07-21T22:27:41.073119+0000","last_active":"2023-07-21T22:27:48.043088+0000","last_peered":"2023-07-21T22:27:48.043088+0000","last_clean":"2023-07-21T22:27:48.043088+0000","last_became_active":"2023-07-21T22:27:41.072813+0000","last_became_peered":"2023-07-21T22:27:41.072813+0000","last_unstale":"2023-07-21T22:27:48.043088+0000","last_undegraded":"2023-07-21T22:27:48.043088+0000","last_fullsized":"2023-07-21T22:27:48.043088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:53:43.622644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176697+0000","last_change":"2023-07-21T22:27:42.096474+0000","last_active":"2023-07-21T22:27:44.176697+0000","last_peered":"2023-07-21T22:27:44.176697+0000","last_clean":"2023-07-21T22:27:44.176697+0000","last_became_active":"2023-07-21T22:27:42.096187+0000","last_became_peered":"2023-07-21T22:27:42.096187+0000","last_unstale":"2023-07-21T22:27:44.176697+0000","last_undegraded":"2023-07-21T22:27:44.176697+0000","last_fullsized":"2023-07-21T22:27:44.176697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:47:09.211204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:23.069146+0000","last_change":"2023-07-21T22:27:41.074125+0000","last_active":"2023-07-21T22:28:23.069146+0000","last_peered":"2023-07-21T22:28:23.069146+0000","last_clean":"2023-07-21T22:28:23.069146+0000","last_became_active":"2023-07-21T22:27:41.073829+0000","last_became_peered":"2023-07-21T22:27:41.073829+0000","last_unstale":"2023-07-21T22:28:23.069146+0000","last_undegraded":"2023-07-21T22:28:23.069146+0000","last_fullsized":"2023-07-21T22:28:23.069146+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:53:14.194907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375574+0000","last_change":"2023-07-21T22:27:42.082970+0000","last_active":"2023-07-21T22:27:43.375574+0000","last_peered":"2023-07-21T22:27:43.375574+0000","last_clean":"2023-07-21T22:27:43.375574+0000","last_became_active":"2023-07-21T22:27:42.082676+0000","last_became_peered":"2023-07-21T22:27:42.082676+0000","last_unstale":"2023-07-21T22:27:43.375574+0000","last_undegraded":"2023-07-21T22:27:43.375574+0000","last_fullsized":"2023-07-21T22:27:43.375574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:47.852129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:47.031735+0000","last_change":"2023-07-21T22:27:41.076844+0000","last_active":"2023-07-21T22:27:47.031735+0000","last_peered":"2023-07-21T22:27:47.031735+0000","last_clean":"2023-07-21T22:27:47.031735+0000","last_became_active":"2023-07-21T22:27:41.076487+0000","last_became_peered":"2023-07-21T22:27:41.076487+0000","last_unstale":"2023-07-21T22:27:47.031735+0000","last_undegraded":"2023-07-21T22:27:47.031735+0000","last_fullsized":"2023-07-21T22:27:47.031735+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:22:04.537934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176398+0000","last_change":"2023-07-21T22:27:42.089996+0000","last_active":"2023-07-21T22:27:44.176398+0000","last_peered":"2023-07-21T22:27:44.176398+0000","last_clean":"2023-07-21T22:27:44.176398+0000","last_became_active":"2023-07-21T22:27:42.089668+0000","last_became_peered":"2023-07-21T22:27:42.089668+0000","last_unstale":"2023-07-21T22:27:44.176398+0000","last_undegraded":"2023-07-21T22:27:44.176398+0000","last_fullsized":"2023-07-21T22:27:44.176398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:31:09.221434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.024979+0000","last_change":"2023-07-21T22:27:41.075864+0000","last_active":"2023-07-21T22:27:46.024979+0000","last_peered":"2023-07-21T22:27:46.024979+0000","last_clean":"2023-07-21T22:27:46.024979+0000","last_became_active":"2023-07-21T22:27:41.075577+0000","last_became_peered":"2023-07-21T22:27:41.075577+0000","last_unstale":"2023-07-21T22:27:46.024979+0000","last_undegraded":"2023-07-21T22:27:46.024979+0000","last_fullsized":"2023-07-21T22:27:46.024979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:54.831786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390617+0000","last_change":"2023-07-21T22:27:42.091730+0000","last_active":"2023-07-21T22:27:44.390617+0000","last_peered":"2023-07-21T22:27:44.390617+0000","last_clean":"2023-07-21T22:27:44.390617+0000","last_became_active":"2023-07-21T22:27:42.091394+0000","last_became_peered":"2023-07-21T22:27:42.091394+0000","last_unstale":"2023-07-21T22:27:44.390617+0000","last_undegraded":"2023-07-21T22:27:44.390617+0000","last_fullsized":"2023-07-21T22:27:44.390617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:17:17.092150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.041728+0000","last_change":"2023-07-21T22:27:41.074822+0000","last_active":"2023-07-21T22:27:48.041728+0000","last_peered":"2023-07-21T22:27:48.041728+0000","last_clean":"2023-07-21T22:27:48.041728+0000","last_became_active":"2023-07-21T22:27:41.074542+0000","last_became_peered":"2023-07-21T22:27:41.074542+0000","last_unstale":"2023-07-21T22:27:48.041728+0000","last_undegraded":"2023-07-21T22:27:48.041728+0000","last_fullsized":"2023-07-21T22:27:48.041728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:44:23.601406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.302989+0000","last_change":"2023-07-21T22:27:42.094351+0000","last_active":"2023-07-21T22:27:43.302989+0000","last_peered":"2023-07-21T22:27:43.302989+0000","last_clean":"2023-07-21T22:27:43.302989+0000","last_became_active":"2023-07-21T22:27:42.094068+0000","last_became_peered":"2023-07-21T22:27:42.094068+0000","last_unstale":"2023-07-21T22:27:43.302989+0000","last_undegraded":"2023-07-21T22:27:43.302989+0000","last_fullsized":"2023-07-21T22:27:43.302989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:27:50.111150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":87,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:21.065807+0000","last_change":"2023-07-21T22:27:41.079188+0000","last_active":"2023-07-21T22:28:21.065807+0000","last_peered":"2023-07-21T22:28:21.065807+0000","last_clean":"2023-07-21T22:28:21.065807+0000","last_became_active":"2023-07-21T22:27:41.078899+0000","last_became_peered":"2023-07-21T22:27:41.078899+0000","last_unstale":"2023-07-21T22:28:21.065807+0000","last_undegraded":"2023-07-21T22:28:21.065807+0000","last_fullsized":"2023-07-21T22:28:21.065807+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:00:37.074692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802626+0000","last_change":"2023-07-21T22:27:42.091194+0000","last_active":"2023-07-21T22:27:44.802626+0000","last_peered":"2023-07-21T22:27:44.802626+0000","last_clean":"2023-07-21T22:27:44.802626+0000","last_became_active":"2023-07-21T22:27:42.090869+0000","last_became_peered":"2023-07-21T22:27:42.090869+0000","last_unstale":"2023-07-21T22:27:44.802626+0000","last_undegraded":"2023-07-21T22:27:44.802626+0000","last_fullsized":"2023-07-21T22:27:44.802626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:26:26.169905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.039753+0000","last_change":"2023-07-21T22:27:41.072701+0000","last_active":"2023-07-21T22:27:48.039753+0000","last_peered":"2023-07-21T22:27:48.039753+0000","last_clean":"2023-07-21T22:27:48.039753+0000","last_became_active":"2023-07-21T22:27:41.072358+0000","last_became_peered":"2023-07-21T22:27:41.072358+0000","last_unstale":"2023-07-21T22:27:48.039753+0000","last_undegraded":"2023-07-21T22:27:48.039753+0000","last_fullsized":"2023-07-21T22:27:48.039753+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:51:15.728117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957974+0000","last_change":"2023-07-21T22:27:42.090808+0000","last_active":"2023-07-21T22:27:43.957974+0000","last_peered":"2023-07-21T22:27:43.957974+0000","last_clean":"2023-07-21T22:27:43.957974+0000","last_became_active":"2023-07-21T22:27:42.090506+0000","last_became_peered":"2023-07-21T22:27:42.090506+0000","last_unstale":"2023-07-21T22:27:43.957974+0000","last_undegraded":"2023-07-21T22:27:43.957974+0000","last_fullsized":"2023-07-21T22:27:43.957974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:59:17.254273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.016315+0000","last_change":"2023-07-21T22:27:41.075643+0000","last_active":"2023-07-21T22:27:46.016315+0000","last_peered":"2023-07-21T22:27:46.016315+0000","last_clean":"2023-07-21T22:27:46.016315+0000","last_became_active":"2023-07-21T22:27:41.075336+0000","last_became_peered":"2023-07-21T22:27:41.075336+0000","last_unstale":"2023-07-21T22:27:46.016315+0000","last_undegraded":"2023-07-21T22:27:46.016315+0000","last_fullsized":"2023-07-21T22:27:46.016315+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:12:56.006860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802567+0000","last_change":"2023-07-21T22:27:42.096722+0000","last_active":"2023-07-21T22:27:44.802567+0000","last_peered":"2023-07-21T22:27:44.802567+0000","last_clean":"2023-07-21T22:27:44.802567+0000","last_became_active":"2023-07-21T22:27:42.096472+0000","last_became_peered":"2023-07-21T22:27:42.096472+0000","last_unstale":"2023-07-21T22:27:44.802567+0000","last_undegraded":"2023-07-21T22:27:44.802567+0000","last_fullsized":"2023-07-21T22:27:44.802567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:25:16.425955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'28","reported_seq":120,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:21.065613+0000","last_change":"2023-07-21T22:27:41.072297+0000","last_active":"2023-07-21T22:28:21.065613+0000","last_peered":"2023-07-21T22:28:21.065613+0000","last_clean":"2023-07-21T22:28:21.065613+0000","last_became_active":"2023-07-21T22:27:41.071971+0000","last_became_peered":"2023-07-21T22:27:41.071971+0000","last_unstale":"2023-07-21T22:28:21.065613+0000","last_undegraded":"2023-07-21T22:28:21.065613+0000","last_fullsized":"2023-07-21T22:28:21.065613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:39:54.361110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54660,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":78,"num_read_kb":92,"num_write":30,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375731+0000","last_change":"2023-07-21T22:27:42.097130+0000","last_active":"2023-07-21T22:27:43.375731+0000","last_peered":"2023-07-21T22:27:43.375731+0000","last_clean":"2023-07-21T22:27:43.375731+0000","last_became_active":"2023-07-21T22:27:42.096853+0000","last_became_peered":"2023-07-21T22:27:42.096853+0000","last_unstale":"2023-07-21T22:27:43.375731+0000","last_undegraded":"2023-07-21T22:27:43.375731+0000","last_fullsized":"2023-07-21T22:27:43.375731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:36:06.680575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.042746+0000","last_change":"2023-07-21T22:27:41.071755+0000","last_active":"2023-07-21T22:27:48.042746+0000","last_peered":"2023-07-21T22:27:48.042746+0000","last_clean":"2023-07-21T22:27:48.042746+0000","last_became_active":"2023-07-21T22:27:41.071245+0000","last_became_peered":"2023-07-21T22:27:41.071245+0000","last_unstale":"2023-07-21T22:27:48.042746+0000","last_undegraded":"2023-07-21T22:27:48.042746+0000","last_fullsized":"2023-07-21T22:27:48.042746+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:09:41.990748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074639+0000","last_change":"2023-07-21T22:27:42.097433+0000","last_active":"2023-07-21T22:27:43.074639+0000","last_peered":"2023-07-21T22:27:43.074639+0000","last_clean":"2023-07-21T22:27:43.074639+0000","last_became_active":"2023-07-21T22:27:42.097100+0000","last_became_peered":"2023-07-21T22:27:42.097100+0000","last_unstale":"2023-07-21T22:27:43.074639+0000","last_undegraded":"2023-07-21T22:27:43.074639+0000","last_fullsized":"2023-07-21T22:27:43.074639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:02:40.214198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.041819+0000","last_change":"2023-07-21T22:27:41.074019+0000","last_active":"2023-07-21T22:27:48.041819+0000","last_peered":"2023-07-21T22:27:48.041819+0000","last_clean":"2023-07-21T22:27:48.041819+0000","last_became_active":"2023-07-21T22:27:41.073699+0000","last_became_peered":"2023-07-21T22:27:41.073699+0000","last_unstale":"2023-07-21T22:27:48.041819+0000","last_undegraded":"2023-07-21T22:27:48.041819+0000","last_fullsized":"2023-07-21T22:27:48.041819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:58:24.226793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172010+0000","last_change":"2023-07-21T22:27:42.098010+0000","last_active":"2023-07-21T22:27:44.172010+0000","last_peered":"2023-07-21T22:27:44.172010+0000","last_clean":"2023-07-21T22:27:44.172010+0000","last_became_active":"2023-07-21T22:27:42.097227+0000","last_became_peered":"2023-07-21T22:27:42.097227+0000","last_unstale":"2023-07-21T22:27:44.172010+0000","last_undegraded":"2023-07-21T22:27:44.172010+0000","last_fullsized":"2023-07-21T22:27:44.172010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:49.592034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:23.095722+0000","last_change":"2023-07-21T22:27:41.072298+0000","last_active":"2023-07-21T22:28:23.095722+0000","last_peered":"2023-07-21T22:28:23.095722+0000","last_clean":"2023-07-21T22:28:23.095722+0000","last_became_active":"2023-07-21T22:27:41.071963+0000","last_became_peered":"2023-07-21T22:27:41.071963+0000","last_unstale":"2023-07-21T22:28:23.095722+0000","last_undegraded":"2023-07-21T22:28:23.095722+0000","last_fullsized":"2023-07-21T22:28:23.095722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:47:00.466196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171795+0000","last_change":"2023-07-21T22:27:42.091276+0000","last_active":"2023-07-21T22:27:44.171795+0000","last_peered":"2023-07-21T22:27:44.171795+0000","last_clean":"2023-07-21T22:27:44.171795+0000","last_became_active":"2023-07-21T22:27:42.090918+0000","last_became_peered":"2023-07-21T22:27:42.090918+0000","last_unstale":"2023-07-21T22:27:44.171795+0000","last_undegraded":"2023-07-21T22:27:44.171795+0000","last_fullsized":"2023-07-21T22:27:44.171795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:07:22.664750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.041117+0000","last_change":"2023-07-21T22:27:41.075633+0000","last_active":"2023-07-21T22:27:48.041117+0000","last_peered":"2023-07-21T22:27:48.041117+0000","last_clean":"2023-07-21T22:27:48.041117+0000","last_became_active":"2023-07-21T22:27:41.075331+0000","last_became_peered":"2023-07-21T22:27:41.075331+0000","last_unstale":"2023-07-21T22:27:48.041117+0000","last_undegraded":"2023-07-21T22:27:48.041117+0000","last_fullsized":"2023-07-21T22:27:48.041117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:28:14.600527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176511+0000","last_change":"2023-07-21T22:27:42.089934+0000","last_active":"2023-07-21T22:27:44.176511+0000","last_peered":"2023-07-21T22:27:44.176511+0000","last_clean":"2023-07-21T22:27:44.176511+0000","last_became_active":"2023-07-21T22:27:42.089652+0000","last_became_peered":"2023-07-21T22:27:42.089652+0000","last_unstale":"2023-07-21T22:27:44.176511+0000","last_undegraded":"2023-07-21T22:27:44.176511+0000","last_fullsized":"2023-07-21T22:27:44.176511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:10:40.658561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.013258+0000","last_change":"2023-07-21T22:27:41.070862+0000","last_active":"2023-07-21T22:27:46.013258+0000","last_peered":"2023-07-21T22:27:46.013258+0000","last_clean":"2023-07-21T22:27:46.013258+0000","last_became_active":"2023-07-21T22:27:41.070507+0000","last_became_peered":"2023-07-21T22:27:41.070507+0000","last_unstale":"2023-07-21T22:27:46.013258+0000","last_undegraded":"2023-07-21T22:27:46.013258+0000","last_fullsized":"2023-07-21T22:27:46.013258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:28:00.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075397+0000","last_change":"2023-07-21T22:27:42.092046+0000","last_active":"2023-07-21T22:27:43.075397+0000","last_peered":"2023-07-21T22:27:43.075397+0000","last_clean":"2023-07-21T22:27:43.075397+0000","last_became_active":"2023-07-21T22:27:42.091069+0000","last_became_peered":"2023-07-21T22:27:42.091069+0000","last_unstale":"2023-07-21T22:27:43.075397+0000","last_undegraded":"2023-07-21T22:27:43.075397+0000","last_fullsized":"2023-07-21T22:27:43.075397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:39:03.277929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.005888+0000","last_change":"2023-07-21T22:27:41.074847+0000","last_active":"2023-07-21T22:27:45.005888+0000","last_peered":"2023-07-21T22:27:45.005888+0000","last_clean":"2023-07-21T22:27:45.005888+0000","last_became_active":"2023-07-21T22:27:41.074530+0000","last_became_peered":"2023-07-21T22:27:41.074530+0000","last_unstale":"2023-07-21T22:27:45.005888+0000","last_undegraded":"2023-07-21T22:27:45.005888+0000","last_fullsized":"2023-07-21T22:27:45.005888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:54:10.420130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802991+0000","last_change":"2023-07-21T22:27:42.089285+0000","last_active":"2023-07-21T22:27:44.802991+0000","last_peered":"2023-07-21T22:27:44.802991+0000","last_clean":"2023-07-21T22:27:44.802991+0000","last_became_active":"2023-07-21T22:27:42.088970+0000","last_became_peered":"2023-07-21T22:27:42.088970+0000","last_unstale":"2023-07-21T22:27:44.802991+0000","last_undegraded":"2023-07-21T22:27:44.802991+0000","last_fullsized":"2023-07-21T22:27:44.802991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:45:46.101767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":52,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:19.085682+0000","last_change":"2023-07-21T22:27:41.077433+0000","last_active":"2023-07-21T22:28:19.085682+0000","last_peered":"2023-07-21T22:28:19.085682+0000","last_clean":"2023-07-21T22:28:19.085682+0000","last_became_active":"2023-07-21T22:27:41.077133+0000","last_became_peered":"2023-07-21T22:27:41.077133+0000","last_unstale":"2023-07-21T22:28:19.085682+0000","last_undegraded":"2023-07-21T22:28:19.085682+0000","last_fullsized":"2023-07-21T22:28:19.085682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:48:14.982483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802503+0000","last_change":"2023-07-21T22:27:42.089833+0000","last_active":"2023-07-21T22:27:44.802503+0000","last_peered":"2023-07-21T22:27:44.802503+0000","last_clean":"2023-07-21T22:27:44.802503+0000","last_became_active":"2023-07-21T22:27:42.089502+0000","last_became_peered":"2023-07-21T22:27:42.089502+0000","last_unstale":"2023-07-21T22:27:44.802503+0000","last_undegraded":"2023-07-21T22:27:44.802503+0000","last_fullsized":"2023-07-21T22:27:44.802503+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:48:49.350741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176463+0000","last_change":"2023-07-21T22:27:42.090033+0000","last_active":"2023-07-21T22:27:44.176463+0000","last_peered":"2023-07-21T22:27:44.176463+0000","last_clean":"2023-07-21T22:27:44.176463+0000","last_became_active":"2023-07-21T22:27:42.089708+0000","last_became_peered":"2023-07-21T22:27:42.089708+0000","last_unstale":"2023-07-21T22:27:44.176463+0000","last_undegraded":"2023-07-21T22:27:44.176463+0000","last_fullsized":"2023-07-21T22:27:44.176463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:52:53.948825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":52,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:19.088045+0000","last_change":"2023-07-21T22:27:41.076784+0000","last_active":"2023-07-21T22:28:19.088045+0000","last_peered":"2023-07-21T22:28:19.088045+0000","last_clean":"2023-07-21T22:28:19.088045+0000","last_became_active":"2023-07-21T22:27:41.076453+0000","last_became_peered":"2023-07-21T22:27:41.076453+0000","last_unstale":"2023-07-21T22:28:19.088045+0000","last_undegraded":"2023-07-21T22:28:19.088045+0000","last_fullsized":"2023-07-21T22:28:19.088045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:32:32.953367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.391049+0000","last_change":"2023-07-21T22:27:42.091286+0000","last_active":"2023-07-21T22:27:44.391049+0000","last_peered":"2023-07-21T22:27:44.391049+0000","last_clean":"2023-07-21T22:27:44.391049+0000","last_became_active":"2023-07-21T22:27:42.090986+0000","last_became_peered":"2023-07-21T22:27:42.090986+0000","last_unstale":"2023-07-21T22:27:44.391049+0000","last_undegraded":"2023-07-21T22:27:44.391049+0000","last_fullsized":"2023-07-21T22:27:44.391049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:07:53.301709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:49.040869+0000","last_change":"2023-07-21T22:27:41.074075+0000","last_active":"2023-07-21T22:27:49.040869+0000","last_peered":"2023-07-21T22:27:49.040869+0000","last_clean":"2023-07-21T22:27:49.040869+0000","last_became_active":"2023-07-21T22:27:41.073757+0000","last_became_peered":"2023-07-21T22:27:41.073757+0000","last_unstale":"2023-07-21T22:27:49.040869+0000","last_undegraded":"2023-07-21T22:27:49.040869+0000","last_fullsized":"2023-07-21T22:27:49.040869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:35:27.586578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171569+0000","last_change":"2023-07-21T22:26:43.405788+0000","last_active":"2023-07-21T22:27:44.171569+0000","last_peered":"2023-07-21T22:27:44.171569+0000","last_clean":"2023-07-21T22:27:44.171569+0000","last_became_active":"2023-07-21T22:26:43.405351+0000","last_became_peered":"2023-07-21T22:26:43.405351+0000","last_unstale":"2023-07-21T22:27:44.171569+0000","last_undegraded":"2023-07-21T22:27:44.171569+0000","last_fullsized":"2023-07-21T22:27:44.171569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_clean_scrub_stamp":"2023-07-21T22:23:52.032210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:54:33.446817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074509+0000","last_change":"2023-07-21T22:27:42.097967+0000","last_active":"2023-07-21T22:27:43.074509+0000","last_peered":"2023-07-21T22:27:43.074509+0000","last_clean":"2023-07-21T22:27:43.074509+0000","last_became_active":"2023-07-21T22:27:42.097708+0000","last_became_peered":"2023-07-21T22:27:42.097708+0000","last_unstale":"2023-07-21T22:27:43.074509+0000","last_undegraded":"2023-07-21T22:27:43.074509+0000","last_fullsized":"2023-07-21T22:27:43.074509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:13:15.609283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.026760+0000","last_change":"2023-07-21T22:27:41.077853+0000","last_active":"2023-07-21T22:27:46.026760+0000","last_peered":"2023-07-21T22:27:46.026760+0000","last_clean":"2023-07-21T22:27:46.026760+0000","last_became_active":"2023-07-21T22:27:41.077539+0000","last_became_peered":"2023-07-21T22:27:41.077539+0000","last_unstale":"2023-07-21T22:27:46.026760+0000","last_undegraded":"2023-07-21T22:27:46.026760+0000","last_fullsized":"2023-07-21T22:27:46.026760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:09:25.539370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:47.031009+0000","last_change":"2023-07-21T22:27:41.076529+0000","last_active":"2023-07-21T22:27:47.031009+0000","last_peered":"2023-07-21T22:27:47.031009+0000","last_clean":"2023-07-21T22:27:47.031009+0000","last_became_active":"2023-07-21T22:27:41.076208+0000","last_became_peered":"2023-07-21T22:27:41.076208+0000","last_unstale":"2023-07-21T22:27:47.031009+0000","last_undegraded":"2023-07-21T22:27:47.031009+0000","last_fullsized":"2023-07-21T22:27:47.031009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:54:46.984912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074958+0000","last_change":"2023-07-21T22:27:42.082587+0000","last_active":"2023-07-21T22:27:43.074958+0000","last_peered":"2023-07-21T22:27:43.074958+0000","last_clean":"2023-07-21T22:27:43.074958+0000","last_became_active":"2023-07-21T22:27:42.082212+0000","last_became_peered":"2023-07-21T22:27:42.082212+0000","last_unstale":"2023-07-21T22:27:43.074958+0000","last_undegraded":"2023-07-21T22:27:43.074958+0000","last_fullsized":"2023-07-21T22:27:43.074958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:38.746984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:50.968129+0000","last_change":"2023-07-21T22:27:41.073736+0000","last_active":"2023-07-21T22:27:50.968129+0000","last_peered":"2023-07-21T22:27:50.968129+0000","last_clean":"2023-07-21T22:27:50.968129+0000","last_became_active":"2023-07-21T22:27:41.073213+0000","last_became_peered":"2023-07-21T22:27:41.073213+0000","last_unstale":"2023-07-21T22:27:50.968129+0000","last_undegraded":"2023-07-21T22:27:50.968129+0000","last_fullsized":"2023-07-21T22:27:50.968129+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:47:44.847798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390871+0000","last_change":"2023-07-21T22:27:42.091313+0000","last_active":"2023-07-21T22:27:44.390871+0000","last_peered":"2023-07-21T22:27:44.390871+0000","last_clean":"2023-07-21T22:27:44.390871+0000","last_became_active":"2023-07-21T22:27:42.091012+0000","last_became_peered":"2023-07-21T22:27:42.091012+0000","last_unstale":"2023-07-21T22:27:44.390871+0000","last_undegraded":"2023-07-21T22:27:44.390871+0000","last_fullsized":"2023-07-21T22:27:44.390871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:27:43.360142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.041975+0000","last_change":"2023-07-21T22:27:41.074644+0000","last_active":"2023-07-21T22:27:48.041975+0000","last_peered":"2023-07-21T22:27:48.041975+0000","last_clean":"2023-07-21T22:27:48.041975+0000","last_became_active":"2023-07-21T22:27:41.074364+0000","last_became_peered":"2023-07-21T22:27:41.074364+0000","last_unstale":"2023-07-21T22:27:48.041975+0000","last_undegraded":"2023-07-21T22:27:48.041975+0000","last_fullsized":"2023-07-21T22:27:48.041975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:31:09.704935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957978+0000","last_change":"2023-07-21T22:27:42.089452+0000","last_active":"2023-07-21T22:27:43.957978+0000","last_peered":"2023-07-21T22:27:43.957978+0000","last_clean":"2023-07-21T22:27:43.957978+0000","last_became_active":"2023-07-21T22:27:42.088859+0000","last_became_peered":"2023-07-21T22:27:42.088859+0000","last_unstale":"2023-07-21T22:27:43.957978+0000","last_undegraded":"2023-07-21T22:27:43.957978+0000","last_fullsized":"2023-07-21T22:27:43.957978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:17:15.973380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.120703+0000","last_change":"2023-07-21T22:27:41.077067+0000","last_active":"2023-07-21T22:27:45.120703+0000","last_peered":"2023-07-21T22:27:45.120703+0000","last_clean":"2023-07-21T22:27:45.120703+0000","last_became_active":"2023-07-21T22:27:41.076768+0000","last_became_peered":"2023-07-21T22:27:41.076768+0000","last_unstale":"2023-07-21T22:27:45.120703+0000","last_undegraded":"2023-07-21T22:27:45.120703+0000","last_fullsized":"2023-07-21T22:27:45.120703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:28:19.457823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375465+0000","last_change":"2023-07-21T22:27:42.088789+0000","last_active":"2023-07-21T22:27:43.375465+0000","last_peered":"2023-07-21T22:27:43.375465+0000","last_clean":"2023-07-21T22:27:43.375465+0000","last_became_active":"2023-07-21T22:27:42.088361+0000","last_became_peered":"2023-07-21T22:27:42.088361+0000","last_unstale":"2023-07-21T22:27:43.375465+0000","last_undegraded":"2023-07-21T22:27:43.375465+0000","last_fullsized":"2023-07-21T22:27:43.375465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:55:58.658869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.039700+0000","last_change":"2023-07-21T22:27:41.069595+0000","last_active":"2023-07-21T22:27:48.039700+0000","last_peered":"2023-07-21T22:27:48.039700+0000","last_clean":"2023-07-21T22:27:48.039700+0000","last_became_active":"2023-07-21T22:27:41.068733+0000","last_became_peered":"2023-07-21T22:27:41.068733+0000","last_unstale":"2023-07-21T22:27:48.039700+0000","last_undegraded":"2023-07-21T22:27:48.039700+0000","last_fullsized":"2023-07-21T22:27:48.039700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:45:30.220542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075228+0000","last_change":"2023-07-21T22:27:42.085701+0000","last_active":"2023-07-21T22:27:43.075228+0000","last_peered":"2023-07-21T22:27:43.075228+0000","last_clean":"2023-07-21T22:27:43.075228+0000","last_became_active":"2023-07-21T22:27:42.085429+0000","last_became_peered":"2023-07-21T22:27:42.085429+0000","last_unstale":"2023-07-21T22:27:43.075228+0000","last_undegraded":"2023-07-21T22:27:43.075228+0000","last_fullsized":"2023-07-21T22:27:43.075228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:48:06.310953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:49.232189+0000","last_change":"2023-07-21T22:27:41.073233+0000","last_active":"2023-07-21T22:27:49.232189+0000","last_peered":"2023-07-21T22:27:49.232189+0000","last_clean":"2023-07-21T22:27:49.232189+0000","last_became_active":"2023-07-21T22:27:41.072866+0000","last_became_peered":"2023-07-21T22:27:41.072866+0000","last_unstale":"2023-07-21T22:27:49.232189+0000","last_undegraded":"2023-07-21T22:27:49.232189+0000","last_fullsized":"2023-07-21T22:27:49.232189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:24:22.198579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390951+0000","last_change":"2023-07-21T22:27:42.100093+0000","last_active":"2023-07-21T22:27:44.390951+0000","last_peered":"2023-07-21T22:27:44.390951+0000","last_clean":"2023-07-21T22:27:44.390951+0000","last_became_active":"2023-07-21T22:27:42.099813+0000","last_became_peered":"2023-07-21T22:27:42.099813+0000","last_unstale":"2023-07-21T22:27:44.390951+0000","last_undegraded":"2023-07-21T22:27:44.390951+0000","last_fullsized":"2023-07-21T22:27:44.390951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:37:26.901693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.121256+0000","last_change":"2023-07-21T22:27:41.075641+0000","last_active":"2023-07-21T22:27:45.121256+0000","last_peered":"2023-07-21T22:27:45.121256+0000","last_clean":"2023-07-21T22:27:45.121256+0000","last_became_active":"2023-07-21T22:27:41.075268+0000","last_became_peered":"2023-07-21T22:27:41.075268+0000","last_unstale":"2023-07-21T22:27:45.121256+0000","last_undegraded":"2023-07-21T22:27:45.121256+0000","last_fullsized":"2023-07-21T22:27:45.121256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:32.377902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172002+0000","last_change":"2023-07-21T22:27:42.097736+0000","last_active":"2023-07-21T22:27:44.172002+0000","last_peered":"2023-07-21T22:27:44.172002+0000","last_clean":"2023-07-21T22:27:44.172002+0000","last_became_active":"2023-07-21T22:27:42.097419+0000","last_became_peered":"2023-07-21T22:27:42.097419+0000","last_unstale":"2023-07-21T22:27:44.172002+0000","last_undegraded":"2023-07-21T22:27:44.172002+0000","last_fullsized":"2023-07-21T22:27:44.172002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:24:11.626848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":55,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:20.095787+0000","last_change":"2023-07-21T22:27:41.074922+0000","last_active":"2023-07-21T22:28:20.095787+0000","last_peered":"2023-07-21T22:28:20.095787+0000","last_clean":"2023-07-21T22:28:20.095787+0000","last_became_active":"2023-07-21T22:27:41.074610+0000","last_became_peered":"2023-07-21T22:27:41.074610+0000","last_unstale":"2023-07-21T22:28:20.095787+0000","last_undegraded":"2023-07-21T22:28:20.095787+0000","last_fullsized":"2023-07-21T22:28:20.095787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:52:46.091639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172068+0000","last_change":"2023-07-21T22:27:42.097312+0000","last_active":"2023-07-21T22:27:44.172068+0000","last_peered":"2023-07-21T22:27:44.172068+0000","last_clean":"2023-07-21T22:27:44.172068+0000","last_became_active":"2023-07-21T22:27:42.096835+0000","last_became_peered":"2023-07-21T22:27:42.096835+0000","last_unstale":"2023-07-21T22:27:44.172068+0000","last_undegraded":"2023-07-21T22:27:44.172068+0000","last_fullsized":"2023-07-21T22:27:44.172068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:46:51.718896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:47.031192+0000","last_change":"2023-07-21T22:27:41.071971+0000","last_active":"2023-07-21T22:27:47.031192+0000","last_peered":"2023-07-21T22:27:47.031192+0000","last_clean":"2023-07-21T22:27:47.031192+0000","last_became_active":"2023-07-21T22:27:41.071550+0000","last_became_peered":"2023-07-21T22:27:41.071550+0000","last_unstale":"2023-07-21T22:27:47.031192+0000","last_undegraded":"2023-07-21T22:27:47.031192+0000","last_fullsized":"2023-07-21T22:27:47.031192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:21:47.017891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.057107+0000","last_change":"2023-07-21T22:27:42.098039+0000","last_active":"2023-07-21T22:27:46.057107+0000","last_peered":"2023-07-21T22:27:46.057107+0000","last_clean":"2023-07-21T22:27:46.057107+0000","last_became_active":"2023-07-21T22:27:42.097737+0000","last_became_peered":"2023-07-21T22:27:42.097737+0000","last_unstale":"2023-07-21T22:27:46.057107+0000","last_undegraded":"2023-07-21T22:27:46.057107+0000","last_fullsized":"2023-07-21T22:27:46.057107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:36:20.727587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.039528+0000","last_change":"2023-07-21T22:27:41.076768+0000","last_active":"2023-07-21T22:27:48.039528+0000","last_peered":"2023-07-21T22:27:48.039528+0000","last_clean":"2023-07-21T22:27:48.039528+0000","last_became_active":"2023-07-21T22:27:41.076451+0000","last_became_peered":"2023-07-21T22:27:41.076451+0000","last_unstale":"2023-07-21T22:27:48.039528+0000","last_undegraded":"2023-07-21T22:27:48.039528+0000","last_fullsized":"2023-07-21T22:27:48.039528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:27:42.924064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172018+0000","last_change":"2023-07-21T22:27:42.097891+0000","last_active":"2023-07-21T22:27:44.172018+0000","last_peered":"2023-07-21T22:27:44.172018+0000","last_clean":"2023-07-21T22:27:44.172018+0000","last_became_active":"2023-07-21T22:27:42.097022+0000","last_became_peered":"2023-07-21T22:27:42.097022+0000","last_unstale":"2023-07-21T22:27:44.172018+0000","last_undegraded":"2023-07-21T22:27:44.172018+0000","last_fullsized":"2023-07-21T22:27:44.172018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:04:32.167698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075273+0000","last_change":"2023-07-21T22:27:42.085774+0000","last_active":"2023-07-21T22:27:43.075273+0000","last_peered":"2023-07-21T22:27:43.075273+0000","last_clean":"2023-07-21T22:27:43.075273+0000","last_became_active":"2023-07-21T22:27:42.085460+0000","last_became_peered":"2023-07-21T22:27:42.085460+0000","last_unstale":"2023-07-21T22:27:43.075273+0000","last_undegraded":"2023-07-21T22:27:43.075273+0000","last_fullsized":"2023-07-21T22:27:43.075273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:20.846694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701118+0000","last_change":"2023-07-21T22:27:42.091630+0000","last_active":"2023-07-21T22:27:43.701118+0000","last_peered":"2023-07-21T22:27:43.701118+0000","last_clean":"2023-07-21T22:27:43.701118+0000","last_became_active":"2023-07-21T22:27:42.090538+0000","last_became_peered":"2023-07-21T22:27:42.090538+0000","last_unstale":"2023-07-21T22:27:43.701118+0000","last_undegraded":"2023-07-21T22:27:43.701118+0000","last_fullsized":"2023-07-21T22:27:43.701118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:05:25.473040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701072+0000","last_change":"2023-07-21T22:27:42.092918+0000","last_active":"2023-07-21T22:27:43.701072+0000","last_peered":"2023-07-21T22:27:43.701072+0000","last_clean":"2023-07-21T22:27:43.701072+0000","last_became_active":"2023-07-21T22:27:42.092484+0000","last_became_peered":"2023-07-21T22:27:42.092484+0000","last_unstale":"2023-07-21T22:27:43.701072+0000","last_undegraded":"2023-07-21T22:27:43.701072+0000","last_fullsized":"2023-07-21T22:27:43.701072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:40:43.563394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.958024+0000","last_change":"2023-07-21T22:27:42.096306+0000","last_active":"2023-07-21T22:27:43.958024+0000","last_peered":"2023-07-21T22:27:43.958024+0000","last_clean":"2023-07-21T22:27:43.958024+0000","last_became_active":"2023-07-21T22:27:42.096010+0000","last_became_peered":"2023-07-21T22:27:42.096010+0000","last_unstale":"2023-07-21T22:27:43.958024+0000","last_undegraded":"2023-07-21T22:27:43.958024+0000","last_fullsized":"2023-07-21T22:27:43.958024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:36:42.362478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074992+0000","last_change":"2023-07-21T22:27:42.097964+0000","last_active":"2023-07-21T22:27:43.074992+0000","last_peered":"2023-07-21T22:27:43.074992+0000","last_clean":"2023-07-21T22:27:43.074992+0000","last_became_active":"2023-07-21T22:27:42.097666+0000","last_became_peered":"2023-07-21T22:27:42.097666+0000","last_unstale":"2023-07-21T22:27:43.074992+0000","last_undegraded":"2023-07-21T22:27:43.074992+0000","last_fullsized":"2023-07-21T22:27:43.074992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:42:54.073891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075488+0000","last_change":"2023-07-21T22:27:42.083225+0000","last_active":"2023-07-21T22:27:43.075488+0000","last_peered":"2023-07-21T22:27:43.075488+0000","last_clean":"2023-07-21T22:27:43.075488+0000","last_became_active":"2023-07-21T22:27:42.082892+0000","last_became_peered":"2023-07-21T22:27:42.082892+0000","last_unstale":"2023-07-21T22:27:43.075488+0000","last_undegraded":"2023-07-21T22:27:43.075488+0000","last_fullsized":"2023-07-21T22:27:43.075488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:17:08.372691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701648+0000","last_change":"2023-07-21T22:27:42.090854+0000","last_active":"2023-07-21T22:27:43.701648+0000","last_peered":"2023-07-21T22:27:43.701648+0000","last_clean":"2023-07-21T22:27:43.701648+0000","last_became_active":"2023-07-21T22:27:42.090501+0000","last_became_peered":"2023-07-21T22:27:42.090501+0000","last_unstale":"2023-07-21T22:27:43.701648+0000","last_undegraded":"2023-07-21T22:27:43.701648+0000","last_fullsized":"2023-07-21T22:27:43.701648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:52:51.633954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390858+0000","last_change":"2023-07-21T22:27:42.094986+0000","last_active":"2023-07-21T22:27:44.390858+0000","last_peered":"2023-07-21T22:27:44.390858+0000","last_clean":"2023-07-21T22:27:44.390858+0000","last_became_active":"2023-07-21T22:27:42.094668+0000","last_became_peered":"2023-07-21T22:27:42.094668+0000","last_unstale":"2023-07-21T22:27:44.390858+0000","last_undegraded":"2023-07-21T22:27:44.390858+0000","last_fullsized":"2023-07-21T22:27:44.390858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:27:19.466686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176209+0000","last_change":"2023-07-21T22:27:42.098033+0000","last_active":"2023-07-21T22:27:44.176209+0000","last_peered":"2023-07-21T22:27:44.176209+0000","last_clean":"2023-07-21T22:27:44.176209+0000","last_became_active":"2023-07-21T22:27:42.097219+0000","last_became_peered":"2023-07-21T22:27:42.097219+0000","last_unstale":"2023-07-21T22:27:44.176209+0000","last_undegraded":"2023-07-21T22:27:44.176209+0000","last_fullsized":"2023-07-21T22:27:44.176209+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:12:40.233218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390721+0000","last_change":"2023-07-21T22:27:42.092136+0000","last_active":"2023-07-21T22:27:44.390721+0000","last_peered":"2023-07-21T22:27:44.390721+0000","last_clean":"2023-07-21T22:27:44.390721+0000","last_became_active":"2023-07-21T22:27:42.091559+0000","last_became_peered":"2023-07-21T22:27:42.091559+0000","last_unstale":"2023-07-21T22:27:44.390721+0000","last_undegraded":"2023-07-21T22:27:44.390721+0000","last_fullsized":"2023-07-21T22:27:44.390721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:30:28.654604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802891+0000","last_change":"2023-07-21T22:27:42.093227+0000","last_active":"2023-07-21T22:27:44.802891+0000","last_peered":"2023-07-21T22:27:44.802891+0000","last_clean":"2023-07-21T22:27:44.802891+0000","last_became_active":"2023-07-21T22:27:42.092918+0000","last_became_peered":"2023-07-21T22:27:42.092918+0000","last_unstale":"2023-07-21T22:27:44.802891+0000","last_undegraded":"2023-07-21T22:27:44.802891+0000","last_fullsized":"2023-07-21T22:27:44.802891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:21:39.475897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074784+0000","last_change":"2023-07-21T22:27:42.086494+0000","last_active":"2023-07-21T22:27:43.074784+0000","last_peered":"2023-07-21T22:27:43.074784+0000","last_clean":"2023-07-21T22:27:43.074784+0000","last_became_active":"2023-07-21T22:27:42.086189+0000","last_became_peered":"2023-07-21T22:27:42.086189+0000","last_unstale":"2023-07-21T22:27:43.074784+0000","last_undegraded":"2023-07-21T22:27:43.074784+0000","last_fullsized":"2023-07-21T22:27:43.074784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:24:31.736443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075473+0000","last_change":"2023-07-21T22:27:42.090874+0000","last_active":"2023-07-21T22:27:43.075473+0000","last_peered":"2023-07-21T22:27:43.075473+0000","last_clean":"2023-07-21T22:27:43.075473+0000","last_became_active":"2023-07-21T22:27:42.090597+0000","last_became_peered":"2023-07-21T22:27:42.090597+0000","last_unstale":"2023-07-21T22:27:43.075473+0000","last_undegraded":"2023-07-21T22:27:43.075473+0000","last_fullsized":"2023-07-21T22:27:43.075473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:10:24.103529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61670,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":386,"num_read_kb":260,"num_write":229,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":185010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481812,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":422619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.79400000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71199999999999997}]},{"osd":5,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.56799999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.747}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.89600000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.83299999999999996}]}]},{"osd":10,"up_from":70,"seq":300647710744,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1581056,"data_stored":1067593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":1,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61599999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.503}]},{"osd":5,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.85499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.879}]},{"osd":9,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.84399999999999997}]},{"osd":11,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.94099999999999995}]}]},{"osd":9,"up_from":65,"seq":279172874269,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":422207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.438,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.0379999999999998,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.36899999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.435}]},{"osd":2,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.129,"5min":3.129,"15min":3.129},"last":0.307},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.51500000000000001,"15min":0.53000000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.66500000000000004,"15min":0.68500000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":2.9710000000000001,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0469999999999999,"5min":1.089,"15min":1.089},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.66400000000000003,"15min":0.68600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93700000000000006,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.504}]},{"osd":8,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.65900000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.623},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.58099999999999996}]}]},{"osd":8,"up_from":57,"seq":244813135904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":422585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.746},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.60899999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.35999999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.45400000000000001,"15min":0.438},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.48099999999999998,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.495}]},{"osd":3,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60099999999999998,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.434,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61599999999999999,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63200000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69999999999999996,"15min":0.69699999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.997,"5min":1.097,"15min":1.097},"last":0.78400000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.55800000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.054,"5min":1.171,"15min":1.171},"last":0.872},{"interface":"front","average":{"1min":0.753,"5min":0.68999999999999995,"15min":0.68000000000000005},"min":{"1min":0.51000000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1830000000000001,"5min":1.232,"15min":1.232},"last":0.83299999999999996}]},{"osd":9,"last update":"Fri Jul 21 22:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.81299999999999994}]},{"osd":10,"last update":"Fri Jul 21 22:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.91900000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":7,"up_from":50,"seq":214748364836,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1507328,"data_stored":1014964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53800000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.433,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46200000000000002,"15min":0.46400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58699999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.40200000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.33900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.86299999999999999,"15min":0.90500000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81999999999999995,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":0.68400000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.81100000000000005,"15min":0.83899999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.6859999999999999,"5min":5.7480000000000002,"15min":5.7480000000000002},"last":0.434},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.83399999999999996,"15min":0.872},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85799999999999998,"5min":5.7649999999999997,"15min":5.7649999999999997},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.38900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.77200000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.621}]},{"osd":5,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67600000000000005,"15min":0.68300000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86599999999999999,"5min":0.873,"15min":0.873},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.55400000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.95299999999999996,"15min":0.998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.90200000000000002,"5min":5.7800000000000002,"15min":5.7800000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.78800000000000003,"15min":0.80000000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.875,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.74399999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.79100000000000004,"15min":0.80700000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85799999999999998,"5min":1.819,"15min":1.819},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.753,"15min":0.76100000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.88300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.53300000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.57599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528360,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":421882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.36499999999999999},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.32000000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.624,"15min":0.56299999999999994},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":6.899,"5min":6.899,"15min":6.899},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54000000000000004,"15min":0.52000000000000002},"min":{"1min":0.44800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.64500000000000002,"15min":0.57499999999999996},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":7.0359999999999996,"5min":7.0359999999999996,"15min":7.0359999999999996},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.496,"15min":0.437},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":7.1289999999999996,"5min":7.1289999999999996,"15min":7.1289999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":1.071,"5min":0.58399999999999996,"15min":0.48599999999999999},"min":{"1min":0.32000000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":12.414999999999999,"5min":12.414999999999999,"15min":12.414999999999999},"last":0.379}]},{"osd":4,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.64900000000000002,"15min":0.58699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.9690000000000003,"5min":6.9690000000000003,"15min":6.9690000000000003},"last":0.629},{"interface":"front","average":{"1min":0.629,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.48099999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.44700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.60199999999999998,"15min":0.58799999999999997},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.63500000000000001,"15min":0.61799999999999999},"min":{"1min":0.47099999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.77600000000000002}]},{"osd":8,"last update":"Fri Jul 21 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.747,"15min":0.69899999999999995},"min":{"1min":0.55000000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":7.0670000000000002,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70099999999999996,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.59999999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.496},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.67300000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.746}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":477449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.502,"15min":0.47999999999999998},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.50800000000000001,"15min":0.47999999999999998},"min":{"1min":0.47099999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.54200000000000004,"15min":0.50900000000000001},"min":{"1min":0.51100000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.79500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.54200000000000004,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76200000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.38300000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.872,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60199999999999998,"15min":0.58999999999999997},"min":{"1min":0.49299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.873}]},{"osd":4,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.436,"15min":0.41899999999999998},"min":{"1min":0.28599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44600000000000001,"15min":0.43099999999999999},"min":{"1min":0.31,"5min":0.248,"15min":0.248},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.495}]},{"osd":5,"last update":"Fri Jul 21 22:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.55800000000000005,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84399999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.75},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.59799999999999998,"15min":0.56200000000000006},"min":{"1min":0.55100000000000005,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84199999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.70899999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.54600000000000004,"15min":0.52600000000000002},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Jul 21 22:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.62}]},{"osd":8,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.69899999999999995,"15min":0.68600000000000005},"min":{"1min":0.44,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78700000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.94199999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":423605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.50700000000000001,"15min":0.48499999999999999},"min":{"1min":0.40899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.497},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81399999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79800000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.33900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67000000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39400000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.439},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.44,"15min":0.432},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.66900000000000004,"5min":0.753,"15min":0.753},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78500000000000003,"5min":0.879,"15min":0.879},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78500000000000003,"5min":1.002,"15min":1.002},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48299999999999998,"15min":0.49199999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58499999999999996,"5min":0.749,"15min":0.749},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79100000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.47199999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66500000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85499999999999998,"5min":1.04,"15min":1.04},"last":0.48499999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.68400000000000005,"15min":0.69099999999999995},"min":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72599999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.61899999999999999,"15min":0.63500000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88600000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.63},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.68200000000000005,"15min":0.70399999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89800000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.53900000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.68300000000000005}]},{"osd":11,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.753},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79000000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":478401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.48899999999999999,"15min":0.47299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.49099999999999999,"15min":0.46100000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.56299999999999994}]},{"osd":1,"last update":"Fri Jul 21 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.52800000000000002,"15min":0.48599999999999999},"min":{"1min":0.36099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.629},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.53700000000000003,"15min":0.51100000000000001},"min":{"1min":0.38300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57799999999999996,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.58199999999999996,"15min":0.54900000000000004},"min":{"1min":0.44600000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55800000000000005,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri Jul 21 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.46800000000000003,"15min":0.45800000000000002},"min":{"1min":0.29999999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.46899999999999997},"min":{"1min":0.33800000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.89300000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.435,"5min":0.432,"15min":0.432},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46899999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.88100000000000001}]},{"osd":9,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.51100000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.98699999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.97299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":421430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35599999999999998,"15min":0.33500000000000002},"min":{"1min":0.24399999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.629,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.26100000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.496,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.44800000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71499999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76000000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.59799999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.48299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.627}]},{"osd":5,"last update":"Fri Jul 21 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73899999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.629,"15min":0.624},"min":{"1min":0.46899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.50800000000000001,"15min":0.501},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66300000000000003,"15min":0.65800000000000003},"min":{"1min":0.498,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70699999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.64300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:27:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.68700000000000006}]},{"osd":10,"last update":"Fri Jul 21 22:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.74299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":421737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.47899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.49299999999999999},"min":{"1min":0.41799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30599999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.40500000000000003,"15min":0.38800000000000001},"min":{"1min":0.32600000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.59199999999999997,"15min":0.58299999999999996},"min":{"1min":0.38300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.46700000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53100000000000003,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59099999999999997}]},{"osd":5,"last update":"Fri Jul 21 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.82899999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.44700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.65900000000000003,"15min":0.60699999999999998},"min":{"1min":0.45300000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":5.6310000000000002,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.68899999999999995,"15min":0.629},"min":{"1min":0.53100000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.74399999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.61099999999999999,"15min":0.59299999999999997},"min":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.59199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.61099999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.51200000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67300000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":0.81299999999999994,"15min":0.755},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":7.6639999999999997,"5min":7.6639999999999997,"15min":7.6639999999999997},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.82999999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.68999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28432,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716720,"statfs":{"total":95995035648,"available":95965921280,"internally_reserved":0,"allocated":1515520,"data_stored":1014407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.33300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85099999999999998,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51500000000000001,"15min":0.504},"min":{"1min":0.35099999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70599999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.65900000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.54400000000000004,"15min":0.52200000000000002},"min":{"1min":0.48999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82099999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55900000000000005,"15min":0.53500000000000003},"min":{"1min":0.436,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.85799999999999998,"5min":1.036,"15min":1.036},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.40500000000000003,"15min":0.38400000000000001},"min":{"1min":0.29899999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.42099999999999999,"15min":0.39900000000000002},"min":{"1min":0.314,"5min":0.255,"15min":0.255},"max":{"1min":0.72599999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.371}]},{"osd":3,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.33000000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77800000000000002,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.54900000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.58999999999999997,"15min":0.56100000000000005},"min":{"1min":0.51500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91000000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.42599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0009999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.60099999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63600000000000001,"15min":0.61299999999999999},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89400000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.57199999999999995,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.63300000000000001}]},{"osd":8,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.64200000000000002,"15min":0.624},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.46400000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.72899999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.749},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:28:27.856 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-21T22:28:28.407 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:29.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:29 smithi132 ceph-mon[97863]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:28:29.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:29 smithi132 ceph-mon[97863]: pgmap v273: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-21T22:28:29.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:29 smithi107 ceph-mon[96543]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:28:29.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:29 smithi107 ceph-mon[96543]: pgmap v273: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-21T22:28:29.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:29 smithi190 ceph-mon[98334]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:28:29.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:29 smithi190 ceph-mon[98334]: pgmap v273: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-21T22:28:29.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.650+0000 7f88ed1c5700 1 -- 172.21.15.107:0/2843688699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88e81005d0 msgr2=0x7f88e8100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:29.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.650+0000 7f88ed1c5700 1 --2- 172.21.15.107:0/2843688699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88e81005d0 0x7f88e8100a40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0099c0 tx=0x7f88dc009cd0 comp rx=0 tx=0).stop 2023-07-21T22:28:29.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.651+0000 7f88ed1c5700 1 -- 172.21.15.107:0/2843688699 shutdown_connections 2023-07-21T22:28:29.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.651+0000 7f88ed1c5700 1 --2- 172.21.15.107:0/2843688699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e8100f80 0x7f88e810ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:29.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.651+0000 7f88ed1c5700 1 --2- 172.21.15.107:0/2843688699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e80fee80 0x7f88e80ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:29.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.651+0000 7f88ed1c5700 1 --2- 172.21.15.107:0/2843688699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88e81005d0 0x7f88e8100a40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.651+0000 7f88ed1c5700 1 -- 172.21.15.107:0/2843688699 >> 172.21.15.107:0/2843688699 conn(0x7f88e8075b60 msgr2=0x7f88e8075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.651+0000 7f88ed1c5700 1 -- 172.21.15.107:0/2843688699 shutdown_connections 2023-07-21T22:28:29.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.651+0000 7f88ed1c5700 1 -- 172.21.15.107:0/2843688699 wait complete. 2023-07-21T22:28:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.651+0000 7f88ed1c5700 1 Processor -- start 2023-07-21T22:28:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88ed1c5700 1 -- start start 2023-07-21T22:28:29.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88ed1c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e80fee80 0x7f88e8196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88ed1c5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88e81005d0 0x7f88e8197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88ed1c5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e8100f80 0x7f88e81a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88ed1c5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e81a2110 con 0x7f88e81005d0 2023-07-21T22:28:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88ed1c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e81a2250 con 0x7f88e80fee80 2023-07-21T22:28:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88ed1c5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e81a2390 con 0x7f88e8100f80 2023-07-21T22:28:29.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88e659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88e81005d0 0x7f88e8197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:29.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88e659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88e81005d0 0x7f88e8197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47066/0 (socket says 172.21.15.107:47066) 2023-07-21T22:28:29.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88e659c700 1 -- 172.21.15.107:0/1759437080 learned_addr learned my addr 172.21.15.107:0/1759437080 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:29.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88e759e700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e8100f80 0x7f88e81a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:29.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88e6d9d700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e80fee80 0x7f88e8196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:29.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88e659c700 1 -- 172.21.15.107:0/1759437080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e8100f80 msgr2=0x7f88e81a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:29.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88e659c700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e8100f80 0x7f88e81a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:29.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88e659c700 1 -- 172.21.15.107:0/1759437080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e80fee80 msgr2=0x7f88e8196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88e659c700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e80fee80 0x7f88e8196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88e659c700 1 -- 172.21.15.107:0/1759437080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88dc0095f0 con 0x7f88e81005d0 2023-07-21T22:28:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88e659c700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88e81005d0 0x7f88e8197090 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f88dc005910 tx=0x7f88dc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.652+0000 7f88cffff700 1 -- 172.21.15.107:0/1759437080 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc01d070 con 0x7f88e81005d0 2023-07-21T22:28:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.653+0000 7f88ed1c5700 1 -- 172.21.15.107:0/1759437080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e81a2590 con 0x7f88e81005d0 2023-07-21T22:28:29.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.653+0000 7f88cffff700 1 -- 172.21.15.107:0/1759437080 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88dc003f90 con 0x7f88e81005d0 2023-07-21T22:28:29.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.653+0000 7f88cffff700 1 -- 172.21.15.107:0/1759437080 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc022bb0 con 0x7f88e81005d0 2023-07-21T22:28:29.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.653+0000 7f88ed1c5700 1 -- 172.21.15.107:0/1759437080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e81a2a30 con 0x7f88e81005d0 2023-07-21T22:28:29.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.653+0000 7f88ed1c5700 1 -- 172.21.15.107:0/1759437080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e804e210 con 0x7f88e81005d0 2023-07-21T22:28:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.655+0000 7f88cffff700 1 -- 172.21.15.107:0/1759437080 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f88dc003ad0 con 0x7f88e81005d0 2023-07-21T22:28:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.655+0000 7f88cffff700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f88d00a11e0 0x7f88d00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.655+0000 7f88e6d9d700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f88d00a11e0 0x7f88d00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.655+0000 7f88cffff700 1 -- 172.21.15.107:0/1759437080 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f88dc0ce300 con 0x7f88e81005d0 2023-07-21T22:28:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.656+0000 7f88e6d9d700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f88d00a11e0 0x7f88d00a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f88d8004630 tx=0x7f88d800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:29.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.658+0000 7f88cffff700 1 -- 172.21.15.107:0/1759437080 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f88dc097b30 con 0x7f88e81005d0 2023-07-21T22:28:29.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.842+0000 7f88ed1c5700 1 -- 172.21.15.107:0/1759437080 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88e8101fa0 con 0x7f88d00a11e0 2023-07-21T22:28:29.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.851+0000 7f88cffff700 1 -- 172.21.15.107:0/1759437080 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340275 (secure 0 0 0) 0x7f88e8101fa0 con 0x7f88d00a11e0 2023-07-21T22:28:29.903 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.854+0000 7f88ed1c5700 1 -- 172.21.15.107:0/1759437080 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f88d00a11e0 msgr2=0x7f88d00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.854+0000 7f88ed1c5700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f88d00a11e0 0x7f88d00a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f88d8004630 tx=0x7f88d800b3c0 comp rx=0 tx=0).stop 2023-07-21T22:28:29.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.854+0000 7f88ed1c5700 1 -- 172.21.15.107:0/1759437080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88e81005d0 msgr2=0x7f88e8197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:29.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.854+0000 7f88ed1c5700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88e81005d0 0x7f88e8197090 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f88dc005910 tx=0x7f88dc009580 comp rx=0 tx=0).stop 2023-07-21T22:28:29.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.854+0000 7f88ed1c5700 1 -- 172.21.15.107:0/1759437080 shutdown_connections 2023-07-21T22:28:29.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.854+0000 7f88ed1c5700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f88d00a11e0 0x7f88d00a3690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:29.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.854+0000 7f88ed1c5700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88e8100f80 0x7f88e81a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.854+0000 7f88ed1c5700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88e80fee80 0x7f88e8196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.854+0000 7f88ed1c5700 1 --2- 172.21.15.107:0/1759437080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88e81005d0 0x7f88e8197090 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.854+0000 7f88ed1c5700 1 -- 172.21.15.107:0/1759437080 >> 172.21.15.107:0/1759437080 conn(0x7f88e8075b60 msgr2=0x7f88e8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.855+0000 7f88ed1c5700 1 -- 172.21.15.107:0/1759437080 shutdown_connections 2023-07-21T22:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:29.855+0000 7f88ed1c5700 1 -- 172.21.15.107:0/1759437080 wait complete. 2023-07-21T22:28:29.907 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2023-07-21T22:28:30.251 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2023-07-21T22:28:29.800451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652284,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":552,"num_read_kb":389,"num_write":412,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335988,"kb_used_data":12564,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605836,"statfs":{"total":1151940427776,"available":1151596376064,"internally_reserved":0,"allocated":12865536,"data_stored":7008879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003166"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075449+0000","last_change":"2023-07-21T22:27:42.091397+0000","last_active":"2023-07-21T22:27:43.075449+0000","last_peered":"2023-07-21T22:27:43.075449+0000","last_clean":"2023-07-21T22:27:43.075449+0000","last_became_active":"2023-07-21T22:27:42.090836+0000","last_became_peered":"2023-07-21T22:27:42.090836+0000","last_unstale":"2023-07-21T22:27:43.075449+0000","last_undegraded":"2023-07-21T22:27:43.075449+0000","last_fullsized":"2023-07-21T22:27:43.075449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:32:29.476744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957881+0000","last_change":"2023-07-21T22:27:42.097003+0000","last_active":"2023-07-21T22:27:43.957881+0000","last_peered":"2023-07-21T22:27:43.957881+0000","last_clean":"2023-07-21T22:27:43.957881+0000","last_became_active":"2023-07-21T22:27:42.096652+0000","last_became_peered":"2023-07-21T22:27:42.096652+0000","last_unstale":"2023-07-21T22:27:43.957881+0000","last_undegraded":"2023-07-21T22:27:43.957881+0000","last_fullsized":"2023-07-21T22:27:43.957881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:46:04.474133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957863+0000","last_change":"2023-07-21T22:27:42.090090+0000","last_active":"2023-07-21T22:27:43.957863+0000","last_peered":"2023-07-21T22:27:43.957863+0000","last_clean":"2023-07-21T22:27:43.957863+0000","last_became_active":"2023-07-21T22:27:42.089802+0000","last_became_peered":"2023-07-21T22:27:42.089802+0000","last_unstale":"2023-07-21T22:27:43.957863+0000","last_undegraded":"2023-07-21T22:27:43.957863+0000","last_fullsized":"2023-07-21T22:27:43.957863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:48:12.562625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075565+0000","last_change":"2023-07-21T22:27:42.096291+0000","last_active":"2023-07-21T22:27:43.075565+0000","last_peered":"2023-07-21T22:27:43.075565+0000","last_clean":"2023-07-21T22:27:43.075565+0000","last_became_active":"2023-07-21T22:27:42.095982+0000","last_became_peered":"2023-07-21T22:27:42.095982+0000","last_unstale":"2023-07-21T22:27:43.075565+0000","last_undegraded":"2023-07-21T22:27:43.075565+0000","last_fullsized":"2023-07-21T22:27:43.075565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:26:52.394622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375575+0000","last_change":"2023-07-21T22:27:42.097495+0000","last_active":"2023-07-21T22:27:43.375575+0000","last_peered":"2023-07-21T22:27:43.375575+0000","last_clean":"2023-07-21T22:27:43.375575+0000","last_became_active":"2023-07-21T22:27:42.095112+0000","last_became_peered":"2023-07-21T22:27:42.095112+0000","last_unstale":"2023-07-21T22:27:43.375575+0000","last_undegraded":"2023-07-21T22:27:43.375575+0000","last_fullsized":"2023-07-21T22:27:43.375575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:10:05.357425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802495+0000","last_change":"2023-07-21T22:27:42.089880+0000","last_active":"2023-07-21T22:27:44.802495+0000","last_peered":"2023-07-21T22:27:44.802495+0000","last_clean":"2023-07-21T22:27:44.802495+0000","last_became_active":"2023-07-21T22:27:42.088614+0000","last_became_peered":"2023-07-21T22:27:42.088614+0000","last_unstale":"2023-07-21T22:27:44.802495+0000","last_undegraded":"2023-07-21T22:27:44.802495+0000","last_fullsized":"2023-07-21T22:27:44.802495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:11.392499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171618+0000","last_change":"2023-07-21T22:27:42.085457+0000","last_active":"2023-07-21T22:27:44.171618+0000","last_peered":"2023-07-21T22:27:44.171618+0000","last_clean":"2023-07-21T22:27:44.171618+0000","last_became_active":"2023-07-21T22:27:42.085146+0000","last_became_peered":"2023-07-21T22:27:42.085146+0000","last_unstale":"2023-07-21T22:27:44.171618+0000","last_undegraded":"2023-07-21T22:27:44.171618+0000","last_fullsized":"2023-07-21T22:27:44.171618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:33:12.389852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390533+0000","last_change":"2023-07-21T22:27:42.089414+0000","last_active":"2023-07-21T22:27:44.390533+0000","last_peered":"2023-07-21T22:27:44.390533+0000","last_clean":"2023-07-21T22:27:44.390533+0000","last_became_active":"2023-07-21T22:27:42.089099+0000","last_became_peered":"2023-07-21T22:27:42.089099+0000","last_unstale":"2023-07-21T22:27:44.390533+0000","last_undegraded":"2023-07-21T22:27:44.390533+0000","last_fullsized":"2023-07-21T22:27:44.390533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:22:36.632124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802605+0000","last_change":"2023-07-21T22:27:42.094223+0000","last_active":"2023-07-21T22:27:44.802605+0000","last_peered":"2023-07-21T22:27:44.802605+0000","last_clean":"2023-07-21T22:27:44.802605+0000","last_became_active":"2023-07-21T22:27:42.093875+0000","last_became_peered":"2023-07-21T22:27:42.093875+0000","last_unstale":"2023-07-21T22:27:44.802605+0000","last_undegraded":"2023-07-21T22:27:44.802605+0000","last_fullsized":"2023-07-21T22:27:44.802605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:42:27.835710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.803127+0000","last_change":"2023-07-21T22:27:42.090442+0000","last_active":"2023-07-21T22:27:44.803127+0000","last_peered":"2023-07-21T22:27:44.803127+0000","last_clean":"2023-07-21T22:27:44.803127+0000","last_became_active":"2023-07-21T22:27:42.090103+0000","last_became_peered":"2023-07-21T22:27:42.090103+0000","last_unstale":"2023-07-21T22:27:44.803127+0000","last_undegraded":"2023-07-21T22:27:44.803127+0000","last_fullsized":"2023-07-21T22:27:44.803127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:48:58.891154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171944+0000","last_change":"2023-07-21T22:27:42.098364+0000","last_active":"2023-07-21T22:27:44.171944+0000","last_peered":"2023-07-21T22:27:44.171944+0000","last_clean":"2023-07-21T22:27:44.171944+0000","last_became_active":"2023-07-21T22:27:42.097378+0000","last_became_peered":"2023-07-21T22:27:42.097378+0000","last_unstale":"2023-07-21T22:27:44.171944+0000","last_undegraded":"2023-07-21T22:27:44.171944+0000","last_fullsized":"2023-07-21T22:27:44.171944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:04:59.958741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075627+0000","last_change":"2023-07-21T22:27:42.103110+0000","last_active":"2023-07-21T22:27:43.075627+0000","last_peered":"2023-07-21T22:27:43.075627+0000","last_clean":"2023-07-21T22:27:43.075627+0000","last_became_active":"2023-07-21T22:27:42.102790+0000","last_became_peered":"2023-07-21T22:27:42.102790+0000","last_unstale":"2023-07-21T22:27:43.075627+0000","last_undegraded":"2023-07-21T22:27:43.075627+0000","last_fullsized":"2023-07-21T22:27:43.075627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:15:45.908285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074861+0000","last_change":"2023-07-21T22:27:42.085176+0000","last_active":"2023-07-21T22:27:43.074861+0000","last_peered":"2023-07-21T22:27:43.074861+0000","last_clean":"2023-07-21T22:27:43.074861+0000","last_became_active":"2023-07-21T22:27:42.084721+0000","last_became_peered":"2023-07-21T22:27:42.084721+0000","last_unstale":"2023-07-21T22:27:43.074861+0000","last_undegraded":"2023-07-21T22:27:43.074861+0000","last_fullsized":"2023-07-21T22:27:43.074861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:54:19.144955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957681+0000","last_change":"2023-07-21T22:27:42.084423+0000","last_active":"2023-07-21T22:27:43.957681+0000","last_peered":"2023-07-21T22:27:43.957681+0000","last_clean":"2023-07-21T22:27:43.957681+0000","last_became_active":"2023-07-21T22:27:42.084074+0000","last_became_peered":"2023-07-21T22:27:42.084074+0000","last_unstale":"2023-07-21T22:27:43.957681+0000","last_undegraded":"2023-07-21T22:27:43.957681+0000","last_fullsized":"2023-07-21T22:27:43.957681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:41:12.520325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957649+0000","last_change":"2023-07-21T22:27:42.091542+0000","last_active":"2023-07-21T22:27:43.957649+0000","last_peered":"2023-07-21T22:27:43.957649+0000","last_clean":"2023-07-21T22:27:43.957649+0000","last_became_active":"2023-07-21T22:27:42.091217+0000","last_became_peered":"2023-07-21T22:27:42.091217+0000","last_unstale":"2023-07-21T22:27:43.957649+0000","last_undegraded":"2023-07-21T22:27:43.957649+0000","last_fullsized":"2023-07-21T22:27:43.957649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:45:58.131728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802320+0000","last_change":"2023-07-21T22:27:42.087505+0000","last_active":"2023-07-21T22:27:44.802320+0000","last_peered":"2023-07-21T22:27:44.802320+0000","last_clean":"2023-07-21T22:27:44.802320+0000","last_became_active":"2023-07-21T22:27:42.087177+0000","last_became_peered":"2023-07-21T22:27:42.087177+0000","last_unstale":"2023-07-21T22:27:44.802320+0000","last_undegraded":"2023-07-21T22:27:44.802320+0000","last_fullsized":"2023-07-21T22:27:44.802320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:58:06.502960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075614+0000","last_change":"2023-07-21T22:27:42.091490+0000","last_active":"2023-07-21T22:27:43.075614+0000","last_peered":"2023-07-21T22:27:43.075614+0000","last_clean":"2023-07-21T22:27:43.075614+0000","last_became_active":"2023-07-21T22:27:42.090782+0000","last_became_peered":"2023-07-21T22:27:42.090782+0000","last_unstale":"2023-07-21T22:27:43.075614+0000","last_undegraded":"2023-07-21T22:27:43.075614+0000","last_fullsized":"2023-07-21T22:27:43.075614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:59:46.774654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.303235+0000","last_change":"2023-07-21T22:27:42.092362+0000","last_active":"2023-07-21T22:27:43.303235+0000","last_peered":"2023-07-21T22:27:43.303235+0000","last_clean":"2023-07-21T22:27:43.303235+0000","last_became_active":"2023-07-21T22:27:42.091915+0000","last_became_peered":"2023-07-21T22:27:42.091915+0000","last_unstale":"2023-07-21T22:27:43.303235+0000","last_undegraded":"2023-07-21T22:27:43.303235+0000","last_fullsized":"2023-07-21T22:27:43.303235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:24:03.180394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701430+0000","last_change":"2023-07-21T22:27:42.098959+0000","last_active":"2023-07-21T22:27:43.701430+0000","last_peered":"2023-07-21T22:27:43.701430+0000","last_clean":"2023-07-21T22:27:43.701430+0000","last_became_active":"2023-07-21T22:27:42.094463+0000","last_became_peered":"2023-07-21T22:27:42.094463+0000","last_unstale":"2023-07-21T22:27:43.701430+0000","last_undegraded":"2023-07-21T22:27:43.701430+0000","last_fullsized":"2023-07-21T22:27:43.701430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:51:18.646837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":64,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:23.089907+0000","last_change":"2023-07-21T22:27:41.074299+0000","last_active":"2023-07-21T22:28:23.089907+0000","last_peered":"2023-07-21T22:28:23.089907+0000","last_clean":"2023-07-21T22:28:23.089907+0000","last_became_active":"2023-07-21T22:27:41.072895+0000","last_became_peered":"2023-07-21T22:27:41.072895+0000","last_unstale":"2023-07-21T22:28:23.089907+0000","last_undegraded":"2023-07-21T22:28:23.089907+0000","last_fullsized":"2023-07-21T22:28:23.089907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:56:42.897768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.391035+0000","last_change":"2023-07-21T22:27:42.090477+0000","last_active":"2023-07-21T22:27:44.391035+0000","last_peered":"2023-07-21T22:27:44.391035+0000","last_clean":"2023-07-21T22:27:44.391035+0000","last_became_active":"2023-07-21T22:27:42.090144+0000","last_became_peered":"2023-07-21T22:27:42.090144+0000","last_unstale":"2023-07-21T22:27:44.391035+0000","last_undegraded":"2023-07-21T22:27:44.391035+0000","last_fullsized":"2023-07-21T22:27:44.391035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:33:32.362360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.391141+0000","last_change":"2023-07-21T22:27:41.075178+0000","last_active":"2023-07-21T22:27:44.391141+0000","last_peered":"2023-07-21T22:27:44.391141+0000","last_clean":"2023-07-21T22:27:44.391141+0000","last_became_active":"2023-07-21T22:27:41.074861+0000","last_became_peered":"2023-07-21T22:27:41.074861+0000","last_unstale":"2023-07-21T22:27:44.391141+0000","last_undegraded":"2023-07-21T22:27:44.391141+0000","last_fullsized":"2023-07-21T22:27:44.391141+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:24:04.625686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.958039+0000","last_change":"2023-07-21T22:27:42.093135+0000","last_active":"2023-07-21T22:27:43.958039+0000","last_peered":"2023-07-21T22:27:43.958039+0000","last_clean":"2023-07-21T22:27:43.958039+0000","last_became_active":"2023-07-21T22:27:42.092457+0000","last_became_peered":"2023-07-21T22:27:42.092457+0000","last_unstale":"2023-07-21T22:27:43.958039+0000","last_undegraded":"2023-07-21T22:27:43.958039+0000","last_fullsized":"2023-07-21T22:27:43.958039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:26:26.440753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:49.040802+0000","last_change":"2023-07-21T22:27:41.076759+0000","last_active":"2023-07-21T22:27:49.040802+0000","last_peered":"2023-07-21T22:27:49.040802+0000","last_clean":"2023-07-21T22:27:49.040802+0000","last_became_active":"2023-07-21T22:27:41.076376+0000","last_became_peered":"2023-07-21T22:27:41.076376+0000","last_unstale":"2023-07-21T22:27:49.040802+0000","last_undegraded":"2023-07-21T22:27:49.040802+0000","last_fullsized":"2023-07-21T22:27:49.040802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:39:40.849097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802286+0000","last_change":"2023-07-21T22:27:42.093643+0000","last_active":"2023-07-21T22:27:44.802286+0000","last_peered":"2023-07-21T22:27:44.802286+0000","last_clean":"2023-07-21T22:27:44.802286+0000","last_became_active":"2023-07-21T22:27:42.093227+0000","last_became_peered":"2023-07-21T22:27:42.093227+0000","last_unstale":"2023-07-21T22:27:44.802286+0000","last_undegraded":"2023-07-21T22:27:44.802286+0000","last_fullsized":"2023-07-21T22:27:44.802286+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:13:02.515776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.043088+0000","last_change":"2023-07-21T22:27:41.073119+0000","last_active":"2023-07-21T22:27:48.043088+0000","last_peered":"2023-07-21T22:27:48.043088+0000","last_clean":"2023-07-21T22:27:48.043088+0000","last_became_active":"2023-07-21T22:27:41.072813+0000","last_became_peered":"2023-07-21T22:27:41.072813+0000","last_unstale":"2023-07-21T22:27:48.043088+0000","last_undegraded":"2023-07-21T22:27:48.043088+0000","last_fullsized":"2023-07-21T22:27:48.043088+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:53:43.622644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176697+0000","last_change":"2023-07-21T22:27:42.096474+0000","last_active":"2023-07-21T22:27:44.176697+0000","last_peered":"2023-07-21T22:27:44.176697+0000","last_clean":"2023-07-21T22:27:44.176697+0000","last_became_active":"2023-07-21T22:27:42.096187+0000","last_became_peered":"2023-07-21T22:27:42.096187+0000","last_unstale":"2023-07-21T22:27:44.176697+0000","last_undegraded":"2023-07-21T22:27:44.176697+0000","last_fullsized":"2023-07-21T22:27:44.176697+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:47:09.211204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:23.069146+0000","last_change":"2023-07-21T22:27:41.074125+0000","last_active":"2023-07-21T22:28:23.069146+0000","last_peered":"2023-07-21T22:28:23.069146+0000","last_clean":"2023-07-21T22:28:23.069146+0000","last_became_active":"2023-07-21T22:27:41.073829+0000","last_became_peered":"2023-07-21T22:27:41.073829+0000","last_unstale":"2023-07-21T22:28:23.069146+0000","last_undegraded":"2023-07-21T22:28:23.069146+0000","last_fullsized":"2023-07-21T22:28:23.069146+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:53:14.194907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375574+0000","last_change":"2023-07-21T22:27:42.082970+0000","last_active":"2023-07-21T22:27:43.375574+0000","last_peered":"2023-07-21T22:27:43.375574+0000","last_clean":"2023-07-21T22:27:43.375574+0000","last_became_active":"2023-07-21T22:27:42.082676+0000","last_became_peered":"2023-07-21T22:27:42.082676+0000","last_unstale":"2023-07-21T22:27:43.375574+0000","last_undegraded":"2023-07-21T22:27:43.375574+0000","last_fullsized":"2023-07-21T22:27:43.375574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:47.852129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:47.031735+0000","last_change":"2023-07-21T22:27:41.076844+0000","last_active":"2023-07-21T22:27:47.031735+0000","last_peered":"2023-07-21T22:27:47.031735+0000","last_clean":"2023-07-21T22:27:47.031735+0000","last_became_active":"2023-07-21T22:27:41.076487+0000","last_became_peered":"2023-07-21T22:27:41.076487+0000","last_unstale":"2023-07-21T22:27:47.031735+0000","last_undegraded":"2023-07-21T22:27:47.031735+0000","last_fullsized":"2023-07-21T22:27:47.031735+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:22:04.537934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176398+0000","last_change":"2023-07-21T22:27:42.089996+0000","last_active":"2023-07-21T22:27:44.176398+0000","last_peered":"2023-07-21T22:27:44.176398+0000","last_clean":"2023-07-21T22:27:44.176398+0000","last_became_active":"2023-07-21T22:27:42.089668+0000","last_became_peered":"2023-07-21T22:27:42.089668+0000","last_unstale":"2023-07-21T22:27:44.176398+0000","last_undegraded":"2023-07-21T22:27:44.176398+0000","last_fullsized":"2023-07-21T22:27:44.176398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:31:09.221434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.024979+0000","last_change":"2023-07-21T22:27:41.075864+0000","last_active":"2023-07-21T22:27:46.024979+0000","last_peered":"2023-07-21T22:27:46.024979+0000","last_clean":"2023-07-21T22:27:46.024979+0000","last_became_active":"2023-07-21T22:27:41.075577+0000","last_became_peered":"2023-07-21T22:27:41.075577+0000","last_unstale":"2023-07-21T22:27:46.024979+0000","last_undegraded":"2023-07-21T22:27:46.024979+0000","last_fullsized":"2023-07-21T22:27:46.024979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:54.831786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390617+0000","last_change":"2023-07-21T22:27:42.091730+0000","last_active":"2023-07-21T22:27:44.390617+0000","last_peered":"2023-07-21T22:27:44.390617+0000","last_clean":"2023-07-21T22:27:44.390617+0000","last_became_active":"2023-07-21T22:27:42.091394+0000","last_became_peered":"2023-07-21T22:27:42.091394+0000","last_unstale":"2023-07-21T22:27:44.390617+0000","last_undegraded":"2023-07-21T22:27:44.390617+0000","last_fullsized":"2023-07-21T22:27:44.390617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:17:17.092150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.041728+0000","last_change":"2023-07-21T22:27:41.074822+0000","last_active":"2023-07-21T22:27:48.041728+0000","last_peered":"2023-07-21T22:27:48.041728+0000","last_clean":"2023-07-21T22:27:48.041728+0000","last_became_active":"2023-07-21T22:27:41.074542+0000","last_became_peered":"2023-07-21T22:27:41.074542+0000","last_unstale":"2023-07-21T22:27:48.041728+0000","last_undegraded":"2023-07-21T22:27:48.041728+0000","last_fullsized":"2023-07-21T22:27:48.041728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:44:23.601406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.302989+0000","last_change":"2023-07-21T22:27:42.094351+0000","last_active":"2023-07-21T22:27:43.302989+0000","last_peered":"2023-07-21T22:27:43.302989+0000","last_clean":"2023-07-21T22:27:43.302989+0000","last_became_active":"2023-07-21T22:27:42.094068+0000","last_became_peered":"2023-07-21T22:27:42.094068+0000","last_unstale":"2023-07-21T22:27:43.302989+0000","last_undegraded":"2023-07-21T22:27:43.302989+0000","last_fullsized":"2023-07-21T22:27:43.302989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:27:50.111150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":97,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:26.072881+0000","last_change":"2023-07-21T22:27:41.079188+0000","last_active":"2023-07-21T22:28:26.072881+0000","last_peered":"2023-07-21T22:28:26.072881+0000","last_clean":"2023-07-21T22:28:26.072881+0000","last_became_active":"2023-07-21T22:27:41.078899+0000","last_became_peered":"2023-07-21T22:27:41.078899+0000","last_unstale":"2023-07-21T22:28:26.072881+0000","last_undegraded":"2023-07-21T22:28:26.072881+0000","last_fullsized":"2023-07-21T22:28:26.072881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:00:37.074692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802626+0000","last_change":"2023-07-21T22:27:42.091194+0000","last_active":"2023-07-21T22:27:44.802626+0000","last_peered":"2023-07-21T22:27:44.802626+0000","last_clean":"2023-07-21T22:27:44.802626+0000","last_became_active":"2023-07-21T22:27:42.090869+0000","last_became_peered":"2023-07-21T22:27:42.090869+0000","last_unstale":"2023-07-21T22:27:44.802626+0000","last_undegraded":"2023-07-21T22:27:44.802626+0000","last_fullsized":"2023-07-21T22:27:44.802626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:26:26.169905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.039753+0000","last_change":"2023-07-21T22:27:41.072701+0000","last_active":"2023-07-21T22:27:48.039753+0000","last_peered":"2023-07-21T22:27:48.039753+0000","last_clean":"2023-07-21T22:27:48.039753+0000","last_became_active":"2023-07-21T22:27:41.072358+0000","last_became_peered":"2023-07-21T22:27:41.072358+0000","last_unstale":"2023-07-21T22:27:48.039753+0000","last_undegraded":"2023-07-21T22:27:48.039753+0000","last_fullsized":"2023-07-21T22:27:48.039753+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:51:15.728117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957974+0000","last_change":"2023-07-21T22:27:42.090808+0000","last_active":"2023-07-21T22:27:43.957974+0000","last_peered":"2023-07-21T22:27:43.957974+0000","last_clean":"2023-07-21T22:27:43.957974+0000","last_became_active":"2023-07-21T22:27:42.090506+0000","last_became_peered":"2023-07-21T22:27:42.090506+0000","last_unstale":"2023-07-21T22:27:43.957974+0000","last_undegraded":"2023-07-21T22:27:43.957974+0000","last_fullsized":"2023-07-21T22:27:43.957974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:59:17.254273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.016315+0000","last_change":"2023-07-21T22:27:41.075643+0000","last_active":"2023-07-21T22:27:46.016315+0000","last_peered":"2023-07-21T22:27:46.016315+0000","last_clean":"2023-07-21T22:27:46.016315+0000","last_became_active":"2023-07-21T22:27:41.075336+0000","last_became_peered":"2023-07-21T22:27:41.075336+0000","last_unstale":"2023-07-21T22:27:46.016315+0000","last_undegraded":"2023-07-21T22:27:46.016315+0000","last_fullsized":"2023-07-21T22:27:46.016315+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:12:56.006860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802567+0000","last_change":"2023-07-21T22:27:42.096722+0000","last_active":"2023-07-21T22:27:44.802567+0000","last_peered":"2023-07-21T22:27:44.802567+0000","last_clean":"2023-07-21T22:27:44.802567+0000","last_became_active":"2023-07-21T22:27:42.096472+0000","last_became_peered":"2023-07-21T22:27:42.096472+0000","last_unstale":"2023-07-21T22:27:44.802567+0000","last_undegraded":"2023-07-21T22:27:44.802567+0000","last_fullsized":"2023-07-21T22:27:44.802567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:25:16.425955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'28","reported_seq":130,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:26.072677+0000","last_change":"2023-07-21T22:27:41.072297+0000","last_active":"2023-07-21T22:28:26.072677+0000","last_peered":"2023-07-21T22:28:26.072677+0000","last_clean":"2023-07-21T22:28:26.072677+0000","last_became_active":"2023-07-21T22:27:41.071971+0000","last_became_peered":"2023-07-21T22:27:41.071971+0000","last_unstale":"2023-07-21T22:28:26.072677+0000","last_undegraded":"2023-07-21T22:28:26.072677+0000","last_fullsized":"2023-07-21T22:28:26.072677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:39:54.361110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54660,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":88,"num_read_kb":97,"num_write":30,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375731+0000","last_change":"2023-07-21T22:27:42.097130+0000","last_active":"2023-07-21T22:27:43.375731+0000","last_peered":"2023-07-21T22:27:43.375731+0000","last_clean":"2023-07-21T22:27:43.375731+0000","last_became_active":"2023-07-21T22:27:42.096853+0000","last_became_peered":"2023-07-21T22:27:42.096853+0000","last_unstale":"2023-07-21T22:27:43.375731+0000","last_undegraded":"2023-07-21T22:27:43.375731+0000","last_fullsized":"2023-07-21T22:27:43.375731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:36:06.680575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.042746+0000","last_change":"2023-07-21T22:27:41.071755+0000","last_active":"2023-07-21T22:27:48.042746+0000","last_peered":"2023-07-21T22:27:48.042746+0000","last_clean":"2023-07-21T22:27:48.042746+0000","last_became_active":"2023-07-21T22:27:41.071245+0000","last_became_peered":"2023-07-21T22:27:41.071245+0000","last_unstale":"2023-07-21T22:27:48.042746+0000","last_undegraded":"2023-07-21T22:27:48.042746+0000","last_fullsized":"2023-07-21T22:27:48.042746+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:09:41.990748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074639+0000","last_change":"2023-07-21T22:27:42.097433+0000","last_active":"2023-07-21T22:27:43.074639+0000","last_peered":"2023-07-21T22:27:43.074639+0000","last_clean":"2023-07-21T22:27:43.074639+0000","last_became_active":"2023-07-21T22:27:42.097100+0000","last_became_peered":"2023-07-21T22:27:42.097100+0000","last_unstale":"2023-07-21T22:27:43.074639+0000","last_undegraded":"2023-07-21T22:27:43.074639+0000","last_fullsized":"2023-07-21T22:27:43.074639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:02:40.214198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.041819+0000","last_change":"2023-07-21T22:27:41.074019+0000","last_active":"2023-07-21T22:27:48.041819+0000","last_peered":"2023-07-21T22:27:48.041819+0000","last_clean":"2023-07-21T22:27:48.041819+0000","last_became_active":"2023-07-21T22:27:41.073699+0000","last_became_peered":"2023-07-21T22:27:41.073699+0000","last_unstale":"2023-07-21T22:27:48.041819+0000","last_undegraded":"2023-07-21T22:27:48.041819+0000","last_fullsized":"2023-07-21T22:27:48.041819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:58:24.226793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172010+0000","last_change":"2023-07-21T22:27:42.098010+0000","last_active":"2023-07-21T22:27:44.172010+0000","last_peered":"2023-07-21T22:27:44.172010+0000","last_clean":"2023-07-21T22:27:44.172010+0000","last_became_active":"2023-07-21T22:27:42.097227+0000","last_became_peered":"2023-07-21T22:27:42.097227+0000","last_unstale":"2023-07-21T22:27:44.172010+0000","last_undegraded":"2023-07-21T22:27:44.172010+0000","last_fullsized":"2023-07-21T22:27:44.172010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:49.592034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:28.104282+0000","last_change":"2023-07-21T22:27:41.072298+0000","last_active":"2023-07-21T22:28:28.104282+0000","last_peered":"2023-07-21T22:28:28.104282+0000","last_clean":"2023-07-21T22:28:28.104282+0000","last_became_active":"2023-07-21T22:27:41.071963+0000","last_became_peered":"2023-07-21T22:27:41.071963+0000","last_unstale":"2023-07-21T22:28:28.104282+0000","last_undegraded":"2023-07-21T22:28:28.104282+0000","last_fullsized":"2023-07-21T22:28:28.104282+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:47:00.466196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171795+0000","last_change":"2023-07-21T22:27:42.091276+0000","last_active":"2023-07-21T22:27:44.171795+0000","last_peered":"2023-07-21T22:27:44.171795+0000","last_clean":"2023-07-21T22:27:44.171795+0000","last_became_active":"2023-07-21T22:27:42.090918+0000","last_became_peered":"2023-07-21T22:27:42.090918+0000","last_unstale":"2023-07-21T22:27:44.171795+0000","last_undegraded":"2023-07-21T22:27:44.171795+0000","last_fullsized":"2023-07-21T22:27:44.171795+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:07:22.664750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.041117+0000","last_change":"2023-07-21T22:27:41.075633+0000","last_active":"2023-07-21T22:27:48.041117+0000","last_peered":"2023-07-21T22:27:48.041117+0000","last_clean":"2023-07-21T22:27:48.041117+0000","last_became_active":"2023-07-21T22:27:41.075331+0000","last_became_peered":"2023-07-21T22:27:41.075331+0000","last_unstale":"2023-07-21T22:27:48.041117+0000","last_undegraded":"2023-07-21T22:27:48.041117+0000","last_fullsized":"2023-07-21T22:27:48.041117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:28:14.600527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176511+0000","last_change":"2023-07-21T22:27:42.089934+0000","last_active":"2023-07-21T22:27:44.176511+0000","last_peered":"2023-07-21T22:27:44.176511+0000","last_clean":"2023-07-21T22:27:44.176511+0000","last_became_active":"2023-07-21T22:27:42.089652+0000","last_became_peered":"2023-07-21T22:27:42.089652+0000","last_unstale":"2023-07-21T22:27:44.176511+0000","last_undegraded":"2023-07-21T22:27:44.176511+0000","last_fullsized":"2023-07-21T22:27:44.176511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:10:40.658561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.013258+0000","last_change":"2023-07-21T22:27:41.070862+0000","last_active":"2023-07-21T22:27:46.013258+0000","last_peered":"2023-07-21T22:27:46.013258+0000","last_clean":"2023-07-21T22:27:46.013258+0000","last_became_active":"2023-07-21T22:27:41.070507+0000","last_became_peered":"2023-07-21T22:27:41.070507+0000","last_unstale":"2023-07-21T22:27:46.013258+0000","last_undegraded":"2023-07-21T22:27:46.013258+0000","last_fullsized":"2023-07-21T22:27:46.013258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:28:00.010282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075397+0000","last_change":"2023-07-21T22:27:42.092046+0000","last_active":"2023-07-21T22:27:43.075397+0000","last_peered":"2023-07-21T22:27:43.075397+0000","last_clean":"2023-07-21T22:27:43.075397+0000","last_became_active":"2023-07-21T22:27:42.091069+0000","last_became_peered":"2023-07-21T22:27:42.091069+0000","last_unstale":"2023-07-21T22:27:43.075397+0000","last_undegraded":"2023-07-21T22:27:43.075397+0000","last_fullsized":"2023-07-21T22:27:43.075397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:39:03.277929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.005888+0000","last_change":"2023-07-21T22:27:41.074847+0000","last_active":"2023-07-21T22:27:45.005888+0000","last_peered":"2023-07-21T22:27:45.005888+0000","last_clean":"2023-07-21T22:27:45.005888+0000","last_became_active":"2023-07-21T22:27:41.074530+0000","last_became_peered":"2023-07-21T22:27:41.074530+0000","last_unstale":"2023-07-21T22:27:45.005888+0000","last_undegraded":"2023-07-21T22:27:45.005888+0000","last_fullsized":"2023-07-21T22:27:45.005888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:54:10.420130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802991+0000","last_change":"2023-07-21T22:27:42.089285+0000","last_active":"2023-07-21T22:27:44.802991+0000","last_peered":"2023-07-21T22:27:44.802991+0000","last_clean":"2023-07-21T22:27:44.802991+0000","last_became_active":"2023-07-21T22:27:42.088970+0000","last_became_peered":"2023-07-21T22:27:42.088970+0000","last_unstale":"2023-07-21T22:27:44.802991+0000","last_undegraded":"2023-07-21T22:27:44.802991+0000","last_fullsized":"2023-07-21T22:27:44.802991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:45:46.101767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:24.091965+0000","last_change":"2023-07-21T22:27:41.077433+0000","last_active":"2023-07-21T22:28:24.091965+0000","last_peered":"2023-07-21T22:28:24.091965+0000","last_clean":"2023-07-21T22:28:24.091965+0000","last_became_active":"2023-07-21T22:27:41.077133+0000","last_became_peered":"2023-07-21T22:27:41.077133+0000","last_unstale":"2023-07-21T22:28:24.091965+0000","last_undegraded":"2023-07-21T22:28:24.091965+0000","last_fullsized":"2023-07-21T22:28:24.091965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:48:14.982483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802503+0000","last_change":"2023-07-21T22:27:42.089833+0000","last_active":"2023-07-21T22:27:44.802503+0000","last_peered":"2023-07-21T22:27:44.802503+0000","last_clean":"2023-07-21T22:27:44.802503+0000","last_became_active":"2023-07-21T22:27:42.089502+0000","last_became_peered":"2023-07-21T22:27:42.089502+0000","last_unstale":"2023-07-21T22:27:44.802503+0000","last_undegraded":"2023-07-21T22:27:44.802503+0000","last_fullsized":"2023-07-21T22:27:44.802503+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:48:49.350741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176463+0000","last_change":"2023-07-21T22:27:42.090033+0000","last_active":"2023-07-21T22:27:44.176463+0000","last_peered":"2023-07-21T22:27:44.176463+0000","last_clean":"2023-07-21T22:27:44.176463+0000","last_became_active":"2023-07-21T22:27:42.089708+0000","last_became_peered":"2023-07-21T22:27:42.089708+0000","last_unstale":"2023-07-21T22:27:44.176463+0000","last_undegraded":"2023-07-21T22:27:44.176463+0000","last_fullsized":"2023-07-21T22:27:44.176463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:52:53.948825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:24.096561+0000","last_change":"2023-07-21T22:27:41.076784+0000","last_active":"2023-07-21T22:28:24.096561+0000","last_peered":"2023-07-21T22:28:24.096561+0000","last_clean":"2023-07-21T22:28:24.096561+0000","last_became_active":"2023-07-21T22:27:41.076453+0000","last_became_peered":"2023-07-21T22:27:41.076453+0000","last_unstale":"2023-07-21T22:28:24.096561+0000","last_undegraded":"2023-07-21T22:28:24.096561+0000","last_fullsized":"2023-07-21T22:28:24.096561+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:32:32.953367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.391049+0000","last_change":"2023-07-21T22:27:42.091286+0000","last_active":"2023-07-21T22:27:44.391049+0000","last_peered":"2023-07-21T22:27:44.391049+0000","last_clean":"2023-07-21T22:27:44.391049+0000","last_became_active":"2023-07-21T22:27:42.090986+0000","last_became_peered":"2023-07-21T22:27:42.090986+0000","last_unstale":"2023-07-21T22:27:44.391049+0000","last_undegraded":"2023-07-21T22:27:44.391049+0000","last_fullsized":"2023-07-21T22:27:44.391049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:07:53.301709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:49.040869+0000","last_change":"2023-07-21T22:27:41.074075+0000","last_active":"2023-07-21T22:27:49.040869+0000","last_peered":"2023-07-21T22:27:49.040869+0000","last_clean":"2023-07-21T22:27:49.040869+0000","last_became_active":"2023-07-21T22:27:41.073757+0000","last_became_peered":"2023-07-21T22:27:41.073757+0000","last_unstale":"2023-07-21T22:27:49.040869+0000","last_undegraded":"2023-07-21T22:27:49.040869+0000","last_fullsized":"2023-07-21T22:27:49.040869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:35:27.586578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.171569+0000","last_change":"2023-07-21T22:26:43.405788+0000","last_active":"2023-07-21T22:27:44.171569+0000","last_peered":"2023-07-21T22:27:44.171569+0000","last_clean":"2023-07-21T22:27:44.171569+0000","last_became_active":"2023-07-21T22:26:43.405351+0000","last_became_peered":"2023-07-21T22:26:43.405351+0000","last_unstale":"2023-07-21T22:27:44.171569+0000","last_undegraded":"2023-07-21T22:27:44.171569+0000","last_fullsized":"2023-07-21T22:27:44.171569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_clean_scrub_stamp":"2023-07-21T22:23:52.032210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:54:33.446817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074509+0000","last_change":"2023-07-21T22:27:42.097967+0000","last_active":"2023-07-21T22:27:43.074509+0000","last_peered":"2023-07-21T22:27:43.074509+0000","last_clean":"2023-07-21T22:27:43.074509+0000","last_became_active":"2023-07-21T22:27:42.097708+0000","last_became_peered":"2023-07-21T22:27:42.097708+0000","last_unstale":"2023-07-21T22:27:43.074509+0000","last_undegraded":"2023-07-21T22:27:43.074509+0000","last_fullsized":"2023-07-21T22:27:43.074509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:13:15.609283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.026760+0000","last_change":"2023-07-21T22:27:41.077853+0000","last_active":"2023-07-21T22:27:46.026760+0000","last_peered":"2023-07-21T22:27:46.026760+0000","last_clean":"2023-07-21T22:27:46.026760+0000","last_became_active":"2023-07-21T22:27:41.077539+0000","last_became_peered":"2023-07-21T22:27:41.077539+0000","last_unstale":"2023-07-21T22:27:46.026760+0000","last_undegraded":"2023-07-21T22:27:46.026760+0000","last_fullsized":"2023-07-21T22:27:46.026760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:09:25.539370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:47.031009+0000","last_change":"2023-07-21T22:27:41.076529+0000","last_active":"2023-07-21T22:27:47.031009+0000","last_peered":"2023-07-21T22:27:47.031009+0000","last_clean":"2023-07-21T22:27:47.031009+0000","last_became_active":"2023-07-21T22:27:41.076208+0000","last_became_peered":"2023-07-21T22:27:41.076208+0000","last_unstale":"2023-07-21T22:27:47.031009+0000","last_undegraded":"2023-07-21T22:27:47.031009+0000","last_fullsized":"2023-07-21T22:27:47.031009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:54:46.984912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074958+0000","last_change":"2023-07-21T22:27:42.082587+0000","last_active":"2023-07-21T22:27:43.074958+0000","last_peered":"2023-07-21T22:27:43.074958+0000","last_clean":"2023-07-21T22:27:43.074958+0000","last_became_active":"2023-07-21T22:27:42.082212+0000","last_became_peered":"2023-07-21T22:27:42.082212+0000","last_unstale":"2023-07-21T22:27:43.074958+0000","last_undegraded":"2023-07-21T22:27:43.074958+0000","last_fullsized":"2023-07-21T22:27:43.074958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:38.746984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:50.968129+0000","last_change":"2023-07-21T22:27:41.073736+0000","last_active":"2023-07-21T22:27:50.968129+0000","last_peered":"2023-07-21T22:27:50.968129+0000","last_clean":"2023-07-21T22:27:50.968129+0000","last_became_active":"2023-07-21T22:27:41.073213+0000","last_became_peered":"2023-07-21T22:27:41.073213+0000","last_unstale":"2023-07-21T22:27:50.968129+0000","last_undegraded":"2023-07-21T22:27:50.968129+0000","last_fullsized":"2023-07-21T22:27:50.968129+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:47:44.847798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390871+0000","last_change":"2023-07-21T22:27:42.091313+0000","last_active":"2023-07-21T22:27:44.390871+0000","last_peered":"2023-07-21T22:27:44.390871+0000","last_clean":"2023-07-21T22:27:44.390871+0000","last_became_active":"2023-07-21T22:27:42.091012+0000","last_became_peered":"2023-07-21T22:27:42.091012+0000","last_unstale":"2023-07-21T22:27:44.390871+0000","last_undegraded":"2023-07-21T22:27:44.390871+0000","last_fullsized":"2023-07-21T22:27:44.390871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:27:43.360142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.041975+0000","last_change":"2023-07-21T22:27:41.074644+0000","last_active":"2023-07-21T22:27:48.041975+0000","last_peered":"2023-07-21T22:27:48.041975+0000","last_clean":"2023-07-21T22:27:48.041975+0000","last_became_active":"2023-07-21T22:27:41.074364+0000","last_became_peered":"2023-07-21T22:27:41.074364+0000","last_unstale":"2023-07-21T22:27:48.041975+0000","last_undegraded":"2023-07-21T22:27:48.041975+0000","last_fullsized":"2023-07-21T22:27:48.041975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:31:09.704935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.957978+0000","last_change":"2023-07-21T22:27:42.089452+0000","last_active":"2023-07-21T22:27:43.957978+0000","last_peered":"2023-07-21T22:27:43.957978+0000","last_clean":"2023-07-21T22:27:43.957978+0000","last_became_active":"2023-07-21T22:27:42.088859+0000","last_became_peered":"2023-07-21T22:27:42.088859+0000","last_unstale":"2023-07-21T22:27:43.957978+0000","last_undegraded":"2023-07-21T22:27:43.957978+0000","last_fullsized":"2023-07-21T22:27:43.957978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:17:15.973380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.120703+0000","last_change":"2023-07-21T22:27:41.077067+0000","last_active":"2023-07-21T22:27:45.120703+0000","last_peered":"2023-07-21T22:27:45.120703+0000","last_clean":"2023-07-21T22:27:45.120703+0000","last_became_active":"2023-07-21T22:27:41.076768+0000","last_became_peered":"2023-07-21T22:27:41.076768+0000","last_unstale":"2023-07-21T22:27:45.120703+0000","last_undegraded":"2023-07-21T22:27:45.120703+0000","last_fullsized":"2023-07-21T22:27:45.120703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:28:19.457823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.375465+0000","last_change":"2023-07-21T22:27:42.088789+0000","last_active":"2023-07-21T22:27:43.375465+0000","last_peered":"2023-07-21T22:27:43.375465+0000","last_clean":"2023-07-21T22:27:43.375465+0000","last_became_active":"2023-07-21T22:27:42.088361+0000","last_became_peered":"2023-07-21T22:27:42.088361+0000","last_unstale":"2023-07-21T22:27:43.375465+0000","last_undegraded":"2023-07-21T22:27:43.375465+0000","last_fullsized":"2023-07-21T22:27:43.375465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:55:58.658869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.039700+0000","last_change":"2023-07-21T22:27:41.069595+0000","last_active":"2023-07-21T22:27:48.039700+0000","last_peered":"2023-07-21T22:27:48.039700+0000","last_clean":"2023-07-21T22:27:48.039700+0000","last_became_active":"2023-07-21T22:27:41.068733+0000","last_became_peered":"2023-07-21T22:27:41.068733+0000","last_unstale":"2023-07-21T22:27:48.039700+0000","last_undegraded":"2023-07-21T22:27:48.039700+0000","last_fullsized":"2023-07-21T22:27:48.039700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:45:30.220542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075228+0000","last_change":"2023-07-21T22:27:42.085701+0000","last_active":"2023-07-21T22:27:43.075228+0000","last_peered":"2023-07-21T22:27:43.075228+0000","last_clean":"2023-07-21T22:27:43.075228+0000","last_became_active":"2023-07-21T22:27:42.085429+0000","last_became_peered":"2023-07-21T22:27:42.085429+0000","last_unstale":"2023-07-21T22:27:43.075228+0000","last_undegraded":"2023-07-21T22:27:43.075228+0000","last_fullsized":"2023-07-21T22:27:43.075228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:48:06.310953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:49.232189+0000","last_change":"2023-07-21T22:27:41.073233+0000","last_active":"2023-07-21T22:27:49.232189+0000","last_peered":"2023-07-21T22:27:49.232189+0000","last_clean":"2023-07-21T22:27:49.232189+0000","last_became_active":"2023-07-21T22:27:41.072866+0000","last_became_peered":"2023-07-21T22:27:41.072866+0000","last_unstale":"2023-07-21T22:27:49.232189+0000","last_undegraded":"2023-07-21T22:27:49.232189+0000","last_fullsized":"2023-07-21T22:27:49.232189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:24:22.198579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390951+0000","last_change":"2023-07-21T22:27:42.100093+0000","last_active":"2023-07-21T22:27:44.390951+0000","last_peered":"2023-07-21T22:27:44.390951+0000","last_clean":"2023-07-21T22:27:44.390951+0000","last_became_active":"2023-07-21T22:27:42.099813+0000","last_became_peered":"2023-07-21T22:27:42.099813+0000","last_unstale":"2023-07-21T22:27:44.390951+0000","last_undegraded":"2023-07-21T22:27:44.390951+0000","last_fullsized":"2023-07-21T22:27:44.390951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:37:26.901693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:45.121256+0000","last_change":"2023-07-21T22:27:41.075641+0000","last_active":"2023-07-21T22:27:45.121256+0000","last_peered":"2023-07-21T22:27:45.121256+0000","last_clean":"2023-07-21T22:27:45.121256+0000","last_became_active":"2023-07-21T22:27:41.075268+0000","last_became_peered":"2023-07-21T22:27:41.075268+0000","last_unstale":"2023-07-21T22:27:45.121256+0000","last_undegraded":"2023-07-21T22:27:45.121256+0000","last_fullsized":"2023-07-21T22:27:45.121256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:32.377902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172002+0000","last_change":"2023-07-21T22:27:42.097736+0000","last_active":"2023-07-21T22:27:44.172002+0000","last_peered":"2023-07-21T22:27:44.172002+0000","last_clean":"2023-07-21T22:27:44.172002+0000","last_became_active":"2023-07-21T22:27:42.097419+0000","last_became_peered":"2023-07-21T22:27:42.097419+0000","last_unstale":"2023-07-21T22:27:44.172002+0000","last_undegraded":"2023-07-21T22:27:44.172002+0000","last_fullsized":"2023-07-21T22:27:44.172002+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:24:11.626848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:28:25.106068+0000","last_change":"2023-07-21T22:27:41.074922+0000","last_active":"2023-07-21T22:28:25.106068+0000","last_peered":"2023-07-21T22:28:25.106068+0000","last_clean":"2023-07-21T22:28:25.106068+0000","last_became_active":"2023-07-21T22:27:41.074610+0000","last_became_peered":"2023-07-21T22:27:41.074610+0000","last_unstale":"2023-07-21T22:28:25.106068+0000","last_undegraded":"2023-07-21T22:28:25.106068+0000","last_fullsized":"2023-07-21T22:28:25.106068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:52:46.091639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172068+0000","last_change":"2023-07-21T22:27:42.097312+0000","last_active":"2023-07-21T22:27:44.172068+0000","last_peered":"2023-07-21T22:27:44.172068+0000","last_clean":"2023-07-21T22:27:44.172068+0000","last_became_active":"2023-07-21T22:27:42.096835+0000","last_became_peered":"2023-07-21T22:27:42.096835+0000","last_unstale":"2023-07-21T22:27:44.172068+0000","last_undegraded":"2023-07-21T22:27:44.172068+0000","last_fullsized":"2023-07-21T22:27:44.172068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:46:51.718896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:47.031192+0000","last_change":"2023-07-21T22:27:41.071971+0000","last_active":"2023-07-21T22:27:47.031192+0000","last_peered":"2023-07-21T22:27:47.031192+0000","last_clean":"2023-07-21T22:27:47.031192+0000","last_became_active":"2023-07-21T22:27:41.071550+0000","last_became_peered":"2023-07-21T22:27:41.071550+0000","last_unstale":"2023-07-21T22:27:47.031192+0000","last_undegraded":"2023-07-21T22:27:47.031192+0000","last_fullsized":"2023-07-21T22:27:47.031192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:21:47.017891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:46.057107+0000","last_change":"2023-07-21T22:27:42.098039+0000","last_active":"2023-07-21T22:27:46.057107+0000","last_peered":"2023-07-21T22:27:46.057107+0000","last_clean":"2023-07-21T22:27:46.057107+0000","last_became_active":"2023-07-21T22:27:42.097737+0000","last_became_peered":"2023-07-21T22:27:42.097737+0000","last_unstale":"2023-07-21T22:27:46.057107+0000","last_undegraded":"2023-07-21T22:27:46.057107+0000","last_fullsized":"2023-07-21T22:27:46.057107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:36:20.727587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:48.039528+0000","last_change":"2023-07-21T22:27:41.076768+0000","last_active":"2023-07-21T22:27:48.039528+0000","last_peered":"2023-07-21T22:27:48.039528+0000","last_clean":"2023-07-21T22:27:48.039528+0000","last_became_active":"2023-07-21T22:27:41.076451+0000","last_became_peered":"2023-07-21T22:27:41.076451+0000","last_unstale":"2023-07-21T22:27:48.039528+0000","last_undegraded":"2023-07-21T22:27:48.039528+0000","last_fullsized":"2023-07-21T22:27:48.039528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:27:42.924064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.172018+0000","last_change":"2023-07-21T22:27:42.097891+0000","last_active":"2023-07-21T22:27:44.172018+0000","last_peered":"2023-07-21T22:27:44.172018+0000","last_clean":"2023-07-21T22:27:44.172018+0000","last_became_active":"2023-07-21T22:27:42.097022+0000","last_became_peered":"2023-07-21T22:27:42.097022+0000","last_unstale":"2023-07-21T22:27:44.172018+0000","last_undegraded":"2023-07-21T22:27:44.172018+0000","last_fullsized":"2023-07-21T22:27:44.172018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:04:32.167698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075273+0000","last_change":"2023-07-21T22:27:42.085774+0000","last_active":"2023-07-21T22:27:43.075273+0000","last_peered":"2023-07-21T22:27:43.075273+0000","last_clean":"2023-07-21T22:27:43.075273+0000","last_became_active":"2023-07-21T22:27:42.085460+0000","last_became_peered":"2023-07-21T22:27:42.085460+0000","last_unstale":"2023-07-21T22:27:43.075273+0000","last_undegraded":"2023-07-21T22:27:43.075273+0000","last_fullsized":"2023-07-21T22:27:43.075273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:20.846694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701118+0000","last_change":"2023-07-21T22:27:42.091630+0000","last_active":"2023-07-21T22:27:43.701118+0000","last_peered":"2023-07-21T22:27:43.701118+0000","last_clean":"2023-07-21T22:27:43.701118+0000","last_became_active":"2023-07-21T22:27:42.090538+0000","last_became_peered":"2023-07-21T22:27:42.090538+0000","last_unstale":"2023-07-21T22:27:43.701118+0000","last_undegraded":"2023-07-21T22:27:43.701118+0000","last_fullsized":"2023-07-21T22:27:43.701118+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:05:25.473040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701072+0000","last_change":"2023-07-21T22:27:42.092918+0000","last_active":"2023-07-21T22:27:43.701072+0000","last_peered":"2023-07-21T22:27:43.701072+0000","last_clean":"2023-07-21T22:27:43.701072+0000","last_became_active":"2023-07-21T22:27:42.092484+0000","last_became_peered":"2023-07-21T22:27:42.092484+0000","last_unstale":"2023-07-21T22:27:43.701072+0000","last_undegraded":"2023-07-21T22:27:43.701072+0000","last_fullsized":"2023-07-21T22:27:43.701072+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:40:43.563394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.958024+0000","last_change":"2023-07-21T22:27:42.096306+0000","last_active":"2023-07-21T22:27:43.958024+0000","last_peered":"2023-07-21T22:27:43.958024+0000","last_clean":"2023-07-21T22:27:43.958024+0000","last_became_active":"2023-07-21T22:27:42.096010+0000","last_became_peered":"2023-07-21T22:27:42.096010+0000","last_unstale":"2023-07-21T22:27:43.958024+0000","last_undegraded":"2023-07-21T22:27:43.958024+0000","last_fullsized":"2023-07-21T22:27:43.958024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:36:42.362478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074992+0000","last_change":"2023-07-21T22:27:42.097964+0000","last_active":"2023-07-21T22:27:43.074992+0000","last_peered":"2023-07-21T22:27:43.074992+0000","last_clean":"2023-07-21T22:27:43.074992+0000","last_became_active":"2023-07-21T22:27:42.097666+0000","last_became_peered":"2023-07-21T22:27:42.097666+0000","last_unstale":"2023-07-21T22:27:43.074992+0000","last_undegraded":"2023-07-21T22:27:43.074992+0000","last_fullsized":"2023-07-21T22:27:43.074992+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:42:54.073891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075488+0000","last_change":"2023-07-21T22:27:42.083225+0000","last_active":"2023-07-21T22:27:43.075488+0000","last_peered":"2023-07-21T22:27:43.075488+0000","last_clean":"2023-07-21T22:27:43.075488+0000","last_became_active":"2023-07-21T22:27:42.082892+0000","last_became_peered":"2023-07-21T22:27:42.082892+0000","last_unstale":"2023-07-21T22:27:43.075488+0000","last_undegraded":"2023-07-21T22:27:43.075488+0000","last_fullsized":"2023-07-21T22:27:43.075488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:17:08.372691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.701648+0000","last_change":"2023-07-21T22:27:42.090854+0000","last_active":"2023-07-21T22:27:43.701648+0000","last_peered":"2023-07-21T22:27:43.701648+0000","last_clean":"2023-07-21T22:27:43.701648+0000","last_became_active":"2023-07-21T22:27:42.090501+0000","last_became_peered":"2023-07-21T22:27:42.090501+0000","last_unstale":"2023-07-21T22:27:43.701648+0000","last_undegraded":"2023-07-21T22:27:43.701648+0000","last_fullsized":"2023-07-21T22:27:43.701648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:52:51.633954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390858+0000","last_change":"2023-07-21T22:27:42.094986+0000","last_active":"2023-07-21T22:27:44.390858+0000","last_peered":"2023-07-21T22:27:44.390858+0000","last_clean":"2023-07-21T22:27:44.390858+0000","last_became_active":"2023-07-21T22:27:42.094668+0000","last_became_peered":"2023-07-21T22:27:42.094668+0000","last_unstale":"2023-07-21T22:27:44.390858+0000","last_undegraded":"2023-07-21T22:27:44.390858+0000","last_fullsized":"2023-07-21T22:27:44.390858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:27:19.466686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.176209+0000","last_change":"2023-07-21T22:27:42.098033+0000","last_active":"2023-07-21T22:27:44.176209+0000","last_peered":"2023-07-21T22:27:44.176209+0000","last_clean":"2023-07-21T22:27:44.176209+0000","last_became_active":"2023-07-21T22:27:42.097219+0000","last_became_peered":"2023-07-21T22:27:42.097219+0000","last_unstale":"2023-07-21T22:27:44.176209+0000","last_undegraded":"2023-07-21T22:27:44.176209+0000","last_fullsized":"2023-07-21T22:27:44.176209+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:12:40.233218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.390721+0000","last_change":"2023-07-21T22:27:42.092136+0000","last_active":"2023-07-21T22:27:44.390721+0000","last_peered":"2023-07-21T22:27:44.390721+0000","last_clean":"2023-07-21T22:27:44.390721+0000","last_became_active":"2023-07-21T22:27:42.091559+0000","last_became_peered":"2023-07-21T22:27:42.091559+0000","last_unstale":"2023-07-21T22:27:44.390721+0000","last_undegraded":"2023-07-21T22:27:44.390721+0000","last_fullsized":"2023-07-21T22:27:44.390721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:30:28.654604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:44.802891+0000","last_change":"2023-07-21T22:27:42.093227+0000","last_active":"2023-07-21T22:27:44.802891+0000","last_peered":"2023-07-21T22:27:44.802891+0000","last_clean":"2023-07-21T22:27:44.802891+0000","last_became_active":"2023-07-21T22:27:42.092918+0000","last_became_peered":"2023-07-21T22:27:42.092918+0000","last_unstale":"2023-07-21T22:27:44.802891+0000","last_undegraded":"2023-07-21T22:27:44.802891+0000","last_fullsized":"2023-07-21T22:27:44.802891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:21:39.475897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.074784+0000","last_change":"2023-07-21T22:27:42.086494+0000","last_active":"2023-07-21T22:27:43.074784+0000","last_peered":"2023-07-21T22:27:43.074784+0000","last_clean":"2023-07-21T22:27:43.074784+0000","last_became_active":"2023-07-21T22:27:42.086189+0000","last_became_peered":"2023-07-21T22:27:42.086189+0000","last_unstale":"2023-07-21T22:27:43.074784+0000","last_undegraded":"2023-07-21T22:27:43.074784+0000","last_fullsized":"2023-07-21T22:27:43.074784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:24:31.736443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-07-21T22:27:43.075473+0000","last_change":"2023-07-21T22:27:42.090874+0000","last_active":"2023-07-21T22:27:43.075473+0000","last_peered":"2023-07-21T22:27:43.075473+0000","last_clean":"2023-07-21T22:27:43.075473+0000","last_became_active":"2023-07-21T22:27:42.090597+0000","last_became_peered":"2023-07-21T22:27:42.090597+0000","last_unstale":"2023-07-21T22:27:43.075473+0000","last_undegraded":"2023-07-21T22:27:43.075473+0000","last_fullsized":"2023-07-21T22:27:43.075473+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:10:24.103529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61670,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":426,"num_read_kb":280,"num_write":229,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":185010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481813,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":422619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.629},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.81100000000000005}]},{"osd":2,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.68400000000000005}]},{"osd":7,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.87}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.75900000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.91200000000000003}]}]},{"osd":10,"up_from":70,"seq":300647710745,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1581056,"data_stored":1067593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.60599999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.58099999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.622},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.92400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.77300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.73299999999999998}]},{"osd":11,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.89200000000000002}]}]},{"osd":9,"up_from":65,"seq":279172874269,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":422207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.438,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.0379999999999998,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.36899999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.435}]},{"osd":2,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.129,"5min":3.129,"15min":3.129},"last":0.307},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.51500000000000001,"15min":0.53000000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.66500000000000004,"15min":0.68500000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":2.9710000000000001,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0469999999999999,"5min":1.089,"15min":1.089},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.66400000000000003,"15min":0.68600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93700000000000006,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.504}]},{"osd":8,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.65900000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.623},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.58099999999999996}]}]},{"osd":8,"up_from":57,"seq":244813135905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":422585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.35999999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.76900000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.45400000000000001,"15min":0.438},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.48099999999999998,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.68400000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60099999999999998,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.82499999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.434,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.87},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61599999999999999,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.88500000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.93500000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69999999999999996,"15min":0.69699999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.997,"5min":1.097,"15min":1.097},"last":0.96599999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.55800000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.054,"5min":1.171,"15min":1.171},"last":1.036},{"interface":"front","average":{"1min":0.753,"5min":0.68999999999999995,"15min":0.68000000000000005},"min":{"1min":0.51000000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1830000000000001,"5min":1.232,"15min":1.232},"last":0.97899999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69799999999999995,"15min":0.68400000000000005},"min":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0680000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.90400000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.018},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.98899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]}]},{"osd":7,"up_from":50,"seq":214748364837,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1507328,"data_stored":1014964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53800000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.433,"15min":0.436},"min":{"1min":0.29299999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46200000000000002,"15min":0.46400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58699999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.434}]},{"osd":2,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.33900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.86299999999999999,"15min":0.90500000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81999999999999995,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.81100000000000005,"15min":0.83899999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.6859999999999999,"5min":5.7480000000000002,"15min":5.7480000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.83399999999999996,"15min":0.872},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85799999999999998,"5min":5.7649999999999997,"15min":5.7649999999999997},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.38900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.77200000000000002,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67600000000000005,"15min":0.68300000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86599999999999999,"5min":0.873,"15min":0.873},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.81699999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.65900000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.95299999999999996,"15min":0.998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.90200000000000002,"5min":5.7800000000000002,"15min":5.7800000000000002},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.78800000000000003,"15min":0.80000000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.875,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.76200000000000001}]},{"osd":9,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.79100000000000004,"15min":0.80700000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85799999999999998,"5min":1.819,"15min":1.819},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.753,"15min":0.76100000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.88300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69499999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.879}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528361,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":421882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.36499999999999999},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.373}]},{"osd":1,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.624,"15min":0.56299999999999994},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":6.899,"5min":6.899,"15min":6.899},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.498}]},{"osd":2,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54000000000000004,"15min":0.52000000000000002},"min":{"1min":0.44800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.64500000000000002,"15min":0.57499999999999996},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":7.0359999999999996,"5min":7.0359999999999996,"15min":7.0359999999999996},"last":0.63}]},{"osd":3,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.496,"15min":0.437},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":7.1289999999999996,"5min":7.1289999999999996,"15min":7.1289999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.071,"5min":0.58399999999999996,"15min":0.48599999999999999},"min":{"1min":0.32000000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":12.414999999999999,"5min":12.414999999999999,"15min":12.414999999999999},"last":0.45400000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.64900000000000002,"15min":0.58699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.9690000000000003,"5min":6.9690000000000003,"15min":6.9690000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.57699999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.44700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.60199999999999998,"15min":0.58799999999999997},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.59099999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.63500000000000001,"15min":0.61799999999999999},"min":{"1min":0.47099999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.56699999999999995}]},{"osd":8,"last update":"Fri Jul 21 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.747,"15min":0.69899999999999995},"min":{"1min":0.55000000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":7.0670000000000002,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70099999999999996,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.65700000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.68000000000000005,"15min":0.66700000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.516,"5min":4.516,"15min":4.516},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.78000000000000003,"15min":0.75600000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":7.1619999999999999,"5min":7.1619999999999999,"15min":7.1619999999999999},"last":0.72499999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":477449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.502,"15min":0.47999999999999998},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.50800000000000001,"15min":0.47999999999999998},"min":{"1min":0.47099999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.54200000000000004,"15min":0.50900000000000001},"min":{"1min":0.51100000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.79500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.54200000000000004,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76200000000000001,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.38300000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.872,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60199999999999998,"15min":0.58999999999999997},"min":{"1min":0.49299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.873}]},{"osd":4,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.436,"15min":0.41899999999999998},"min":{"1min":0.28599999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44600000000000001,"15min":0.43099999999999999},"min":{"1min":0.31,"5min":0.248,"15min":0.248},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.495}]},{"osd":5,"last update":"Fri Jul 21 22:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.56999999999999995,"15min":0.53400000000000003},"min":{"1min":0.55800000000000005,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84399999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.75},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.59799999999999998,"15min":0.56200000000000006},"min":{"1min":0.55100000000000005,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84199999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.70899999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.54600000000000004,"15min":0.52600000000000002},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Jul 21 22:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.39300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.62}]},{"osd":8,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.69899999999999995,"15min":0.68600000000000005},"min":{"1min":0.44,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78700000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.94199999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":423605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.50700000000000001,"15min":0.48499999999999999},"min":{"1min":0.40899999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51600000000000001,"15min":0.497},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81399999999999995,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52600000000000002,"15min":0.52100000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79800000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.33900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67000000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39400000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67600000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.439},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.44,"15min":0.432},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.66900000000000004,"5min":0.753,"15min":0.753},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58999999999999997,"15min":0.58899999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78500000000000003,"5min":0.879,"15min":0.879},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78500000000000003,"5min":1.002,"15min":1.002},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48299999999999998,"15min":0.49199999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58499999999999996,"5min":0.749,"15min":0.749},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79100000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.47199999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66500000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85499999999999998,"5min":1.04,"15min":1.04},"last":0.48499999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.68400000000000005,"15min":0.69099999999999995},"min":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72599999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.61899999999999999,"15min":0.63500000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88600000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.63},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.68200000000000005,"15min":0.70399999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89800000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.53900000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.68300000000000005}]},{"osd":11,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.753},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.79000000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":478401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.48899999999999999,"15min":0.47299999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.629},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.49099999999999999,"15min":0.46100000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64400000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.52800000000000002,"15min":0.48599999999999999},"min":{"1min":0.36099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.53700000000000003,"15min":0.51100000000000001},"min":{"1min":0.38300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.77900000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57799999999999996,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.58199999999999996,"15min":0.54900000000000004},"min":{"1min":0.44600000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55800000000000005,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.73999999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.46800000000000003,"15min":0.45800000000000002},"min":{"1min":0.29999999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.46899999999999997},"min":{"1min":0.33800000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.435,"5min":0.432,"15min":0.432},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46899999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.66100000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.51100000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.76400000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.90400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":421430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35599999999999998,"15min":0.33500000000000002},"min":{"1min":0.24399999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.629,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.26100000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.496,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.434}]},{"osd":1,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71499999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.73599999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76000000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Jul 21 22:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.44500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.48299999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.752}]},{"osd":5,"last update":"Fri Jul 21 22:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73899999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.629,"15min":0.624},"min":{"1min":0.46899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60399999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.50800000000000001,"15min":0.501},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.55300000000000005}]},{"osd":7,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.59199999999999997,"15min":0.59899999999999998},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.751,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66300000000000003,"15min":0.65800000000000003},"min":{"1min":0.498,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.78500000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.64300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.996,"5min":1.038,"15min":1.038},"last":0.62}]},{"osd":10,"last update":"Fri Jul 21 22:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.76500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":421737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.47899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.49299999999999999},"min":{"1min":0.41799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.30599999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.40500000000000003,"15min":0.38800000000000001},"min":{"1min":0.32600000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.38300000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.59199999999999997,"15min":0.58299999999999996},"min":{"1min":0.38300000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.69499999999999995}]},{"osd":3,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.46700000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53100000000000003,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53500000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.82899999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.60899999999999999,"15min":0.58699999999999997},"min":{"1min":0.44700000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.65900000000000003,"15min":0.60699999999999998},"min":{"1min":0.45300000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":5.6310000000000002,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.68899999999999995,"15min":0.629},"min":{"1min":0.53100000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.61099999999999999,"15min":0.59299999999999997},"min":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60099999999999998,"15min":0.59199999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.59499999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.51200000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.52200000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72699999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":0.81299999999999994,"15min":0.755},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":7.6639999999999997,"5min":7.6639999999999997,"15min":7.6639999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.64000000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.496}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28432,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716720,"statfs":{"total":95995035648,"available":95965921280,"internally_reserved":0,"allocated":1515520,"data_stored":1014407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.33300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.85099999999999998,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51500000000000001,"15min":0.504},"min":{"1min":0.35099999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70599999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.65900000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.54400000000000004,"15min":0.52200000000000002},"min":{"1min":0.48999999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82099999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55900000000000005,"15min":0.53500000000000003},"min":{"1min":0.436,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.85799999999999998,"5min":1.036,"15min":1.036},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.40500000000000003,"15min":0.38400000000000001},"min":{"1min":0.29899999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.42099999999999999,"15min":0.39900000000000002},"min":{"1min":0.314,"5min":0.255,"15min":0.255},"max":{"1min":0.72599999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.371}]},{"osd":3,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.33000000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77800000000000002,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.54900000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.58999999999999997,"15min":0.56100000000000005},"min":{"1min":0.51500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91000000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.42599999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0009999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.60099999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63600000000000001,"15min":0.61299999999999999},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89400000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.57199999999999995,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.63300000000000001}]},{"osd":8,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.64200000000000002,"15min":0.624},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.46400000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:27:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.72899999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.749},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:28:30.260 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-21T22:28:30.260 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-21T22:28:30.260 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-21T22:28:30.261 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph health --format=json 2023-07-21T22:28:30.812 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:31.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:31 smithi107 ceph-mon[96543]: pgmap v274: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:28:31.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:31 smithi107 ceph-mon[96543]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:28:31.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:31 smithi132 ceph-mon[97863]: pgmap v274: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:28:31.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:31 smithi132 ceph-mon[97863]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:28:31.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:31 smithi190 ceph-mon[98334]: pgmap v274: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:28:31.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:31 smithi190 ceph-mon[98334]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:28:32.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.078+0000 7f7353d42700 1 -- 172.21.15.107:0/3365260657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f734c1005d0 msgr2=0x7f734c100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:32.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.078+0000 7f7353d42700 1 --2- 172.21.15.107:0/3365260657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f734c1005d0 0x7f734c100a40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f73480099c0 tx=0x7f7348009cd0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.078+0000 7f7353d42700 1 -- 172.21.15.107:0/3365260657 shutdown_connections 2023-07-21T22:28:32.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.078+0000 7f7353d42700 1 --2- 172.21.15.107:0/3365260657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f734c100f80 0x7f734c10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.078+0000 7f7353d42700 1 --2- 172.21.15.107:0/3365260657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f734c0fee80 0x7f734c0ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.078+0000 7f7353d42700 1 --2- 172.21.15.107:0/3365260657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f734c1005d0 0x7f734c100a40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.078+0000 7f7353d42700 1 -- 172.21.15.107:0/3365260657 >> 172.21.15.107:0/3365260657 conn(0x7f734c075b60 msgr2=0x7f734c075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:32.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.078+0000 7f7353d42700 1 -- 172.21.15.107:0/3365260657 shutdown_connections 2023-07-21T22:28:32.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.078+0000 7f7353d42700 1 -- 172.21.15.107:0/3365260657 wait complete. 2023-07-21T22:28:32.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f7353d42700 1 Processor -- start 2023-07-21T22:28:32.127 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f7353d42700 1 -- start start 2023-07-21T22:28:32.127 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f7353d42700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f734c0fee80 0x7f734c195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:32.127 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f7353d42700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f734c1005d0 0x7f734c1961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:32.127 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f7353d42700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f734c100f80 0x7f734c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:32.127 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f7353d42700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c199bb0 con 0x7f734c100f80 2023-07-21T22:28:32.127 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f7353d42700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c199d20 con 0x7f734c0fee80 2023-07-21T22:28:32.127 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f7353d42700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c199ee0 con 0x7f734c1005d0 2023-07-21T22:28:32.127 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f73522df700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f734c100f80 0x7f734c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:32.128 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f73522df700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f734c100f80 0x7f734c1995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47080/0 (socket says 172.21.15.107:47080) 2023-07-21T22:28:32.128 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f73522df700 1 -- 172.21.15.107:0/1725395440 learned_addr learned my addr 172.21.15.107:0/1725395440 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:32.128 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f73512dd700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f734c1005d0 0x7f734c1961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:32.128 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f7351ade700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f734c0fee80 0x7f734c195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:32.128 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f73522df700 1 -- 172.21.15.107:0/1725395440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f734c1005d0 msgr2=0x7f734c1961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:32.128 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f73522df700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f734c1005d0 0x7f734c1961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f73522df700 1 -- 172.21.15.107:0/1725395440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f734c0fee80 msgr2=0x7f734c195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:32.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f73522df700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f734c0fee80 0x7f734c195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f73522df700 1 -- 172.21.15.107:0/1725395440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480095f0 con 0x7f734c100f80 2023-07-21T22:28:32.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.079+0000 7f7351ade700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f734c0fee80 0x7f734c195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:32.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.080+0000 7f73522df700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f734c100f80 0x7f734c1995b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7344007d70 tx=0x7f734400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:32.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.080+0000 7f7342ffd700 1 -- 172.21.15.107:0/1725395440 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734400ad40 con 0x7f734c100f80 2023-07-21T22:28:32.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.080+0000 7f7353d42700 1 -- 172.21.15.107:0/1725395440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734c19a1c0 con 0x7f734c100f80 2023-07-21T22:28:32.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.080+0000 7f7342ffd700 1 -- 172.21.15.107:0/1725395440 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73440128e0 con 0x7f734c100f80 2023-07-21T22:28:32.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.080+0000 7f7342ffd700 1 -- 172.21.15.107:0/1725395440 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344011780 con 0x7f734c100f80 2023-07-21T22:28:32.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.080+0000 7f7353d42700 1 -- 172.21.15.107:0/1725395440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734c1a2b10 con 0x7f734c100f80 2023-07-21T22:28:32.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.081+0000 7f7353d42700 1 -- 172.21.15.107:0/1725395440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734c04e210 con 0x7f734c100f80 2023-07-21T22:28:32.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.082+0000 7f7342ffd700 1 -- 172.21.15.107:0/1725395440 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f7344012a50 con 0x7f734c100f80 2023-07-21T22:28:32.131 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.083+0000 7f7342ffd700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f73380a11e0 0x7f73380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:32.131 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.083+0000 7f7351ade700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f73380a11e0 0x7f73380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:32.131 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.083+0000 7f7342ffd700 1 -- 172.21.15.107:0/1725395440 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f73440ccf40 con 0x7f734c100f80 2023-07-21T22:28:32.131 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.083+0000 7f7351ade700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f73380a11e0 0x7f73380a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f733c007900 tx=0x7f733c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:32.132 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.086+0000 7f7342ffd700 1 -- 172.21.15.107:0/1725395440 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f73440967f0 con 0x7f734c100f80 2023-07-21T22:28:32.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.317+0000 7f7353d42700 1 -- 172.21.15.107:0/1725395440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f734c196fd0 con 0x7f734c100f80 2023-07-21T22:28:32.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.317+0000 7f7342ffd700 1 -- 172.21.15.107:0/1725395440 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7344099e10 con 0x7f734c100f80 2023-07-21T22:28:32.363 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:32.363 INFO:teuthology.orchestra.run.smithi107.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-21T22:28:32.365 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.319+0000 7f7353d42700 1 -- 172.21.15.107:0/1725395440 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f73380a11e0 msgr2=0x7f73380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:32.365 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.319+0000 7f7353d42700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f73380a11e0 0x7f73380a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f733c007900 tx=0x7f733c008040 comp rx=0 tx=0).stop 2023-07-21T22:28:32.365 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.319+0000 7f7353d42700 1 -- 172.21.15.107:0/1725395440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f734c100f80 msgr2=0x7f734c1995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:32.365 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.319+0000 7f7353d42700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f734c100f80 0x7f734c1995b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7344007d70 tx=0x7f734400a340 comp rx=0 tx=0).stop 2023-07-21T22:28:32.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.320+0000 7f7353d42700 1 -- 172.21.15.107:0/1725395440 shutdown_connections 2023-07-21T22:28:32.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.320+0000 7f7353d42700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f734c100f80 0x7f734c1995b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.320+0000 7f7353d42700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f734c0fee80 0x7f734c195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.320+0000 7f7353d42700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f73380a11e0 0x7f73380a3690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.320+0000 7f7353d42700 1 --2- 172.21.15.107:0/1725395440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f734c1005d0 0x7f734c1961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.320+0000 7f7353d42700 1 -- 172.21.15.107:0/1725395440 >> 172.21.15.107:0/1725395440 conn(0x7f734c075b60 msgr2=0x7f734c1047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:32.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.320+0000 7f7353d42700 1 -- 172.21.15.107:0/1725395440 shutdown_connections 2023-07-21T22:28:32.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.320+0000 7f7353d42700 1 -- 172.21.15.107:0/1725395440 wait complete. 2023-07-21T22:28:32.766 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-21T22:28:32.766 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:28:32.894 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.845+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4226694390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc64101680 msgr2=0x7fcc64107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:32.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.845+0000 7fcc6a59e700 1 --2- 172.21.15.107:0/4226694390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc64101680 0x7fcc64107f40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00b0a0 tx=0x7fcc5c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.845+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4226694390 shutdown_connections 2023-07-21T22:28:32.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.845+0000 7fcc6a59e700 1 --2- 172.21.15.107:0/4226694390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc64101680 0x7fcc64107f40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.845+0000 7fcc6a59e700 1 --2- 172.21.15.107:0/4226694390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc640ffb00 0x7fcc640fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.845+0000 7fcc6a59e700 1 --2- 172.21.15.107:0/4226694390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc64100cf0 0x7fcc64101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.845+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4226694390 >> 172.21.15.107:0/4226694390 conn(0x7fcc640fb0e0 msgr2=0x7fcc640fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:32.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.845+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4226694390 shutdown_connections 2023-07-21T22:28:32.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.845+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4226694390 wait complete. 2023-07-21T22:28:32.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.846+0000 7fcc6a59e700 1 Processor -- start 2023-07-21T22:28:32.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.846+0000 7fcc6a59e700 1 -- start start 2023-07-21T22:28:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.846+0000 7fcc6a59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc640ffb00 0x7fcc64195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.846+0000 7fcc6a59e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc64100cf0 0x7fcc64195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.846+0000 7fcc6a59e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc64101680 0x7fcc6419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.846+0000 7fcc6a59e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6419d410 con 0x7fcc64101680 2023-07-21T22:28:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.846+0000 7fcc6a59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6419d550 con 0x7fcc640ffb00 2023-07-21T22:28:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.846+0000 7fcc6a59e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6419d6e0 con 0x7fcc64100cf0 2023-07-21T22:28:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.846+0000 7fcc68b3b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc64101680 0x7fcc6419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:32.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.846+0000 7fcc68b3b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc64101680 0x7fcc6419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47092/0 (socket says 172.21.15.107:47092) 2023-07-21T22:28:32.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc68b3b700 1 -- 172.21.15.107:0/4182250291 learned_addr learned my addr 172.21.15.107:0/4182250291 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:32.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc63fff700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc640ffb00 0x7fcc64195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:32.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc637fe700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc64100cf0 0x7fcc64195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:32.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc68b3b700 1 -- 172.21.15.107:0/4182250291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc64100cf0 msgr2=0x7fcc64195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:32.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc68b3b700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc64100cf0 0x7fcc64195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc68b3b700 1 -- 172.21.15.107:0/4182250291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc640ffb00 msgr2=0x7fcc64195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:32.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc68b3b700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc640ffb00 0x7fcc64195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:32.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc68b3b700 1 -- 172.21.15.107:0/4182250291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c009d00 con 0x7fcc64101680 2023-07-21T22:28:32.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc637fe700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc64100cf0 0x7fcc64195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:32.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc68b3b700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc64101680 0x7fcc6419ce40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00b780 tx=0x7fcc5c00bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:32.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc617fa700 1 -- 172.21.15.107:0/4182250291 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c00e040 con 0x7fcc64101680 2023-07-21T22:28:32.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc617fa700 1 -- 172.21.15.107:0/4182250291 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc5c004e60 con 0x7fcc64101680 2023-07-21T22:28:32.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4182250291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6419d960 con 0x7fcc64101680 2023-07-21T22:28:32.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc617fa700 1 -- 172.21.15.107:0/4182250291 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c01b740 con 0x7fcc64101680 2023-07-21T22:28:32.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.847+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4182250291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6419dde0 con 0x7fcc64101680 2023-07-21T22:28:32.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.848+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4182250291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc64106f50 con 0x7fcc64101680 2023-07-21T22:28:32.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.849+0000 7fcc617fa700 1 -- 172.21.15.107:0/4182250291 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fcc5c01da40 con 0x7fcc64101680 2023-07-21T22:28:32.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.850+0000 7fcc617fa700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc4c0a1220 0x7fcc4c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:32.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.850+0000 7fcc63fff700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc4c0a1220 0x7fcc4c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:32.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.850+0000 7fcc617fa700 1 -- 172.21.15.107:0/4182250291 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcc5c0ce260 con 0x7fcc64101680 2023-07-21T22:28:32.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.850+0000 7fcc63fff700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc4c0a1220 0x7fcc4c0a36d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcc64100b50 tx=0x7fcc54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:32.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:32.854+0000 7fcc617fa700 1 -- 172.21.15.107:0/4182250291 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcc5c097a90 con 0x7fcc64101680 2023-07-21T22:28:33.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.046+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4182250291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc6402caa0 con 0x7fcc64101680 2023-07-21T22:28:33.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.047+0000 7fcc617fa700 1 -- 172.21.15.107:0/4182250291 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22201 (secure 0 0 0) 0x7fcc5c09b0b0 con 0x7fcc64101680 2023-07-21T22:28:33.093 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:33.093 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:27:43.071566+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:28:33.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.050+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4182250291 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc4c0a1220 msgr2=0x7fcc4c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.050+0000 7fcc6a59e700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc4c0a1220 0x7fcc4c0a36d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcc64100b50 tx=0x7fcc54009260 comp rx=0 tx=0).stop 2023-07-21T22:28:33.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.050+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4182250291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc64101680 msgr2=0x7fcc6419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.050+0000 7fcc6a59e700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc64101680 0x7fcc6419ce40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00b780 tx=0x7fcc5c00bd00 comp rx=0 tx=0).stop 2023-07-21T22:28:33.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.050+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4182250291 shutdown_connections 2023-07-21T22:28:33.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.050+0000 7fcc6a59e700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc64101680 0x7fcc6419ce40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.050+0000 7fcc6a59e700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc640ffb00 0x7fcc64195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.050+0000 7fcc6a59e700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc4c0a1220 0x7fcc4c0a36d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.050+0000 7fcc6a59e700 1 --2- 172.21.15.107:0/4182250291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc64100cf0 0x7fcc64195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.050+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4182250291 >> 172.21.15.107:0/4182250291 conn(0x7fcc640fb0e0 msgr2=0x7fcc64104090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:33.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.050+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4182250291 shutdown_connections 2023-07-21T22:28:33.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.050+0000 7fcc6a59e700 1 -- 172.21.15.107:0/4182250291 wait complete. 2023-07-21T22:28:33.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:23:52.031819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:27:39.494209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:27:40.672303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:28:33.108 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:28:33.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.188+0000 7f3755a8a700 1 -- 172.21.15.107:0/179884475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37500fff60 msgr2=0x7f3750100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.188+0000 7f3755a8a700 1 --2- 172.21.15.107:0/179884475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37500fff60 0x7f3750100370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3744009990 tx=0x7f3744009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.188+0000 7f3755a8a700 1 -- 172.21.15.107:0/179884475 shutdown_connections 2023-07-21T22:28:33.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.188+0000 7f3755a8a700 1 --2- 172.21.15.107:0/179884475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37500ff620 0x7f3750106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.188+0000 7f3755a8a700 1 --2- 172.21.15.107:0/179884475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37500fff60 0x7f3750100370 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.188+0000 7f3755a8a700 1 --2- 172.21.15.107:0/179884475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37500fec90 0x7f37500ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.188+0000 7f3755a8a700 1 -- 172.21.15.107:0/179884475 >> 172.21.15.107:0/179884475 conn(0x7f37500f9770 msgr2=0x7f37500fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:33.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.188+0000 7f3755a8a700 1 -- 172.21.15.107:0/179884475 shutdown_connections 2023-07-21T22:28:33.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.188+0000 7f3755a8a700 1 -- 172.21.15.107:0/179884475 wait complete. 2023-07-21T22:28:33.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.188+0000 7f3755a8a700 1 Processor -- start 2023-07-21T22:28:33.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f3755a8a700 1 -- start start 2023-07-21T22:28:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f3755a8a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37500fec90 0x7f37501957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f3755a8a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37500ff620 0x7f3750195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f3755a8a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37500fff60 0x7f375019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f3755a8a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375019d340 con 0x7f37500ff620 2023-07-21T22:28:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f3755a8a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375019d480 con 0x7f37500fff60 2023-07-21T22:28:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f3755a8a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375019d610 con 0x7f37500fec90 2023-07-21T22:28:33.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37500ff620 0x7f3750195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37500ff620 0x7f3750195d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47104/0 (socket says 172.21.15.107:47104) 2023-07-21T22:28:33.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374effd700 1 -- 172.21.15.107:0/2387539875 learned_addr learned my addr 172.21.15.107:0/2387539875 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:33.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374ffff700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37500fff60 0x7f375019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374f7fe700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37500fec90 0x7f37501957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374effd700 1 -- 172.21.15.107:0/2387539875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37500fec90 msgr2=0x7f37501957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374effd700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37500fec90 0x7f37501957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374effd700 1 -- 172.21.15.107:0/2387539875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37500fff60 msgr2=0x7f375019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374effd700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37500fff60 0x7f375019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374effd700 1 -- 172.21.15.107:0/2387539875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37440095f0 con 0x7f37500ff620 2023-07-21T22:28:33.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374ffff700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37500fff60 0x7f375019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:33.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374f7fe700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37500fec90 0x7f37501957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:33.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374effd700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37500ff620 0x7f3750195d00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f374000d950 tx=0x7f374000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:33.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374cff9700 1 -- 172.21.15.107:0/2387539875 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740009b40 con 0x7f37500ff620 2023-07-21T22:28:33.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f3755a8a700 1 -- 172.21.15.107:0/2387539875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375019d8f0 con 0x7f37500ff620 2023-07-21T22:28:33.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.189+0000 7f374cff9700 1 -- 172.21.15.107:0/2387539875 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3740009ca0 con 0x7f37500ff620 2023-07-21T22:28:33.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.190+0000 7f374cff9700 1 -- 172.21.15.107:0/2387539875 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374000f620 con 0x7f37500ff620 2023-07-21T22:28:33.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.190+0000 7f3755a8a700 1 -- 172.21.15.107:0/2387539875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375019de00 con 0x7f37500ff620 2023-07-21T22:28:33.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.191+0000 7f3755a8a700 1 -- 172.21.15.107:0/2387539875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3750106b60 con 0x7f37500ff620 2023-07-21T22:28:33.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.192+0000 7f374cff9700 1 -- 172.21.15.107:0/2387539875 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3740010460 con 0x7f37500ff620 2023-07-21T22:28:33.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.192+0000 7f374cff9700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f37380a11e0 0x7f37380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.192+0000 7f374f7fe700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f37380a11e0 0x7f37380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.192+0000 7f374cff9700 1 -- 172.21.15.107:0/2387539875 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f37400ccc70 con 0x7f37500ff620 2023-07-21T22:28:33.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.193+0000 7f374f7fe700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f37380a11e0 0x7f37380a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f374400b5c0 tx=0x7f3744005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:33.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.197+0000 7f374cff9700 1 -- 172.21.15.107:0/2387539875 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3740096410 con 0x7f37500ff620 2023-07-21T22:28:33.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:33 smithi132 ceph-mon[97863]: pgmap v275: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T22:28:33.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:33 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1725395440' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T22:28:33.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:33 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/4182250291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:33.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.386+0000 7f3755a8a700 1 -- 172.21.15.107:0/2387539875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f375002ca70 con 0x7f37500ff620 2023-07-21T22:28:33.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.386+0000 7f374cff9700 1 -- 172.21.15.107:0/2387539875 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f37400200a0 con 0x7f37500ff620 2023-07-21T22:28:33.433 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2023-07-21T22:28:33.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.388+0000 7f3755a8a700 1 -- 172.21.15.107:0/2387539875 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f37380a11e0 msgr2=0x7f37380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.388+0000 7f3755a8a700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f37380a11e0 0x7f37380a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f374400b5c0 tx=0x7f3744005fb0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.388+0000 7f3755a8a700 1 -- 172.21.15.107:0/2387539875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37500ff620 msgr2=0x7f3750195d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.388+0000 7f3755a8a700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37500ff620 0x7f3750195d00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f374000d950 tx=0x7f374000dd10 comp rx=0 tx=0).stop 2023-07-21T22:28:33.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.389+0000 7f3755a8a700 1 -- 172.21.15.107:0/2387539875 shutdown_connections 2023-07-21T22:28:33.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.389+0000 7f3755a8a700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37500fff60 0x7f375019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.389+0000 7f3755a8a700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f37380a11e0 0x7f37380a3690 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.389+0000 7f3755a8a700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37500fec90 0x7f37501957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.389+0000 7f3755a8a700 1 --2- 172.21.15.107:0/2387539875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37500ff620 0x7f3750195d00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.389+0000 7f3755a8a700 1 -- 172.21.15.107:0/2387539875 >> 172.21.15.107:0/2387539875 conn(0x7f37500f9770 msgr2=0x7f37500fb970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:33.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.389+0000 7f3755a8a700 1 -- 172.21.15.107:0/2387539875 shutdown_connections 2023-07-21T22:28:33.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.389+0000 7f3755a8a700 1 -- 172.21.15.107:0/2387539875 wait complete. 2023-07-21T22:28:33.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:33 smithi107 ceph-mon[96543]: pgmap v275: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T22:28:33.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:33 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1725395440' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T22:28:33.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:33 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/4182250291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:33.447 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:28:33.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.526+0000 7f443c477700 1 -- 172.21.15.107:0/1466413948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4434100590 msgr2=0x7f4434106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.526+0000 7f443c477700 1 --2- 172.21.15.107:0/1466413948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4434100590 0x7f4434106e50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f442c00b0a0 tx=0x7f442c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.526+0000 7f443c477700 1 -- 172.21.15.107:0/1466413948 shutdown_connections 2023-07-21T22:28:33.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.526+0000 7f443c477700 1 --2- 172.21.15.107:0/1466413948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4434100590 0x7f4434106e50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.526+0000 7f443c477700 1 --2- 172.21.15.107:0/1466413948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44340fea10 0x7f44340fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.526+0000 7f443c477700 1 --2- 172.21.15.107:0/1466413948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44340ffc00 0x7f4434100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.526+0000 7f443c477700 1 -- 172.21.15.107:0/1466413948 >> 172.21.15.107:0/1466413948 conn(0x7f44340fa030 msgr2=0x7f44340fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:33.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.526+0000 7f443c477700 1 -- 172.21.15.107:0/1466413948 shutdown_connections 2023-07-21T22:28:33.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.526+0000 7f443c477700 1 -- 172.21.15.107:0/1466413948 wait complete. 2023-07-21T22:28:33.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443c477700 1 Processor -- start 2023-07-21T22:28:33.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443c477700 1 -- start start 2023-07-21T22:28:33.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443c477700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44340fea10 0x7f44340691b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443c477700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44340ffc00 0x7f443419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443c477700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4434100590 0x7f443419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443c477700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443419d430 con 0x7f4434100590 2023-07-21T22:28:33.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443c477700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443419d5a0 con 0x7f44340ffc00 2023-07-21T22:28:33.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443c477700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443419d760 con 0x7f44340fea10 2023-07-21T22:28:33.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443aa14700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4434100590 0x7f443419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443aa14700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4434100590 0x7f443419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47120/0 (socket says 172.21.15.107:47120) 2023-07-21T22:28:33.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443aa14700 1 -- 172.21.15.107:0/2300719904 learned_addr learned my addr 172.21.15.107:0/2300719904 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:33.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f4439a12700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44340ffc00 0x7f443419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443a213700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44340fea10 0x7f44340691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443aa14700 1 -- 172.21.15.107:0/2300719904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44340fea10 msgr2=0x7f44340691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443aa14700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44340fea10 0x7f44340691b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443aa14700 1 -- 172.21.15.107:0/2300719904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44340ffc00 msgr2=0x7f443419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443aa14700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44340ffc00 0x7f443419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443aa14700 1 -- 172.21.15.107:0/2300719904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442c009d00 con 0x7f4434100590 2023-07-21T22:28:33.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443a213700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44340fea10 0x7f44340691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:33.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f4439a12700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44340ffc00 0x7f443419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:33.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f443aa14700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4434100590 0x7f443419cc90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f442c00b830 tx=0x7f442c004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:33.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.527+0000 7f44277fe700 1 -- 172.21.15.107:0/2300719904 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c00e040 con 0x7f4434100590 2023-07-21T22:28:33.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.528+0000 7f443c477700 1 -- 172.21.15.107:0/2300719904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443419d9e0 con 0x7f4434100590 2023-07-21T22:28:33.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.528+0000 7f44277fe700 1 -- 172.21.15.107:0/2300719904 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f442c0094c0 con 0x7f4434100590 2023-07-21T22:28:33.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.528+0000 7f44277fe700 1 -- 172.21.15.107:0/2300719904 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c01bc70 con 0x7f4434100590 2023-07-21T22:28:33.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.528+0000 7f443c477700 1 -- 172.21.15.107:0/2300719904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443419deb0 con 0x7f4434100590 2023-07-21T22:28:33.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.528+0000 7f443c477700 1 -- 172.21.15.107:0/2300719904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4434105f10 con 0x7f4434100590 2023-07-21T22:28:33.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.529+0000 7f44277fe700 1 -- 172.21.15.107:0/2300719904 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f442c01d890 con 0x7f4434100590 2023-07-21T22:28:33.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.530+0000 7f44277fe700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f44200a11e0 0x7f44200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.530+0000 7f443a213700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f44200a11e0 0x7f44200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.530+0000 7f44277fe700 1 -- 172.21.15.107:0/2300719904 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f442c0ce9e0 con 0x7f4434100590 2023-07-21T22:28:33.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.530+0000 7f443a213700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f44200a11e0 0x7f44200a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4430005fd0 tx=0x7f4430005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:33.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.534+0000 7f44277fe700 1 -- 172.21.15.107:0/2300719904 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f442c098290 con 0x7f4434100590 2023-07-21T22:28:33.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:33 smithi190 ceph-mon[98334]: pgmap v275: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T22:28:33.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:33 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1725395440' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T22:28:33.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:33 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/4182250291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:33.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.721+0000 7f443c477700 1 -- 172.21.15.107:0/2300719904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f443402ca60 con 0x7f4434100590 2023-07-21T22:28:33.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.722+0000 7f44277fe700 1 -- 172.21.15.107:0/2300719904 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f442c021020 con 0x7f4434100590 2023-07-21T22:28:33.768 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2023-07-21T22:28:33.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.724+0000 7f443c477700 1 -- 172.21.15.107:0/2300719904 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f44200a11e0 msgr2=0x7f44200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.724+0000 7f443c477700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f44200a11e0 0x7f44200a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4430005fd0 tx=0x7f4430005dc0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.724+0000 7f443c477700 1 -- 172.21.15.107:0/2300719904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4434100590 msgr2=0x7f443419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.724+0000 7f443c477700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4434100590 0x7f443419cc90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f442c00b830 tx=0x7f442c004d30 comp rx=0 tx=0).stop 2023-07-21T22:28:33.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.724+0000 7f443c477700 1 -- 172.21.15.107:0/2300719904 shutdown_connections 2023-07-21T22:28:33.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.724+0000 7f443c477700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4434100590 0x7f443419cc90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.724+0000 7f443c477700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f44200a11e0 0x7f44200a3690 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.724+0000 7f443c477700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44340fea10 0x7f44340691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.724+0000 7f443c477700 1 --2- 172.21.15.107:0/2300719904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44340ffc00 0x7f443419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.724+0000 7f443c477700 1 -- 172.21.15.107:0/2300719904 >> 172.21.15.107:0/2300719904 conn(0x7f44340fa030 msgr2=0x7f4434103050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:33.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.724+0000 7f443c477700 1 -- 172.21.15.107:0/2300719904 shutdown_connections 2023-07-21T22:28:33.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.724+0000 7f443c477700 1 -- 172.21.15.107:0/2300719904 wait complete. 2023-07-21T22:28:33.781 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:28:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.859+0000 7f8448878700 1 -- 172.21.15.107:0/2859414446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84441057d0 msgr2=0x7f8444107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.859+0000 7f8448878700 1 --2- 172.21.15.107:0/2859414446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84441057d0 0x7f8444107c30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f843c00b0a0 tx=0x7f843c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 -- 172.21.15.107:0/2859414446 shutdown_connections 2023-07-21T22:28:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 --2- 172.21.15.107:0/2859414446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84441057d0 0x7f8444107c30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 --2- 172.21.15.107:0/2859414446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8444100590 0x7f8444102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 --2- 172.21.15.107:0/2859414446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8444102eb0 0x7f8444105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 -- 172.21.15.107:0/2859414446 >> 172.21.15.107:0/2859414446 conn(0x7f84440fa030 msgr2=0x7f84440fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 -- 172.21.15.107:0/2859414446 shutdown_connections 2023-07-21T22:28:33.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 -- 172.21.15.107:0/2859414446 wait complete. 2023-07-21T22:28:33.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 Processor -- start 2023-07-21T22:28:33.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 -- start start 2023-07-21T22:28:33.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8444100590 0x7f8444195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8444102eb0 0x7f8444195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84441057d0 0x7f844419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844419d4c0 con 0x7f8444100590 2023-07-21T22:28:33.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844419d600 con 0x7f84441057d0 2023-07-21T22:28:33.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.860+0000 7f8448878700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844419d790 con 0x7f8444102eb0 2023-07-21T22:28:33.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f844259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8444100590 0x7f8444195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f844259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8444100590 0x7f8444195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47130/0 (socket says 172.21.15.107:47130) 2023-07-21T22:28:33.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f844259c700 1 -- 172.21.15.107:0/2535078538 learned_addr learned my addr 172.21.15.107:0/2535078538 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:33.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f844259c700 1 -- 172.21.15.107:0/2535078538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8444102eb0 msgr2=0x7f8444195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f844259c700 1 --2- 172.21.15.107:0/2535078538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8444102eb0 0x7f8444195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.911 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f844259c700 1 -- 172.21.15.107:0/2535078538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84441057d0 msgr2=0x7f844419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:33.911 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f844259c700 1 --2- 172.21.15.107:0/2535078538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84441057d0 0x7f844419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:33.911 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f844259c700 1 -- 172.21.15.107:0/2535078538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c009d00 con 0x7f8444100590 2023-07-21T22:28:33.911 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f844259c700 1 --2- 172.21.15.107:0/2535078538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8444100590 0x7f8444195940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f843400c920 tx=0x7f843400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:33.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f84337fe700 1 -- 172.21.15.107:0/2535078538 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434007b00 con 0x7f8444100590 2023-07-21T22:28:33.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f84337fe700 1 -- 172.21.15.107:0/2535078538 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8434004cf0 con 0x7f8444100590 2023-07-21T22:28:33.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f8448878700 1 -- 172.21.15.107:0/2535078538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844419da70 con 0x7f8444100590 2023-07-21T22:28:33.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f84337fe700 1 -- 172.21.15.107:0/2535078538 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434005610 con 0x7f8444100590 2023-07-21T22:28:33.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.861+0000 7f8448878700 1 -- 172.21.15.107:0/2535078538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844419df80 con 0x7f8444100590 2023-07-21T22:28:33.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.862+0000 7f8448878700 1 -- 172.21.15.107:0/2535078538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84440fbba0 con 0x7f8444100590 2023-07-21T22:28:33.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.863+0000 7f84337fe700 1 -- 172.21.15.107:0/2535078538 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8434004750 con 0x7f8444100590 2023-07-21T22:28:33.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.864+0000 7f84337fe700 1 --2- 172.21.15.107:0/2535078538 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f842c0a11e0 0x7f842c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:33.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.864+0000 7f84337fe700 1 -- 172.21.15.107:0/2535078538 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f84340cc8e0 con 0x7f8444100590 2023-07-21T22:28:33.914 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.864+0000 7f8441d9b700 1 --2- 172.21.15.107:0/2535078538 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f842c0a11e0 0x7f842c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:33.914 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.864+0000 7f8441d9b700 1 --2- 172.21.15.107:0/2535078538 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f842c0a11e0 0x7f842c0a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f843800a990 tx=0x7f8438005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:33.914 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:33.868+0000 7f84337fe700 1 -- 172.21.15.107:0/2535078538 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8434096080 con 0x7f8444100590 2023-07-21T22:28:34.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.057+0000 7f8448878700 1 -- 172.21.15.107:0/2535078538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f844402cb00 con 0x7f8444100590 2023-07-21T22:28:34.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.058+0000 7f84337fe700 1 -- 172.21.15.107:0/2535078538 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f843401d020 con 0x7f8444100590 2023-07-21T22:28:34.103 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2023-07-21T22:28:34.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.060+0000 7f8448878700 1 -- 172.21.15.107:0/2535078538 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f842c0a11e0 msgr2=0x7f842c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.060+0000 7f8448878700 1 --2- 172.21.15.107:0/2535078538 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f842c0a11e0 0x7f842c0a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f843800a990 tx=0x7f8438005c80 comp rx=0 tx=0).stop 2023-07-21T22:28:34.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.060+0000 7f8448878700 1 -- 172.21.15.107:0/2535078538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8444100590 msgr2=0x7f8444195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.060+0000 7f8448878700 1 --2- 172.21.15.107:0/2535078538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8444100590 0x7f8444195940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f843400c920 tx=0x7f843400cce0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.060+0000 7f8448878700 1 -- 172.21.15.107:0/2535078538 shutdown_connections 2023-07-21T22:28:34.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.060+0000 7f8448878700 1 --2- 172.21.15.107:0/2535078538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84441057d0 0x7f844419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.060+0000 7f8448878700 1 --2- 172.21.15.107:0/2535078538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8444100590 0x7f8444195940 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.060+0000 7f8448878700 1 --2- 172.21.15.107:0/2535078538 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f842c0a11e0 0x7f842c0a3690 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.060+0000 7f8448878700 1 --2- 172.21.15.107:0/2535078538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8444102eb0 0x7f8444195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.060+0000 7f8448878700 1 -- 172.21.15.107:0/2535078538 >> 172.21.15.107:0/2535078538 conn(0x7f84440fa030 msgr2=0x7f8444101210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:34.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.060+0000 7f8448878700 1 -- 172.21.15.107:0/2535078538 shutdown_connections 2023-07-21T22:28:34.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.060+0000 7f8448878700 1 -- 172.21.15.107:0/2535078538 wait complete. 2023-07-21T22:28:34.117 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:28:34.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.193+0000 7f3289d2e700 1 -- 172.21.15.107:0/2491724826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3284100550 msgr2=0x7f3284106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.193+0000 7f3289d2e700 1 --2- 172.21.15.107:0/2491724826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3284100550 0x7f3284106e10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f327c00b0a0 tx=0x7f327c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 -- 172.21.15.107:0/2491724826 shutdown_connections 2023-07-21T22:28:34.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 --2- 172.21.15.107:0/2491724826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3284100550 0x7f3284106e10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 --2- 172.21.15.107:0/2491724826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32840fe9d0 0x7f32840fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 --2- 172.21.15.107:0/2491724826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32840ffbc0 0x7f3284100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 -- 172.21.15.107:0/2491724826 >> 172.21.15.107:0/2491724826 conn(0x7f32840f9fd0 msgr2=0x7f32840fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:34.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 -- 172.21.15.107:0/2491724826 shutdown_connections 2023-07-21T22:28:34.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 -- 172.21.15.107:0/2491724826 wait complete. 2023-07-21T22:28:34.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 Processor -- start 2023-07-21T22:28:34.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 -- start start 2023-07-21T22:28:34.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32840fe9d0 0x7f3284197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:34.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32840ffbc0 0x7f3284197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:34.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3284100550 0x7f328419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:34.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328419f5f0 con 0x7f32840fe9d0 2023-07-21T22:28:34.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328419f730 con 0x7f3284100550 2023-07-21T22:28:34.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.194+0000 7f3289d2e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328419f8c0 con 0x7f32840ffbc0 2023-07-21T22:28:34.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f32837fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32840fe9d0 0x7f3284197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:34.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f32837fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32840fe9d0 0x7f3284197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47144/0 (socket says 172.21.15.107:47144) 2023-07-21T22:28:34.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f32837fe700 1 -- 172.21.15.107:0/3515459237 learned_addr learned my addr 172.21.15.107:0/3515459237 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:34.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f3283fff700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3284100550 0x7f328419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:34.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f3282ffd700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32840ffbc0 0x7f3284197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:34.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f32837fe700 1 -- 172.21.15.107:0/3515459237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32840ffbc0 msgr2=0x7f3284197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f32837fe700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32840ffbc0 0x7f3284197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f32837fe700 1 -- 172.21.15.107:0/3515459237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3284100550 msgr2=0x7f328419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f32837fe700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3284100550 0x7f328419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f32837fe700 1 -- 172.21.15.107:0/3515459237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c009d00 con 0x7f32840fe9d0 2023-07-21T22:28:34.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f3283fff700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3284100550 0x7f328419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:34.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f3282ffd700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32840ffbc0 0x7f3284197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:34.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f32837fe700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32840fe9d0 0x7f3284197a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f327800c970 tx=0x7f327800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:34.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f3280ff9700 1 -- 172.21.15.107:0/3515459237 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278004220 con 0x7f32840fe9d0 2023-07-21T22:28:34.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f3289d2e700 1 -- 172.21.15.107:0/3515459237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328419fba0 con 0x7f32840fe9d0 2023-07-21T22:28:34.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f3280ff9700 1 -- 172.21.15.107:0/3515459237 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3278007d60 con 0x7f32840fe9d0 2023-07-21T22:28:34.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f3280ff9700 1 -- 172.21.15.107:0/3515459237 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3278003d00 con 0x7f32840fe9d0 2023-07-21T22:28:34.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.195+0000 7f3289d2e700 1 -- 172.21.15.107:0/3515459237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32841a00b0 con 0x7f32840fe9d0 2023-07-21T22:28:34.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.196+0000 7f3289d2e700 1 -- 172.21.15.107:0/3515459237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3284109170 con 0x7f32840fe9d0 2023-07-21T22:28:34.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.197+0000 7f3280ff9700 1 -- 172.21.15.107:0/3515459237 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3278004380 con 0x7f32840fe9d0 2023-07-21T22:28:34.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.198+0000 7f3280ff9700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f326c0a1210 0x7f326c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:34.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.198+0000 7f3280ff9700 1 -- 172.21.15.107:0/3515459237 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f32780cc9d0 con 0x7f32840fe9d0 2023-07-21T22:28:34.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.198+0000 7f3282ffd700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f326c0a1210 0x7f326c0a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:34.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.198+0000 7f3282ffd700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f326c0a1210 0x7f326c0a36c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3274005fd0 tx=0x7f3274005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:34.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.201+0000 7f3280ff9700 1 -- 172.21.15.107:0/3515459237 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3278096070 con 0x7f32840fe9d0 2023-07-21T22:28:34.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:34 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2387539875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:28:34.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:34 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2300719904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:28:34.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:34 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2535078538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:28:34.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.432+0000 7f3289d2e700 1 -- 172.21.15.107:0/3515459237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3284105730 con 0x7f32840fe9d0 2023-07-21T22:28:34.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.432+0000 7f3280ff9700 1 -- 172.21.15.107:0/3515459237 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10868 (secure 0 0 0) 0x7f3278099690 con 0x7f32840fe9d0 2023-07-21T22:28:34.478 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:34.478 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:27:49.034257+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:28:34.480 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.434+0000 7f3289d2e700 1 -- 172.21.15.107:0/3515459237 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f326c0a1210 msgr2=0x7f326c0a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.480 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.434+0000 7f3289d2e700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f326c0a1210 0x7f326c0a36c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3274005fd0 tx=0x7f3274005ee0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.480 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.434+0000 7f3289d2e700 1 -- 172.21.15.107:0/3515459237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32840fe9d0 msgr2=0x7f3284197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.480 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.434+0000 7f3289d2e700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32840fe9d0 0x7f3284197a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f327800c970 tx=0x7f327800cd30 comp rx=0 tx=0).stop 2023-07-21T22:28:34.480 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.434+0000 7f3289d2e700 1 -- 172.21.15.107:0/3515459237 shutdown_connections 2023-07-21T22:28:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.434+0000 7f3289d2e700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3284100550 0x7f328419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.434+0000 7f3289d2e700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32840fe9d0 0x7f3284197a70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.434+0000 7f3289d2e700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f326c0a1210 0x7f326c0a36c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.434+0000 7f3289d2e700 1 --2- 172.21.15.107:0/3515459237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32840ffbc0 0x7f3284197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.434+0000 7f3289d2e700 1 -- 172.21.15.107:0/3515459237 >> 172.21.15.107:0/3515459237 conn(0x7f32840f9fd0 msgr2=0x7f32840fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.435+0000 7f3289d2e700 1 -- 172.21.15.107:0/3515459237 shutdown_connections 2023-07-21T22:28:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.435+0000 7f3289d2e700 1 -- 172.21.15.107:0/3515459237 wait complete. 2023-07-21T22:28:34.482 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2023-07-21T22:28:34.493 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:28:34.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.568+0000 7f91365af700 1 -- 172.21.15.107:0/2728923769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9130105790 msgr2=0x7f9130107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.568+0000 7f91365af700 1 --2- 172.21.15.107:0/2728923769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9130105790 0x7f9130107bf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f912800b0a0 tx=0x7f912800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.568+0000 7f91365af700 1 -- 172.21.15.107:0/2728923769 shutdown_connections 2023-07-21T22:28:34.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.568+0000 7f91365af700 1 --2- 172.21.15.107:0/2728923769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9130105790 0x7f9130107bf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.568+0000 7f91365af700 1 --2- 172.21.15.107:0/2728923769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9130100550 0x7f9130102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.568+0000 7f91365af700 1 --2- 172.21.15.107:0/2728923769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9130102e70 0x7f9130105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.568+0000 7f91365af700 1 -- 172.21.15.107:0/2728923769 >> 172.21.15.107:0/2728923769 conn(0x7f91300f9fd0 msgr2=0x7f91300fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:34.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.568+0000 7f91365af700 1 -- 172.21.15.107:0/2728923769 shutdown_connections 2023-07-21T22:28:34.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.568+0000 7f91365af700 1 -- 172.21.15.107:0/2728923769 wait complete. 2023-07-21T22:28:34.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.569+0000 7f91365af700 1 Processor -- start 2023-07-21T22:28:34.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.569+0000 7f91365af700 1 -- start start 2023-07-21T22:28:34.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.569+0000 7f91365af700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9130100550 0x7f91300ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:34.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.569+0000 7f91365af700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9130102e70 0x7f91300fe500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:34.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.569+0000 7f91365af700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9130105790 0x7f91300fea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:34.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.569+0000 7f91365af700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91300ff1e0 con 0x7f9130100550 2023-07-21T22:28:34.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.569+0000 7f91365af700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91300ff350 con 0x7f9130102e70 2023-07-21T22:28:34.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.569+0000 7f91365af700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913019f8c0 con 0x7f9130105790 2023-07-21T22:28:34.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.569+0000 7f912ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9130100550 0x7f91300ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:34.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.569+0000 7f912ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9130100550 0x7f91300ffec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47160/0 (socket says 172.21.15.107:47160) 2023-07-21T22:28:34.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.569+0000 7f912ffff700 1 -- 172.21.15.107:0/2453403257 learned_addr learned my addr 172.21.15.107:0/2453403257 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:34.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.570+0000 7f912f7fe700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9130102e70 0x7f91300fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:34.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.570+0000 7f9134b4c700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9130105790 0x7f91300fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:34.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.570+0000 7f912ffff700 1 -- 172.21.15.107:0/2453403257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9130105790 msgr2=0x7f91300fea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.570+0000 7f912ffff700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9130105790 0x7f91300fea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.570+0000 7f912ffff700 1 -- 172.21.15.107:0/2453403257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9130102e70 msgr2=0x7f91300fe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.570+0000 7f912ffff700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9130102e70 0x7f91300fe500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.570+0000 7f912ffff700 1 -- 172.21.15.107:0/2453403257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9128009d00 con 0x7f9130100550 2023-07-21T22:28:34.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.570+0000 7f912ffff700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9130100550 0x7f91300ffec0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f912000d950 tx=0x7f912000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:34.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.570+0000 7f912d7fa700 1 -- 172.21.15.107:0/2453403257 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120009940 con 0x7f9130100550 2023-07-21T22:28:34.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.570+0000 7f912d7fa700 1 -- 172.21.15.107:0/2453403257 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9120010460 con 0x7f9130100550 2023-07-21T22:28:34.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.570+0000 7f91365af700 1 -- 172.21.15.107:0/2453403257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913019fba0 con 0x7f9130100550 2023-07-21T22:28:34.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.570+0000 7f912d7fa700 1 -- 172.21.15.107:0/2453403257 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912000f620 con 0x7f9130100550 2023-07-21T22:28:34.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.570+0000 7f91365af700 1 -- 172.21.15.107:0/2453403257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91301a00b0 con 0x7f9130100550 2023-07-21T22:28:34.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.571+0000 7f91365af700 1 -- 172.21.15.107:0/2453403257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91300fbb40 con 0x7f9130100550 2023-07-21T22:28:34.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.572+0000 7f912d7fa700 1 -- 172.21.15.107:0/2453403257 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f91200108d0 con 0x7f9130100550 2023-07-21T22:28:34.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.573+0000 7f912d7fa700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91180a1220 0x7f91180a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:34.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.573+0000 7f912d7fa700 1 -- 172.21.15.107:0/2453403257 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f91200cc780 con 0x7f9130100550 2023-07-21T22:28:34.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.573+0000 7f912f7fe700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91180a1220 0x7f91180a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:34.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.573+0000 7f912f7fe700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91180a1220 0x7f91180a36d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9124005950 tx=0x7f912400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:34.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.576+0000 7f912d7fa700 1 -- 172.21.15.107:0/2453403257 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9120095f20 con 0x7f9130100550 2023-07-21T22:28:34.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:34 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2387539875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:28:34.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:34 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2300719904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:28:34.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:34 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2535078538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:28:34.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:34 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2387539875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:28:34.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:34 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2300719904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:28:34.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:34 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2535078538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:28:34.812 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.766+0000 7f91365af700 1 -- 172.21.15.107:0/2453403257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f913002c850 con 0x7f9130100550 2023-07-21T22:28:34.813 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.768+0000 7f912d7fa700 1 -- 172.21.15.107:0/2453403257 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22201 (secure 0 0 0) 0x7f9120016090 con 0x7f9130100550 2023-07-21T22:28:34.814 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:34.814 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:27:43.071566+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:28:34.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.770+0000 7f91365af700 1 -- 172.21.15.107:0/2453403257 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91180a1220 msgr2=0x7f91180a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.770+0000 7f91365af700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91180a1220 0x7f91180a36d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9124005950 tx=0x7f912400b410 comp rx=0 tx=0).stop 2023-07-21T22:28:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.770+0000 7f91365af700 1 -- 172.21.15.107:0/2453403257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9130100550 msgr2=0x7f91300ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.770+0000 7f91365af700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9130100550 0x7f91300ffec0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f912000d950 tx=0x7f912000dc60 comp rx=0 tx=0).stop 2023-07-21T22:28:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.770+0000 7f91365af700 1 -- 172.21.15.107:0/2453403257 shutdown_connections 2023-07-21T22:28:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.770+0000 7f91365af700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91180a1220 0x7f91180a36d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.770+0000 7f91365af700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9130105790 0x7f91300fea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.770+0000 7f91365af700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9130100550 0x7f91300ffec0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.770+0000 7f91365af700 1 --2- 172.21.15.107:0/2453403257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9130102e70 0x7f91300fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.770+0000 7f91365af700 1 -- 172.21.15.107:0/2453403257 >> 172.21.15.107:0/2453403257 conn(0x7f91300f9fd0 msgr2=0x7f91301011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:34.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.770+0000 7f91365af700 1 -- 172.21.15.107:0/2453403257 shutdown_connections 2023-07-21T22:28:34.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.770+0000 7f91365af700 1 -- 172.21.15.107:0/2453403257 wait complete. 2023-07-21T22:28:34.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:23:52.031819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:27:39.494209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:27:40.672303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:28:34.829 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:28:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.907+0000 7efdd49b2700 1 -- 172.21.15.107:0/3360883835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd00ffc10 msgr2=0x7efdd0100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.907+0000 7efdd49b2700 1 --2- 172.21.15.107:0/3360883835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd00ffc10 0x7efdd0100060 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009990 tx=0x7efdc0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.907+0000 7efdd49b2700 1 -- 172.21.15.107:0/3360883835 shutdown_connections 2023-07-21T22:28:34.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.907+0000 7efdd49b2700 1 --2- 172.21.15.107:0/3360883835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd01005a0 0x7efdd0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.907+0000 7efdd49b2700 1 --2- 172.21.15.107:0/3360883835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd00fea20 0x7efdd00fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.907+0000 7efdd49b2700 1 --2- 172.21.15.107:0/3360883835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd00ffc10 0x7efdd0100060 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.907+0000 7efdd49b2700 1 -- 172.21.15.107:0/3360883835 >> 172.21.15.107:0/3360883835 conn(0x7efdd00fa020 msgr2=0x7efdd00fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:34.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.907+0000 7efdd49b2700 1 -- 172.21.15.107:0/3360883835 shutdown_connections 2023-07-21T22:28:34.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.907+0000 7efdd49b2700 1 -- 172.21.15.107:0/3360883835 wait complete. 2023-07-21T22:28:34.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdd49b2700 1 Processor -- start 2023-07-21T22:28:34.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdd49b2700 1 -- start start 2023-07-21T22:28:34.956 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdd49b2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd00fea20 0x7efdd00693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:34.957 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdd49b2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd00ffc10 0x7efdd019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:34.957 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdd49b2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd01005a0 0x7efdd019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:34.957 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdd49b2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd019d3e0 con 0x7efdd00ffc10 2023-07-21T22:28:34.957 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdd49b2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd019d550 con 0x7efdd00fea20 2023-07-21T22:28:34.957 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdd49b2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd019d710 con 0x7efdd01005a0 2023-07-21T22:28:34.957 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdcdd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd00ffc10 0x7efdd019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:34.957 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdcdd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd00ffc10 0x7efdd019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47176/0 (socket says 172.21.15.107:47176) 2023-07-21T22:28:34.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdcdd9b700 1 -- 172.21.15.107:0/3636968322 learned_addr learned my addr 172.21.15.107:0/3636968322 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:34.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdced9d700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd01005a0 0x7efdd019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:34.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdce59c700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd00fea20 0x7efdd00693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:34.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdcdd9b700 1 -- 172.21.15.107:0/3636968322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd01005a0 msgr2=0x7efdd019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdcdd9b700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd01005a0 0x7efdd019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdcdd9b700 1 -- 172.21.15.107:0/3636968322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd00fea20 msgr2=0x7efdd00693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:34.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdcdd9b700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd00fea20 0x7efdd00693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:34.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdcdd9b700 1 -- 172.21.15.107:0/3636968322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc00095f0 con 0x7efdd00ffc10 2023-07-21T22:28:34.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.908+0000 7efdcdd9b700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd00ffc10 0x7efdd019a5c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efdc0004760 tx=0x7efdc0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:34.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.909+0000 7efdc77fe700 1 -- 172.21.15.107:0/3636968322 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc001d070 con 0x7efdd00ffc10 2023-07-21T22:28:34.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.909+0000 7efdc77fe700 1 -- 172.21.15.107:0/3636968322 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdc0004cf0 con 0x7efdd00ffc10 2023-07-21T22:28:34.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.909+0000 7efdd49b2700 1 -- 172.21.15.107:0/3636968322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd019d990 con 0x7efdd00ffc10 2023-07-21T22:28:34.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.909+0000 7efdc77fe700 1 -- 172.21.15.107:0/3636968322 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc0005280 con 0x7efdd00ffc10 2023-07-21T22:28:34.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.909+0000 7efdd49b2700 1 -- 172.21.15.107:0/3636968322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd019de60 con 0x7efdd00ffc10 2023-07-21T22:28:34.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.909+0000 7efdd49b2700 1 -- 172.21.15.107:0/3636968322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd0105f70 con 0x7efdd00ffc10 2023-07-21T22:28:34.960 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.911+0000 7efdc77fe700 1 -- 172.21.15.107:0/3636968322 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7efdc00053e0 con 0x7efdd00ffc10 2023-07-21T22:28:34.960 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.911+0000 7efdc77fe700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efdbc0a11e0 0x7efdbc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:34.960 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.911+0000 7efdce59c700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efdbc0a11e0 0x7efdbc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:34.960 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.911+0000 7efdc77fe700 1 -- 172.21.15.107:0/3636968322 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efdc00cde70 con 0x7efdd00ffc10 2023-07-21T22:28:34.961 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.912+0000 7efdce59c700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efdbc0a11e0 0x7efdbc0a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efdb8005950 tx=0x7efdb80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:34.961 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:34.914+0000 7efdc77fe700 1 -- 172.21.15.107:0/3636968322 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7efdc00976a0 con 0x7efdd00ffc10 2023-07-21T22:28:35.147 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.102+0000 7efdd49b2700 1 -- 172.21.15.107:0/3636968322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efdd010a2e0 con 0x7efdd00ffc10 2023-07-21T22:28:35.148 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.102+0000 7efdc77fe700 1 -- 172.21.15.107:0/3636968322 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7efdc009acc0 con 0x7efdd00ffc10 2023-07-21T22:28:35.148 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2023-07-21T22:28:35.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.104+0000 7efdd49b2700 1 -- 172.21.15.107:0/3636968322 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efdbc0a11e0 msgr2=0x7efdbc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.104+0000 7efdd49b2700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efdbc0a11e0 0x7efdbc0a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efdb8005950 tx=0x7efdb80058e0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.104+0000 7efdd49b2700 1 -- 172.21.15.107:0/3636968322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd00ffc10 msgr2=0x7efdd019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.104+0000 7efdd49b2700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd00ffc10 0x7efdd019a5c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efdc0004760 tx=0x7efdc0004790 comp rx=0 tx=0).stop 2023-07-21T22:28:35.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.104+0000 7efdd49b2700 1 -- 172.21.15.107:0/3636968322 shutdown_connections 2023-07-21T22:28:35.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.104+0000 7efdd49b2700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efdbc0a11e0 0x7efdbc0a3690 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.104+0000 7efdd49b2700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdd01005a0 0x7efdd019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.104+0000 7efdd49b2700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdd00fea20 0x7efdd00693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.104+0000 7efdd49b2700 1 --2- 172.21.15.107:0/3636968322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdd00ffc10 0x7efdd019a5c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.104+0000 7efdd49b2700 1 -- 172.21.15.107:0/3636968322 >> 172.21.15.107:0/3636968322 conn(0x7efdd00fa020 msgr2=0x7efdd01030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:35.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.104+0000 7efdd49b2700 1 -- 172.21.15.107:0/3636968322 shutdown_connections 2023-07-21T22:28:35.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.104+0000 7efdd49b2700 1 -- 172.21.15.107:0/3636968322 wait complete. 2023-07-21T22:28:35.163 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:28:35.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.243+0000 7ff6c6700700 1 -- 172.21.15.107:0/3840026489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6c0100cd0 msgr2=0x7ff6c0101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.243+0000 7ff6c6700700 1 --2- 172.21.15.107:0/3840026489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6c0100cd0 0x7ff6c0101120 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8009990 tx=0x7ff6a8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.243+0000 7ff6c6700700 1 -- 172.21.15.107:0/3840026489 shutdown_connections 2023-07-21T22:28:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.243+0000 7ff6c6700700 1 --2- 172.21.15.107:0/3840026489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6c0101660 0x7ff6c0107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.243+0000 7ff6c6700700 1 --2- 172.21.15.107:0/3840026489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6c00ffae0 0x7ff6c00ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.243+0000 7ff6c6700700 1 --2- 172.21.15.107:0/3840026489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6c0100cd0 0x7ff6c0101120 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.243+0000 7ff6c6700700 1 -- 172.21.15.107:0/3840026489 >> 172.21.15.107:0/3840026489 conn(0x7ff6c00fb120 msgr2=0x7ff6c00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.243+0000 7ff6c6700700 1 -- 172.21.15.107:0/3840026489 shutdown_connections 2023-07-21T22:28:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.243+0000 7ff6c6700700 1 -- 172.21.15.107:0/3840026489 wait complete. 2023-07-21T22:28:35.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.243+0000 7ff6c6700700 1 Processor -- start 2023-07-21T22:28:35.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.243+0000 7ff6c6700700 1 -- start start 2023-07-21T22:28:35.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6c6700700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6c00ffae0 0x7ff6c0195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:35.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6c6700700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6c0100cd0 0x7ff6c0195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:35.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6c6700700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6c0101660 0x7ff6c019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:35.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6c6700700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c019d3a0 con 0x7ff6c0100cd0 2023-07-21T22:28:35.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6c6700700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c019d4e0 con 0x7ff6c0101660 2023-07-21T22:28:35.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6c6700700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c019d670 con 0x7ff6c00ffae0 2023-07-21T22:28:35.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6bf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6c0100cd0 0x7ff6c0195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:35.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6bf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6c0100cd0 0x7ff6c0195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47188/0 (socket says 172.21.15.107:47188) 2023-07-21T22:28:35.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6bf7fe700 1 -- 172.21.15.107:0/2558707986 learned_addr learned my addr 172.21.15.107:0/2558707986 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:35.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6bffff700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6c00ffae0 0x7ff6c0195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:35.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6c4c9d700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6c0101660 0x7ff6c019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:35.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6bf7fe700 1 -- 172.21.15.107:0/2558707986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6c00ffae0 msgr2=0x7ff6c0195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6bf7fe700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6c00ffae0 0x7ff6c0195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6bf7fe700 1 -- 172.21.15.107:0/2558707986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6c0101660 msgr2=0x7ff6c019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6bf7fe700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6c0101660 0x7ff6c019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6bf7fe700 1 -- 172.21.15.107:0/2558707986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a80095f0 con 0x7ff6c0100cd0 2023-07-21T22:28:35.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6bf7fe700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6c0100cd0 0x7ff6c0195d60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8000c00 tx=0x7ff6a80047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:35.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6bd7fa700 1 -- 172.21.15.107:0/2558707986 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a801d070 con 0x7ff6c0100cd0 2023-07-21T22:28:35.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6bd7fa700 1 -- 172.21.15.107:0/2558707986 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6a8022470 con 0x7ff6c0100cd0 2023-07-21T22:28:35.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6c6700700 1 -- 172.21.15.107:0/2558707986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c019d8f0 con 0x7ff6c0100cd0 2023-07-21T22:28:35.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6bd7fa700 1 -- 172.21.15.107:0/2558707986 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8017650 con 0x7ff6c0100cd0 2023-07-21T22:28:35.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.244+0000 7ff6c6700700 1 -- 172.21.15.107:0/2558707986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c019dd70 con 0x7ff6c0100cd0 2023-07-21T22:28:35.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.245+0000 7ff6c6700700 1 -- 172.21.15.107:0/2558707986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c0106a10 con 0x7ff6c0100cd0 2023-07-21T22:28:35.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.247+0000 7ff6bd7fa700 1 -- 172.21.15.107:0/2558707986 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff6a80225e0 con 0x7ff6c0100cd0 2023-07-21T22:28:35.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.247+0000 7ff6bd7fa700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff6ac0a1260 0x7ff6ac0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:35.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.247+0000 7ff6bffff700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff6ac0a1260 0x7ff6ac0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:35.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.247+0000 7ff6bd7fa700 1 -- 172.21.15.107:0/2558707986 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff6a80ce180 con 0x7ff6c0100cd0 2023-07-21T22:28:35.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.248+0000 7ff6bffff700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff6ac0a1260 0x7ff6ac0a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0100b30 tx=0x7ff6b000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:35.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.251+0000 7ff6bd7fa700 1 -- 172.21.15.107:0/2558707986 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff6a80979b0 con 0x7ff6c0100cd0 2023-07-21T22:28:35.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:35 smithi107 ceph-mon[96543]: pgmap v276: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:28:35.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:35 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3515459237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:35.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:35 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2453403257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:35.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:35 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3636968322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:28:35.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.440+0000 7ff6c6700700 1 -- 172.21.15.107:0/2558707986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff6c002c9e0 con 0x7ff6c0100cd0 2023-07-21T22:28:35.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.440+0000 7ff6bd7fa700 1 -- 172.21.15.107:0/2558707986 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6a809afd0 con 0x7ff6c0100cd0 2023-07-21T22:28:35.486 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2023-07-21T22:28:35.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.442+0000 7ff6c6700700 1 -- 172.21.15.107:0/2558707986 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff6ac0a1260 msgr2=0x7ff6ac0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.442+0000 7ff6c6700700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff6ac0a1260 0x7ff6ac0a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0100b30 tx=0x7ff6b000b410 comp rx=0 tx=0).stop 2023-07-21T22:28:35.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.442+0000 7ff6c6700700 1 -- 172.21.15.107:0/2558707986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6c0100cd0 msgr2=0x7ff6c0195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.442+0000 7ff6c6700700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6c0100cd0 0x7ff6c0195d60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8000c00 tx=0x7ff6a80047a0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.443+0000 7ff6c6700700 1 -- 172.21.15.107:0/2558707986 shutdown_connections 2023-07-21T22:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.443+0000 7ff6c6700700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6c0101660 0x7ff6c019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.443+0000 7ff6c6700700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff6ac0a1260 0x7ff6ac0a3710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.443+0000 7ff6c6700700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6c00ffae0 0x7ff6c0195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.443+0000 7ff6c6700700 1 --2- 172.21.15.107:0/2558707986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6c0100cd0 0x7ff6c0195d60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.443+0000 7ff6c6700700 1 -- 172.21.15.107:0/2558707986 >> 172.21.15.107:0/2558707986 conn(0x7ff6c00fb120 msgr2=0x7ff6c00fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.443+0000 7ff6c6700700 1 -- 172.21.15.107:0/2558707986 shutdown_connections 2023-07-21T22:28:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.443+0000 7ff6c6700700 1 -- 172.21.15.107:0/2558707986 wait complete. 2023-07-21T22:28:35.501 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:28:35.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.580+0000 7f975329c700 1 -- 172.21.15.107:0/1446276705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974c0ff5c0 msgr2=0x7f974c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.580+0000 7f975329c700 1 --2- 172.21.15.107:0/1446276705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974c0ff5c0 0x7f974c106e70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f974400b0a0 tx=0x7f974400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.580+0000 7f975329c700 1 -- 172.21.15.107:0/1446276705 shutdown_connections 2023-07-21T22:28:35.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.580+0000 7f975329c700 1 --2- 172.21.15.107:0/1446276705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974c0ff5c0 0x7f974c106e70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.580+0000 7f975329c700 1 --2- 172.21.15.107:0/1446276705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f974c100050 0x7f974c100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.580+0000 7f975329c700 1 --2- 172.21.15.107:0/1446276705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c0fec10 0x7f974c0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.580+0000 7f975329c700 1 -- 172.21.15.107:0/1446276705 >> 172.21.15.107:0/1446276705 conn(0x7f974c0f9770 msgr2=0x7f974c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:35.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.580+0000 7f975329c700 1 -- 172.21.15.107:0/1446276705 shutdown_connections 2023-07-21T22:28:35.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.580+0000 7f975329c700 1 -- 172.21.15.107:0/1446276705 wait complete. 2023-07-21T22:28:35.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f975329c700 1 Processor -- start 2023-07-21T22:28:35.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f975329c700 1 -- start start 2023-07-21T22:28:35.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f975329c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974c0fec10 0x7f974c0fe690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:35.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f975329c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f974c0ff5c0 0x7f974c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:35.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f975329c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c100050 0x7f974c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:35.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f975329c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c19d430 con 0x7f974c0fec10 2023-07-21T22:28:35.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f975329c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c19d5a0 con 0x7f974c0ff5c0 2023-07-21T22:28:35.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f975329c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c19d760 con 0x7f974c100050 2023-07-21T22:28:35.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f9751038700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974c0fec10 0x7f974c0fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:35.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f9751038700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974c0fec10 0x7f974c0fe690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47200/0 (socket says 172.21.15.107:47200) 2023-07-21T22:28:35.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f9751038700 1 -- 172.21.15.107:0/806291567 learned_addr learned my addr 172.21.15.107:0/806291567 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:35.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f9750837700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f974c0ff5c0 0x7f974c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:35.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f9751839700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c100050 0x7f974c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:35.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f9751038700 1 -- 172.21.15.107:0/806291567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c100050 msgr2=0x7f974c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.581+0000 7f9751038700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c100050 0x7f974c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.582+0000 7f9751038700 1 -- 172.21.15.107:0/806291567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f974c0ff5c0 msgr2=0x7f974c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.582+0000 7f9751038700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f974c0ff5c0 0x7f974c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.582+0000 7f9751038700 1 -- 172.21.15.107:0/806291567 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9744009d00 con 0x7f974c0fec10 2023-07-21T22:28:35.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.582+0000 7f9751038700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974c0fec10 0x7f974c0fe690 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f974800e9a0 tx=0x7f974800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:35.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.582+0000 7f97427fc700 1 -- 172.21.15.107:0/806291567 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748009790 con 0x7f974c0fec10 2023-07-21T22:28:35.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.582+0000 7f97427fc700 1 -- 172.21.15.107:0/806291567 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9748010430 con 0x7f974c0fec10 2023-07-21T22:28:35.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.582+0000 7f975329c700 1 -- 172.21.15.107:0/806291567 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c19da40 con 0x7f974c0fec10 2023-07-21T22:28:35.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.582+0000 7f97427fc700 1 -- 172.21.15.107:0/806291567 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748018680 con 0x7f974c0fec10 2023-07-21T22:28:35.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.582+0000 7f975329c700 1 -- 172.21.15.107:0/806291567 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c19df50 con 0x7f974c0fec10 2023-07-21T22:28:35.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.583+0000 7f975329c700 1 -- 172.21.15.107:0/806291567 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c105960 con 0x7f974c0fec10 2023-07-21T22:28:35.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.584+0000 7f97427fc700 1 -- 172.21.15.107:0/806291567 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f97480105a0 con 0x7f974c0fec10 2023-07-21T22:28:35.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.584+0000 7f97427fc700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f97380a11e0 0x7f97380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:35.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.584+0000 7f9750837700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f97380a11e0 0x7f97380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:35.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.584+0000 7f97427fc700 1 -- 172.21.15.107:0/806291567 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9748014070 con 0x7f974c0fec10 2023-07-21T22:28:35.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.585+0000 7f9750837700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f97380a11e0 0x7f97380a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f974c0fe400 tx=0x7f973c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:35.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.588+0000 7f97427fc700 1 -- 172.21.15.107:0/806291567 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9748051770 con 0x7f974c0fec10 2023-07-21T22:28:35.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:35 smithi190 ceph-mon[98334]: pgmap v276: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:28:35.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:35 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3515459237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:35.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:35 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2453403257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:35.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:35 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3636968322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:28:35.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:35 smithi132 ceph-mon[97863]: pgmap v276: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:28:35.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:35 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3515459237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:35.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:35 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2453403257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:35.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:35 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3636968322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:28:35.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.777+0000 7f975329c700 1 -- 172.21.15.107:0/806291567 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f974c02ca70 con 0x7f974c0fec10 2023-07-21T22:28:35.823 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.777+0000 7f97427fc700 1 -- 172.21.15.107:0/806291567 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9748096250 con 0x7f974c0fec10 2023-07-21T22:28:35.823 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2023-07-21T22:28:35.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.779+0000 7f975329c700 1 -- 172.21.15.107:0/806291567 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f97380a11e0 msgr2=0x7f97380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.779+0000 7f975329c700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f97380a11e0 0x7f97380a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f974c0fe400 tx=0x7f973c009260 comp rx=0 tx=0).stop 2023-07-21T22:28:35.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.779+0000 7f975329c700 1 -- 172.21.15.107:0/806291567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974c0fec10 msgr2=0x7f974c0fe690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.779+0000 7f975329c700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974c0fec10 0x7f974c0fe690 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f974800e9a0 tx=0x7f974800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.779+0000 7f975329c700 1 -- 172.21.15.107:0/806291567 shutdown_connections 2023-07-21T22:28:35.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.779+0000 7f975329c700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f97380a11e0 0x7f97380a3690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.779+0000 7f975329c700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f974c100050 0x7f974c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.779+0000 7f975329c700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974c0fec10 0x7f974c0fe690 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.779+0000 7f975329c700 1 --2- 172.21.15.107:0/806291567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f974c0ff5c0 0x7f974c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.779+0000 7f975329c700 1 -- 172.21.15.107:0/806291567 >> 172.21.15.107:0/806291567 conn(0x7f974c0f9770 msgr2=0x7f974c0fb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:35.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.779+0000 7f975329c700 1 -- 172.21.15.107:0/806291567 shutdown_connections 2023-07-21T22:28:35.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.779+0000 7f975329c700 1 -- 172.21.15.107:0/806291567 wait complete. 2023-07-21T22:28:35.837 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:28:35.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.912+0000 7f10ec8ea700 1 -- 172.21.15.107:0/1113627472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10e8100050 msgr2=0x7f10e8100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.912+0000 7f10ec8ea700 1 --2- 172.21.15.107:0/1113627472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10e8100050 0x7f10e8100460 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009990 tx=0x7f10d8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.912+0000 7f10ec8ea700 1 -- 172.21.15.107:0/1113627472 shutdown_connections 2023-07-21T22:28:35.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.912+0000 7f10ec8ea700 1 --2- 172.21.15.107:0/1113627472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10e80ff5c0 0x7f10e8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.912+0000 7f10ec8ea700 1 --2- 172.21.15.107:0/1113627472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10e8100050 0x7f10e8100460 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.912+0000 7f10ec8ea700 1 --2- 172.21.15.107:0/1113627472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10e80fec10 0x7f10e80ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.912+0000 7f10ec8ea700 1 -- 172.21.15.107:0/1113627472 >> 172.21.15.107:0/1113627472 conn(0x7f10e80f9770 msgr2=0x7f10e80fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:35.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.912+0000 7f10ec8ea700 1 -- 172.21.15.107:0/1113627472 shutdown_connections 2023-07-21T22:28:35.960 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.912+0000 7f10ec8ea700 1 -- 172.21.15.107:0/1113627472 wait complete. 2023-07-21T22:28:35.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.913+0000 7f10ec8ea700 1 Processor -- start 2023-07-21T22:28:35.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.913+0000 7f10ec8ea700 1 -- start start 2023-07-21T22:28:35.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.913+0000 7f10ec8ea700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10e80fec10 0x7f10e8195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:35.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.913+0000 7f10ec8ea700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10e80ff5c0 0x7f10e8195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:35.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.913+0000 7f10ec8ea700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10e8100050 0x7f10e819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:35.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.913+0000 7f10ec8ea700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e819d310 con 0x7f10e80fec10 2023-07-21T22:28:35.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.913+0000 7f10ec8ea700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e819d450 con 0x7f10e8100050 2023-07-21T22:28:35.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.913+0000 7f10ec8ea700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e819d5e0 con 0x7f10e80ff5c0 2023-07-21T22:28:35.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.913+0000 7f10e659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10e80fec10 0x7f10e8195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:35.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.913+0000 7f10e659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10e80fec10 0x7f10e8195790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47214/0 (socket says 172.21.15.107:47214) 2023-07-21T22:28:35.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.913+0000 7f10e659c700 1 -- 172.21.15.107:0/3213844770 learned_addr learned my addr 172.21.15.107:0/3213844770 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:35.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10e6d9d700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10e8100050 0x7f10e819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:35.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10e659c700 1 -- 172.21.15.107:0/3213844770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10e80ff5c0 msgr2=0x7f10e8195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10e5d9b700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10e80ff5c0 0x7f10e8195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:35.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10e659c700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10e80ff5c0 0x7f10e8195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10e659c700 1 -- 172.21.15.107:0/3213844770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10e8100050 msgr2=0x7f10e819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:35.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10e659c700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10e8100050 0x7f10e819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:35.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10e659c700 1 -- 172.21.15.107:0/3213844770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d80095f0 con 0x7f10e80fec10 2023-07-21T22:28:35.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10e6d9d700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10e8100050 0x7f10e819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:35.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10e659c700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10e80fec10 0x7f10e8195790 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f10d8000c00 tx=0x7f10d800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:35.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10d77fe700 1 -- 172.21.15.107:0/3213844770 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d801d070 con 0x7f10e80fec10 2023-07-21T22:28:35.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10ec8ea700 1 -- 172.21.15.107:0/3213844770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e819d860 con 0x7f10e80fec10 2023-07-21T22:28:35.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10d77fe700 1 -- 172.21.15.107:0/3213844770 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10d800fc00 con 0x7f10e80fec10 2023-07-21T22:28:35.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10d77fe700 1 -- 172.21.15.107:0/3213844770 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d8017650 con 0x7f10e80fec10 2023-07-21T22:28:35.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.914+0000 7f10ec8ea700 1 -- 172.21.15.107:0/3213844770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e819dd60 con 0x7f10e80fec10 2023-07-21T22:28:35.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.915+0000 7f10ec8ea700 1 -- 172.21.15.107:0/3213844770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e81059b0 con 0x7f10e80fec10 2023-07-21T22:28:35.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.916+0000 7f10d77fe700 1 -- 172.21.15.107:0/3213844770 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f10d8022470 con 0x7f10e80fec10 2023-07-21T22:28:35.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.917+0000 7f10d77fe700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10d00a12a0 0x7f10d00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:35.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.917+0000 7f10e5d9b700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10d00a12a0 0x7f10d00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:35.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.917+0000 7f10d77fe700 1 -- 172.21.15.107:0/3213844770 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f10d80cdf60 con 0x7f10e80fec10 2023-07-21T22:28:35.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.917+0000 7f10e5d9b700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10d00a12a0 0x7f10d00a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10dc005950 tx=0x7f10dc005cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:35.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:35.920+0000 7f10d77fe700 1 -- 172.21.15.107:0/3213844770 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f10d8097690 con 0x7f10e80fec10 2023-07-21T22:28:36.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.153+0000 7f10ec8ea700 1 -- 172.21.15.107:0/3213844770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10e802c850 con 0x7f10e80fec10 2023-07-21T22:28:36.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.154+0000 7f10d77fe700 1 -- 172.21.15.107:0/3213844770 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10868 (secure 0 0 0) 0x7f10d809acb0 con 0x7f10e80fec10 2023-07-21T22:28:36.200 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:36.200 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:27:49.034257+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:28:36.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.156+0000 7f10ec8ea700 1 -- 172.21.15.107:0/3213844770 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10d00a12a0 msgr2=0x7f10d00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:36.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.156+0000 7f10ec8ea700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10d00a12a0 0x7f10d00a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10dc005950 tx=0x7f10dc005cf0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.156+0000 7f10ec8ea700 1 -- 172.21.15.107:0/3213844770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10e80fec10 msgr2=0x7f10e8195790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:36.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.156+0000 7f10ec8ea700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10e80fec10 0x7f10e8195790 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f10d8000c00 tx=0x7f10d800f710 comp rx=0 tx=0).stop 2023-07-21T22:28:36.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.156+0000 7f10ec8ea700 1 -- 172.21.15.107:0/3213844770 shutdown_connections 2023-07-21T22:28:36.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.156+0000 7f10ec8ea700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10e8100050 0x7f10e819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.156+0000 7f10ec8ea700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10e80fec10 0x7f10e8195790 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.156+0000 7f10ec8ea700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10d00a12a0 0x7f10d00a3750 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.156+0000 7f10ec8ea700 1 --2- 172.21.15.107:0/3213844770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10e80ff5c0 0x7f10e8195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.156+0000 7f10ec8ea700 1 -- 172.21.15.107:0/3213844770 >> 172.21.15.107:0/3213844770 conn(0x7f10e80f9770 msgr2=0x7f10e80fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.156+0000 7f10ec8ea700 1 -- 172.21.15.107:0/3213844770 shutdown_connections 2023-07-21T22:28:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.156+0000 7f10ec8ea700 1 -- 172.21.15.107:0/3213844770 wait complete. 2023-07-21T22:28:36.205 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2023-07-21T22:28:36.214 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:28:36.338 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.290+0000 7f30a43e4700 1 -- 172.21.15.107:0/1047769399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f309c100550 msgr2=0x7f309c106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:36.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.290+0000 7f30a43e4700 1 --2- 172.21.15.107:0/1047769399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f309c100550 0x7f309c106e10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f309400b0a0 tx=0x7f309400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.290+0000 7f30a43e4700 1 -- 172.21.15.107:0/1047769399 shutdown_connections 2023-07-21T22:28:36.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.290+0000 7f30a43e4700 1 --2- 172.21.15.107:0/1047769399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f309c100550 0x7f309c106e10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.290+0000 7f30a43e4700 1 --2- 172.21.15.107:0/1047769399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f309c0fe9d0 0x7f309c0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.290+0000 7f30a43e4700 1 --2- 172.21.15.107:0/1047769399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f309c0ffbc0 0x7f309c100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.290+0000 7f30a43e4700 1 -- 172.21.15.107:0/1047769399 >> 172.21.15.107:0/1047769399 conn(0x7f309c0f9fd0 msgr2=0x7f309c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:36.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.290+0000 7f30a43e4700 1 -- 172.21.15.107:0/1047769399 shutdown_connections 2023-07-21T22:28:36.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.290+0000 7f30a43e4700 1 -- 172.21.15.107:0/1047769399 wait complete. 2023-07-21T22:28:36.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.291+0000 7f30a43e4700 1 Processor -- start 2023-07-21T22:28:36.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.291+0000 7f30a43e4700 1 -- start start 2023-07-21T22:28:36.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.291+0000 7f30a43e4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f309c0fe9d0 0x7f309c1947a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:36.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.291+0000 7f30a43e4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f309c0ffbc0 0x7f309c194ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:36.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.291+0000 7f30a43e4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f309c100550 0x7f309c19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:36.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.291+0000 7f30a43e4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c19c320 con 0x7f309c100550 2023-07-21T22:28:36.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.291+0000 7f30a43e4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c19c460 con 0x7f309c0ffbc0 2023-07-21T22:28:36.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.291+0000 7f30a43e4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c19c5f0 con 0x7f309c0fe9d0 2023-07-21T22:28:36.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.291+0000 7f30a2981700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f309c100550 0x7f309c19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:36.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.291+0000 7f30a2981700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f309c100550 0x7f309c19bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47232/0 (socket says 172.21.15.107:47232) 2023-07-21T22:28:36.342 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.291+0000 7f30a2981700 1 -- 172.21.15.107:0/3782659529 learned_addr learned my addr 172.21.15.107:0/3782659529 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:36.342 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.291+0000 7f30a197f700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f309c0ffbc0 0x7f309c194ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:36.342 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.292+0000 7f30a2180700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f309c0fe9d0 0x7f309c1947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:36.342 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.292+0000 7f30a2981700 1 -- 172.21.15.107:0/3782659529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f309c0fe9d0 msgr2=0x7f309c1947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:36.342 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.292+0000 7f30a2981700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f309c0fe9d0 0x7f309c1947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.342 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.292+0000 7f30a2981700 1 -- 172.21.15.107:0/3782659529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f309c0ffbc0 msgr2=0x7f309c194ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:36.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.292+0000 7f30a2981700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f309c0ffbc0 0x7f309c194ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.292+0000 7f30a2981700 1 -- 172.21.15.107:0/3782659529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3094009d00 con 0x7f309c100550 2023-07-21T22:28:36.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.292+0000 7f30a2180700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f309c0fe9d0 0x7f309c1947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:36.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.292+0000 7f30a2981700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f309c100550 0x7f309c19bd50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f309400b830 tx=0x7f3094004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:36.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.292+0000 7f30937fe700 1 -- 172.21.15.107:0/3782659529 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309400e040 con 0x7f309c100550 2023-07-21T22:28:36.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.292+0000 7f30937fe700 1 -- 172.21.15.107:0/3782659529 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30940094c0 con 0x7f309c100550 2023-07-21T22:28:36.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.292+0000 7f30a43e4700 1 -- 172.21.15.107:0/3782659529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c19c870 con 0x7f309c100550 2023-07-21T22:28:36.344 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.292+0000 7f30937fe700 1 -- 172.21.15.107:0/3782659529 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309401bc70 con 0x7f309c100550 2023-07-21T22:28:36.344 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.292+0000 7f30a43e4700 1 -- 172.21.15.107:0/3782659529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c19ccf0 con 0x7f309c100550 2023-07-21T22:28:36.344 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.293+0000 7f30a43e4700 1 -- 172.21.15.107:0/3782659529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c10a1b0 con 0x7f309c100550 2023-07-21T22:28:36.345 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.294+0000 7f30937fe700 1 -- 172.21.15.107:0/3782659529 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f309401d890 con 0x7f309c100550 2023-07-21T22:28:36.345 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.295+0000 7f30937fe700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30880a1220 0x7f30880a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:36.346 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.295+0000 7f30a2180700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30880a1220 0x7f30880a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:36.346 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.295+0000 7f30937fe700 1 -- 172.21.15.107:0/3782659529 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f30940ced20 con 0x7f309c100550 2023-07-21T22:28:36.346 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.295+0000 7f30a2180700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30880a1220 0x7f30880a36d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f308c005fd0 tx=0x7f308c005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:36.346 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.298+0000 7f30937fe700 1 -- 172.21.15.107:0/3782659529 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3094098450 con 0x7f309c100550 2023-07-21T22:28:36.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.489+0000 7f30a43e4700 1 -- 172.21.15.107:0/3782659529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f309c1057f0 con 0x7f309c100550 2023-07-21T22:28:36.536 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.490+0000 7f30937fe700 1 -- 172.21.15.107:0/3782659529 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22201 (secure 0 0 0) 0x7f3094021090 con 0x7f309c100550 2023-07-21T22:28:36.537 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:36.537 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:27:43.071566+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:28:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.493+0000 7f30a43e4700 1 -- 172.21.15.107:0/3782659529 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30880a1220 msgr2=0x7f30880a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.493+0000 7f30a43e4700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30880a1220 0x7f30880a36d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f308c005fd0 tx=0x7f308c005dc0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.493+0000 7f30a43e4700 1 -- 172.21.15.107:0/3782659529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f309c100550 msgr2=0x7f309c19bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:36.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.493+0000 7f30a43e4700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f309c100550 0x7f309c19bd50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f309400b830 tx=0x7f3094004d30 comp rx=0 tx=0).stop 2023-07-21T22:28:36.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.493+0000 7f30a43e4700 1 -- 172.21.15.107:0/3782659529 shutdown_connections 2023-07-21T22:28:36.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.493+0000 7f30a43e4700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f309c100550 0x7f309c19bd50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.493+0000 7f30a43e4700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30880a1220 0x7f30880a36d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.493+0000 7f30a43e4700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f309c0fe9d0 0x7f309c1947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.493+0000 7f30a43e4700 1 --2- 172.21.15.107:0/3782659529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f309c0ffbc0 0x7f309c194ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.493+0000 7f30a43e4700 1 -- 172.21.15.107:0/3782659529 >> 172.21.15.107:0/3782659529 conn(0x7f309c0f9fd0 msgr2=0x7f309c0fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:36.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.493+0000 7f30a43e4700 1 -- 172.21.15.107:0/3782659529 shutdown_connections 2023-07-21T22:28:36.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.493+0000 7f30a43e4700 1 -- 172.21.15.107:0/3782659529 wait complete. 2023-07-21T22:28:36.549 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:36 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2558707986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:28:36.550 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:36 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/806291567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:28:36.551 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:28:36.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:36 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2558707986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:28:36.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:36 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/806291567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:28:36.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.626+0000 7f685080c700 1 -- 172.21.15.107:0/2455972609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f684c100e20 msgr2=0x7f684c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:36.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.626+0000 7f685080c700 1 --2- 172.21.15.107:0/2455972609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f684c100e20 0x7f684c101270 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f683c009990 tx=0x7f683c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.626+0000 7f685080c700 1 -- 172.21.15.107:0/2455972609 shutdown_connections 2023-07-21T22:28:36.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.626+0000 7f685080c700 1 --2- 172.21.15.107:0/2455972609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f684c1017b0 0x7f684c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.626+0000 7f685080c700 1 --2- 172.21.15.107:0/2455972609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f684c0fddb0 0x7f684c0fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.626+0000 7f685080c700 1 --2- 172.21.15.107:0/2455972609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f684c100e20 0x7f684c101270 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.626+0000 7f685080c700 1 -- 172.21.15.107:0/2455972609 >> 172.21.15.107:0/2455972609 conn(0x7f684c0f9770 msgr2=0x7f684c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:36.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.626+0000 7f685080c700 1 -- 172.21.15.107:0/2455972609 shutdown_connections 2023-07-21T22:28:36.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.626+0000 7f685080c700 1 -- 172.21.15.107:0/2455972609 wait complete. 2023-07-21T22:28:36.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.627+0000 7f685080c700 1 Processor -- start 2023-07-21T22:28:36.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.627+0000 7f685080c700 1 -- start start 2023-07-21T22:28:36.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.627+0000 7f685080c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f684c0fddb0 0x7f684c195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:36.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.627+0000 7f685080c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f684c100e20 0x7f684c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:36.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.627+0000 7f685080c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f684c1017b0 0x7f684c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:36.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.627+0000 7f685080c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c19d3f0 con 0x7f684c1017b0 2023-07-21T22:28:36.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.627+0000 7f685080c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c19d530 con 0x7f684c0fddb0 2023-07-21T22:28:36.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.627+0000 7f685080c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c19d6c0 con 0x7f684c100e20 2023-07-21T22:28:36.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.627+0000 7f684ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f684c1017b0 0x7f684c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:36.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.627+0000 7f684ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f684c1017b0 0x7f684c19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47244/0 (socket says 172.21.15.107:47244) 2023-07-21T22:28:36.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.627+0000 7f684ad9d700 1 -- 172.21.15.107:0/507631016 learned_addr learned my addr 172.21.15.107:0/507631016 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:36.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.628+0000 7f684a59c700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f684c0fddb0 0x7f684c195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:36.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.628+0000 7f6849d9b700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f684c100e20 0x7f684c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:36.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.628+0000 7f684ad9d700 1 -- 172.21.15.107:0/507631016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f684c100e20 msgr2=0x7f684c195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:36.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.628+0000 7f684ad9d700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f684c100e20 0x7f684c195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.628+0000 7f684ad9d700 1 -- 172.21.15.107:0/507631016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f684c0fddb0 msgr2=0x7f684c195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:36.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.628+0000 7f684ad9d700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f684c0fddb0 0x7f684c195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.628+0000 7f684ad9d700 1 -- 172.21.15.107:0/507631016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c0095f0 con 0x7f684c1017b0 2023-07-21T22:28:36.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.628+0000 7f684ad9d700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f684c1017b0 0x7f684c19ce20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f684400b970 tx=0x7f684400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:36.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.628+0000 7f68437fe700 1 -- 172.21.15.107:0/507631016 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684400c930 con 0x7f684c1017b0 2023-07-21T22:28:36.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.628+0000 7f68437fe700 1 -- 172.21.15.107:0/507631016 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6844014440 con 0x7f684c1017b0 2023-07-21T22:28:36.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.628+0000 7f685080c700 1 -- 172.21.15.107:0/507631016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c19d9a0 con 0x7f684c1017b0 2023-07-21T22:28:36.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.628+0000 7f68437fe700 1 -- 172.21.15.107:0/507631016 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68440075d0 con 0x7f684c1017b0 2023-07-21T22:28:36.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.628+0000 7f685080c700 1 -- 172.21.15.107:0/507631016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c19dee0 con 0x7f684c1017b0 2023-07-21T22:28:36.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.629+0000 7f685080c700 1 -- 172.21.15.107:0/507631016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c10b3b0 con 0x7f684c1017b0 2023-07-21T22:28:36.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.630+0000 7f68437fe700 1 -- 172.21.15.107:0/507631016 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f68440145b0 con 0x7f684c1017b0 2023-07-21T22:28:36.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.630+0000 7f68437fe700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f68380a1220 0x7f68380a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:36.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.631+0000 7f684a59c700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f68380a1220 0x7f68380a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:36.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.631+0000 7f68437fe700 1 -- 172.21.15.107:0/507631016 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6844006e40 con 0x7f684c1017b0 2023-07-21T22:28:36.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.631+0000 7f684a59c700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f68380a1220 0x7f68380a36d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f684c0673b0 tx=0x7f6834009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:36.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.635+0000 7f68437fe700 1 -- 172.21.15.107:0/507631016 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f684404e770 con 0x7f684c1017b0 2023-07-21T22:28:36.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:36 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2558707986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:28:36.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:36 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/806291567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:28:36.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.866+0000 7f685080c700 1 -- 172.21.15.107:0/507631016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f684c19e550 con 0x7f684c1017b0 2023-07-21T22:28:36.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.867+0000 7f68437fe700 1 -- 172.21.15.107:0/507631016 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10868 (secure 0 0 0) 0x7f6844095730 con 0x7f684c1017b0 2023-07-21T22:28:36.913 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:36.913 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:27:49.034257+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:28:36.914 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.869+0000 7f685080c700 1 -- 172.21.15.107:0/507631016 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f68380a1220 msgr2=0x7f68380a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:36.915 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.869+0000 7f685080c700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f68380a1220 0x7f68380a36d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f684c0673b0 tx=0x7f6834009190 comp rx=0 tx=0).stop 2023-07-21T22:28:36.915 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.869+0000 7f685080c700 1 -- 172.21.15.107:0/507631016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f684c1017b0 msgr2=0x7f684c19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:36.915 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.869+0000 7f685080c700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f684c1017b0 0x7f684c19ce20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f684400b970 tx=0x7f684400bc80 comp rx=0 tx=0).stop 2023-07-21T22:28:36.915 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.869+0000 7f685080c700 1 -- 172.21.15.107:0/507631016 shutdown_connections 2023-07-21T22:28:36.915 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.869+0000 7f685080c700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f684c1017b0 0x7f684c19ce20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.915 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.869+0000 7f685080c700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f684c0fddb0 0x7f684c195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.916 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.869+0000 7f685080c700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f68380a1220 0x7f68380a36d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.916 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.869+0000 7f685080c700 1 --2- 172.21.15.107:0/507631016 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f684c100e20 0x7f684c195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:36.916 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.869+0000 7f685080c700 1 -- 172.21.15.107:0/507631016 >> 172.21.15.107:0/507631016 conn(0x7f684c0f9770 msgr2=0x7f684c0fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:36.916 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.869+0000 7f685080c700 1 -- 172.21.15.107:0/507631016 shutdown_connections 2023-07-21T22:28:36.916 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:36.869+0000 7f685080c700 1 -- 172.21.15.107:0/507631016 wait complete. 2023-07-21T22:28:36.917 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2023-07-21T22:28:36.927 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-21T22:27:41.478391+0000', 'modified': '2023-07-21T22:27:49.034257+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24533, 'mds_1': 24493, 'mds_2': 24466, 'mds_3': 24455, 'mds_4': 24437}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14646': {'gid': 14646, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6839/1956222571', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 1956222571}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 1956222571}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14658': {'gid': 14658, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6841/1878776680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6840', 'nonce': 1878776680}, {'type': 'v1', 'addr': '172.21.15.107:6841', 'nonce': 1878776680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6833/253562772', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 253562772}, {'type': 'v1', 'addr': '172.21.15.190:6833', 'nonce': 253562772}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24437': {'gid': 24437, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.132:6833/3293513462', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6832', 'nonce': 3293513462}, {'type': 'v1', 'addr': '172.21.15.132:6833', 'nonce': 3293513462}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.132:6835/1022724305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6834', 'nonce': 1022724305}, {'type': 'v1', 'addr': '172.21.15.132:6835', 'nonce': 1022724305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.190:6835/2286851849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 2286851849}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 2286851849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6837/3774629948', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 3774629948}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 3774629948}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6837/1492770170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6836', 'nonce': 1492770170}, {'type': 'v1', 'addr': '172.21.15.132:6837', 'nonce': 1492770170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.190:6839/2960500982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 2960500982}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 2960500982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24533': {'gid': 24533, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6839/523672176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6838', 'nonce': 523672176}, {'type': 'v1', 'addr': '172.21.15.132:6839', 'nonce': 523672176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-21T22:28:36.927 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-07-21T22:28:36.928 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-07-21T22:28:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.004+0000 7f5962982700 1 -- 172.21.15.107:0/1608321482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c1005f0 msgr2=0x7f595c1029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.004+0000 7f5962982700 1 --2- 172.21.15.107:0/1608321482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c1005f0 0x7f595c1029d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5944009990 tx=0x7f5944009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.004+0000 7f5962982700 1 -- 172.21.15.107:0/1608321482 shutdown_connections 2023-07-21T22:28:37.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.004+0000 7f5962982700 1 --2- 172.21.15.107:0/1608321482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f595c105830 0x7f595c107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.004+0000 7f5962982700 1 --2- 172.21.15.107:0/1608321482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c1005f0 0x7f595c1029d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.004+0000 7f5962982700 1 --2- 172.21.15.107:0/1608321482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c102f10 0x7f595c1052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.004+0000 7f5962982700 1 -- 172.21.15.107:0/1608321482 >> 172.21.15.107:0/1608321482 conn(0x7f595c0fa030 msgr2=0x7f595c0fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.004+0000 7f5962982700 1 -- 172.21.15.107:0/1608321482 shutdown_connections 2023-07-21T22:28:37.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.004+0000 7f5962982700 1 -- 172.21.15.107:0/1608321482 wait complete. 2023-07-21T22:28:37.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.004+0000 7f5962982700 1 Processor -- start 2023-07-21T22:28:37.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.004+0000 7f5962982700 1 -- start start 2023-07-21T22:28:37.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5962982700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f595c1005f0 0x7f595c195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5962982700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c102f10 0x7f595c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5962982700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c105830 0x7f595c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5962982700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c19d3a0 con 0x7f595c105830 2023-07-21T22:28:37.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5962982700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c19d4e0 con 0x7f595c102f10 2023-07-21T22:28:37.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5962982700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c19d670 con 0x7f595c1005f0 2023-07-21T22:28:37.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5960f1f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c105830 0x7f595c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5960f1f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c105830 0x7f595c19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47258/0 (socket says 172.21.15.107:47258) 2023-07-21T22:28:37.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5960f1f700 1 -- 172.21.15.107:0/2289585649 learned_addr learned my addr 172.21.15.107:0/2289585649 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f595bfff700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f595c1005f0 0x7f595c195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f595b7fe700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c102f10 0x7f595c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5960f1f700 1 -- 172.21.15.107:0/2289585649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f595c1005f0 msgr2=0x7f595c195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5960f1f700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f595c1005f0 0x7f595c195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5960f1f700 1 -- 172.21.15.107:0/2289585649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c102f10 msgr2=0x7f595c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5960f1f700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c102f10 0x7f595c195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f5960f1f700 1 -- 172.21.15.107:0/2289585649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59440095f0 con 0x7f595c105830 2023-07-21T22:28:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.005+0000 7f595b7fe700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c102f10 0x7f595c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.006+0000 7f5960f1f700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c105830 0x7f595c19cdd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f595400ada0 tx=0x7f595400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.006+0000 7f59597fa700 1 -- 172.21.15.107:0/2289585649 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595400f070 con 0x7f595c105830 2023-07-21T22:28:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.006+0000 7f5962982700 1 -- 172.21.15.107:0/2289585649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595c19d950 con 0x7f595c105830 2023-07-21T22:28:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.006+0000 7f59597fa700 1 -- 172.21.15.107:0/2289585649 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5954007b30 con 0x7f595c105830 2023-07-21T22:28:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.006+0000 7f59597fa700 1 -- 172.21.15.107:0/2289585649 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954005440 con 0x7f595c105830 2023-07-21T22:28:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.006+0000 7f5962982700 1 -- 172.21.15.107:0/2289585649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595c19de10 con 0x7f595c105830 2023-07-21T22:28:37.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.008+0000 7f59597fa700 1 -- 172.21.15.107:0/2289585649 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f5954004ad0 con 0x7f595c105830 2023-07-21T22:28:37.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.008+0000 7f59597fa700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f59480a11e0 0x7f59480a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.008+0000 7f595bfff700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f59480a11e0 0x7f59480a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.008+0000 7f59597fa700 1 -- 172.21.15.107:0/2289585649 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f59540cbfd0 con 0x7f595c105830 2023-07-21T22:28:37.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.009+0000 7f595bfff700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f59480a11e0 0x7f59480a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5944006010 tx=0x7f594400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.009+0000 7f596eb07b80 1 Processor -- start 2023-07-21T22:28:37.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.009+0000 7f596eb07b80 1 -- start start 2023-07-21T22:28:37.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.009+0000 7f596eb07b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a87e0 0x55edfc8a8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.009+0000 7f596eb07b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a9150 0x55edfc8a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.009+0000 7f596eb07b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8a7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.009+0000 7f596eb07b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edfc7acc60 con 0x55edfc8a79c0 2023-07-21T22:28:37.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.009+0000 7f596eb07b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edfc7ff9f0 con 0x55edfc8a87e0 2023-07-21T22:28:37.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.009+0000 7f596eb07b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edfc7f8e80 con 0x55edfc8a9150 2023-07-21T22:28:37.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.009+0000 7f5960f1f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8a7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.009+0000 7f5960f1f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8a7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47266/0 (socket says 172.21.15.107:47266) 2023-07-21T22:28:37.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.009+0000 7f5960f1f700 1 -- 172.21.15.107:0/2942770826 learned_addr learned my addr 172.21.15.107:0/2942770826 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.009+0000 7f595bfff700 1 --2- 172.21.15.107:0/2942770826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a87e0 0x55edfc8a8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.010+0000 7f595b7fe700 1 --2- 172.21.15.107:0/2942770826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a9150 0x55edfc8a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.010+0000 7f5960f1f700 1 -- 172.21.15.107:0/2942770826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a9150 msgr2=0x55edfc8a73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.010+0000 7f5960f1f700 1 --2- 172.21.15.107:0/2942770826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a9150 0x55edfc8a73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.010+0000 7f5960f1f700 1 -- 172.21.15.107:0/2942770826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a87e0 msgr2=0x55edfc8a8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.010+0000 7f5960f1f700 1 --2- 172.21.15.107:0/2942770826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a87e0 0x55edfc8a8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.010+0000 7f5960f1f700 1 -- 172.21.15.107:0/2942770826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edfc626d90 con 0x55edfc8a79c0 2023-07-21T22:28:37.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.010+0000 7f5960f1f700 1 --2- 172.21.15.107:0/2942770826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8a7e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5954095f30 tx=0x7f59540b9050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1ddc111442229b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.010+0000 7f59527fc700 1 -- 172.21.15.107:0/2942770826 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540ae020 con 0x55edfc8a79c0 2023-07-21T22:28:37.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.010+0000 7f59527fc700 1 -- 172.21.15.107:0/2942770826 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f595409d4f0 con 0x55edfc8a79c0 2023-07-21T22:28:37.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.010+0000 7f59527fc700 1 -- 172.21.15.107:0/2942770826 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954099920 con 0x55edfc8a79c0 2023-07-21T22:28:37.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 -- 172.21.15.107:0/2942770826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 msgr2=0x55edfc8a7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 --2- 172.21.15.107:0/2942770826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8a7e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5954095f30 tx=0x7f59540b9050 comp rx=0 tx=0).stop 2023-07-21T22:28:37.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 -- 172.21.15.107:0/2942770826 shutdown_connections 2023-07-21T22:28:37.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 --2- 172.21.15.107:0/2942770826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8a7e40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 --2- 172.21.15.107:0/2942770826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a87e0 0x55edfc8a8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 --2- 172.21.15.107:0/2942770826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a9150 0x55edfc8a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 -- 172.21.15.107:0/2942770826 >> 172.21.15.107:0/2942770826 conn(0x55edfc89cea0 msgr2=0x55edfc8a2310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 -- 172.21.15.107:0/2942770826 shutdown_connections 2023-07-21T22:28:37.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 -- 172.21.15.107:0/2942770826 wait complete. 2023-07-21T22:28:37.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 -- start start 2023-07-21T22:28:37.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 Processor -- start 2023-07-21T22:28:37.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a87e0 0x55edfc8b1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a9150 0x55edfc8b8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edfc798a70 con 0x55edfc8a79c0 2023-07-21T22:28:37.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edfc798770 con 0x55edfc8a87e0 2023-07-21T22:28:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edfc67c8d0 con 0x55edfc8a9150 2023-07-21T22:28:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.011+0000 7f596eb07b80 10 client.0 _ll_register_callbacks cb 0x55edfc796710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f595bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f595bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8b10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47268/0 (socket says 172.21.15.107:47268) 2023-07-21T22:28:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f595bfff700 1 -- 172.21.15.107:0/1175531590 learned_addr learned my addr 172.21.15.107:0/1175531590 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f595b7fe700 1 --2- 172.21.15.107:0/1175531590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a87e0 0x55edfc8b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f595bfff700 1 -- 172.21.15.107:0/1175531590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a9150 msgr2=0x55edfc8b8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f595bfff700 1 --2- 172.21.15.107:0/1175531590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a9150 0x55edfc8b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f595bfff700 1 -- 172.21.15.107:0/1175531590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a87e0 msgr2=0x55edfc8b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f595bfff700 1 --2- 172.21.15.107:0/1175531590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a87e0 0x55edfc8b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f595bfff700 1 -- 172.21.15.107:0/1175531590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595c19de10 con 0x55edfc8a79c0 2023-07-21T22:28:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f595bfff700 1 --2- 172.21.15.107:0/1175531590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8b10e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5944009990 tx=0x7f594401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bebf81fd22f7e23 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f593b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:37.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f593b7fe700 1 -- 172.21.15.107:0/1175531590 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594402d040 con 0x55edfc8a79c0 2023-07-21T22:28:37.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f593b7fe700 1 -- 172.21.15.107:0/1175531590 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5944024820 con 0x55edfc8a79c0 2023-07-21T22:28:37.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f596eb07b80 1 -- 172.21.15.107:0/1175531590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edfc7524a0 con 0x55edfc8a79c0 2023-07-21T22:28:37.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f593b7fe700 1 -- 172.21.15.107:0/1175531590 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59440249e0 con 0x55edfc8a79c0 2023-07-21T22:28:37.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f596eb07b80 10 client.15072 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:37.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f596eb07b80 1 -- 172.21.15.107:0/1175531590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55edfc60fc10 con 0x55edfc8a79c0 2023-07-21T22:28:37.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f593b7fe700 1 -- 172.21.15.107:0/1175531590 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55edfc7524a0 con 0x55edfc8a79c0 2023-07-21T22:28:37.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.012+0000 7f593b7fe700 1 -- 172.21.15.107:0/1175531590 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f594403c070 con 0x55edfc8a79c0 2023-07-21T22:28:37.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.013+0000 7f596eb07b80 10 client.15072 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:37.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.013+0000 7f596eb07b80 10 client.15072 resolve_mds: resolved name 'k' to daemon mds.k 2023-07-21T22:28:37.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.013+0000 7f596eb07b80 20 client.15072 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:37.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.013+0000 7f596eb07b80 1 --2- 172.21.15.107:0/1175531590 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55edfc8bc420 0x55edfc8bb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.013+0000 7f596eb07b80 4 client.15072 mds_command: new command op to 24437 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:37.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.013+0000 7f596eb07b80 1 -- 172.21.15.107:0/1175531590 --> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55edfc7f1e60 con 0x55edfc8bc420 2023-07-21T22:28:37.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.013+0000 7f595b7fe700 1 --2- 172.21.15.107:0/1175531590 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55edfc8bc420 0x55edfc8bb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.013+0000 7f593b7fe700 10 client.15072 ms_handle_connect on v2:172.21.15.132:6832/3293513462 2023-07-21T22:28:37.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.013+0000 7f595b7fe700 1 --2- 172.21.15.107:0/1175531590 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55edfc8bc420 0x55edfc8bb7b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=787268fadccc8d59 server_cookie=f0090b54036d0aa2 in_seq=0 out_seq=0 2023-07-21T22:28:37.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f593b7fe700 1 -- 172.21.15.107:0/1175531590 <== mds.4 v2:172.21.15.132:6832/3293513462 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55edfc7f1e60 con 0x55edfc8bc420 2023-07-21T22:28:37.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f593b7fe700 10 client.15072 handle_command_reply: tid=0 2023-07-21T22:28:37.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f596eb07b80 1 client.15072 shutdown 2023-07-21T22:28:37.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f596eb07b80 1 -- 172.21.15.107:0/1175531590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 msgr2=0x55edfc8b10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f596eb07b80 1 --2- 172.21.15.107:0/1175531590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8b10e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5944009990 tx=0x7f594401f000 comp rx=0 tx=0).stop 2023-07-21T22:28:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f596eb07b80 1 -- 172.21.15.107:0/1175531590 shutdown_connections 2023-07-21T22:28:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f596eb07b80 1 --2- 172.21.15.107:0/1175531590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a9150 0x55edfc8b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f596eb07b80 1 --2- 172.21.15.107:0/1175531590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8b10e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f596eb07b80 1 --2- 172.21.15.107:0/1175531590 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55edfc8bc420 0x55edfc8bb7b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f596eb07b80 1 --2- 172.21.15.107:0/1175531590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a87e0 0x55edfc8b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f593b7fe700 0 client.15072 ms_handle_reset on v2:172.21.15.132:6832/3293513462 2023-07-21T22:28:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f596eb07b80 1 -- 172.21.15.107:0/1175531590 >> 172.21.15.107:0/1175531590 conn(0x55edfc89cea0 msgr2=0x55edfc8a19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f596eb07b80 1 -- 172.21.15.107:0/1175531590 shutdown_connections 2023-07-21T22:28:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f596eb07b80 1 -- 172.21.15.107:0/1175531590 wait complete. 2023-07-21T22:28:37.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.014+0000 7f596eb07b80 20 client.15072 trim_cache size 0 max 16384 2023-07-21T22:28:37.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.031+0000 7f596eb07b80 1 Processor -- start 2023-07-21T22:28:37.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f596eb07b80 1 -- start start 2023-07-21T22:28:37.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f596eb07b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a79c0 0x55edfc8a7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f596eb07b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a87e0 0x55edfc8b8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f596eb07b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a9150 0x55edfc8aef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f596eb07b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edfc8b22d0 con 0x55edfc8a9150 2023-07-21T22:28:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f596eb07b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edfc7c7790 con 0x55edfc8a79c0 2023-07-21T22:28:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f596eb07b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edfc7b7480 con 0x55edfc8a87e0 2023-07-21T22:28:37.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f5960f1f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a9150 0x55edfc8aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f5960f1f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a9150 0x55edfc8aef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47280/0 (socket says 172.21.15.107:47280) 2023-07-21T22:28:37.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f5960f1f700 1 -- 172.21.15.107:0/1171610676 learned_addr learned my addr 172.21.15.107:0/1171610676 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f595bfff700 1 --2- 172.21.15.107:0/1171610676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a79c0 0x55edfc8a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f595b7fe700 1 --2- 172.21.15.107:0/1171610676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a87e0 0x55edfc8b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f5960f1f700 1 -- 172.21.15.107:0/1171610676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a87e0 msgr2=0x55edfc8b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f5960f1f700 1 --2- 172.21.15.107:0/1171610676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a87e0 0x55edfc8b8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f5960f1f700 1 -- 172.21.15.107:0/1171610676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a79c0 msgr2=0x55edfc8a7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f5960f1f700 1 --2- 172.21.15.107:0/1171610676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a79c0 0x55edfc8a7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f5960f1f700 1 -- 172.21.15.107:0/1171610676 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edfc64d200 con 0x55edfc8a9150 2023-07-21T22:28:37.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f595bfff700 1 --2- 172.21.15.107:0/1171610676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a79c0 0x55edfc8a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:37.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f5960f1f700 1 --2- 172.21.15.107:0/1171610676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a9150 0x55edfc8aef60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f595400a0a0 tx=0x7f5954004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501dce09e90e05ab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f5952ffd700 1 -- 172.21.15.107:0/1171610676 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540ae050 con 0x55edfc8a9150 2023-07-21T22:28:37.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f5952ffd700 1 -- 172.21.15.107:0/1171610676 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59540c9a70 con 0x55edfc8a9150 2023-07-21T22:28:37.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.032+0000 7f5952ffd700 1 -- 172.21.15.107:0/1171610676 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954098c10 con 0x55edfc8a9150 2023-07-21T22:28:37.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.033+0000 7f596eb07b80 1 -- 172.21.15.107:0/1171610676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a9150 msgr2=0x55edfc8aef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.033+0000 7f596eb07b80 1 --2- 172.21.15.107:0/1171610676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a9150 0x55edfc8aef60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f595400a0a0 tx=0x7f5954004ed0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.033+0000 7f596eb07b80 1 -- 172.21.15.107:0/1171610676 shutdown_connections 2023-07-21T22:28:37.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.033+0000 7f596eb07b80 1 --2- 172.21.15.107:0/1171610676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a9150 0x55edfc8aef60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.033+0000 7f596eb07b80 1 --2- 172.21.15.107:0/1171610676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a79c0 0x55edfc8a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.033+0000 7f596eb07b80 1 --2- 172.21.15.107:0/1171610676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a87e0 0x55edfc8b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.033+0000 7f596eb07b80 1 -- 172.21.15.107:0/1171610676 >> 172.21.15.107:0/1171610676 conn(0x55edfc89cea0 msgr2=0x55edfc8b8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.033+0000 7f596eb07b80 1 -- 172.21.15.107:0/1171610676 shutdown_connections 2023-07-21T22:28:37.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.033+0000 7f596eb07b80 1 -- 172.21.15.107:0/1171610676 wait complete. 2023-07-21T22:28:37.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.033+0000 7f596eb07b80 1 -- start start 2023-07-21T22:28:37.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f596eb07b80 1 Processor -- start 2023-07-21T22:28:37.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f596eb07b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f596eb07b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a87e0 0x55edfc8b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f596eb07b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a9150 0x55edfc9102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f596eb07b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edfc8b87c0 con 0x55edfc8a79c0 2023-07-21T22:28:37.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f596eb07b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edfc8b8930 con 0x55edfc8a9150 2023-07-21T22:28:37.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f596eb07b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edfc8b8af0 con 0x55edfc8a87e0 2023-07-21T22:28:37.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f596eb07b80 10 client.0 _ll_register_callbacks cb 0x7f594400f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:37.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f595bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f595bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47282/0 (socket says 172.21.15.107:47282) 2023-07-21T22:28:37.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f595bfff700 1 -- 172.21.15.107:0/3849340289 learned_addr learned my addr 172.21.15.107:0/3849340289 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f5960f1f700 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a9150 0x55edfc9102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f595b7fe700 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a87e0 0x55edfc8b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f595bfff700 1 -- 172.21.15.107:0/3849340289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a87e0 msgr2=0x55edfc8b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f595bfff700 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a87e0 0x55edfc8b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f595bfff700 1 -- 172.21.15.107:0/3849340289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a9150 msgr2=0x55edfc9102e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f595bfff700 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a9150 0x55edfc9102e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f595bfff700 1 -- 172.21.15.107:0/3849340289 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edfc626d90 con 0x55edfc8a79c0 2023-07-21T22:28:37.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f5960f1f700 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a9150 0x55edfc9102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:37.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f595bfff700 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8b3860 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5944032d10 tx=0x7f594400b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e935901bd252830 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f593b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:37.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f593b7fe700 1 -- 172.21.15.107:0/3849340289 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594402d040 con 0x55edfc8a79c0 2023-07-21T22:28:37.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f593b7fe700 1 -- 172.21.15.107:0/3849340289 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5944032630 con 0x55edfc8a79c0 2023-07-21T22:28:37.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.034+0000 7f596eb07b80 1 -- 172.21.15.107:0/3849340289 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f595409d4f0 con 0x55edfc8a79c0 2023-07-21T22:28:37.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.035+0000 7f593b7fe700 1 -- 172.21.15.107:0/3849340289 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59440327f0 con 0x55edfc8a79c0 2023-07-21T22:28:37.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.035+0000 7f596eb07b80 10 client.15078 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:37.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.035+0000 7f596eb07b80 1 -- 172.21.15.107:0/3849340289 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55edfc625260 con 0x55edfc8a79c0 2023-07-21T22:28:37.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.035+0000 7f593b7fe700 1 -- 172.21.15.107:0/3849340289 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f595409d4f0 con 0x55edfc8a79c0 2023-07-21T22:28:37.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.035+0000 7f593b7fe700 1 -- 172.21.15.107:0/3849340289 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f594403c050 con 0x55edfc8a79c0 2023-07-21T22:28:37.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.035+0000 7f596eb07b80 10 client.15078 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:37.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.035+0000 7f596eb07b80 10 client.15078 resolve_mds: resolved name 'k' to daemon mds.k 2023-07-21T22:28:37.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.035+0000 7f596eb07b80 20 client.15078 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:37.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.035+0000 7f596eb07b80 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55edfc915010 0x55edfc911980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.035+0000 7f596eb07b80 4 client.15078 mds_command: new command op to 24437 tid=0[{"prefix": "status"}] 2023-07-21T22:28:37.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.035+0000 7f596eb07b80 1 -- 172.21.15.107:0/3849340289 --> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55edfc7b4000 con 0x55edfc915010 2023-07-21T22:28:37.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.035+0000 7f595b7fe700 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55edfc915010 0x55edfc911980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f595b7fe700 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55edfc915010 0x55edfc911980 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21d7977b1ad66091 server_cookie=db8479243e4b5ed in_seq=0 out_seq=0 2023-07-21T22:28:37.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f593b7fe700 10 client.15078 ms_handle_connect on v2:172.21.15.132:6832/3293513462 2023-07-21T22:28:37.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f593b7fe700 1 -- 172.21.15.107:0/3849340289 <== mds.4 v2:172.21.15.132:6832/3293513462 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55edfc7b4000 con 0x55edfc915010 2023-07-21T22:28:37.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f593b7fe700 10 client.15078 handle_command_reply: tid=0 2023-07-21T22:28:37.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f596eb07b80 1 client.15078 shutdown 2023-07-21T22:28:37.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f596eb07b80 1 -- 172.21.15.107:0/3849340289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 msgr2=0x55edfc8b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.098 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f596eb07b80 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8b3860 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5944032d10 tx=0x7f594400b980 comp rx=0 tx=0).stop 2023-07-21T22:28:37.098 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f596eb07b80 1 -- 172.21.15.107:0/3849340289 shutdown_connections 2023-07-21T22:28:37.098 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f596eb07b80 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55edfc915010 0x55edfc911980 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.098 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f596eb07b80 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55edfc8a9150 0x55edfc9102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.098 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f596eb07b80 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55edfc8a79c0 0x55edfc8b3860 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.098 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f593b7fe700 0 client.15078 ms_handle_reset on v2:172.21.15.132:6832/3293513462 2023-07-21T22:28:37.098 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f596eb07b80 1 --2- 172.21.15.107:0/3849340289 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55edfc8a87e0 0x55edfc8b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.099 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f596eb07b80 1 -- 172.21.15.107:0/3849340289 >> 172.21.15.107:0/3849340289 conn(0x55edfc89cea0 msgr2=0x55edfc8aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.099 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f596eb07b80 1 -- 172.21.15.107:0/3849340289 shutdown_connections 2023-07-21T22:28:37.099 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f596eb07b80 1 -- 172.21.15.107:0/3849340289 wait complete. 2023-07-21T22:28:37.099 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.036+0000 7f596eb07b80 20 client.15078 trim_cache size 0 max 16384 2023-07-21T22:28:37.099 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:28:37.100 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "af99f5d0-2814-11ee-9b34-001a4aab830c", 2023-07-21T22:28:37.100 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 4, 2023-07-21T22:28:37.100 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24437, 2023-07-21T22:28:37.100 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2023-07-21T22:28:37.100 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2023-07-21T22:28:37.100 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2023-07-21T22:28:37.101 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 49.004413069000002, 2023-07-21T22:28:37.101 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2023-07-21T22:28:37.101 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 82, 2023-07-21T22:28:37.101 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:28:37.101 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 83.044682041000002 2023-07-21T22:28:37.101 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:28:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.037+0000 7f5962982700 1 -- 172.21.15.107:0/2289585649 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f59480a11e0 msgr2=0x7f59480a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.037+0000 7f5962982700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f59480a11e0 0x7f59480a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5944006010 tx=0x7f594400b540 comp rx=0 tx=0).stop 2023-07-21T22:28:37.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.037+0000 7f5962982700 1 -- 172.21.15.107:0/2289585649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c105830 msgr2=0x7f595c19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.037+0000 7f5962982700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c105830 0x7f595c19cdd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f595400ada0 tx=0x7f595400c330 comp rx=0 tx=0).stop 2023-07-21T22:28:37.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.037+0000 7f5962982700 1 -- 172.21.15.107:0/2289585649 shutdown_connections 2023-07-21T22:28:37.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.037+0000 7f5962982700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c105830 0x7f595c19cdd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.037+0000 7f5962982700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f59480a11e0 0x7f59480a3690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.037+0000 7f5962982700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f595c1005f0 0x7f595c195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.104 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.037+0000 7f5962982700 1 --2- 172.21.15.107:0/2289585649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c102f10 0x7f595c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.104 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.037+0000 7f5962982700 1 -- 172.21.15.107:0/2289585649 >> 172.21.15.107:0/2289585649 conn(0x7f595c0fa030 msgr2=0x7f595c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.104 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.037+0000 7f5962982700 1 -- 172.21.15.107:0/2289585649 shutdown_connections 2023-07-21T22:28:37.104 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.037+0000 7f5962982700 1 -- 172.21.15.107:0/2289585649 wait complete. 2023-07-21T22:28:37.104 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-07-21T22:28:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.183+0000 7f30b2a1e700 1 -- 172.21.15.107:0/3178333076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac100560 msgr2=0x7f30ac106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.183+0000 7f30b2a1e700 1 --2- 172.21.15.107:0/3178333076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac100560 0x7f30ac106e20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f30a400b0a0 tx=0x7f30a400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.183+0000 7f30b2a1e700 1 -- 172.21.15.107:0/3178333076 shutdown_connections 2023-07-21T22:28:37.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.183+0000 7f30b2a1e700 1 --2- 172.21.15.107:0/3178333076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac100560 0x7f30ac106e20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.183+0000 7f30b2a1e700 1 --2- 172.21.15.107:0/3178333076 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30ac0fe9e0 0x7f30ac0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.183+0000 7f30b2a1e700 1 --2- 172.21.15.107:0/3178333076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30ac0ffbd0 0x7f30ac100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.183+0000 7f30b2a1e700 1 -- 172.21.15.107:0/3178333076 >> 172.21.15.107:0/3178333076 conn(0x7f30ac0fa000 msgr2=0x7f30ac0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.183+0000 7f30b2a1e700 1 -- 172.21.15.107:0/3178333076 shutdown_connections 2023-07-21T22:28:37.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.183+0000 7f30b2a1e700 1 -- 172.21.15.107:0/3178333076 wait complete. 2023-07-21T22:28:37.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.184+0000 7f30b2a1e700 1 Processor -- start 2023-07-21T22:28:37.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.184+0000 7f30b2a1e700 1 -- start start 2023-07-21T22:28:37.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30b2a1e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30ac0fe9e0 0x7f30ac194840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30b2a1e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac0ffbd0 0x7f30ac194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30b2a1e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30ac100560 0x7f30ac19bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30b2a1e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac19c3c0 con 0x7f30ac0ffbd0 2023-07-21T22:28:37.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30b2a1e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac19c500 con 0x7f30ac100560 2023-07-21T22:28:37.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30b2a1e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac19c690 con 0x7f30ac0fe9e0 2023-07-21T22:28:37.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30ab7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac0ffbd0 0x7f30ac194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30ab7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac0ffbd0 0x7f30ac194d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47296/0 (socket says 172.21.15.107:47296) 2023-07-21T22:28:37.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30ab7fe700 1 -- 172.21.15.107:0/3484190190 learned_addr learned my addr 172.21.15.107:0/3484190190 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30b0fbb700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30ac100560 0x7f30ac19bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30abfff700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30ac0fe9e0 0x7f30ac194840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30ab7fe700 1 -- 172.21.15.107:0/3484190190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30ac0fe9e0 msgr2=0x7f30ac194840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30ab7fe700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30ac0fe9e0 0x7f30ac194840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30ab7fe700 1 -- 172.21.15.107:0/3484190190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30ac100560 msgr2=0x7f30ac19bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30ab7fe700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30ac100560 0x7f30ac19bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30ab7fe700 1 -- 172.21.15.107:0/3484190190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a4009d00 con 0x7f30ac0ffbd0 2023-07-21T22:28:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30abfff700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30ac0fe9e0 0x7f30ac194840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30b0fbb700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30ac100560 0x7f30ac19bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30ab7fe700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac0ffbd0 0x7f30ac194d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f30a000e9a0 tx=0x7f30a000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30a97fa700 1 -- 172.21.15.107:0/3484190190 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0009790 con 0x7f30ac0ffbd0 2023-07-21T22:28:37.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30a97fa700 1 -- 172.21.15.107:0/3484190190 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30a000cbd0 con 0x7f30ac0ffbd0 2023-07-21T22:28:37.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30b2a1e700 1 -- 172.21.15.107:0/3484190190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ac19c970 con 0x7f30ac0ffbd0 2023-07-21T22:28:37.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30a97fa700 1 -- 172.21.15.107:0/3484190190 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0018680 con 0x7f30ac0ffbd0 2023-07-21T22:28:37.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.185+0000 7f30b2a1e700 1 -- 172.21.15.107:0/3484190190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ac19ce80 con 0x7f30ac0ffbd0 2023-07-21T22:28:37.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.187+0000 7f30a97fa700 1 -- 172.21.15.107:0/3484190190 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f30a000cd40 con 0x7f30ac0ffbd0 2023-07-21T22:28:37.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.188+0000 7f30a97fa700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30940a11e0 0x7f30940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.188+0000 7f30abfff700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30940a11e0 0x7f30940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.188+0000 7f30a97fa700 1 -- 172.21.15.107:0/3484190190 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f30a0014070 con 0x7f30ac0ffbd0 2023-07-21T22:28:37.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.188+0000 7f30abfff700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30940a11e0 0x7f30940a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f309c005fd0 tx=0x7f309c005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30beba3b80 1 Processor -- start 2023-07-21T22:28:37.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30beba3b80 1 -- start start 2023-07-21T22:28:37.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30beba3b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 0x55b2f3254bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30beba3b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f3255150 0x55b2f32533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30beba3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32539c0 0x55b2f3253e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30beba3b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f3158c60 con 0x55b2f32547e0 2023-07-21T22:28:37.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30beba3b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f31ab9f0 con 0x55b2f3255150 2023-07-21T22:28:37.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30beba3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f31a4e80 con 0x55b2f32539c0 2023-07-21T22:28:37.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30abfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 0x55b2f3254bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30abfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 0x55b2f3254bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47302/0 (socket says 172.21.15.107:47302) 2023-07-21T22:28:37.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30abfff700 1 -- 172.21.15.107:0/2136766792 learned_addr learned my addr 172.21.15.107:0/2136766792 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30b0fbb700 1 --2- 172.21.15.107:0/2136766792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32539c0 0x55b2f3253e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30ab7fe700 1 --2- 172.21.15.107:0/2136766792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f3255150 0x55b2f32533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30abfff700 1 -- 172.21.15.107:0/2136766792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32539c0 msgr2=0x55b2f3253e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30abfff700 1 --2- 172.21.15.107:0/2136766792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32539c0 0x55b2f3253e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30abfff700 1 -- 172.21.15.107:0/2136766792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f3255150 msgr2=0x55b2f32533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30abfff700 1 --2- 172.21.15.107:0/2136766792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f3255150 0x55b2f32533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.189+0000 7f30abfff700 1 -- 172.21.15.107:0/2136766792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2f2fd2d90 con 0x55b2f32547e0 2023-07-21T22:28:37.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.190+0000 7f30abfff700 1 --2- 172.21.15.107:0/2136766792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 0x55b2f3254bf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f309c00edb0 tx=0x7f309c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a18bcacb3b47406 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.190+0000 7f30927fc700 1 -- 172.21.15.107:0/2136766792 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c011c90 con 0x55b2f32547e0 2023-07-21T22:28:37.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.190+0000 7f30927fc700 1 -- 172.21.15.107:0/2136766792 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f309c011df0 con 0x55b2f32547e0 2023-07-21T22:28:37.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.190+0000 7f30927fc700 1 -- 172.21.15.107:0/2136766792 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c010710 con 0x55b2f32547e0 2023-07-21T22:28:37.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.190+0000 7f30beba3b80 1 -- 172.21.15.107:0/2136766792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 msgr2=0x55b2f3254bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.190+0000 7f30beba3b80 1 --2- 172.21.15.107:0/2136766792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 0x55b2f3254bf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f309c00edb0 tx=0x7f309c011000 comp rx=0 tx=0).stop 2023-07-21T22:28:37.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.190+0000 7f30beba3b80 1 -- 172.21.15.107:0/2136766792 shutdown_connections 2023-07-21T22:28:37.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.190+0000 7f30beba3b80 1 --2- 172.21.15.107:0/2136766792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32539c0 0x55b2f3253e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.190+0000 7f30beba3b80 1 --2- 172.21.15.107:0/2136766792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 0x55b2f3254bf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.190+0000 7f30beba3b80 1 --2- 172.21.15.107:0/2136766792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f3255150 0x55b2f32533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.190+0000 7f30beba3b80 1 -- 172.21.15.107:0/2136766792 >> 172.21.15.107:0/2136766792 conn(0x55b2f3248ea0 msgr2=0x55b2f324e310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.190+0000 7f30beba3b80 1 -- 172.21.15.107:0/2136766792 shutdown_connections 2023-07-21T22:28:37.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.190+0000 7f30beba3b80 1 -- 172.21.15.107:0/2136766792 wait complete. 2023-07-21T22:28:37.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.191+0000 7f30beba3b80 1 -- start start 2023-07-21T22:28:37.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.191+0000 7f30beba3b80 1 Processor -- start 2023-07-21T22:28:37.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.191+0000 7f30beba3b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32539c0 0x55b2f325d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.191+0000 7f30beba3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32547e0 0x55b2f325d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.191+0000 7f30beba3b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f3255150 0x55b2f32645a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.191+0000 7f30beba3b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f3144a70 con 0x55b2f32539c0 2023-07-21T22:28:37.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.191+0000 7f30beba3b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f3144770 con 0x55b2f3255150 2023-07-21T22:28:37.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.191+0000 7f30beba3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f30288d0 con 0x55b2f32547e0 2023-07-21T22:28:37.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.191+0000 7f30beba3b80 10 client.0 _ll_register_callbacks cb 0x55b2f3142710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:37.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.191+0000 7f30abfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32539c0 0x55b2f325d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.191+0000 7f30abfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32539c0 0x55b2f325d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47308/0 (socket says 172.21.15.107:47308) 2023-07-21T22:28:37.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.191+0000 7f30abfff700 1 -- 172.21.15.107:0/4275055216 learned_addr learned my addr 172.21.15.107:0/4275055216 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.191+0000 7f30b0fbb700 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f3255150 0x55b2f32645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30ab7fe700 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32547e0 0x55b2f325d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30abfff700 1 -- 172.21.15.107:0/4275055216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32547e0 msgr2=0x55b2f325d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30abfff700 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32547e0 0x55b2f325d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30abfff700 1 -- 172.21.15.107:0/4275055216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f3255150 msgr2=0x55b2f32645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30abfff700 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f3255150 0x55b2f32645a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30abfff700 1 -- 172.21.15.107:0/4275055216 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c00ea90 con 0x55b2f32539c0 2023-07-21T22:28:37.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30b0fbb700 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f3255150 0x55b2f32645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:37.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30ab7fe700 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32547e0 0x55b2f325d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:37.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30abfff700 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32539c0 0x55b2f325d020 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f309c011ae0 tx=0x7f309c01ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e232b13553d0d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f308b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:37.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f308b7fe700 1 -- 172.21.15.107:0/4275055216 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c011c90 con 0x55b2f32539c0 2023-07-21T22:28:37.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f308b7fe700 1 -- 172.21.15.107:0/4275055216 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f309c005050 con 0x55b2f32539c0 2023-07-21T22:28:37.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30beba3b80 1 -- 172.21.15.107:0/4275055216 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2f30fe4a0 con 0x55b2f32539c0 2023-07-21T22:28:37.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f308b7fe700 1 -- 172.21.15.107:0/4275055216 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f309c005210 con 0x55b2f32539c0 2023-07-21T22:28:37.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30beba3b80 10 client.15090 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:37.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30beba3b80 1 -- 172.21.15.107:0/4275055216 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b2f2fbbc10 con 0x55b2f32539c0 2023-07-21T22:28:37.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f308b7fe700 1 -- 172.21.15.107:0/4275055216 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b2f30fe4a0 con 0x55b2f32539c0 2023-07-21T22:28:37.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f308b7fe700 1 -- 172.21.15.107:0/4275055216 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f309c0104b0 con 0x55b2f32539c0 2023-07-21T22:28:37.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30beba3b80 10 client.15090 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:37.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30beba3b80 10 client.15090 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-21T22:28:37.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30beba3b80 20 client.15090 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:37.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30beba3b80 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55b2f3268240 0x55b2f3267650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30beba3b80 4 client.15090 mds_command: new command op to 24455 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:37.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.192+0000 7f30beba3b80 1 -- 172.21.15.107:0/4275055216 --> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2f319de60 con 0x55b2f3268240 2023-07-21T22:28:37.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.193+0000 7f30ab7fe700 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55b2f3268240 0x55b2f3267650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.193+0000 7f30ab7fe700 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55b2f3268240 0x55b2f3267650 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=70956a0980753ef9 server_cookie=7164382f0fba0724 in_seq=0 out_seq=0 2023-07-21T22:28:37.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.193+0000 7f308b7fe700 10 client.15090 ms_handle_connect on v2:172.21.15.132:6834/1022724305 2023-07-21T22:28:37.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f308b7fe700 1 -- 172.21.15.107:0/4275055216 <== mds.3 v2:172.21.15.132:6834/1022724305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b2f319de60 con 0x55b2f3268240 2023-07-21T22:28:37.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f308b7fe700 10 client.15090 handle_command_reply: tid=0 2023-07-21T22:28:37.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f30beba3b80 1 client.15090 shutdown 2023-07-21T22:28:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f30beba3b80 1 -- 172.21.15.107:0/4275055216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32539c0 msgr2=0x55b2f325d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f30beba3b80 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32539c0 0x55b2f325d020 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f309c011ae0 tx=0x7f309c01ea60 comp rx=0 tx=0).stop 2023-07-21T22:28:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f30beba3b80 1 -- 172.21.15.107:0/4275055216 shutdown_connections 2023-07-21T22:28:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f30beba3b80 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55b2f3268240 0x55b2f3267650 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f30beba3b80 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f3255150 0x55b2f32645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f308b7fe700 0 client.15090 ms_handle_reset on v2:172.21.15.132:6834/1022724305 2023-07-21T22:28:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f30beba3b80 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32539c0 0x55b2f325d020 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f30beba3b80 1 --2- 172.21.15.107:0/4275055216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32547e0 0x55b2f325d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f30beba3b80 1 -- 172.21.15.107:0/4275055216 >> 172.21.15.107:0/4275055216 conn(0x55b2f3248ea0 msgr2=0x55b2f324d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f30beba3b80 1 -- 172.21.15.107:0/4275055216 shutdown_connections 2023-07-21T22:28:37.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f30beba3b80 1 -- 172.21.15.107:0/4275055216 wait complete. 2023-07-21T22:28:37.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.194+0000 7f30beba3b80 20 client.15090 trim_cache size 0 max 16384 2023-07-21T22:28:37.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.211+0000 7f30beba3b80 1 Processor -- start 2023-07-21T22:28:37.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.211+0000 7f30beba3b80 1 -- start start 2023-07-21T22:28:37.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.211+0000 7f30beba3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32539c0 0x55b2f3253300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30beba3b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f32547e0 0x55b2f3264dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30beba3b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f3255150 0x55b2f3267650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30beba3b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f32544a0 con 0x55b2f3255150 2023-07-21T22:28:37.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30beba3b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f319de60 con 0x55b2f32547e0 2023-07-21T22:28:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30beba3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f30fe4a0 con 0x55b2f32539c0 2023-07-21T22:28:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30b0fbb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f3255150 0x55b2f3267650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30b0fbb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f3255150 0x55b2f3267650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47320/0 (socket says 172.21.15.107:47320) 2023-07-21T22:28:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30b0fbb700 1 -- 172.21.15.107:0/1857178397 learned_addr learned my addr 172.21.15.107:0/1857178397 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30b0fbb700 1 -- 172.21.15.107:0/1857178397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32539c0 msgr2=0x55b2f3253300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30b0fbb700 1 --2- 172.21.15.107:0/1857178397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32539c0 0x55b2f3253300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30b0fbb700 1 -- 172.21.15.107:0/1857178397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f32547e0 msgr2=0x55b2f3264dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30b0fbb700 1 --2- 172.21.15.107:0/1857178397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f32547e0 0x55b2f3264dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30b0fbb700 1 -- 172.21.15.107:0/1857178397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2f325df70 con 0x55b2f3255150 2023-07-21T22:28:37.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f30b0fbb700 1 --2- 172.21.15.107:0/1857178397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f3255150 0x55b2f3267650 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f30a400b0a0 tx=0x7f30a40093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58627a21f0f81516 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f3092ffd700 1 -- 172.21.15.107:0/1857178397 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a400e040 con 0x55b2f3255150 2023-07-21T22:28:37.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f3092ffd700 1 -- 172.21.15.107:0/1857178397 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30a4020070 con 0x55b2f3255150 2023-07-21T22:28:37.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.212+0000 7f3092ffd700 1 -- 172.21.15.107:0/1857178397 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4004e30 con 0x55b2f3255150 2023-07-21T22:28:37.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 -- 172.21.15.107:0/1857178397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f3255150 msgr2=0x55b2f3267650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 --2- 172.21.15.107:0/1857178397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f3255150 0x55b2f3267650 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f30a400b0a0 tx=0x7f30a40093c0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 -- 172.21.15.107:0/1857178397 shutdown_connections 2023-07-21T22:28:37.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 --2- 172.21.15.107:0/1857178397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f3255150 0x55b2f3267650 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 --2- 172.21.15.107:0/1857178397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f32539c0 0x55b2f3253300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 --2- 172.21.15.107:0/1857178397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f32547e0 0x55b2f3264dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 -- 172.21.15.107:0/1857178397 >> 172.21.15.107:0/1857178397 conn(0x55b2f3248ea0 msgr2=0x55b2f3253d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 -- 172.21.15.107:0/1857178397 shutdown_connections 2023-07-21T22:28:37.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 -- 172.21.15.107:0/1857178397 wait complete. 2023-07-21T22:28:37.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 -- start start 2023-07-21T22:28:37.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 Processor -- start 2023-07-21T22:28:37.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f32539c0 0x55b2f325a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 0x55b2f32bc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f3255150 0x55b2f32be610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f3264870 con 0x55b2f32547e0 2023-07-21T22:28:37.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f32649e0 con 0x55b2f32539c0 2023-07-21T22:28:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.213+0000 7f30beba3b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f32beb50 con 0x55b2f3255150 2023-07-21T22:28:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30beba3b80 10 client.0 _ll_register_callbacks cb 0x7f309c00ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30ab7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 0x55b2f32bc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30ab7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 0x55b2f32bc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47324/0 (socket says 172.21.15.107:47324) 2023-07-21T22:28:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30ab7fe700 1 -- 172.21.15.107:0/1561611001 learned_addr learned my addr 172.21.15.107:0/1561611001 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30b0fbb700 1 --2- 172.21.15.107:0/1561611001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f3255150 0x55b2f32be610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30abfff700 1 --2- 172.21.15.107:0/1561611001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f32539c0 0x55b2f325a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30ab7fe700 1 -- 172.21.15.107:0/1561611001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f3255150 msgr2=0x55b2f32be610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30ab7fe700 1 --2- 172.21.15.107:0/1561611001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f3255150 0x55b2f32be610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30ab7fe700 1 -- 172.21.15.107:0/1561611001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f32539c0 msgr2=0x55b2f325a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30ab7fe700 1 --2- 172.21.15.107:0/1561611001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f32539c0 0x55b2f325a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30ab7fe700 1 -- 172.21.15.107:0/1561611001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ac107750 con 0x55b2f32547e0 2023-07-21T22:28:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30ab7fe700 1 --2- 172.21.15.107:0/1561611001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 0x55b2f32bc0c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f30a0096b40 tx=0x7f30a000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f7c7fba4f8ed2a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f308b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:37.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f308b7fe700 1 -- 172.21.15.107:0/1561611001 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a00a2090 con 0x55b2f32547e0 2023-07-21T22:28:37.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f308b7fe700 1 -- 172.21.15.107:0/1561611001 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f30a00a21f0 con 0x55b2f32547e0 2023-07-21T22:28:37.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30beba3b80 1 -- 172.21.15.107:0/1561611001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f309c011df0 con 0x55b2f32547e0 2023-07-21T22:28:37.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f308b7fe700 1 -- 172.21.15.107:0/1561611001 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30a0095ec0 con 0x55b2f32547e0 2023-07-21T22:28:37.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30beba3b80 10 client.15096 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:37.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f30beba3b80 1 -- 172.21.15.107:0/1561611001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b2f2ff9200 con 0x55b2f32547e0 2023-07-21T22:28:37.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.214+0000 7f308b7fe700 1 -- 172.21.15.107:0/1561611001 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f309c011df0 con 0x55b2f32547e0 2023-07-21T22:28:37.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.215+0000 7f308b7fe700 1 -- 172.21.15.107:0/1561611001 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f30a00991a0 con 0x55b2f32547e0 2023-07-21T22:28:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.215+0000 7f30beba3b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.215+0000 7f30beba3b80 10 client.15096 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-21T22:28:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.215+0000 7f30beba3b80 20 client.15096 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.215+0000 7f30beba3b80 1 --2- 172.21.15.107:0/1561611001 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55b2f32c3860 0x55b2f32c0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.215+0000 7f30beba3b80 4 client.15096 mds_command: new command op to 24455 tid=0[{"prefix": "status"}] 2023-07-21T22:28:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.215+0000 7f30beba3b80 1 -- 172.21.15.107:0/1561611001 --> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b2f3160000 con 0x55b2f32c3860 2023-07-21T22:28:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.215+0000 7f30abfff700 1 --2- 172.21.15.107:0/1561611001 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55b2f32c3860 0x55b2f32c0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.215+0000 7f308b7fe700 10 client.15096 ms_handle_connect on v2:172.21.15.132:6834/1022724305 2023-07-21T22:28:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.215+0000 7f30abfff700 1 --2- 172.21.15.107:0/1561611001 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55b2f32c3860 0x55b2f32c0210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c31ad0571e728d8 server_cookie=44c5448053149a23 in_seq=0 out_seq=0 2023-07-21T22:28:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.215+0000 7f308b7fe700 1 -- 172.21.15.107:0/1561611001 <== mds.3 v2:172.21.15.132:6834/1022724305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55b2f3160000 con 0x55b2f32c3860 2023-07-21T22:28:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.215+0000 7f308b7fe700 10 client.15096 handle_command_reply: tid=0 2023-07-21T22:28:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.215+0000 7f30beba3b80 1 client.15096 shutdown 2023-07-21T22:28:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30beba3b80 1 -- 172.21.15.107:0/1561611001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 msgr2=0x55b2f32bc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30beba3b80 1 --2- 172.21.15.107:0/1561611001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 0x55b2f32bc0c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f30a0096b40 tx=0x7f30a000c6e0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30beba3b80 1 -- 172.21.15.107:0/1561611001 shutdown_connections 2023-07-21T22:28:37.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30beba3b80 1 --2- 172.21.15.107:0/1561611001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b2f3255150 0x55b2f32be610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30beba3b80 1 --2- 172.21.15.107:0/1561611001 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55b2f32c3860 0x55b2f32c0210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30beba3b80 1 --2- 172.21.15.107:0/1561611001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b2f32539c0 0x55b2f325a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30beba3b80 1 --2- 172.21.15.107:0/1561611001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2f32547e0 0x55b2f32bc0c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f308b7fe700 0 client.15096 ms_handle_reset on v2:172.21.15.132:6834/1022724305 2023-07-21T22:28:37.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30beba3b80 1 -- 172.21.15.107:0/1561611001 >> 172.21.15.107:0/1561611001 conn(0x55b2f3248ea0 msgr2=0x55b2f3264dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30beba3b80 1 -- 172.21.15.107:0/1561611001 shutdown_connections 2023-07-21T22:28:37.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30beba3b80 1 -- 172.21.15.107:0/1561611001 wait complete. 2023-07-21T22:28:37.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30beba3b80 20 client.15096 trim_cache size 0 max 16384 2023-07-21T22:28:37.272 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:28:37.272 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "af99f5d0-2814-11ee-9b34-001a4aab830c", 2023-07-21T22:28:37.272 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 3, 2023-07-21T22:28:37.272 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24455, 2023-07-21T22:28:37.273 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2023-07-21T22:28:37.273 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2023-07-21T22:28:37.273 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2023-07-21T22:28:37.273 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 50.193465169, 2023-07-21T22:28:37.273 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2023-07-21T22:28:37.273 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 82, 2023-07-21T22:28:37.273 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:28:37.273 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 76.716276614999998 2023-07-21T22:28:37.274 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:28:37.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30b2a1e700 1 -- 172.21.15.107:0/3484190190 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30940a11e0 msgr2=0x7f30940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30b2a1e700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30940a11e0 0x7f30940a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f309c005fd0 tx=0x7f309c005f00 comp rx=0 tx=0).stop 2023-07-21T22:28:37.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30b2a1e700 1 -- 172.21.15.107:0/3484190190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac0ffbd0 msgr2=0x7f30ac194d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30b2a1e700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac0ffbd0 0x7f30ac194d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f30a000e9a0 tx=0x7f30a000ed60 comp rx=0 tx=0).stop 2023-07-21T22:28:37.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30b2a1e700 1 -- 172.21.15.107:0/3484190190 shutdown_connections 2023-07-21T22:28:37.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30b2a1e700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30ac100560 0x7f30ac19bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30b2a1e700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30940a11e0 0x7f30940a3690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30b2a1e700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30ac0fe9e0 0x7f30ac194840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30b2a1e700 1 --2- 172.21.15.107:0/3484190190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac0ffbd0 0x7f30ac194d80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30b2a1e700 1 -- 172.21.15.107:0/3484190190 >> 172.21.15.107:0/3484190190 conn(0x7f30ac0fa000 msgr2=0x7f30ac0fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30b2a1e700 1 -- 172.21.15.107:0/3484190190 shutdown_connections 2023-07-21T22:28:37.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.216+0000 7f30b2a1e700 1 -- 172.21.15.107:0/3484190190 wait complete. 2023-07-21T22:28:37.277 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-07-21T22:28:37.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.353+0000 7f29a044c700 1 -- 172.21.15.107:0/2644283317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29980ff5c0 msgr2=0x7f2998106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.353+0000 7f29a044c700 1 --2- 172.21.15.107:0/2644283317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29980ff5c0 0x7f2998106e70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f299000b0a0 tx=0x7f299000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.353+0000 7f29a044c700 1 -- 172.21.15.107:0/2644283317 shutdown_connections 2023-07-21T22:28:37.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.353+0000 7f29a044c700 1 --2- 172.21.15.107:0/2644283317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29980ff5c0 0x7f2998106e70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.353+0000 7f29a044c700 1 --2- 172.21.15.107:0/2644283317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2998100050 0x7f2998100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.353+0000 7f29a044c700 1 --2- 172.21.15.107:0/2644283317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29980fec10 0x7f29980ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.353+0000 7f29a044c700 1 -- 172.21.15.107:0/2644283317 >> 172.21.15.107:0/2644283317 conn(0x7f29980f9770 msgr2=0x7f29980fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.353+0000 7f29a044c700 1 -- 172.21.15.107:0/2644283317 shutdown_connections 2023-07-21T22:28:37.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.353+0000 7f29a044c700 1 -- 172.21.15.107:0/2644283317 wait complete. 2023-07-21T22:28:37.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.354+0000 7f29a044c700 1 Processor -- start 2023-07-21T22:28:37.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.354+0000 7f29a044c700 1 -- start start 2023-07-21T22:28:37.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.354+0000 7f29a044c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29980fec10 0x7f2998193740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.354+0000 7f29a044c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29980ff5c0 0x7f2998193c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.354+0000 7f29a044c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2998100050 0x7f299819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.354+0000 7f29a044c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299819b2c0 con 0x7f2998100050 2023-07-21T22:28:37.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.354+0000 7f29a044c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299819b400 con 0x7f29980ff5c0 2023-07-21T22:28:37.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.354+0000 7f29a044c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299819b590 con 0x7f29980fec10 2023-07-21T22:28:37.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.354+0000 7f299e9e9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2998100050 0x7f299819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.354+0000 7f299e9e9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2998100050 0x7f299819acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47340/0 (socket says 172.21.15.107:47340) 2023-07-21T22:28:37.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.354+0000 7f299e9e9700 1 -- 172.21.15.107:0/393213383 learned_addr learned my addr 172.21.15.107:0/393213383 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.354+0000 7f299d9e7700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29980ff5c0 0x7f2998193c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.354+0000 7f299e1e8700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29980fec10 0x7f2998193740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.355+0000 7f299e9e9700 1 -- 172.21.15.107:0/393213383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29980fec10 msgr2=0x7f2998193740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.355+0000 7f299e9e9700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29980fec10 0x7f2998193740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.355+0000 7f299e9e9700 1 -- 172.21.15.107:0/393213383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29980ff5c0 msgr2=0x7f2998193c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.355+0000 7f299e9e9700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29980ff5c0 0x7f2998193c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.355+0000 7f299e9e9700 1 -- 172.21.15.107:0/393213383 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2990009d00 con 0x7f2998100050 2023-07-21T22:28:37.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.355+0000 7f299e9e9700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2998100050 0x7f299819acf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f299000b070 tx=0x7f2990004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.355+0000 7f298f7fe700 1 -- 172.21.15.107:0/393213383 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299000e040 con 0x7f2998100050 2023-07-21T22:28:37.405 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.355+0000 7f298f7fe700 1 -- 172.21.15.107:0/393213383 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2990004e80 con 0x7f2998100050 2023-07-21T22:28:37.405 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.355+0000 7f29a044c700 1 -- 172.21.15.107:0/393213383 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299819b810 con 0x7f2998100050 2023-07-21T22:28:37.405 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.355+0000 7f298f7fe700 1 -- 172.21.15.107:0/393213383 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299001b6b0 con 0x7f2998100050 2023-07-21T22:28:37.405 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.355+0000 7f29a044c700 1 -- 172.21.15.107:0/393213383 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299819bce0 con 0x7f2998100050 2023-07-21T22:28:37.410 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.357+0000 7f298f7fe700 1 -- 172.21.15.107:0/393213383 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f2990003a00 con 0x7f2998100050 2023-07-21T22:28:37.410 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.358+0000 7f298f7fe700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f29840a1260 0x7f29840a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.410 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.358+0000 7f299e1e8700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f29840a1260 0x7f29840a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.410 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.358+0000 7f298f7fe700 1 -- 172.21.15.107:0/393213383 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f29900cdab0 con 0x7f2998100050 2023-07-21T22:28:37.410 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.359+0000 7f299e1e8700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f29840a1260 0x7f29840a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2988005950 tx=0x7f29880058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.411 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.359+0000 7f29ac5d1b80 1 Processor -- start 2023-07-21T22:28:37.411 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.359+0000 7f29ac5d1b80 1 -- start start 2023-07-21T22:28:37.411 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.359+0000 7f29ac5d1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c07e0 0x5640d84c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.411 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.359+0000 7f29ac5d1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84c1150 0x5640d84bf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.411 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.359+0000 7f29ac5d1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84bf9c0 0x5640d84bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.411 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.359+0000 7f29ac5d1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640d83c4c60 con 0x5640d84bf9c0 2023-07-21T22:28:37.411 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.359+0000 7f29ac5d1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640d84179f0 con 0x5640d84c1150 2023-07-21T22:28:37.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.359+0000 7f29ac5d1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640d8410e80 con 0x5640d84c07e0 2023-07-21T22:28:37.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.359+0000 7f299e9e9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84bf9c0 0x5640d84bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.359+0000 7f299e9e9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84bf9c0 0x5640d84bfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47348/0 (socket says 172.21.15.107:47348) 2023-07-21T22:28:37.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.359+0000 7f299e9e9700 1 -- 172.21.15.107:0/586082806 learned_addr learned my addr 172.21.15.107:0/586082806 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.359+0000 7f299d9e7700 1 --2- 172.21.15.107:0/586082806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84c1150 0x5640d84bf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.360+0000 7f299e1e8700 1 --2- 172.21.15.107:0/586082806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c07e0 0x5640d84c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.360+0000 7f299e9e9700 1 -- 172.21.15.107:0/586082806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c07e0 msgr2=0x5640d84c0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.360+0000 7f299e9e9700 1 --2- 172.21.15.107:0/586082806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c07e0 0x5640d84c0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.360+0000 7f299e9e9700 1 -- 172.21.15.107:0/586082806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84c1150 msgr2=0x5640d84bf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.360+0000 7f299e9e9700 1 --2- 172.21.15.107:0/586082806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84c1150 0x5640d84bf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.360+0000 7f299e9e9700 1 -- 172.21.15.107:0/586082806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640d823ed90 con 0x5640d84bf9c0 2023-07-21T22:28:37.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.360+0000 7f299e1e8700 1 --2- 172.21.15.107:0/586082806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c07e0 0x5640d84c0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:37.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.360+0000 7f299e9e9700 1 --2- 172.21.15.107:0/586082806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84bf9c0 0x5640d84bfe40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f29900cb660 tx=0x7f299000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ca7f8079c9fcb5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.360+0000 7f298cdac700 1 -- 172.21.15.107:0/586082806 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299009afa0 con 0x5640d84bf9c0 2023-07-21T22:28:37.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.360+0000 7f298cdac700 1 -- 172.21.15.107:0/586082806 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f299009efd0 con 0x5640d84bf9c0 2023-07-21T22:28:37.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.360+0000 7f298cdac700 1 -- 172.21.15.107:0/586082806 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299009a490 con 0x5640d84bf9c0 2023-07-21T22:28:37.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/586082806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84bf9c0 msgr2=0x5640d84bfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/586082806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84bf9c0 0x5640d84bfe40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f29900cb660 tx=0x7f299000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/586082806 shutdown_connections 2023-07-21T22:28:37.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/586082806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84bf9c0 0x5640d84bfe40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/586082806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c07e0 0x5640d84c0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/586082806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84c1150 0x5640d84bf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/586082806 >> 172.21.15.107:0/586082806 conn(0x5640d84b4ea0 msgr2=0x5640d84ba310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/586082806 shutdown_connections 2023-07-21T22:28:37.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/586082806 wait complete. 2023-07-21T22:28:37.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 -- start start 2023-07-21T22:28:37.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 Processor -- start 2023-07-21T22:28:37.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 0x5640d84c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 0x5640d84c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c1150 0x5640d84d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640d83b0a70 con 0x5640d84c07e0 2023-07-21T22:28:37.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640d83b0770 con 0x5640d84bf9c0 2023-07-21T22:28:37.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640d82948d0 con 0x5640d84c1150 2023-07-21T22:28:37.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f29ac5d1b80 10 client.0 _ll_register_callbacks cb 0x5640d83ae710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:37.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f299d9e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 0x5640d84c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f299d9e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 0x5640d84c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47352/0 (socket says 172.21.15.107:47352) 2023-07-21T22:28:37.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.361+0000 7f299d9e7700 1 -- 172.21.15.107:0/1369514677 learned_addr learned my addr 172.21.15.107:0/1369514677 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f299e1e8700 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 0x5640d84c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f299e9e9700 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c1150 0x5640d84d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f299d9e7700 1 -- 172.21.15.107:0/1369514677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c1150 msgr2=0x5640d84d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f299d9e7700 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c1150 0x5640d84d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f299d9e7700 1 -- 172.21.15.107:0/1369514677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 msgr2=0x5640d84c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f299d9e7700 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 0x5640d84c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f299d9e7700 1 -- 172.21.15.107:0/1369514677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299819bce0 con 0x5640d84c07e0 2023-07-21T22:28:37.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f299e9e9700 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c1150 0x5640d84d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:37.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f299d9e7700 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 0x5640d84c92e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2994017d90 tx=0x7f2994017e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0795d9e1dffabe1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f2981ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:37.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f2981ffb700 1 -- 172.21.15.107:0/1369514677 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994004250 con 0x5640d84c07e0 2023-07-21T22:28:37.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f2981ffb700 1 -- 172.21.15.107:0/1369514677 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f29940043b0 con 0x5640d84c07e0 2023-07-21T22:28:37.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/1369514677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640d836a4a0 con 0x5640d84c07e0 2023-07-21T22:28:37.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f2981ffb700 1 -- 172.21.15.107:0/1369514677 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2994004570 con 0x5640d84c07e0 2023-07-21T22:28:37.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f29ac5d1b80 10 client.15108 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:37.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/1369514677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5640d8227c10 con 0x5640d84c07e0 2023-07-21T22:28:37.419 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.362+0000 7f2981ffb700 1 -- 172.21.15.107:0/1369514677 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5640d836a4a0 con 0x5640d84c07e0 2023-07-21T22:28:37.419 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.363+0000 7f2981ffb700 1 -- 172.21.15.107:0/1369514677 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2994020020 con 0x5640d84c07e0 2023-07-21T22:28:37.419 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.363+0000 7f29ac5d1b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:37.419 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.363+0000 7f29ac5d1b80 10 client.15108 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-21T22:28:37.419 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.363+0000 7f29ac5d1b80 20 client.15108 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:37.419 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.363+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5640d84d4090 0x5640d84d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.419 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.363+0000 7f29ac5d1b80 4 client.15108 mds_command: new command op to 24466 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:37.419 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.363+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/1369514677 --> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640d8409e60 con 0x5640d84d4090 2023-07-21T22:28:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.363+0000 7f299e1e8700 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5640d84d4090 0x5640d84d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.363+0000 7f2981ffb700 10 client.15108 ms_handle_connect on v2:172.21.15.190:6834/2286851849 2023-07-21T22:28:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.363+0000 7f299e1e8700 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5640d84d4090 0x5640d84d3440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f295ae2ab76d03c server_cookie=2cc5d93e2f5c3725 in_seq=0 out_seq=0 2023-07-21T22:28:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f2981ffb700 1 -- 172.21.15.107:0/1369514677 <== mds.2 v2:172.21.15.190:6834/2286851849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5640d8409e60 con 0x5640d84d4090 2023-07-21T22:28:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f2981ffb700 10 client.15108 handle_command_reply: tid=0 2023-07-21T22:28:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f29ac5d1b80 1 client.15108 shutdown 2023-07-21T22:28:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/1369514677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 msgr2=0x5640d84c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 0x5640d84c92e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2994017d90 tx=0x7f2994017e70 comp rx=0 tx=0).stop 2023-07-21T22:28:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/1369514677 shutdown_connections 2023-07-21T22:28:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5640d84d4090 0x5640d84d3440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c1150 0x5640d84d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 0x5640d84c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/1369514677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 0x5640d84c92e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f2981ffb700 0 client.15108 ms_handle_reset on v2:172.21.15.190:6834/2286851849 2023-07-21T22:28:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/1369514677 >> 172.21.15.107:0/1369514677 conn(0x5640d84b4ea0 msgr2=0x5640d84b99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/1369514677 shutdown_connections 2023-07-21T22:28:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.364+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/1369514677 wait complete. 2023-07-21T22:28:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.365+0000 7f29ac5d1b80 20 client.15108 trim_cache size 0 max 16384 2023-07-21T22:28:37.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f29ac5d1b80 1 Processor -- start 2023-07-21T22:28:37.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f29ac5d1b80 1 -- start start 2023-07-21T22:28:37.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f29ac5d1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 0x5640d84d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f29ac5d1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 0x5640d84d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f29ac5d1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c1150 0x5640d84c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f29ac5d1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640d8409e60 con 0x5640d84c07e0 2023-07-21T22:28:37.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f29ac5d1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640d836a4a0 con 0x5640d84bf9c0 2023-07-21T22:28:37.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f29ac5d1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640d83df790 con 0x5640d84c1150 2023-07-21T22:28:37.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f299d9e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 0x5640d84d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f299d9e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 0x5640d84d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47360/0 (socket says 172.21.15.107:47360) 2023-07-21T22:28:37.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f299d9e7700 1 -- 172.21.15.107:0/3170060837 learned_addr learned my addr 172.21.15.107:0/3170060837 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f299e1e8700 1 --2- 172.21.15.107:0/3170060837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 0x5640d84d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f299e9e9700 1 --2- 172.21.15.107:0/3170060837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c1150 0x5640d84c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f299d9e7700 1 -- 172.21.15.107:0/3170060837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c1150 msgr2=0x5640d84c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f299d9e7700 1 --2- 172.21.15.107:0/3170060837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c1150 0x5640d84c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f299d9e7700 1 -- 172.21.15.107:0/3170060837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 msgr2=0x5640d84d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f299d9e7700 1 --2- 172.21.15.107:0/3170060837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 0x5640d84d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f299d9e7700 1 -- 172.21.15.107:0/3170060837 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640d8265200 con 0x5640d84c07e0 2023-07-21T22:28:37.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f299e9e9700 1 --2- 172.21.15.107:0/3170060837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c1150 0x5640d84c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:37.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f299d9e7700 1 --2- 172.21.15.107:0/3170060837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 0x5640d84d02e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2994000c00 tx=0x7f2994015590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fd1a581da80417 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.382+0000 7f298d5ad700 1 -- 172.21.15.107:0/3170060837 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994003880 con 0x5640d84c07e0 2023-07-21T22:28:37.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.383+0000 7f298d5ad700 1 -- 172.21.15.107:0/3170060837 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2994013040 con 0x5640d84c07e0 2023-07-21T22:28:37.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.383+0000 7f298d5ad700 1 -- 172.21.15.107:0/3170060837 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994015d40 con 0x5640d84c07e0 2023-07-21T22:28:37.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.383+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/3170060837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 msgr2=0x5640d84d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.383+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/3170060837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 0x5640d84d02e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2994000c00 tx=0x7f2994015590 comp rx=0 tx=0).stop 2023-07-21T22:28:37.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.383+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/3170060837 shutdown_connections 2023-07-21T22:28:37.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.383+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/3170060837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c1150 0x5640d84c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.383+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/3170060837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 0x5640d84d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.383+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/3170060837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c07e0 0x5640d84d02e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.383+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/3170060837 >> 172.21.15.107:0/3170060837 conn(0x5640d84b4ea0 msgr2=0x5640d84dce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.383+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/3170060837 shutdown_connections 2023-07-21T22:28:37.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.383+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/3170060837 wait complete. 2023-07-21T22:28:37.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.383+0000 7f29ac5d1b80 1 -- start start 2023-07-21T22:28:37.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.383+0000 7f29ac5d1b80 1 Processor -- start 2023-07-21T22:28:37.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f29ac5d1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 0x5640d84cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f29ac5d1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c07e0 0x5640d84cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f29ac5d1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c1150 0x5640d84cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f29ac5d1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640d84cf5c0 con 0x5640d84c1150 2023-07-21T22:28:37.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f29ac5d1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640d84cf730 con 0x5640d84bf9c0 2023-07-21T22:28:37.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f29ac5d1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640d84cf8f0 con 0x5640d84c07e0 2023-07-21T22:28:37.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f29ac5d1b80 10 client.0 _ll_register_callbacks cb 0x7f2994017650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:37.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f299e9e9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c1150 0x5640d84cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f299e9e9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c1150 0x5640d84cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47368/0 (socket says 172.21.15.107:47368) 2023-07-21T22:28:37.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f299e9e9700 1 -- 172.21.15.107:0/3881755304 learned_addr learned my addr 172.21.15.107:0/3881755304 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f299e1e8700 1 --2- 172.21.15.107:0/3881755304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 0x5640d84cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f299e9e9700 1 -- 172.21.15.107:0/3881755304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c07e0 msgr2=0x5640d84cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f299e9e9700 1 --2- 172.21.15.107:0/3881755304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c07e0 0x5640d84cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f299e9e9700 1 -- 172.21.15.107:0/3881755304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 msgr2=0x5640d84cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f299e9e9700 1 --2- 172.21.15.107:0/3881755304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 0x5640d84cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f299e9e9700 1 -- 172.21.15.107:0/3881755304 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640d8227c10 con 0x5640d84c1150 2023-07-21T22:28:37.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f299e1e8700 1 --2- 172.21.15.107:0/3881755304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 0x5640d84cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:37.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f299e9e9700 1 --2- 172.21.15.107:0/3881755304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c1150 0x5640d84cf080 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f299009aca0 tx=0x7f299009aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9613fb459cddbe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f2981ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:37.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f2981ffb700 1 -- 172.21.15.107:0/3881755304 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29900a3360 con 0x5640d84c1150 2023-07-21T22:28:37.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f2981ffb700 1 -- 172.21.15.107:0/3881755304 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f299009b410 con 0x5640d84c1150 2023-07-21T22:28:37.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/3881755304 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f299009efd0 con 0x5640d84c1150 2023-07-21T22:28:37.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f2981ffb700 1 -- 172.21.15.107:0/3881755304 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29900b9250 con 0x5640d84c1150 2023-07-21T22:28:37.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f29ac5d1b80 10 client.15114 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.384+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/3881755304 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5640d823d260 con 0x5640d84c1150 2023-07-21T22:28:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.385+0000 7f2981ffb700 1 -- 172.21.15.107:0/3881755304 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29900cb020 con 0x5640d84c1150 2023-07-21T22:28:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.385+0000 7f2981ffb700 1 -- 172.21.15.107:0/3881755304 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2990095bb0 con 0x5640d84c1150 2023-07-21T22:28:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.385+0000 7f29ac5d1b80 10 client.15114 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.385+0000 7f29ac5d1b80 10 client.15114 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-21T22:28:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.385+0000 7f29ac5d1b80 20 client.15114 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.385+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/3881755304 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5640d852d930 0x5640d852a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.385+0000 7f29ac5d1b80 4 client.15114 mds_command: new command op to 24466 tid=0[{"prefix": "status"}] 2023-07-21T22:28:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.385+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/3881755304 --> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5640d83cc000 con 0x5640d852d930 2023-07-21T22:28:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.385+0000 7f299e1e8700 1 --2- 172.21.15.107:0/3881755304 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5640d852d930 0x5640d852a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.385+0000 7f2981ffb700 10 client.15114 ms_handle_connect on v2:172.21.15.190:6834/2286851849 2023-07-21T22:28:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.385+0000 7f299e1e8700 1 --2- 172.21.15.107:0/3881755304 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5640d852d930 0x5640d852a2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19dc73d048fd87f3 server_cookie=2f891ca7674cb2fa in_seq=0 out_seq=0 2023-07-21T22:28:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f2981ffb700 1 -- 172.21.15.107:0/3881755304 <== mds.2 v2:172.21.15.190:6834/2286851849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5640d83cc000 con 0x5640d852d930 2023-07-21T22:28:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f2981ffb700 10 client.15114 handle_command_reply: tid=0 2023-07-21T22:28:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29ac5d1b80 1 client.15114 shutdown 2023-07-21T22:28:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/3881755304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c1150 msgr2=0x5640d84cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/3881755304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c1150 0x5640d84cf080 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f299009aca0 tx=0x7f299009aa30 comp rx=0 tx=0).stop 2023-07-21T22:28:37.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/3881755304 shutdown_connections 2023-07-21T22:28:37.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/3881755304 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5640d852d930 0x5640d852a2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/3881755304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5640d84c1150 0x5640d84cf080 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f2981ffb700 0 client.15114 ms_handle_reset on v2:172.21.15.190:6834/2286851849 2023-07-21T22:28:37.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/3881755304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640d84bf9c0 0x5640d84cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29ac5d1b80 1 --2- 172.21.15.107:0/3881755304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5640d84c07e0 0x5640d84cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/3881755304 >> 172.21.15.107:0/3881755304 conn(0x5640d84b4ea0 msgr2=0x5640d84c6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/3881755304 shutdown_connections 2023-07-21T22:28:37.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29ac5d1b80 1 -- 172.21.15.107:0/3881755304 wait complete. 2023-07-21T22:28:37.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29ac5d1b80 20 client.15114 trim_cache size 0 max 16384 2023-07-21T22:28:37.443 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:28:37.444 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "af99f5d0-2814-11ee-9b34-001a4aab830c", 2023-07-21T22:28:37.444 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 2, 2023-07-21T22:28:37.444 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24466, 2023-07-21T22:28:37.444 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2023-07-21T22:28:37.444 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2023-07-21T22:28:37.444 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2023-07-21T22:28:37.444 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 51.369754743999998, 2023-07-21T22:28:37.445 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2023-07-21T22:28:37.445 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 82, 2023-07-21T22:28:37.445 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:28:37.445 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 74.793721058000003 2023-07-21T22:28:37.445 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:28:37.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29a044c700 1 -- 172.21.15.107:0/393213383 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f29840a1260 msgr2=0x7f29840a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29a044c700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f29840a1260 0x7f29840a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2988005950 tx=0x7f29880058e0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29a044c700 1 -- 172.21.15.107:0/393213383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2998100050 msgr2=0x7f299819acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29a044c700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2998100050 0x7f299819acf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f299000b070 tx=0x7f2990004a50 comp rx=0 tx=0).stop 2023-07-21T22:28:37.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29a044c700 1 -- 172.21.15.107:0/393213383 shutdown_connections 2023-07-21T22:28:37.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29a044c700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2998100050 0x7f299819acf0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29a044c700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f29840a1260 0x7f29840a3710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29a044c700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29980fec10 0x7f2998193740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.386+0000 7f29a044c700 1 --2- 172.21.15.107:0/393213383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29980ff5c0 0x7f2998193c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.387+0000 7f29a044c700 1 -- 172.21.15.107:0/393213383 >> 172.21.15.107:0/393213383 conn(0x7f29980f9770 msgr2=0x7f29980fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.387+0000 7f29a044c700 1 -- 172.21.15.107:0/393213383 shutdown_connections 2023-07-21T22:28:37.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.387+0000 7f29a044c700 1 -- 172.21.15.107:0/393213383 wait complete. 2023-07-21T22:28:37.448 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-21T22:28:37.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:37 smithi107 ceph-mon[96543]: pgmap v277: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:28:37.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:37 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3213844770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:37.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:37 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3782659529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:37.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:37 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/507631016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.524+0000 7f26a7fd4700 1 -- 172.21.15.107:0/932005895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26a0100d00 msgr2=0x7f26a0101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.524+0000 7f26a7fd4700 1 --2- 172.21.15.107:0/932005895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26a0100d00 0x7f26a0101150 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f269c009990 tx=0x7f269c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.524+0000 7f26a7fd4700 1 -- 172.21.15.107:0/932005895 shutdown_connections 2023-07-21T22:28:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.524+0000 7f26a7fd4700 1 --2- 172.21.15.107:0/932005895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26a0101690 0x7f26a0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.524+0000 7f26a7fd4700 1 --2- 172.21.15.107:0/932005895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26a00ffb10 0x7f26a00fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.524+0000 7f26a7fd4700 1 --2- 172.21.15.107:0/932005895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26a0100d00 0x7f26a0101150 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.524+0000 7f26a7fd4700 1 -- 172.21.15.107:0/932005895 >> 172.21.15.107:0/932005895 conn(0x7f26a00fb110 msgr2=0x7f26a00fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.524+0000 7f26a7fd4700 1 -- 172.21.15.107:0/932005895 shutdown_connections 2023-07-21T22:28:37.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.524+0000 7f26a7fd4700 1 -- 172.21.15.107:0/932005895 wait complete. 2023-07-21T22:28:37.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.524+0000 7f26a7fd4700 1 Processor -- start 2023-07-21T22:28:37.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.524+0000 7f26a7fd4700 1 -- start start 2023-07-21T22:28:37.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a7fd4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26a00ffb10 0x7f26a01957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a7fd4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26a0100d00 0x7f26a0195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a7fd4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26a0101690 0x7f26a019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a7fd4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a019d370 con 0x7f26a0100d00 2023-07-21T22:28:37.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a7fd4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a019d4b0 con 0x7f26a0101690 2023-07-21T22:28:37.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a7fd4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a019d640 con 0x7f26a00ffb10 2023-07-21T22:28:37.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a556f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26a0100d00 0x7f26a0195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a556f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26a0100d00 0x7f26a0195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47380/0 (socket says 172.21.15.107:47380) 2023-07-21T22:28:37.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a556f700 1 -- 172.21.15.107:0/2245222889 learned_addr learned my addr 172.21.15.107:0/2245222889 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a5d70700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26a00ffb10 0x7f26a01957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a6571700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26a0101690 0x7f26a019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a556f700 1 -- 172.21.15.107:0/2245222889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26a00ffb10 msgr2=0x7f26a01957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a556f700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26a00ffb10 0x7f26a01957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a556f700 1 -- 172.21.15.107:0/2245222889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26a0101690 msgr2=0x7f26a019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a556f700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26a0101690 0x7f26a019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a556f700 1 -- 172.21.15.107:0/2245222889 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c0095f0 con 0x7f26a0100d00 2023-07-21T22:28:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a6571700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26a0101690 0x7f26a019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a556f700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26a0100d00 0x7f26a0195d30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f269c004760 tx=0x7f269c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f2696ffd700 1 -- 172.21.15.107:0/2245222889 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c01d070 con 0x7f26a0100d00 2023-07-21T22:28:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a7fd4700 1 -- 172.21.15.107:0/2245222889 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a019d8c0 con 0x7f26a0100d00 2023-07-21T22:28:37.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f2696ffd700 1 -- 172.21.15.107:0/2245222889 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f269c0041d0 con 0x7f26a0100d00 2023-07-21T22:28:37.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f2696ffd700 1 -- 172.21.15.107:0/2245222889 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c0053a0 con 0x7f26a0100d00 2023-07-21T22:28:37.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.525+0000 7f26a7fd4700 1 -- 172.21.15.107:0/2245222889 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a019dd60 con 0x7f26a0100d00 2023-07-21T22:28:37.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.527+0000 7f2696ffd700 1 -- 172.21.15.107:0/2245222889 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f269c005500 con 0x7f26a0100d00 2023-07-21T22:28:37.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.528+0000 7f2696ffd700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f268c0a1260 0x7f268c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.528+0000 7f2696ffd700 1 -- 172.21.15.107:0/2245222889 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f269c0cdca0 con 0x7f26a0100d00 2023-07-21T22:28:37.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.528+0000 7f26a5d70700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f268c0a1260 0x7f268c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.528+0000 7f26a5d70700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f268c0a1260 0x7f268c0a3710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f26a0100b60 tx=0x7f269000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.529+0000 7f26b4159b80 1 Processor -- start 2023-07-21T22:28:37.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.529+0000 7f26b4159b80 1 -- start start 2023-07-21T22:28:37.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.529+0000 7f26b4159b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 0x55dd248a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.529+0000 7f26b4159b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a3150 0x55dd248a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.529+0000 7f26b4159b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 0x55dd248a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.529+0000 7f26b4159b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd247a6c60 con 0x55dd248a27e0 2023-07-21T22:28:37.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.529+0000 7f26b4159b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd247f99f0 con 0x55dd248a3150 2023-07-21T22:28:37.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.529+0000 7f26b4159b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd247f2e80 con 0x55dd248a19c0 2023-07-21T22:28:37.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.529+0000 7f26a5d70700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 0x55dd248a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.529+0000 7f26a5d70700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 0x55dd248a2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47386/0 (socket says 172.21.15.107:47386) 2023-07-21T22:28:37.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.529+0000 7f26a5d70700 1 -- 172.21.15.107:0/2713728041 learned_addr learned my addr 172.21.15.107:0/2713728041 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.529+0000 7f26a556f700 1 --2- 172.21.15.107:0/2713728041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a3150 0x55dd248a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.529+0000 7f26a6571700 1 --2- 172.21.15.107:0/2713728041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 0x55dd248a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.530+0000 7f26a5d70700 1 -- 172.21.15.107:0/2713728041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 msgr2=0x55dd248a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.530+0000 7f26a5d70700 1 --2- 172.21.15.107:0/2713728041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 0x55dd248a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.530+0000 7f26a5d70700 1 -- 172.21.15.107:0/2713728041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a3150 msgr2=0x55dd248a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.530+0000 7f26a5d70700 1 --2- 172.21.15.107:0/2713728041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a3150 0x55dd248a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.530+0000 7f26a5d70700 1 -- 172.21.15.107:0/2713728041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd24620d90 con 0x55dd248a27e0 2023-07-21T22:28:37.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.530+0000 7f26a556f700 1 --2- 172.21.15.107:0/2713728041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a3150 0x55dd248a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:37.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.530+0000 7f26a6571700 1 --2- 172.21.15.107:0/2713728041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 0x55dd248a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:37.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.530+0000 7f26a5d70700 1 --2- 172.21.15.107:0/2713728041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 0x55dd248a2bf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f269000fd80 tx=0x7f2690012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0895db8d2ad84c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.530+0000 7f268bfff700 1 -- 172.21.15.107:0/2713728041 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2690010ac0 con 0x55dd248a27e0 2023-07-21T22:28:37.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.530+0000 7f268bfff700 1 -- 172.21.15.107:0/2713728041 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2690020070 con 0x55dd248a27e0 2023-07-21T22:28:37.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.530+0000 7f268bfff700 1 -- 172.21.15.107:0/2713728041 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269001be50 con 0x55dd248a27e0 2023-07-21T22:28:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.530+0000 7f26b4159b80 1 -- 172.21.15.107:0/2713728041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 msgr2=0x55dd248a2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.530+0000 7f26b4159b80 1 --2- 172.21.15.107:0/2713728041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 0x55dd248a2bf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f269000fd80 tx=0x7f2690012000 comp rx=0 tx=0).stop 2023-07-21T22:28:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.531+0000 7f26b4159b80 1 -- 172.21.15.107:0/2713728041 shutdown_connections 2023-07-21T22:28:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.531+0000 7f26b4159b80 1 --2- 172.21.15.107:0/2713728041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 0x55dd248a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.531+0000 7f26b4159b80 1 --2- 172.21.15.107:0/2713728041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 0x55dd248a2bf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.531+0000 7f26b4159b80 1 --2- 172.21.15.107:0/2713728041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a3150 0x55dd248a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.531+0000 7f26b4159b80 1 -- 172.21.15.107:0/2713728041 >> 172.21.15.107:0/2713728041 conn(0x55dd24896ea0 msgr2=0x55dd2489c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.531+0000 7f26b4159b80 1 -- 172.21.15.107:0/2713728041 shutdown_connections 2023-07-21T22:28:37.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.531+0000 7f26b4159b80 1 -- 172.21.15.107:0/2713728041 wait complete. 2023-07-21T22:28:37.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.531+0000 7f26b4159b80 1 -- start start 2023-07-21T22:28:37.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.531+0000 7f26b4159b80 1 Processor -- start 2023-07-21T22:28:37.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26b4159b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 0x55dd248ab050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26b4159b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 0x55dd248ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26b4159b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a3150 0x55dd248b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26b4159b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd24792a70 con 0x55dd248a27e0 2023-07-21T22:28:37.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26b4159b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd24792770 con 0x55dd248a3150 2023-07-21T22:28:37.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26b4159b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd246768d0 con 0x55dd248a19c0 2023-07-21T22:28:37.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26b4159b80 10 client.0 _ll_register_callbacks cb 0x55dd24790710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:37.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26a556f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 0x55dd248ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26a556f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 0x55dd248ab590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47394/0 (socket says 172.21.15.107:47394) 2023-07-21T22:28:37.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26a556f700 1 -- 172.21.15.107:0/3130693577 learned_addr learned my addr 172.21.15.107:0/3130693577 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26a5d70700 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 0x55dd248ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26a6571700 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a3150 0x55dd248b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26a556f700 1 -- 172.21.15.107:0/3130693577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 msgr2=0x55dd248ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26a556f700 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 0x55dd248ab050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26a556f700 1 -- 172.21.15.107:0/3130693577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a3150 msgr2=0x55dd248b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.590 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26a556f700 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a3150 0x55dd248b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.590 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26a556f700 1 -- 172.21.15.107:0/3130693577 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269000fa60 con 0x55dd248a27e0 2023-07-21T22:28:37.590 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26a6571700 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a3150 0x55dd248b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:37.590 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26a556f700 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 0x55dd248ab590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f269c097590 tx=0x7f269c09b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9120b7283130c9f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.590 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26897fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:37.590 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26897fa700 1 -- 172.21.15.107:0/3130693577 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c0a31c0 con 0x55dd248a27e0 2023-07-21T22:28:37.590 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26897fa700 1 -- 172.21.15.107:0/3130693577 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f269c09f0c0 con 0x55dd248a27e0 2023-07-21T22:28:37.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26b4159b80 1 -- 172.21.15.107:0/3130693577 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd2474c4a0 con 0x55dd248a27e0 2023-07-21T22:28:37.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26897fa700 1 -- 172.21.15.107:0/3130693577 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f269c0a3450 con 0x55dd248a27e0 2023-07-21T22:28:37.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26b4159b80 10 client.15126 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:37.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.532+0000 7f26b4159b80 1 -- 172.21.15.107:0/3130693577 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dd24609c10 con 0x55dd248a27e0 2023-07-21T22:28:37.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.533+0000 7f26897fa700 1 -- 172.21.15.107:0/3130693577 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd2474c4a0 con 0x55dd248a27e0 2023-07-21T22:28:37.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.533+0000 7f26897fa700 1 -- 172.21.15.107:0/3130693577 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f269c099ff0 con 0x55dd248a27e0 2023-07-21T22:28:37.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.533+0000 7f26b4159b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:37.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.533+0000 7f26b4159b80 10 client.15126 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T22:28:37.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.533+0000 7f26b4159b80 20 client.15126 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:37.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.533+0000 7f26b4159b80 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55dd248b5df0 0x55dd248b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.533+0000 7f26b4159b80 4 client.15126 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:37.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.533+0000 7f26b4159b80 1 -- 172.21.15.107:0/3130693577 --> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd247ebe60 con 0x55dd248b5df0 2023-07-21T22:28:37.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.533+0000 7f26a5d70700 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55dd248b5df0 0x55dd248b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26897fa700 10 client.15126 ms_handle_connect on v2:172.21.15.190:6838/2960500982 2023-07-21T22:28:37.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26a5d70700 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55dd248b5df0 0x55dd248b5470 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa39ae52b8e35725 server_cookie=83a70a29087e3d70 in_seq=0 out_seq=0 2023-07-21T22:28:37.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26897fa700 1 -- 172.21.15.107:0/3130693577 <== mds.1 v2:172.21.15.190:6838/2960500982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dd247ebe60 con 0x55dd248b5df0 2023-07-21T22:28:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26897fa700 10 client.15126 handle_command_reply: tid=0 2023-07-21T22:28:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26b4159b80 1 client.15126 shutdown 2023-07-21T22:28:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26b4159b80 1 -- 172.21.15.107:0/3130693577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 msgr2=0x55dd248ab590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26b4159b80 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 0x55dd248ab590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f269c097590 tx=0x7f269c09b5d0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26b4159b80 1 -- 172.21.15.107:0/3130693577 shutdown_connections 2023-07-21T22:28:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26b4159b80 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55dd248b5df0 0x55dd248b5470 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26b4159b80 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a3150 0x55dd248b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26897fa700 0 client.15126 ms_handle_reset on v2:172.21.15.190:6838/2960500982 2023-07-21T22:28:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26b4159b80 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 0x55dd248ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26b4159b80 1 --2- 172.21.15.107:0/3130693577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a27e0 0x55dd248ab590 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.534+0000 7f26b4159b80 1 -- 172.21.15.107:0/3130693577 >> 172.21.15.107:0/3130693577 conn(0x55dd24896ea0 msgr2=0x55dd2489b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.535+0000 7f26b4159b80 1 -- 172.21.15.107:0/3130693577 shutdown_connections 2023-07-21T22:28:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.535+0000 7f26b4159b80 1 -- 172.21.15.107:0/3130693577 wait complete. 2023-07-21T22:28:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.535+0000 7f26b4159b80 20 client.15126 trim_cache size 0 max 16384 2023-07-21T22:28:37.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26b4159b80 1 Processor -- start 2023-07-21T22:28:37.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26b4159b80 1 -- start start 2023-07-21T22:28:37.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26b4159b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 0x55dd248a8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26b4159b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a27e0 0x55dd248a9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26b4159b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a3150 0x55dd248a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26b4159b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2489d5e0 con 0x55dd248a3150 2023-07-21T22:28:37.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26b4159b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd24894050 con 0x55dd248a27e0 2023-07-21T22:28:37.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26b4159b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd248b70d0 con 0x55dd248a19c0 2023-07-21T22:28:37.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26a6571700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a3150 0x55dd248a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26a6571700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a3150 0x55dd248a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47404/0 (socket says 172.21.15.107:47404) 2023-07-21T22:28:37.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26a6571700 1 -- 172.21.15.107:0/782825563 learned_addr learned my addr 172.21.15.107:0/782825563 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26a5d70700 1 --2- 172.21.15.107:0/782825563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 0x55dd248a8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26a6571700 1 -- 172.21.15.107:0/782825563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 msgr2=0x55dd248a8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26a6571700 1 --2- 172.21.15.107:0/782825563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 0x55dd248a8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26a6571700 1 -- 172.21.15.107:0/782825563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a27e0 msgr2=0x55dd248a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26a6571700 1 --2- 172.21.15.107:0/782825563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a27e0 0x55dd248a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26a6571700 1 -- 172.21.15.107:0/782825563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd24647200 con 0x55dd248a3150 2023-07-21T22:28:37.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.552+0000 7f26a6571700 1 --2- 172.21.15.107:0/782825563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a3150 0x55dd248a5a40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f26980084c0 tx=0x7f2698008880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442b3e33d555a1e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.553+0000 7f2694ff9700 1 -- 172.21.15.107:0/782825563 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698014600 con 0x55dd248a3150 2023-07-21T22:28:37.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.553+0000 7f2694ff9700 1 -- 172.21.15.107:0/782825563 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f269800e030 con 0x55dd248a3150 2023-07-21T22:28:37.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.553+0000 7f2694ff9700 1 -- 172.21.15.107:0/782825563 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698013a10 con 0x55dd248a3150 2023-07-21T22:28:37.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.553+0000 7f26b4159b80 1 -- 172.21.15.107:0/782825563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a3150 msgr2=0x55dd248a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.553+0000 7f26b4159b80 1 --2- 172.21.15.107:0/782825563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a3150 0x55dd248a5a40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f26980084c0 tx=0x7f2698008880 comp rx=0 tx=0).stop 2023-07-21T22:28:37.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.553+0000 7f26b4159b80 1 -- 172.21.15.107:0/782825563 shutdown_connections 2023-07-21T22:28:37.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.553+0000 7f26b4159b80 1 --2- 172.21.15.107:0/782825563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a3150 0x55dd248a5a40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.553+0000 7f26b4159b80 1 --2- 172.21.15.107:0/782825563 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a19c0 0x55dd248a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.553+0000 7f26b4159b80 1 --2- 172.21.15.107:0/782825563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a27e0 0x55dd248a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.553+0000 7f26b4159b80 1 -- 172.21.15.107:0/782825563 >> 172.21.15.107:0/782825563 conn(0x55dd24896ea0 msgr2=0x55dd248b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.553+0000 7f26b4159b80 1 -- 172.21.15.107:0/782825563 shutdown_connections 2023-07-21T22:28:37.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.553+0000 7f26b4159b80 1 -- 172.21.15.107:0/782825563 wait complete. 2023-07-21T22:28:37.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26b4159b80 1 -- start start 2023-07-21T22:28:37.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26b4159b80 1 Processor -- start 2023-07-21T22:28:37.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26b4159b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a19c0 0x55dd248ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26b4159b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a27e0 0x55dd248ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26b4159b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a3150 0x55dd2490e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26b4159b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2490e550 con 0x55dd248a19c0 2023-07-21T22:28:37.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26b4159b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2490e690 con 0x55dd248a27e0 2023-07-21T22:28:37.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26b4159b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd2490e850 con 0x55dd248a3150 2023-07-21T22:28:37.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26b4159b80 10 client.0 _ll_register_callbacks cb 0x7f269c09aef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:37.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26a5d70700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a19c0 0x55dd248ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26a5d70700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a19c0 0x55dd248ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47406/0 (socket says 172.21.15.107:47406) 2023-07-21T22:28:37.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26a5d70700 1 -- 172.21.15.107:0/2570688392 learned_addr learned my addr 172.21.15.107:0/2570688392 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26a556f700 1 --2- 172.21.15.107:0/2570688392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a27e0 0x55dd248ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26a6571700 1 --2- 172.21.15.107:0/2570688392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a3150 0x55dd2490e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26a5d70700 1 -- 172.21.15.107:0/2570688392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a3150 msgr2=0x55dd2490e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26a5d70700 1 --2- 172.21.15.107:0/2570688392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a3150 0x55dd2490e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26a5d70700 1 -- 172.21.15.107:0/2570688392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a27e0 msgr2=0x55dd248ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26a5d70700 1 --2- 172.21.15.107:0/2570688392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a27e0 0x55dd248ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.554+0000 7f26a5d70700 1 -- 172.21.15.107:0/2570688392 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2698007d20 con 0x55dd248a19c0 2023-07-21T22:28:37.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26a5d70700 1 --2- 172.21.15.107:0/2570688392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a19c0 0x55dd248ad340 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f269000fd50 tx=0x7f269001bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd2985af8d8f74c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26897fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:37.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26897fa700 1 -- 172.21.15.107:0/2570688392 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269002a040 con 0x55dd248a19c0 2023-07-21T22:28:37.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26897fa700 1 -- 172.21.15.107:0/2570688392 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2690032050 con 0x55dd248a19c0 2023-07-21T22:28:37.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26b4159b80 1 -- 172.21.15.107:0/2570688392 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2690020070 con 0x55dd248a19c0 2023-07-21T22:28:37.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26897fa700 1 -- 172.21.15.107:0/2570688392 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2690032210 con 0x55dd248a19c0 2023-07-21T22:28:37.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26b4159b80 10 client.15132 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:37.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26b4159b80 1 -- 172.21.15.107:0/2570688392 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dd2461f260 con 0x55dd248a19c0 2023-07-21T22:28:37.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26897fa700 1 -- 172.21.15.107:0/2570688392 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2690043020 con 0x55dd248a19c0 2023-07-21T22:28:37.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26897fa700 1 -- 172.21.15.107:0/2570688392 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f269001b540 con 0x55dd248a19c0 2023-07-21T22:28:37.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26b4159b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:37.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26b4159b80 10 client.15132 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T22:28:37.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26b4159b80 20 client.15132 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:37.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26b4159b80 1 --2- 172.21.15.107:0/2570688392 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55dd248b5df0 0x55dd2490ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26b4159b80 4 client.15132 mds_command: new command op to 24493 tid=0[{"prefix": "status"}] 2023-07-21T22:28:37.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.555+0000 7f26b4159b80 1 -- 172.21.15.107:0/2570688392 --> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dd247ae000 con 0x55dd248b5df0 2023-07-21T22:28:37.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26a556f700 1 --2- 172.21.15.107:0/2570688392 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55dd248b5df0 0x55dd2490ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26a556f700 1 --2- 172.21.15.107:0/2570688392 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55dd248b5df0 0x55dd2490ff00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a5612a3c86bbc7c server_cookie=acb737edae977181 in_seq=0 out_seq=0 2023-07-21T22:28:37.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26897fa700 10 client.15132 ms_handle_connect on v2:172.21.15.190:6838/2960500982 2023-07-21T22:28:37.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26897fa700 1 -- 172.21.15.107:0/2570688392 <== mds.1 v2:172.21.15.190:6838/2960500982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55dd247ae000 con 0x55dd248b5df0 2023-07-21T22:28:37.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26897fa700 10 client.15132 handle_command_reply: tid=0 2023-07-21T22:28:37.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26b4159b80 1 client.15132 shutdown 2023-07-21T22:28:37.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26b4159b80 1 -- 172.21.15.107:0/2570688392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a19c0 msgr2=0x55dd248ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26b4159b80 1 --2- 172.21.15.107:0/2570688392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a19c0 0x55dd248ad340 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f269000fd50 tx=0x7f269001bdf0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26b4159b80 1 -- 172.21.15.107:0/2570688392 shutdown_connections 2023-07-21T22:28:37.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26b4159b80 1 --2- 172.21.15.107:0/2570688392 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55dd248b5df0 0x55dd2490ff00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26b4159b80 1 --2- 172.21.15.107:0/2570688392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dd248a3150 0x55dd2490e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26897fa700 0 client.15132 ms_handle_reset on v2:172.21.15.190:6838/2960500982 2023-07-21T22:28:37.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26b4159b80 1 --2- 172.21.15.107:0/2570688392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd248a19c0 0x55dd248ad340 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26b4159b80 1 --2- 172.21.15.107:0/2570688392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dd248a27e0 0x55dd248ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26b4159b80 1 -- 172.21.15.107:0/2570688392 >> 172.21.15.107:0/2570688392 conn(0x55dd24896ea0 msgr2=0x55dd248a8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26b4159b80 1 -- 172.21.15.107:0/2570688392 shutdown_connections 2023-07-21T22:28:37.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.556+0000 7f26b4159b80 1 -- 172.21.15.107:0/2570688392 wait complete. 2023-07-21T22:28:37.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.557+0000 7f26b4159b80 20 client.15132 trim_cache size 0 max 16384 2023-07-21T22:28:37.612 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:28:37.613 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "af99f5d0-2814-11ee-9b34-001a4aab830c", 2023-07-21T22:28:37.613 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 1, 2023-07-21T22:28:37.613 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24493, 2023-07-21T22:28:37.613 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2023-07-21T22:28:37.613 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2023-07-21T22:28:37.613 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2023-07-21T22:28:37.613 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 52.445665312000003, 2023-07-21T22:28:37.614 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2023-07-21T22:28:37.614 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 82, 2023-07-21T22:28:37.614 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:28:37.614 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 68.328236324000002 2023-07-21T22:28:37.614 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:28:37.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.557+0000 7f26a7fd4700 1 -- 172.21.15.107:0/2245222889 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f268c0a1260 msgr2=0x7f268c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.557+0000 7f26a7fd4700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f268c0a1260 0x7f268c0a3710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f26a0100b60 tx=0x7f269000a380 comp rx=0 tx=0).stop 2023-07-21T22:28:37.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.557+0000 7f26a7fd4700 1 -- 172.21.15.107:0/2245222889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26a0100d00 msgr2=0x7f26a0195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.557+0000 7f26a7fd4700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26a0100d00 0x7f26a0195d30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f269c004760 tx=0x7f269c004790 comp rx=0 tx=0).stop 2023-07-21T22:28:37.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.557+0000 7f26a7fd4700 1 -- 172.21.15.107:0/2245222889 shutdown_connections 2023-07-21T22:28:37.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.557+0000 7f26a7fd4700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26a0101690 0x7f26a019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.557+0000 7f26a7fd4700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f268c0a1260 0x7f268c0a3710 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.557+0000 7f26a7fd4700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26a00ffb10 0x7f26a01957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.557+0000 7f26a7fd4700 1 --2- 172.21.15.107:0/2245222889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26a0100d00 0x7f26a0195d30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.557+0000 7f26a7fd4700 1 -- 172.21.15.107:0/2245222889 >> 172.21.15.107:0/2245222889 conn(0x7f26a00fb110 msgr2=0x7f26a01041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.557+0000 7f26a7fd4700 1 -- 172.21.15.107:0/2245222889 shutdown_connections 2023-07-21T22:28:37.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.557+0000 7f26a7fd4700 1 -- 172.21.15.107:0/2245222889 wait complete. 2023-07-21T22:28:37.617 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-21T22:28:37.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:37 smithi190 ceph-mon[98334]: pgmap v277: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:28:37.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:37 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3213844770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:37.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:37 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3782659529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:37.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:37 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/507631016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:37.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:37 smithi132 ceph-mon[97863]: pgmap v277: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:28:37.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:37 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3213844770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:37.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:37 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3782659529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:37.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:37 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/507631016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:37.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.731+0000 7f3b6fc1f700 1 -- 172.21.15.107:0/2089712128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b680fe9e0 msgr2=0x7f3b680fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.731+0000 7f3b6fc1f700 1 --2- 172.21.15.107:0/2089712128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b680fe9e0 0x7f3b680fedf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009990 tx=0x7f3b58009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.731+0000 7f3b6fc1f700 1 -- 172.21.15.107:0/2089712128 shutdown_connections 2023-07-21T22:28:37.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.731+0000 7f3b6fc1f700 1 --2- 172.21.15.107:0/2089712128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b68100560 0x7f3b68106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.731+0000 7f3b6fc1f700 1 --2- 172.21.15.107:0/2089712128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b680fe9e0 0x7f3b680fedf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.731+0000 7f3b6fc1f700 1 --2- 172.21.15.107:0/2089712128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b680ffbd0 0x7f3b68100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.731+0000 7f3b6fc1f700 1 -- 172.21.15.107:0/2089712128 >> 172.21.15.107:0/2089712128 conn(0x7f3b680fa000 msgr2=0x7f3b680fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.731+0000 7f3b6fc1f700 1 -- 172.21.15.107:0/2089712128 shutdown_connections 2023-07-21T22:28:37.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.731+0000 7f3b6fc1f700 1 -- 172.21.15.107:0/2089712128 wait complete. 2023-07-21T22:28:37.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6fc1f700 1 Processor -- start 2023-07-21T22:28:37.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6fc1f700 1 -- start start 2023-07-21T22:28:37.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6fc1f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b680fe9e0 0x7f3b68194790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6fc1f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b680ffbd0 0x7f3b68194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6fc1f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b68100560 0x7f3b6819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6fc1f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6819c310 con 0x7f3b68100560 2023-07-21T22:28:37.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6fc1f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6819c450 con 0x7f3b680fe9e0 2023-07-21T22:28:37.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6fc1f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6819c5e0 con 0x7f3b680ffbd0 2023-07-21T22:28:37.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6e1bc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b68100560 0x7f3b6819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6e1bc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b68100560 0x7f3b6819bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47424/0 (socket says 172.21.15.107:47424) 2023-07-21T22:28:37.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6e1bc700 1 -- 172.21.15.107:0/512554061 learned_addr learned my addr 172.21.15.107:0/512554061 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6d9bb700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b680fe9e0 0x7f3b68194790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6d1ba700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b680ffbd0 0x7f3b68194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6e1bc700 1 -- 172.21.15.107:0/512554061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b680ffbd0 msgr2=0x7f3b68194cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6e1bc700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b680ffbd0 0x7f3b68194cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6e1bc700 1 -- 172.21.15.107:0/512554061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b680fe9e0 msgr2=0x7f3b68194790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6e1bc700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b680fe9e0 0x7f3b68194790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.732+0000 7f3b6e1bc700 1 -- 172.21.15.107:0/512554061 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b580095f0 con 0x7f3b68100560 2023-07-21T22:28:37.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.733+0000 7f3b6e1bc700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b68100560 0x7f3b6819bd40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3b60007ee0 tx=0x7f3b6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.733+0000 7f3b5effd700 1 -- 172.21.15.107:0/512554061 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b60017070 con 0x7f3b68100560 2023-07-21T22:28:37.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.733+0000 7f3b5effd700 1 -- 172.21.15.107:0/512554061 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b6000f040 con 0x7f3b68100560 2023-07-21T22:28:37.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.733+0000 7f3b6fc1f700 1 -- 172.21.15.107:0/512554061 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6819c8c0 con 0x7f3b68100560 2023-07-21T22:28:37.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.733+0000 7f3b5effd700 1 -- 172.21.15.107:0/512554061 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b60005580 con 0x7f3b68100560 2023-07-21T22:28:37.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.733+0000 7f3b6fc1f700 1 -- 172.21.15.107:0/512554061 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6819cd80 con 0x7f3b68100560 2023-07-21T22:28:37.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.735+0000 7f3b5effd700 1 -- 172.21.15.107:0/512554061 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3b6000db20 con 0x7f3b68100560 2023-07-21T22:28:37.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.735+0000 7f3b5effd700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3b540a11e0 0x7f3b540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.735+0000 7f3b6d9bb700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3b540a11e0 0x7f3b540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.735+0000 7f3b5effd700 1 -- 172.21.15.107:0/512554061 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3b600cc020 con 0x7f3b68100560 2023-07-21T22:28:37.791 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.736+0000 7f3b6d9bb700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3b540a11e0 0x7f3b540a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800b5c0 tx=0x7f3b58005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.791 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.736+0000 7f3b7bda4b80 1 Processor -- start 2023-07-21T22:28:37.791 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.736+0000 7f3b7bda4b80 1 -- start start 2023-07-21T22:28:37.791 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b7bda4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d107e0 0x55ab57d10bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.791 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b7bda4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d0f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.791 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b7bda4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 0x55ab57d0fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.791 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b7bda4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab57c14c60 con 0x55ab57d0f9c0 2023-07-21T22:28:37.792 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b7bda4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab57c679f0 con 0x55ab57d11150 2023-07-21T22:28:37.792 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b7bda4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab57c60e80 con 0x55ab57d107e0 2023-07-21T22:28:37.792 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b6e1bc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 0x55ab57d0fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.792 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b6e1bc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 0x55ab57d0fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47432/0 (socket says 172.21.15.107:47432) 2023-07-21T22:28:37.792 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b6e1bc700 1 -- 172.21.15.107:0/3102532532 learned_addr learned my addr 172.21.15.107:0/3102532532 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.792 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b6d1ba700 1 --2- 172.21.15.107:0/3102532532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d0f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.793 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b6d9bb700 1 --2- 172.21.15.107:0/3102532532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d107e0 0x55ab57d10bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.793 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b6e1bc700 1 -- 172.21.15.107:0/3102532532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d107e0 msgr2=0x55ab57d10bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.793 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b6e1bc700 1 --2- 172.21.15.107:0/3102532532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d107e0 0x55ab57d10bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.793 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b6e1bc700 1 -- 172.21.15.107:0/3102532532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 msgr2=0x55ab57d0f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.793 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b6e1bc700 1 --2- 172.21.15.107:0/3102532532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d0f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.793 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b6e1bc700 1 -- 172.21.15.107:0/3102532532 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab57a8ed90 con 0x55ab57d0f9c0 2023-07-21T22:28:37.793 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b6e1bc700 1 --2- 172.21.15.107:0/3102532532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 0x55ab57d0fe40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3b600a13a0 tx=0x7f3b60095840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d40deeb86d54f02 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.794 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b53fff700 1 -- 172.21.15.107:0/3102532532 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b600a18d0 con 0x55ab57d0f9c0 2023-07-21T22:28:37.794 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b53fff700 1 -- 172.21.15.107:0/3102532532 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b600975c0 con 0x55ab57d0f9c0 2023-07-21T22:28:37.794 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.737+0000 7f3b53fff700 1 -- 172.21.15.107:0/3102532532 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b60095b90 con 0x55ab57d0f9c0 2023-07-21T22:28:37.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.738+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/3102532532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 msgr2=0x55ab57d0fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.738+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/3102532532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 0x55ab57d0fe40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3b600a13a0 tx=0x7f3b60095840 comp rx=0 tx=0).stop 2023-07-21T22:28:37.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.738+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/3102532532 shutdown_connections 2023-07-21T22:28:37.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.738+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/3102532532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 0x55ab57d0fe40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.738+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/3102532532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d107e0 0x55ab57d10bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.738+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/3102532532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d0f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.738+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/3102532532 >> 172.21.15.107:0/3102532532 conn(0x55ab57d04ea0 msgr2=0x55ab57d0a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.738+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/3102532532 shutdown_connections 2023-07-21T22:28:37.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.738+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/3102532532 wait complete. 2023-07-21T22:28:37.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.738+0000 7f3b7bda4b80 1 -- start start 2023-07-21T22:28:37.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.738+0000 7f3b7bda4b80 1 Processor -- start 2023-07-21T22:28:37.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b7bda4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 0x55ab57d1b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b7bda4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d107e0 0x55ab57d1b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b7bda4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d22870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b7bda4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab57c00a70 con 0x55ab57d0f9c0 2023-07-21T22:28:37.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b7bda4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab57c00770 con 0x55ab57d11150 2023-07-21T22:28:37.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b7bda4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab57ae48d0 con 0x55ab57d107e0 2023-07-21T22:28:37.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b7bda4b80 10 client.0 _ll_register_callbacks cb 0x55ab57bfe710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:37.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b6d9bb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 0x55ab57d1b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b6d9bb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 0x55ab57d1b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47434/0 (socket says 172.21.15.107:47434) 2023-07-21T22:28:37.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b6d9bb700 1 -- 172.21.15.107:0/4142067117 learned_addr learned my addr 172.21.15.107:0/4142067117 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b6d1ba700 1 --2- 172.21.15.107:0/4142067117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d107e0 0x55ab57d1b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b6e1bc700 1 --2- 172.21.15.107:0/4142067117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d22870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b6d9bb700 1 -- 172.21.15.107:0/4142067117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d107e0 msgr2=0x55ab57d1b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b6d9bb700 1 --2- 172.21.15.107:0/4142067117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d107e0 0x55ab57d1b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b6d9bb700 1 -- 172.21.15.107:0/4142067117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 msgr2=0x55ab57d22870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b6d9bb700 1 --2- 172.21.15.107:0/4142067117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d22870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b6d9bb700 1 -- 172.21.15.107:0/4142067117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6819cd80 con 0x55ab57d0f9c0 2023-07-21T22:28:37.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b6d9bb700 1 --2- 172.21.15.107:0/4142067117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 0x55ab57d1b2f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3b680ffa30 tx=0x7f3b5801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91a491db506ebdb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b517fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:37.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.739+0000 7f3b517fa700 1 -- 172.21.15.107:0/4142067117 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5800be70 con 0x55ab57d0f9c0 2023-07-21T22:28:37.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b517fa700 1 -- 172.21.15.107:0/4142067117 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3b58003f00 con 0x55ab57d0f9c0 2023-07-21T22:28:37.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/4142067117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab57bba4a0 con 0x55ab57d0f9c0 2023-07-21T22:28:37.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b517fa700 1 -- 172.21.15.107:0/4142067117 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b580040c0 con 0x55ab57d0f9c0 2023-07-21T22:28:37.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b7bda4b80 10 client.15144 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:37.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/4142067117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ab57a77c10 con 0x55ab57d0f9c0 2023-07-21T22:28:37.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b517fa700 1 -- 172.21.15.107:0/4142067117 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab57bba4a0 con 0x55ab57d0f9c0 2023-07-21T22:28:37.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b517fa700 1 -- 172.21.15.107:0/4142067117 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3b5800f460 con 0x55ab57d0f9c0 2023-07-21T22:28:37.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b7bda4b80 10 client.15144 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:37.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b7bda4b80 10 client.15144 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:28:37.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b7bda4b80 20 client.15144 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:37.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/4142067117 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ab57d26570 0x55ab57d25970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b7bda4b80 4 client.15144 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:37.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/4142067117 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab57c59e60 con 0x55ab57d26570 2023-07-21T22:28:37.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.740+0000 7f3b6d1ba700 1 --2- 172.21.15.107:0/4142067117 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ab57d26570 0x55ab57d25970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.741+0000 7f3b517fa700 10 client.15144 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:28:37.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.741+0000 7f3b6d1ba700 1 --2- 172.21.15.107:0/4142067117 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ab57d26570 0x55ab57d25970 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36d9ed1e5f55e1e1 server_cookie=de319f3d1ed7f4af in_seq=0 out_seq=0 2023-07-21T22:28:37.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.741+0000 7f3b517fa700 1 -- 172.21.15.107:0/4142067117 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ab57c59e60 con 0x55ab57d26570 2023-07-21T22:28:37.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.741+0000 7f3b517fa700 10 client.15144 handle_command_reply: tid=0 2023-07-21T22:28:37.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.742+0000 7f3b7bda4b80 1 client.15144 shutdown 2023-07-21T22:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.742+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/4142067117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 msgr2=0x55ab57d1b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.742+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/4142067117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 0x55ab57d1b2f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3b680ffa30 tx=0x7f3b5801a050 comp rx=0 tx=0).stop 2023-07-21T22:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.742+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/4142067117 shutdown_connections 2023-07-21T22:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.742+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/4142067117 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ab57d26570 0x55ab57d25970 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.742+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/4142067117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d22870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.742+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/4142067117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d0f9c0 0x55ab57d1b2f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.742+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/4142067117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d107e0 0x55ab57d1b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.742+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/4142067117 >> 172.21.15.107:0/4142067117 conn(0x55ab57d04ea0 msgr2=0x55ab57d099b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.742+0000 7f3b517fa700 0 client.15144 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:28:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.742+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/4142067117 shutdown_connections 2023-07-21T22:28:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.742+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/4142067117 wait complete. 2023-07-21T22:28:37.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.742+0000 7f3b7bda4b80 20 client.15144 trim_cache size 0 max 16384 2023-07-21T22:28:37.806 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.759+0000 7f3b7bda4b80 1 Processor -- start 2023-07-21T22:28:37.806 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.759+0000 7f3b7bda4b80 1 -- start start 2023-07-21T22:28:37.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.759+0000 7f3b7bda4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d0f9c0 0x55ab57d25970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.759+0000 7f3b7bda4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 0x55ab57d25eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.759+0000 7f3b7bda4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d22830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.759+0000 7f3b7bda4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab57bba4a0 con 0x55ab57d107e0 2023-07-21T22:28:37.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.759+0000 7f3b7bda4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab57d24560 con 0x55ab57d11150 2023-07-21T22:28:37.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.759+0000 7f3b7bda4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab57c2f790 con 0x55ab57d0f9c0 2023-07-21T22:28:37.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.759+0000 7f3b6d1ba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 0x55ab57d25eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.759+0000 7f3b6d1ba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 0x55ab57d25eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47444/0 (socket says 172.21.15.107:47444) 2023-07-21T22:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.759+0000 7f3b6d1ba700 1 -- 172.21.15.107:0/3112551671 learned_addr learned my addr 172.21.15.107:0/3112551671 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.760+0000 7f3b6d9bb700 1 --2- 172.21.15.107:0/3112551671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d0f9c0 0x55ab57d25970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.760+0000 7f3b6e1bc700 1 --2- 172.21.15.107:0/3112551671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d22830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.760+0000 7f3b6d1ba700 1 -- 172.21.15.107:0/3112551671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d0f9c0 msgr2=0x55ab57d25970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.760+0000 7f3b6d1ba700 1 --2- 172.21.15.107:0/3112551671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d0f9c0 0x55ab57d25970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.760+0000 7f3b6d1ba700 1 -- 172.21.15.107:0/3112551671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 msgr2=0x55ab57d22830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.760+0000 7f3b6d1ba700 1 --2- 172.21.15.107:0/3112551671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d22830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.760+0000 7f3b6d1ba700 1 -- 172.21.15.107:0/3112551671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab57ab5200 con 0x55ab57d107e0 2023-07-21T22:28:37.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.760+0000 7f3b6d1ba700 1 --2- 172.21.15.107:0/3112551671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 0x55ab57d25eb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3b64005950 tx=0x7f3b6400a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf9723ba5902921 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.760+0000 7f3b5cdac700 1 -- 172.21.15.107:0/3112551671 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64015040 con 0x55ab57d107e0 2023-07-21T22:28:37.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.760+0000 7f3b5cdac700 1 -- 172.21.15.107:0/3112551671 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b64016040 con 0x55ab57d107e0 2023-07-21T22:28:37.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.760+0000 7f3b5cdac700 1 -- 172.21.15.107:0/3112551671 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6401e780 con 0x55ab57d107e0 2023-07-21T22:28:37.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/3112551671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 msgr2=0x55ab57d25eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.811 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/3112551671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 0x55ab57d25eb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3b64005950 tx=0x7f3b6400a300 comp rx=0 tx=0).stop 2023-07-21T22:28:37.811 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/3112551671 shutdown_connections 2023-07-21T22:28:37.811 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/3112551671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d22830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.811 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/3112551671 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d0f9c0 0x55ab57d25970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.811 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/3112551671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 0x55ab57d25eb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.811 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/3112551671 >> 172.21.15.107:0/3112551671 conn(0x55ab57d04ea0 msgr2=0x55ab57d17d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.811 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/3112551671 shutdown_connections 2023-07-21T22:28:37.811 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/3112551671 wait complete. 2023-07-21T22:28:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 -- start start 2023-07-21T22:28:37.812 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 Processor -- start 2023-07-21T22:28:37.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d0f9c0 0x55ab57d1d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 0x55ab57d1da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d7a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab57d1dff0 con 0x55ab57d107e0 2023-07-21T22:28:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab57d1e130 con 0x55ab57d11150 2023-07-21T22:28:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab57d7a950 con 0x55ab57d0f9c0 2023-07-21T22:28:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b7bda4b80 10 client.0 _ll_register_callbacks cb 0x7f3b5800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.761+0000 7f3b6d1ba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 0x55ab57d1da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b6d1ba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 0x55ab57d1da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47450/0 (socket says 172.21.15.107:47450) 2023-07-21T22:28:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b6d1ba700 1 -- 172.21.15.107:0/1939146641 learned_addr learned my addr 172.21.15.107:0/1939146641 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b6e1bc700 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d7a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b6d9bb700 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d0f9c0 0x55ab57d1d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b6d1ba700 1 -- 172.21.15.107:0/1939146641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d0f9c0 msgr2=0x55ab57d1d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b6d1ba700 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d0f9c0 0x55ab57d1d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b6d1ba700 1 -- 172.21.15.107:0/1939146641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 msgr2=0x55ab57d7a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:37.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b6d1ba700 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d7a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b6d1ba700 1 -- 172.21.15.107:0/1939146641 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6400ab20 con 0x55ab57d107e0 2023-07-21T22:28:37.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b6d9bb700 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d0f9c0 0x55ab57d1d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:37.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b6d1ba700 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 0x55ab57d1da70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3b64026000 tx=0x7f3b64010aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36cbdcae2913773 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:37.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b517fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:37.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b517fa700 1 -- 172.21.15.107:0/1939146641 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64004220 con 0x55ab57d107e0 2023-07-21T22:28:37.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b517fa700 1 -- 172.21.15.107:0/1939146641 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3b64004380 con 0x55ab57d107e0 2023-07-21T22:28:37.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/1939146641 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b600975c0 con 0x55ab57d107e0 2023-07-21T22:28:37.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b517fa700 1 -- 172.21.15.107:0/1939146641 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b64004540 con 0x55ab57d107e0 2023-07-21T22:28:37.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b7bda4b80 10 client.15150 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:37.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/1939146641 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ab57a8d260 con 0x55ab57d107e0 2023-07-21T22:28:37.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.762+0000 7f3b517fa700 1 -- 172.21.15.107:0/1939146641 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b600975c0 con 0x55ab57d107e0 2023-07-21T22:28:37.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b517fa700 1 -- 172.21.15.107:0/1939146641 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3b64003680 con 0x55ab57d107e0 2023-07-21T22:28:37.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b7bda4b80 10 client.15150 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:37.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b7bda4b80 10 client.15150 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:28:37.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b7bda4b80 20 client.15150 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:37.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ab57d7f5d0 0x55ab57d7bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:37.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b7bda4b80 4 client.15150 mds_command: new command op to 24533 tid=0[{"prefix": "status"}] 2023-07-21T22:28:37.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/1939146641 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ab57c1c000 con 0x55ab57d7f5d0 2023-07-21T22:28:37.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b6d9bb700 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ab57d7f5d0 0x55ab57d7bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:37.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b517fa700 10 client.15150 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:28:37.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b6d9bb700 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ab57d7f5d0 0x55ab57d7bf20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c7bc244d0303849 server_cookie=5f7e3bcbb3181585 in_seq=0 out_seq=0 2023-07-21T22:28:37.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b517fa700 1 -- 172.21.15.107:0/1939146641 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ab57c1c000 con 0x55ab57d7f5d0 2023-07-21T22:28:37.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b517fa700 10 client.15150 handle_command_reply: tid=0 2023-07-21T22:28:37.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b7bda4b80 1 client.15150 shutdown 2023-07-21T22:28:37.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/1939146641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 msgr2=0x55ab57d1da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.763+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 0x55ab57d1da70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3b64026000 tx=0x7f3b64010aa0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/1939146641 shutdown_connections 2023-07-21T22:28:37.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ab57d7f5d0 0x55ab57d7bf20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab57d11150 0x55ab57d7a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ab57d0f9c0 0x55ab57d1d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b7bda4b80 1 --2- 172.21.15.107:0/1939146641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ab57d107e0 0x55ab57d1da70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b517fa700 0 client.15150 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:28:37.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/1939146641 >> 172.21.15.107:0/1939146641 conn(0x55ab57d04ea0 msgr2=0x55ab57d13a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/1939146641 shutdown_connections 2023-07-21T22:28:37.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b7bda4b80 1 -- 172.21.15.107:0/1939146641 wait complete. 2023-07-21T22:28:37.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b7bda4b80 20 client.15150 trim_cache size 0 max 16384 2023-07-21T22:28:37.822 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:28:37.822 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "af99f5d0-2814-11ee-9b34-001a4aab830c", 2023-07-21T22:28:37.822 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2023-07-21T22:28:37.822 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24533, 2023-07-21T22:28:37.822 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2023-07-21T22:28:37.822 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2023-07-21T22:28:37.823 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2023-07-21T22:28:37.823 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 55.691368351999998, 2023-07-21T22:28:37.823 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2023-07-21T22:28:37.823 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 82, 2023-07-21T22:28:37.823 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:28:37.823 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 61.799669590000001 2023-07-21T22:28:37.823 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:28:37.824 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b6fc1f700 1 -- 172.21.15.107:0/512554061 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3b540a11e0 msgr2=0x7f3b540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b6fc1f700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3b540a11e0 0x7f3b540a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800b5c0 tx=0x7f3b58005fb0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b6fc1f700 1 -- 172.21.15.107:0/512554061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b68100560 msgr2=0x7f3b6819bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:37.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b6fc1f700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b68100560 0x7f3b6819bd40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3b60007ee0 tx=0x7f3b6000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b6fc1f700 1 -- 172.21.15.107:0/512554061 shutdown_connections 2023-07-21T22:28:37.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b6fc1f700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b68100560 0x7f3b6819bd40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b6fc1f700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b680fe9e0 0x7f3b68194790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b6fc1f700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3b540a11e0 0x7f3b540a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b6fc1f700 1 --2- 172.21.15.107:0/512554061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b680ffbd0 0x7f3b68194cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:37.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b6fc1f700 1 -- 172.21.15.107:0/512554061 >> 172.21.15.107:0/512554061 conn(0x7f3b680fa000 msgr2=0x7f3b680fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:37.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b6fc1f700 1 -- 172.21.15.107:0/512554061 shutdown_connections 2023-07-21T22:28:37.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:37.764+0000 7f3b6fc1f700 1 -- 172.21.15.107:0/512554061 wait complete. 2023-07-21T22:28:37.826 INFO:tasks.cephadm:Setup complete, yielding 2023-07-21T22:28:37.827 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-21T22:28:37.837 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2023-07-21T22:28:37.838 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- bash -c 'ceph orch status' 2023-07-21T22:28:38.386 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:39.463 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:39 smithi107 ceph-mon[96543]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-21T22:28:39.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.559+0000 7f6d24028700 1 -- 172.21.15.107:0/2421263757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0ffc80 msgr2=0x7f6d1c1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:39.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.559+0000 7f6d24028700 1 --2- 172.21.15.107:0/2421263757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0ffc80 0x7f6d1c1000d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009990 tx=0x7f6d0c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:39.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.559+0000 7f6d24028700 1 -- 172.21.15.107:0/2421263757 shutdown_connections 2023-07-21T22:28:39.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.559+0000 7f6d24028700 1 --2- 172.21.15.107:0/2421263757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c100610 0x7f6d1c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:39.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.559+0000 7f6d24028700 1 --2- 172.21.15.107:0/2421263757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d1c0fea80 0x7f6d1c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:39.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.559+0000 7f6d24028700 1 --2- 172.21.15.107:0/2421263757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0ffc80 0x7f6d1c1000d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:39.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.559+0000 7f6d24028700 1 -- 172.21.15.107:0/2421263757 >> 172.21.15.107:0/2421263757 conn(0x7f6d1c0fa0b0 msgr2=0x7f6d1c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:39.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.559+0000 7f6d24028700 1 -- 172.21.15.107:0/2421263757 shutdown_connections 2023-07-21T22:28:39.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.559+0000 7f6d24028700 1 -- 172.21.15.107:0/2421263757 wait complete. 2023-07-21T22:28:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.559+0000 7f6d24028700 1 Processor -- start 2023-07-21T22:28:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.559+0000 7f6d24028700 1 -- start start 2023-07-21T22:28:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d24028700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c0fea80 0x7f6d1c194740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d24028700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0ffc80 0x7f6d1c194c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d24028700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d1c100610 0x7f6d1c19bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d24028700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c19c2d0 con 0x7f6d1c0ffc80 2023-07-21T22:28:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d24028700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c19c410 con 0x7f6d1c100610 2023-07-21T22:28:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d24028700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c19c5a0 con 0x7f6d1c0fea80 2023-07-21T22:28:39.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d215c3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0ffc80 0x7f6d1c194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:39.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d215c3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0ffc80 0x7f6d1c194c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47460/0 (socket says 172.21.15.107:47460) 2023-07-21T22:28:39.609 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d215c3700 1 -- 172.21.15.107:0/2062283687 learned_addr learned my addr 172.21.15.107:0/2062283687 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d225c5700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d1c100610 0x7f6d1c19bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d21dc4700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c0fea80 0x7f6d1c194740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d215c3700 1 -- 172.21.15.107:0/2062283687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c0fea80 msgr2=0x7f6d1c194740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:39.610 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d215c3700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c0fea80 0x7f6d1c194740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d215c3700 1 -- 172.21.15.107:0/2062283687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d1c100610 msgr2=0x7f6d1c19bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d215c3700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d1c100610 0x7f6d1c19bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d215c3700 1 -- 172.21.15.107:0/2062283687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c0095f0 con 0x7f6d1c0ffc80 2023-07-21T22:28:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d225c5700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d1c100610 0x7f6d1c19bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d21dc4700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c0fea80 0x7f6d1c194740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d215c3700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0ffc80 0x7f6d1c194c80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c005910 tx=0x7f6d0c00f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d12ffd700 1 -- 172.21.15.107:0/2062283687 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c01d070 con 0x7f6d1c0ffc80 2023-07-21T22:28:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d24028700 1 -- 172.21.15.107:0/2062283687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c19c820 con 0x7f6d1c0ffc80 2023-07-21T22:28:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d12ffd700 1 -- 172.21.15.107:0/2062283687 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d0c00fd10 con 0x7f6d1c0ffc80 2023-07-21T22:28:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d12ffd700 1 -- 172.21.15.107:0/2062283687 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0176e0 con 0x7f6d1c0ffc80 2023-07-21T22:28:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.560+0000 7f6d24028700 1 -- 172.21.15.107:0/2062283687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c067fd0 con 0x7f6d1c0ffc80 2023-07-21T22:28:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.561+0000 7f6d24028700 1 -- 172.21.15.107:0/2062283687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c10a2c0 con 0x7f6d1c0ffc80 2023-07-21T22:28:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.562+0000 7f6d12ffd700 1 -- 172.21.15.107:0/2062283687 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6d0c022470 con 0x7f6d1c0ffc80 2023-07-21T22:28:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.563+0000 7f6d12ffd700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6d080a33f0 0x7f6d080a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.563+0000 7f6d21dc4700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6d080a33f0 0x7f6d080a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.563+0000 7f6d12ffd700 1 -- 172.21.15.107:0/2062283687 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d0c0ce250 con 0x7f6d1c0ffc80 2023-07-21T22:28:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.563+0000 7f6d21dc4700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6d080a33f0 0x7f6d080a58a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d18005fd0 tx=0x7f6d18005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.566+0000 7f6d12ffd700 1 -- 172.21.15.107:0/2062283687 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6d0c097a80 con 0x7f6d1c0ffc80 2023-07-21T22:28:39.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:39 smithi190 ceph-mon[98334]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-21T22:28:39.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:39 smithi132 ceph-mon[97863]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-21T22:28:39.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.770+0000 7f6d24028700 1 -- 172.21.15.107:0/2062283687 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d1c105ad0 con 0x7f6d080a33f0 2023-07-21T22:28:39.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.771+0000 7f6d12ffd700 1 -- 172.21.15.107:0/2062283687 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6d1c105ad0 con 0x7f6d080a33f0 2023-07-21T22:28:39.817 INFO:teuthology.orchestra.run.smithi107.stdout:Backend: cephadm 2023-07-21T22:28:39.817 INFO:teuthology.orchestra.run.smithi107.stdout:Available: Yes 2023-07-21T22:28:39.817 INFO:teuthology.orchestra.run.smithi107.stdout:Paused: No 2023-07-21T22:28:39.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.773+0000 7f6d24028700 1 -- 172.21.15.107:0/2062283687 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6d080a33f0 msgr2=0x7f6d080a58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:39.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.773+0000 7f6d24028700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6d080a33f0 0x7f6d080a58a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d18005fd0 tx=0x7f6d18005e40 comp rx=0 tx=0).stop 2023-07-21T22:28:39.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.773+0000 7f6d24028700 1 -- 172.21.15.107:0/2062283687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0ffc80 msgr2=0x7f6d1c194c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:39.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.773+0000 7f6d24028700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0ffc80 0x7f6d1c194c80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c005910 tx=0x7f6d0c00f830 comp rx=0 tx=0).stop 2023-07-21T22:28:39.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.773+0000 7f6d24028700 1 -- 172.21.15.107:0/2062283687 shutdown_connections 2023-07-21T22:28:39.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.773+0000 7f6d24028700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d1c100610 0x7f6d1c19bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:39.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.773+0000 7f6d24028700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6d080a33f0 0x7f6d080a58a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:39.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.773+0000 7f6d24028700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d1c0fea80 0x7f6d1c194740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:39.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.773+0000 7f6d24028700 1 --2- 172.21.15.107:0/2062283687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0ffc80 0x7f6d1c194c80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:39.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.773+0000 7f6d24028700 1 -- 172.21.15.107:0/2062283687 >> 172.21.15.107:0/2062283687 conn(0x7f6d1c0fa0b0 msgr2=0x7f6d1c103120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:39.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.773+0000 7f6d24028700 1 -- 172.21.15.107:0/2062283687 shutdown_connections 2023-07-21T22:28:39.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:39.773+0000 7f6d24028700 1 -- 172.21.15.107:0/2062283687 wait complete. 2023-07-21T22:28:40.327 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-21T22:28:40.876 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:41.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:41 smithi107 ceph-mon[96543]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:41.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:41 smithi107 ceph-mon[96543]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:28:41.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:41 smithi190 ceph-mon[98334]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:41.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:41 smithi190 ceph-mon[98334]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:28:41.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:41 smithi132 ceph-mon[97863]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:41.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:41 smithi132 ceph-mon[97863]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:28:42.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.070+0000 7f0a29b27700 1 -- 172.21.15.107:0/4074379764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a24100d90 msgr2=0x7f0a241011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:42.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.070+0000 7f0a29b27700 1 --2- 172.21.15.107:0/4074379764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a24100d90 0x7f0a241011e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0a14009990 tx=0x7f0a14009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:42.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.070+0000 7f0a29b27700 1 -- 172.21.15.107:0/4074379764 shutdown_connections 2023-07-21T22:28:42.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.070+0000 7f0a29b27700 1 --2- 172.21.15.107:0/4074379764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a24101720 0x7f0a24107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:42.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.070+0000 7f0a29b27700 1 --2- 172.21.15.107:0/4074379764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a240ffb90 0x7f0a240fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:42.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.070+0000 7f0a29b27700 1 --2- 172.21.15.107:0/4074379764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a24100d90 0x7f0a241011e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:42.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.070+0000 7f0a29b27700 1 -- 172.21.15.107:0/4074379764 >> 172.21.15.107:0/4074379764 conn(0x7f0a240fb1a0 msgr2=0x7f0a240fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:42.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.070+0000 7f0a29b27700 1 -- 172.21.15.107:0/4074379764 shutdown_connections 2023-07-21T22:28:42.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.070+0000 7f0a29b27700 1 -- 172.21.15.107:0/4074379764 wait complete. 2023-07-21T22:28:42.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.071+0000 7f0a29b27700 1 Processor -- start 2023-07-21T22:28:42.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.071+0000 7f0a29b27700 1 -- start start 2023-07-21T22:28:42.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.071+0000 7f0a29b27700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a240ffb90 0x7f0a240742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:42.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.071+0000 7f0a29b27700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a24100d90 0x7f0a24072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:42.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.071+0000 7f0a29b27700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a24101720 0x7f0a24072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:42.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.071+0000 7f0a29b27700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240735c0 con 0x7f0a24101720 2023-07-21T22:28:42.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.071+0000 7f0a29b27700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a24073730 con 0x7f0a24100d90 2023-07-21T22:28:42.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.071+0000 7f0a29b27700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2419f840 con 0x7f0a240ffb90 2023-07-21T22:28:42.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.071+0000 7f0a23fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a24101720 0x7f0a24072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:42.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.071+0000 7f0a23fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a24101720 0x7f0a24072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47474/0 (socket says 172.21.15.107:47474) 2023-07-21T22:28:42.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.071+0000 7f0a23fff700 1 -- 172.21.15.107:0/2218149147 learned_addr learned my addr 172.21.15.107:0/2218149147 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:42.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a22ffd700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a24100d90 0x7f0a24072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:42.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a237fe700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a240ffb90 0x7f0a240742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:42.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a23fff700 1 -- 172.21.15.107:0/2218149147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a240ffb90 msgr2=0x7f0a240742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:42.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a23fff700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a240ffb90 0x7f0a240742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:42.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a23fff700 1 -- 172.21.15.107:0/2218149147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a24100d90 msgr2=0x7f0a24072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:42.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a23fff700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a24100d90 0x7f0a24072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:42.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a23fff700 1 -- 172.21.15.107:0/2218149147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a140095f0 con 0x7f0a24101720 2023-07-21T22:28:42.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a22ffd700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a24100d90 0x7f0a24072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:42.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a23fff700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a24101720 0x7f0a24072e40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c007d70 tx=0x7f0a1c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:42.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a20ff9700 1 -- 172.21.15.107:0/2218149147 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c00ad40 con 0x7f0a24101720 2023-07-21T22:28:42.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a29b27700 1 -- 172.21.15.107:0/2218149147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2419fb20 con 0x7f0a24101720 2023-07-21T22:28:42.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a20ff9700 1 -- 172.21.15.107:0/2218149147 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a1c0128e0 con 0x7f0a24101720 2023-07-21T22:28:42.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a20ff9700 1 -- 172.21.15.107:0/2218149147 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c011780 con 0x7f0a24101720 2023-07-21T22:28:42.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.072+0000 7f0a29b27700 1 -- 172.21.15.107:0/2218149147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a241a0020 con 0x7f0a24101720 2023-07-21T22:28:42.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.073+0000 7f0a29b27700 1 -- 172.21.15.107:0/2218149147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a24107100 con 0x7f0a24101720 2023-07-21T22:28:42.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.074+0000 7f0a20ff9700 1 -- 172.21.15.107:0/2218149147 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f0a1c012a50 con 0x7f0a24101720 2023-07-21T22:28:42.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.075+0000 7f0a20ff9700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0a0c0a11e0 0x7f0a0c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:42.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.075+0000 7f0a237fe700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0a0c0a11e0 0x7f0a0c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:42.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.075+0000 7f0a20ff9700 1 -- 172.21.15.107:0/2218149147 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0a1c0ccf90 con 0x7f0a24101720 2023-07-21T22:28:42.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.075+0000 7f0a237fe700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0a0c0a11e0 0x7f0a0c0a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0a24100bf0 tx=0x7f0a1800b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:42.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.078+0000 7f0a20ff9700 1 -- 172.21.15.107:0/2218149147 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0a1c096740 con 0x7f0a24101720 2023-07-21T22:28:42.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.275+0000 7f0a29b27700 1 -- 172.21.15.107:0/2218149147 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a24060950 con 0x7f0a0c0a11e0 2023-07-21T22:28:42.334 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.288+0000 7f0a20ff9700 1 -- 172.21.15.107:0/2218149147 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f0a24060950 con 0x7f0a0c0a11e0 2023-07-21T22:28:42.334 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-21T22:28:42.335 INFO:teuthology.orchestra.run.smithi107.stdout:mds.a smithi107 running (69s) 59s ago 68s 13.2M - 18.0.0-4735-gbffe049d fa7da5dee971 4e99dd7a0a2c 2023-07-21T22:28:42.335 INFO:teuthology.orchestra.run.smithi107.stdout:mds.b smithi132 running (66s) 60s ago 66s 14.0M - 18.0.0-4735-gbffe049d fa7da5dee971 9c013ed61989 2023-07-21T22:28:42.335 INFO:teuthology.orchestra.run.smithi107.stdout:mds.c smithi190 running (77s) 60s ago 77s 13.4M - 18.0.0-4735-gbffe049d fa7da5dee971 2a7e428e6dae 2023-07-21T22:28:42.335 INFO:teuthology.orchestra.run.smithi107.stdout:mds.d smithi107 running (84s) 59s ago 83s 14.3M - 18.0.0-4735-gbffe049d fa7da5dee971 05ed6aa1af96 2023-07-21T22:28:42.335 INFO:teuthology.orchestra.run.smithi107.stdout:mds.e smithi132 running (82s) 60s ago 81s 15.7M - 18.0.0-4735-gbffe049d fa7da5dee971 77b76fdf4b5d 2023-07-21T22:28:42.335 INFO:teuthology.orchestra.run.smithi107.stdout:mds.f smithi190 running (80s) 60s ago 79s 15.7M - 18.0.0-4735-gbffe049d fa7da5dee971 679d63805bc2 2023-07-21T22:28:42.335 INFO:teuthology.orchestra.run.smithi107.stdout:mds.g smithi107 running (71s) 59s ago 70s 14.3M - 18.0.0-4735-gbffe049d fa7da5dee971 6e9e5728cecf 2023-07-21T22:28:42.336 INFO:teuthology.orchestra.run.smithi107.stdout:mds.h smithi132 running (75s) 60s ago 75s 14.4M - 18.0.0-4735-gbffe049d fa7da5dee971 121f654ce162 2023-07-21T22:28:42.336 INFO:teuthology.orchestra.run.smithi107.stdout:mds.i smithi190 running (73s) 60s ago 73s 14.3M - 18.0.0-4735-gbffe049d fa7da5dee971 b470e884a143 2023-07-21T22:28:42.336 INFO:teuthology.orchestra.run.smithi107.stdout:mds.j smithi107 running (86s) 59s ago 86s 16.8M - 18.0.0-4735-gbffe049d fa7da5dee971 333e551620de 2023-07-21T22:28:42.336 INFO:teuthology.orchestra.run.smithi107.stdout:mds.k smithi132 running (88s) 60s ago 88s 13.3M - 18.0.0-4735-gbffe049d fa7da5dee971 45df9a83c818 2023-07-21T22:28:42.336 INFO:teuthology.orchestra.run.smithi107.stdout:mds.l smithi190 running (90s) 60s ago 90s 13.5M - 18.0.0-4735-gbffe049d fa7da5dee971 388fa1c51875 2023-07-21T22:28:42.336 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.x smithi107 *:9283,8765 running (7m) 59s ago 7m 510M - 18.0.0-4735-gbffe049d fa7da5dee971 dcfdbbf70d28 2023-07-21T22:28:42.336 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.y smithi132 *:8443,8765 running (5m) 60s ago 5m 447M - 18.0.0-4735-gbffe049d fa7da5dee971 186e0c403623 2023-07-21T22:28:42.337 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.z smithi190 *:8443,8765 running (5m) 60s ago 5m 447M - 18.0.0-4735-gbffe049d fa7da5dee971 ebf3befdbd6c 2023-07-21T22:28:42.337 INFO:teuthology.orchestra.run.smithi107.stdout:mon.a smithi107 running (7m) 59s ago 7m 86.0M 2048M 18.0.0-4735-gbffe049d fa7da5dee971 1b44ab1ea505 2023-07-21T22:28:42.337 INFO:teuthology.orchestra.run.smithi107.stdout:mon.b smithi132 running (6m) 60s ago 6m 57.6M 2048M 18.0.0-4735-gbffe049d fa7da5dee971 a2ce252f5187 2023-07-21T22:28:42.337 INFO:teuthology.orchestra.run.smithi107.stdout:mon.c smithi190 running (6m) 60s ago 6m 60.6M 2048M 18.0.0-4735-gbffe049d fa7da5dee971 799a149ed230 2023-07-21T22:28:42.337 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (5m) 59s ago 5m 58.8M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 cb8042469ea7 2023-07-21T22:28:42.337 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi132 running (5m) 60s ago 5m 60.5M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 12b561c23c10 2023-07-21T22:28:42.337 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi190 running (4m) 60s ago 4m 57.4M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 9126fda397ff 2023-07-21T22:28:42.337 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi107 running (4m) 59s ago 4m 51.6M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 13feb548db94 2023-07-21T22:28:42.338 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi132 running (4m) 60s ago 4m 51.0M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 158327a529e0 2023-07-21T22:28:42.338 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi190 running (3m) 60s ago 3m 51.2M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 0b64c108b71c 2023-07-21T22:28:42.338 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6 smithi107 running (3m) 59s ago 3m 47.2M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 6086c7aff3fc 2023-07-21T22:28:42.338 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 smithi132 running (3m) 60s ago 3m 54.7M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 f8a9681c6fd6 2023-07-21T22:28:42.338 INFO:teuthology.orchestra.run.smithi107.stdout:osd.8 smithi190 running (2m) 60s ago 2m 52.2M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 98454ea255ac 2023-07-21T22:28:42.338 INFO:teuthology.orchestra.run.smithi107.stdout:osd.9 smithi107 running (2m) 59s ago 2m 49.5M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 44268d699490 2023-07-21T22:28:42.338 INFO:teuthology.orchestra.run.smithi107.stdout:osd.10 smithi132 running (2m) 60s ago 2m 48.6M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 9f4542592915 2023-07-21T22:28:42.338 INFO:teuthology.orchestra.run.smithi107.stdout:osd.11 smithi190 running (108s) 60s ago 107s 45.0M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 552afcc3326d 2023-07-21T22:28:42.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.290+0000 7f0a29b27700 1 -- 172.21.15.107:0/2218149147 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0a0c0a11e0 msgr2=0x7f0a0c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:42.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.290+0000 7f0a29b27700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0a0c0a11e0 0x7f0a0c0a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0a24100bf0 tx=0x7f0a1800b5b0 comp rx=0 tx=0).stop 2023-07-21T22:28:42.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.290+0000 7f0a29b27700 1 -- 172.21.15.107:0/2218149147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a24101720 msgr2=0x7f0a24072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:42.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.290+0000 7f0a29b27700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a24101720 0x7f0a24072e40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c007d70 tx=0x7f0a1c00a340 comp rx=0 tx=0).stop 2023-07-21T22:28:42.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.290+0000 7f0a29b27700 1 -- 172.21.15.107:0/2218149147 shutdown_connections 2023-07-21T22:28:42.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.290+0000 7f0a29b27700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a24101720 0x7f0a24072e40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:42.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.290+0000 7f0a29b27700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0a0c0a11e0 0x7f0a0c0a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:42.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.290+0000 7f0a29b27700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a240ffb90 0x7f0a240742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:42.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.290+0000 7f0a29b27700 1 --2- 172.21.15.107:0/2218149147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a24100d90 0x7f0a24072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:42.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.290+0000 7f0a29b27700 1 -- 172.21.15.107:0/2218149147 >> 172.21.15.107:0/2218149147 conn(0x7f0a240fb1a0 msgr2=0x7f0a24104240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:42.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.290+0000 7f0a29b27700 1 -- 172.21.15.107:0/2218149147 shutdown_connections 2023-07-21T22:28:42.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:42.290+0000 7f0a29b27700 1 -- 172.21.15.107:0/2218149147 wait complete. 2023-07-21T22:28:42.721 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-21T22:28:43.285 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:43.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:43 smithi190 ceph-mon[98334]: pgmap v280: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T22:28:43.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:43 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:28:43.673 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:43 smithi107 ceph-mon[96543]: pgmap v280: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T22:28:43.673 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:43 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:28:43.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:43 smithi132 ceph-mon[97863]: pgmap v280: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-21T22:28:43.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:43 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:28:44.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:44 smithi107 ceph-mon[96543]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:44.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:44 smithi190 ceph-mon[98334]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:44.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:44 smithi132 ceph-mon[97863]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.809+0000 7f671a3a8700 1 -- 172.21.15.107:0/640848466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6714100610 msgr2=0x7f6714106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.809+0000 7f671a3a8700 1 --2- 172.21.15.107:0/640848466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6714100610 0x7f6714106ed0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f670c00b0a0 tx=0x7f670c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.809+0000 7f671a3a8700 1 -- 172.21.15.107:0/640848466 shutdown_connections 2023-07-21T22:28:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.809+0000 7f671a3a8700 1 --2- 172.21.15.107:0/640848466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6714100610 0x7f6714106ed0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.809+0000 7f671a3a8700 1 --2- 172.21.15.107:0/640848466 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67140fea80 0x7f67140fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:44.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.809+0000 7f671a3a8700 1 --2- 172.21.15.107:0/640848466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67140ffc80 0x7f67141000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:44.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.809+0000 7f671a3a8700 1 -- 172.21.15.107:0/640848466 >> 172.21.15.107:0/640848466 conn(0x7f67140fa0b0 msgr2=0x7f67140fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:44.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.809+0000 7f671a3a8700 1 -- 172.21.15.107:0/640848466 shutdown_connections 2023-07-21T22:28:44.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.809+0000 7f671a3a8700 1 -- 172.21.15.107:0/640848466 wait complete. 2023-07-21T22:28:44.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.809+0000 7f671a3a8700 1 Processor -- start 2023-07-21T22:28:44.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.809+0000 7f671a3a8700 1 -- start start 2023-07-21T22:28:44.859 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f671a3a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67140fea80 0x7f6714068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:44.859 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f671a3a8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67140ffc80 0x7f671419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:44.859 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f671a3a8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6714100610 0x7f671419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:44.859 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f671a3a8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671419f680 con 0x7f67140fea80 2023-07-21T22:28:44.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f671a3a8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671419f7f0 con 0x7f67140ffc80 2023-07-21T22:28:44.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f671a3a8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671419f9b0 con 0x7f6714100610 2023-07-21T22:28:44.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f6713fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67140fea80 0x7f6714068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:44.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f6713fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67140fea80 0x7f6714068b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47480/0 (socket says 172.21.15.107:47480) 2023-07-21T22:28:44.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f6713fff700 1 -- 172.21.15.107:0/3977346279 learned_addr learned my addr 172.21.15.107:0/3977346279 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:44.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f6718945700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6714100610 0x7f671419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:44.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f67137fe700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67140ffc80 0x7f671419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:44.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f6713fff700 1 -- 172.21.15.107:0/3977346279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6714100610 msgr2=0x7f671419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:44.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f6713fff700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6714100610 0x7f671419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:44.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f6713fff700 1 -- 172.21.15.107:0/3977346279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67140ffc80 msgr2=0x7f671419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:44.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f6713fff700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67140ffc80 0x7f671419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:44.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f6713fff700 1 -- 172.21.15.107:0/3977346279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c009d00 con 0x7f67140fea80 2023-07-21T22:28:44.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.810+0000 7f6713fff700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67140fea80 0x7f6714068b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00e970 tx=0x7f66fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:44.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.811+0000 7f67117fa700 1 -- 172.21.15.107:0/3977346279 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc009790 con 0x7f67140fea80 2023-07-21T22:28:44.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.811+0000 7f671a3a8700 1 -- 172.21.15.107:0/3977346279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671419fc90 con 0x7f67140fea80 2023-07-21T22:28:44.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.811+0000 7f67117fa700 1 -- 172.21.15.107:0/3977346279 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66fc00c7e0 con 0x7f67140fea80 2023-07-21T22:28:44.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.811+0000 7f67117fa700 1 -- 172.21.15.107:0/3977346279 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc003c70 con 0x7f67140fea80 2023-07-21T22:28:44.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.811+0000 7f671a3a8700 1 -- 172.21.15.107:0/3977346279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67141a0190 con 0x7f67140fea80 2023-07-21T22:28:44.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.812+0000 7f671a3a8700 1 -- 172.21.15.107:0/3977346279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6714106030 con 0x7f67140fea80 2023-07-21T22:28:44.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.813+0000 7f67117fa700 1 -- 172.21.15.107:0/3977346279 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f66fc00cc50 con 0x7f67140fea80 2023-07-21T22:28:44.863 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.813+0000 7f67117fa700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f67000a11e0 0x7f67000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:44.863 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.813+0000 7f67137fe700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f67000a11e0 0x7f67000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:44.863 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.813+0000 7f67117fa700 1 -- 172.21.15.107:0/3977346279 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f66fc014070 con 0x7f67140fea80 2023-07-21T22:28:44.863 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.814+0000 7f67137fe700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f67000a11e0 0x7f67000a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6704007be0 tx=0x7f67040073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:44.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:44.816+0000 7f67117fa700 1 -- 172.21.15.107:0/3977346279 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f66fc0963a0 con 0x7f67140fea80 2023-07-21T22:28:45.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.015+0000 7f671a3a8700 1 -- 172.21.15.107:0/3977346279 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6714060950 con 0x7f67000a11e0 2023-07-21T22:28:45.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.021+0000 7f67117fa700 1 -- 172.21.15.107:0/3977346279 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f6714060950 con 0x7f67000a11e0 2023-07-21T22:28:45.068 INFO:teuthology.orchestra.run.smithi107.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-21T22:28:45.068 INFO:teuthology.orchestra.run.smithi107.stdout:mds.all 12/12 62s ago 95s smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi190=c;smithi190=f;smithi190=i;smithi190=l;count:12 2023-07-21T22:28:45.069 INFO:teuthology.orchestra.run.smithi107.stdout:mgr 3/3 62s ago 6m smithi107=x;smithi132=y;smithi190=z;count:3 2023-07-21T22:28:45.069 INFO:teuthology.orchestra.run.smithi107.stdout:mon 3/3 62s ago 6m smithi107:172.21.15.107=a;smithi132:172.21.15.132=b;smithi190:172.21.15.190=c;count:3 2023-07-21T22:28:45.069 INFO:teuthology.orchestra.run.smithi107.stdout:osd 12 62s ago - 2023-07-21T22:28:45.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.023+0000 7f671a3a8700 1 -- 172.21.15.107:0/3977346279 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f67000a11e0 msgr2=0x7f67000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:45.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.023+0000 7f671a3a8700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f67000a11e0 0x7f67000a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6704007be0 tx=0x7f67040073d0 comp rx=0 tx=0).stop 2023-07-21T22:28:45.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.023+0000 7f671a3a8700 1 -- 172.21.15.107:0/3977346279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67140fea80 msgr2=0x7f6714068b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:45.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.023+0000 7f671a3a8700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67140fea80 0x7f6714068b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00e970 tx=0x7f66fc00ec80 comp rx=0 tx=0).stop 2023-07-21T22:28:45.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.023+0000 7f671a3a8700 1 -- 172.21.15.107:0/3977346279 shutdown_connections 2023-07-21T22:28:45.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.023+0000 7f671a3a8700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f67000a11e0 0x7f67000a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:45.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.023+0000 7f671a3a8700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6714100610 0x7f671419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:45.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.023+0000 7f671a3a8700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67140fea80 0x7f6714068b90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:45.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.023+0000 7f671a3a8700 1 --2- 172.21.15.107:0/3977346279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67140ffc80 0x7f671419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:45.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.023+0000 7f671a3a8700 1 -- 172.21.15.107:0/3977346279 >> 172.21.15.107:0/3977346279 conn(0x7f67140fa0b0 msgr2=0x7f6714103170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:45.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.023+0000 7f671a3a8700 1 -- 172.21.15.107:0/3977346279 shutdown_connections 2023-07-21T22:28:45.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:45.023+0000 7f671a3a8700 1 -- 172.21.15.107:0/3977346279 wait complete. 2023-07-21T22:28:45.407 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:45 smithi107 ceph-mon[96543]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:28:45.498 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-21T22:28:45.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:45 smithi190 ceph-mon[98334]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:28:45.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:45 smithi132 ceph-mon[97863]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:28:46.057 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:46.508 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:46.509 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.509 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.509 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.509 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.510 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.510 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.510 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2023-07-21T22:28:46.510 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-07-21T22:28:46.510 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-07-21T22:28:46.511 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-07-21T22:28:46.511 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-07-21T22:28:46.511 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2023-07-21T22:28:46.512 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2023-07-21T22:28:46.512 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-07-21T22:28:46.512 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-07-21T22:28:46.512 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-07-21T22:28:46.513 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:28:46.513 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:28:46.513 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:46.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2023-07-21T22:28:46.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-07-21T22:28:46.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-07-21T22:28:46.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-07-21T22:28:46.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-07-21T22:28:46.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2023-07-21T22:28:46.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2023-07-21T22:28:46.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-07-21T22:28:46.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-07-21T22:28:46.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-07-21T22:28:46.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:28:46.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:28:46.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:46.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:46.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2023-07-21T22:28:46.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-07-21T22:28:46.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-07-21T22:28:46.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2023-07-21T22:28:46.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-07-21T22:28:46.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]: dispatch 2023-07-21T22:28:46.685 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]: dispatch 2023-07-21T22:28:46.685 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-07-21T22:28:46.685 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-07-21T22:28:46.685 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-07-21T22:28:46.685 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:28:46.685 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:28:46.685 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:28:47.159 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.111+0000 7f196e6dd700 1 -- 172.21.15.107:0/2961093515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19680ffb90 msgr2=0x7f19680fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.111+0000 7f196e6dd700 1 --2- 172.21.15.107:0/2961093515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19680ffb90 0x7f19680fffa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1958009990 tx=0x7f1958009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.111+0000 7f196e6dd700 1 -- 172.21.15.107:0/2961093515 shutdown_connections 2023-07-21T22:28:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.111+0000 7f196e6dd700 1 --2- 172.21.15.107:0/2961093515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1968101720 0x7f1968107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.111+0000 7f196e6dd700 1 --2- 172.21.15.107:0/2961093515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19680ffb90 0x7f19680fffa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.111+0000 7f196e6dd700 1 --2- 172.21.15.107:0/2961093515 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1968100d90 0x7f19681011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.111+0000 7f196e6dd700 1 -- 172.21.15.107:0/2961093515 >> 172.21.15.107:0/2961093515 conn(0x7f19680fb1a0 msgr2=0x7f19680fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.111+0000 7f196e6dd700 1 -- 172.21.15.107:0/2961093515 shutdown_connections 2023-07-21T22:28:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.111+0000 7f196e6dd700 1 -- 172.21.15.107:0/2961093515 wait complete. 2023-07-21T22:28:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f196e6dd700 1 Processor -- start 2023-07-21T22:28:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f196e6dd700 1 -- start start 2023-07-21T22:28:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f196e6dd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19680ffb90 0x7f19681959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f196e6dd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1968100d90 0x7f1968195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f196e6dd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1968101720 0x7f196819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f196e6dd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196819d570 con 0x7f19680ffb90 2023-07-21T22:28:47.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f196e6dd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196819d6b0 con 0x7f1968100d90 2023-07-21T22:28:47.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f196e6dd700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196819d840 con 0x7f1968101720 2023-07-21T22:28:47.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f1967fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19680ffb90 0x7f19681959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f1967fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19680ffb90 0x7f19681959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47490/0 (socket says 172.21.15.107:47490) 2023-07-21T22:28:47.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f1967fff700 1 -- 172.21.15.107:0/395958391 learned_addr learned my addr 172.21.15.107:0/395958391 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:47.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f196cc7a700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1968101720 0x7f196819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f1967fff700 1 -- 172.21.15.107:0/395958391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1968101720 msgr2=0x7f196819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f1967fff700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1968101720 0x7f196819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f1967fff700 1 -- 172.21.15.107:0/395958391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1968100d90 msgr2=0x7f1968195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f19677fe700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1968100d90 0x7f1968195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f1967fff700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1968100d90 0x7f1968195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f1967fff700 1 -- 172.21.15.107:0/395958391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19580095f0 con 0x7f19680ffb90 2023-07-21T22:28:47.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.112+0000 7f196cc7a700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1968101720 0x7f196819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.113+0000 7f1967fff700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19680ffb90 0x7f19681959e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f195800f760 tx=0x7f195800f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.113+0000 7f19657fa700 1 -- 172.21.15.107:0/395958391 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195801d070 con 0x7f19680ffb90 2023-07-21T22:28:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.113+0000 7f196e6dd700 1 -- 172.21.15.107:0/395958391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196819dac0 con 0x7f19680ffb90 2023-07-21T22:28:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.113+0000 7f19657fa700 1 -- 172.21.15.107:0/395958391 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f195800fd20 con 0x7f19680ffb90 2023-07-21T22:28:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.113+0000 7f19657fa700 1 -- 172.21.15.107:0/395958391 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19580056f0 con 0x7f19680ffb90 2023-07-21T22:28:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.113+0000 7f196e6dd700 1 -- 172.21.15.107:0/395958391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196819df60 con 0x7f19680ffb90 2023-07-21T22:28:47.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.114+0000 7f196e6dd700 1 -- 172.21.15.107:0/395958391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196810b4a0 con 0x7f19680ffb90 2023-07-21T22:28:47.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.115+0000 7f19657fa700 1 -- 172.21.15.107:0/395958391 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f19580047e0 con 0x7f19680ffb90 2023-07-21T22:28:47.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.115+0000 7f19657fa700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f19540a11e0 0x7f19540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:47.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.116+0000 7f19677fe700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f19540a11e0 0x7f19540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:47.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.116+0000 7f19657fa700 1 -- 172.21.15.107:0/395958391 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f19580ce1f0 con 0x7f19680ffb90 2023-07-21T22:28:47.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.116+0000 7f19677fe700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f19540a11e0 0x7f19540a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1950006fd0 tx=0x7f1950008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:47.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.119+0000 7f19657fa700 1 -- 172.21.15.107:0/395958391 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1958097850 con 0x7f19680ffb90 2023-07-21T22:28:47.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.319+0000 7f196e6dd700 1 -- 172.21.15.107:0/395958391 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1968060950 con 0x7f19540a11e0 2023-07-21T22:28:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:HOST ADDR LABELS STATUS 2023-07-21T22:28:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 172.21.15.107 2023-07-21T22:28:47.367 INFO:teuthology.orchestra.run.smithi107.stdout:smithi132 172.21.15.132 2023-07-21T22:28:47.368 INFO:teuthology.orchestra.run.smithi107.stdout:smithi190 172.21.15.190 2023-07-21T22:28:47.368 INFO:teuthology.orchestra.run.smithi107.stdout:3 hosts in cluster 2023-07-21T22:28:47.368 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.320+0000 7f19657fa700 1 -- 172.21.15.107:0/395958391 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f1968060950 con 0x7f19540a11e0 2023-07-21T22:28:47.369 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.323+0000 7f196e6dd700 1 -- 172.21.15.107:0/395958391 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f19540a11e0 msgr2=0x7f19540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.323+0000 7f196e6dd700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f19540a11e0 0x7f19540a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1950006fd0 tx=0x7f1950008040 comp rx=0 tx=0).stop 2023-07-21T22:28:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.323+0000 7f196e6dd700 1 -- 172.21.15.107:0/395958391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19680ffb90 msgr2=0x7f19681959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.323+0000 7f196e6dd700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19680ffb90 0x7f19681959e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f195800f760 tx=0x7f195800f840 comp rx=0 tx=0).stop 2023-07-21T22:28:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.323+0000 7f196e6dd700 1 -- 172.21.15.107:0/395958391 shutdown_connections 2023-07-21T22:28:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.323+0000 7f196e6dd700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f19540a11e0 0x7f19540a3690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.323+0000 7f196e6dd700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1968101720 0x7f196819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.323+0000 7f196e6dd700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19680ffb90 0x7f19681959e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.323+0000 7f196e6dd700 1 --2- 172.21.15.107:0/395958391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1968100d90 0x7f1968195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:47.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.323+0000 7f196e6dd700 1 -- 172.21.15.107:0/395958391 >> 172.21.15.107:0/395958391 conn(0x7f19680fb1a0 msgr2=0x7f19680fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:47.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.323+0000 7f196e6dd700 1 -- 172.21.15.107:0/395958391 shutdown_connections 2023-07-21T22:28:47.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:47.323+0000 7f196e6dd700 1 -- 172.21.15.107:0/395958391 wait complete. 2023-07-21T22:28:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:47 smithi190 ceph-mon[98334]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:28:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2023-07-21T22:28:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-07-21T22:28:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-07-21T22:28:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-07-21T22:28:47.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-07-21T22:28:47.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2023-07-21T22:28:47.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2023-07-21T22:28:47.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-07-21T22:28:47.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-07-21T22:28:47.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:47 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-07-21T22:28:47.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:47 smithi190 ceph-mon[98334]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T22:28:47.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:47 smithi132 ceph-mon[97863]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:28:47.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2023-07-21T22:28:47.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-07-21T22:28:47.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-07-21T22:28:47.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-07-21T22:28:47.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-07-21T22:28:47.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2023-07-21T22:28:47.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2023-07-21T22:28:47.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-07-21T22:28:47.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-07-21T22:28:47.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:47 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-07-21T22:28:47.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:47 smithi132 ceph-mon[97863]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T22:28:47.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:47 smithi107 ceph-mon[96543]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-21T22:28:47.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2023-07-21T22:28:47.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-07-21T22:28:47.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-07-21T22:28:47.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2023-07-21T22:28:47.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-07-21T22:28:47.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 6]}]': finished 2023-07-21T22:28:47.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 7]}]': finished 2023-07-21T22:28:47.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-07-21T22:28:47.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-07-21T22:28:47.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:47 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-07-21T22:28:47.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:47 smithi107 ceph-mon[96543]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T22:28:47.703 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-21T22:28:48.253 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:48.606 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:48 smithi107 ceph-mon[96543]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T22:28:48.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:48 smithi190 ceph-mon[98334]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T22:28:48.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:48 smithi132 ceph-mon[97863]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T22:28:49.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 -- 172.21.15.107:0/3829691132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3104100e30 msgr2=0x7f3104101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:49.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 --2- 172.21.15.107:0/3829691132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3104100e30 0x7f3104101280 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009990 tx=0x7f30f0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:49.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 -- 172.21.15.107:0/3829691132 shutdown_connections 2023-07-21T22:28:49.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 --2- 172.21.15.107:0/3829691132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31041017c0 0x7f3104108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:49.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 --2- 172.21.15.107:0/3829691132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31040fde50 0x7f31040fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:49.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 --2- 172.21.15.107:0/3829691132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3104100e30 0x7f3104101280 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:49.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 -- 172.21.15.107:0/3829691132 >> 172.21.15.107:0/3829691132 conn(0x7f31040f9860 msgr2=0x7f31040fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:49.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 -- 172.21.15.107:0/3829691132 shutdown_connections 2023-07-21T22:28:49.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 -- 172.21.15.107:0/3829691132 wait complete. 2023-07-21T22:28:49.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 Processor -- start 2023-07-21T22:28:49.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 -- start start 2023-07-21T22:28:49.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31040fde50 0x7f3104100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3104100e30 0x7f31040fede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31041017c0 0x7f31040ff320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31040ffaa0 con 0x7f3104100e30 2023-07-21T22:28:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31040ffc10 con 0x7f31041017c0 2023-07-21T22:28:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.275+0000 7f310acfc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310419f8f0 con 0x7f31040fde50 2023-07-21T22:28:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3103fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3104100e30 0x7f31040fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3103fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3104100e30 0x7f31040fede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47504/0 (socket says 172.21.15.107:47504) 2023-07-21T22:28:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3103fff700 1 -- 172.21.15.107:0/3320040940 learned_addr learned my addr 172.21.15.107:0/3320040940 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3109299700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31041017c0 0x7f31040ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3108a98700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31040fde50 0x7f3104100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3103fff700 1 -- 172.21.15.107:0/3320040940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31040fde50 msgr2=0x7f3104100790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3103fff700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31040fde50 0x7f3104100790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3103fff700 1 -- 172.21.15.107:0/3320040940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31041017c0 msgr2=0x7f31040ff320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3103fff700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31041017c0 0x7f31040ff320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3103fff700 1 -- 172.21.15.107:0/3320040940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f00095f0 con 0x7f3104100e30 2023-07-21T22:28:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3109299700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31041017c0 0x7f31040ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3103fff700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3104100e30 0x7f31040fede0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f30f0000c00 tx=0x7f30f000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3101ffb700 1 -- 172.21.15.107:0/3320040940 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f001d070 con 0x7f3104100e30 2023-07-21T22:28:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f310acfc700 1 -- 172.21.15.107:0/3320040940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310419fb70 con 0x7f3104100e30 2023-07-21T22:28:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3101ffb700 1 -- 172.21.15.107:0/3320040940 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30f000fc30 con 0x7f3104100e30 2023-07-21T22:28:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f3101ffb700 1 -- 172.21.15.107:0/3320040940 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0017770 con 0x7f3104100e30 2023-07-21T22:28:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.276+0000 7f310acfc700 1 -- 172.21.15.107:0/3320040940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310419ffe0 con 0x7f3104100e30 2023-07-21T22:28:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.277+0000 7f310acfc700 1 -- 172.21.15.107:0/3320040940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3104107120 con 0x7f3104100e30 2023-07-21T22:28:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.278+0000 7f3101ffb700 1 -- 172.21.15.107:0/3320040940 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f30f0022a30 con 0x7f3104100e30 2023-07-21T22:28:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.279+0000 7f3101ffb700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30f40a11e0 0x7f30f40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.279+0000 7f3108a98700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30f40a11e0 0x7f30f40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.279+0000 7f3101ffb700 1 -- 172.21.15.107:0/3320040940 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f30f00ce660 con 0x7f3104100e30 2023-07-21T22:28:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.280+0000 7f3108a98700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30f40a11e0 0x7f30f40a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f30f8006fd0 tx=0x7f30f8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.283+0000 7f3101ffb700 1 -- 172.21.15.107:0/3320040940 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f30f0097cc0 con 0x7f3104100e30 2023-07-21T22:28:49.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:49 smithi107 ceph-mon[96543]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:49.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:49 smithi107 ceph-mon[96543]: pgmap v285: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2023-07-21T22:28:49.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:49 smithi107 ceph-mon[96543]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2023-07-21T22:28:49.524 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.478+0000 7f310acfc700 1 -- 172.21.15.107:0/3320040940 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3104060950 con 0x7f30f40a11e0 2023-07-21T22:28:49.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.479+0000 7f3101ffb700 1 -- 172.21.15.107:0/3320040940 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f3104060950 con 0x7f30f40a11e0 2023-07-21T22:28:49.526 INFO:teuthology.orchestra.run.smithi107.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-21T22:28:49.526 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 2m ago LVM detected, locked 2023-07-21T22:28:49.526 INFO:teuthology.orchestra.run.smithi107.stdout:smithi132 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232000B400BGN 372G No 2m ago LVM detected, locked 2023-07-21T22:28:49.526 INFO:teuthology.orchestra.run.smithi107.stdout:smithi190 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT63210012400BGN 372G No 109s ago LVM detected, locked 2023-07-21T22:28:49.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.481+0000 7f310acfc700 1 -- 172.21.15.107:0/3320040940 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30f40a11e0 msgr2=0x7f30f40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:49.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.481+0000 7f310acfc700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30f40a11e0 0x7f30f40a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f30f8006fd0 tx=0x7f30f8008040 comp rx=0 tx=0).stop 2023-07-21T22:28:49.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.481+0000 7f310acfc700 1 -- 172.21.15.107:0/3320040940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3104100e30 msgr2=0x7f31040fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:49.528 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.481+0000 7f310acfc700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3104100e30 0x7f31040fede0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f30f0000c00 tx=0x7f30f000f790 comp rx=0 tx=0).stop 2023-07-21T22:28:49.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.481+0000 7f310acfc700 1 -- 172.21.15.107:0/3320040940 shutdown_connections 2023-07-21T22:28:49.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.481+0000 7f310acfc700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31041017c0 0x7f31040ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:49.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.481+0000 7f310acfc700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f30f40a11e0 0x7f30f40a3690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:49.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.481+0000 7f310acfc700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31040fde50 0x7f3104100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:49.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.481+0000 7f310acfc700 1 --2- 172.21.15.107:0/3320040940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3104100e30 0x7f31040fede0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:49.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.481+0000 7f310acfc700 1 -- 172.21.15.107:0/3320040940 >> 172.21.15.107:0/3320040940 conn(0x7f31040f9860 msgr2=0x7f3104104260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:49.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.482+0000 7f310acfc700 1 -- 172.21.15.107:0/3320040940 shutdown_connections 2023-07-21T22:28:49.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:49.482+0000 7f310acfc700 1 -- 172.21.15.107:0/3320040940 wait complete. 2023-07-21T22:28:49.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:49 smithi190 ceph-mon[98334]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:49.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:49 smithi190 ceph-mon[98334]: pgmap v285: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2023-07-21T22:28:49.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:49 smithi190 ceph-mon[98334]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2023-07-21T22:28:49.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:49 smithi132 ceph-mon[97863]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:49.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:49 smithi132 ceph-mon[97863]: pgmap v285: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2023-07-21T22:28:49.684 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:49 smithi132 ceph-mon[97863]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2023-07-21T22:28:49.896 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-21T22:28:49.906 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2023-07-21T22:28:49.906 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-21T22:28:50.457 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:51.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:51 smithi107 ceph-mon[96543]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:51.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:51 smithi107 ceph-mon[96543]: pgmap v286: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 7 op/s 2023-07-21T22:28:51.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:51 smithi190 ceph-mon[98334]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:51.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:51 smithi190 ceph-mon[98334]: pgmap v286: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 7 op/s 2023-07-21T22:28:51.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:51 smithi132 ceph-mon[97863]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:28:51.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:51 smithi132 ceph-mon[97863]: pgmap v286: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 7 op/s 2023-07-21T22:28:51.813 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.766+0000 7f12e2620700 1 -- 172.21.15.107:0/2950728266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12dc100610 msgr2=0x7f12dc106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:51.813 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.766+0000 7f12e2620700 1 --2- 172.21.15.107:0/2950728266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12dc100610 0x7f12dc106ed0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f12d400b0a0 tx=0x7f12d400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:51.813 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.766+0000 7f12e2620700 1 -- 172.21.15.107:0/2950728266 shutdown_connections 2023-07-21T22:28:51.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.766+0000 7f12e2620700 1 --2- 172.21.15.107:0/2950728266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12dc100610 0x7f12dc106ed0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:51.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.766+0000 7f12e2620700 1 --2- 172.21.15.107:0/2950728266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12dc0fea80 0x7f12dc0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:51.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.766+0000 7f12e2620700 1 --2- 172.21.15.107:0/2950728266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12dc0ffc80 0x7f12dc1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:51.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.766+0000 7f12e2620700 1 -- 172.21.15.107:0/2950728266 >> 172.21.15.107:0/2950728266 conn(0x7f12dc0fa0b0 msgr2=0x7f12dc0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:51.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.766+0000 7f12e2620700 1 -- 172.21.15.107:0/2950728266 shutdown_connections 2023-07-21T22:28:51.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.766+0000 7f12e2620700 1 -- 172.21.15.107:0/2950728266 wait complete. 2023-07-21T22:28:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.767+0000 7f12e2620700 1 Processor -- start 2023-07-21T22:28:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.767+0000 7f12e2620700 1 -- start start 2023-07-21T22:28:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.767+0000 7f12e2620700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12dc0fea80 0x7f12dc194850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.767+0000 7f12e2620700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12dc0ffc80 0x7f12dc194d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.767+0000 7f12e2620700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12dc100610 0x7f12dc19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.767+0000 7f12e2620700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc19c3e0 con 0x7f12dc100610 2023-07-21T22:28:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.767+0000 7f12e2620700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc067fd0 con 0x7f12dc0ffc80 2023-07-21T22:28:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.767+0000 7f12e2620700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc068190 con 0x7f12dc0fea80 2023-07-21T22:28:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.767+0000 7f12e0bbd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12dc100610 0x7f12dc19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.767+0000 7f12e0bbd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12dc100610 0x7f12dc19be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47520/0 (socket says 172.21.15.107:47520) 2023-07-21T22:28:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.767+0000 7f12e0bbd700 1 -- 172.21.15.107:0/1780829094 learned_addr learned my addr 172.21.15.107:0/1780829094 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12db7fe700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12dc0ffc80 0x7f12dc194d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:51.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12dbfff700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12dc0fea80 0x7f12dc194850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:51.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12e0bbd700 1 -- 172.21.15.107:0/1780829094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12dc0fea80 msgr2=0x7f12dc194850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:51.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12e0bbd700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12dc0fea80 0x7f12dc194850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:51.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12e0bbd700 1 -- 172.21.15.107:0/1780829094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12dc0ffc80 msgr2=0x7f12dc194d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:51.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12e0bbd700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12dc0ffc80 0x7f12dc194d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:51.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12e0bbd700 1 -- 172.21.15.107:0/1780829094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d4009d00 con 0x7f12dc100610 2023-07-21T22:28:51.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12db7fe700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12dc0ffc80 0x7f12dc194d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:51.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12dbfff700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12dc0fea80 0x7f12dc194850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:51.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12e0bbd700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12dc100610 0x7f12dc19be10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f12d40062a0 tx=0x7f12d4004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:51.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12d97fa700 1 -- 172.21.15.107:0/1780829094 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d400e040 con 0x7f12dc100610 2023-07-21T22:28:51.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12d97fa700 1 -- 172.21.15.107:0/1780829094 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12d401db50 con 0x7f12dc100610 2023-07-21T22:28:51.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12e2620700 1 -- 172.21.15.107:0/1780829094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12dc068410 con 0x7f12dc100610 2023-07-21T22:28:51.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.768+0000 7f12d97fa700 1 -- 172.21.15.107:0/1780829094 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d401bc40 con 0x7f12dc100610 2023-07-21T22:28:51.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.769+0000 7f12e2620700 1 -- 172.21.15.107:0/1780829094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12dc068910 con 0x7f12dc100610 2023-07-21T22:28:51.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.770+0000 7f12e2620700 1 -- 172.21.15.107:0/1780829094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12dc105fe0 con 0x7f12dc100610 2023-07-21T22:28:51.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.770+0000 7f12d97fa700 1 -- 172.21.15.107:0/1780829094 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f12d401d690 con 0x7f12dc100610 2023-07-21T22:28:51.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.771+0000 7f12d97fa700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f12c40a11e0 0x7f12c40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:51.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.771+0000 7f12dbfff700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f12c40a11e0 0x7f12c40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:51.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.771+0000 7f12d97fa700 1 -- 172.21.15.107:0/1780829094 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f12d40ce9e0 con 0x7f12dc100610 2023-07-21T22:28:51.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.771+0000 7f12dbfff700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f12c40a11e0 0x7f12c40a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0ffae0 tx=0x7f12cc009670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:51.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:51.775+0000 7f12d97fa700 1 -- 172.21.15.107:0/1780829094 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f12d4097fe0 con 0x7f12dc100610 2023-07-21T22:28:52.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.005+0000 7f12e2620700 1 -- 172.21.15.107:0/1780829094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f12dc068c70 con 0x7f12dc100610 2023-07-21T22:28:52.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.005+0000 7f12d97fa700 1 -- 172.21.15.107:0/1780829094 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2936 (secure 0 0 0) 0x7f12d409b600 con 0x7f12dc100610 2023-07-21T22:28:52.052 INFO:teuthology.orchestra.run.smithi107.stdout:e27 2023-07-21T22:28:52.053 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T22:28:52.053 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:28:52.053 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2023-07-21T22:28:52.053 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:52.053 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2023-07-21T22:28:52.053 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2023-07-21T22:28:52.054 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 27 2023-07-21T22:28:52.054 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-21T22:28:52.054 INFO:teuthology.orchestra.run.smithi107.stdout:created 2023-07-21T22:27:41.478391+0000 2023-07-21T22:28:52.054 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2023-07-21T22:27:49.034257+0000 2023-07-21T22:28:52.054 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2023-07-21T22:28:52.054 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2023-07-21T22:28:52.054 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 300 2023-07-21T22:28:52.054 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2023-07-21T22:28:52.055 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2023-07-21T22:28:52.055 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2023-07-21T22:28:52.055 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2023-07-21T22:28:52.055 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2023-07-21T22:28:52.055 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 0 2023-07-21T22:28:52.055 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:28:52.055 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 5 2023-07-21T22:28:52.055 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1,2,3,4 2023-07-21T22:28:52.056 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24533,1=24493,2=24466,3=24455,4=24437} 2023-07-21T22:28:52.056 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2023-07-21T22:28:52.056 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2023-07-21T22:28:52.056 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2023-07-21T22:28:52.056 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2023-07-21T22:28:52.056 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2023-07-21T22:28:52.056 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2023-07-21T22:28:52.057 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2023-07-21T22:28:52.057 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2023-07-21T22:28:52.057 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 0 2023-07-21T22:28:52.057 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b{0:24533} state up:active seq 3 addr [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:28:52.057 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.h{0:24491} state up:standby-replay seq 1 addr [v2:172.21.15.132:6836/1492770170,v1:172.21.15.132:6837/1492770170] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:28:52.057 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.i{1:24493} state up:active seq 5 addr [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:28:52.057 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c{1:24475} state up:standby-replay seq 1 addr [v2:172.21.15.190:6836/3774629948,v1:172.21.15.190:6837/3774629948] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:28:52.057 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.f{2:24466} state up:active seq 7 addr [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:28:52.058 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.l{2:24424} state up:standby-replay seq 1 addr [v2:172.21.15.190:6832/253562772,v1:172.21.15.190:6833/253562772] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:28:52.058 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.e{3:24455} state up:active seq 8 addr [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:28:52.058 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a{3:14658} state up:standby-replay seq 1 addr [v2:172.21.15.107:6840/1878776680,v1:172.21.15.107:6841/1878776680] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:28:52.058 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.k{4:24437} state up:active seq 10 addr [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:28:52.058 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.g{4:14646} state up:standby-replay seq 1 addr [v2:172.21.15.107:6838/1956222571,v1:172.21.15.107:6839/1956222571] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:28:52.058 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:52.058 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:52.058 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2023-07-21T22:28:52.059 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:52.059 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.j{-1:14568} state up:standby seq 1 addr [v2:172.21.15.107:6834/219129239,v1:172.21.15.107:6835/219129239] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:28:52.059 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d{-1:14574} state up:standby seq 1 addr [v2:172.21.15.107:6836/2102134479,v1:172.21.15.107:6837/2102134479] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:28:52.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.007+0000 7f12e2620700 1 -- 172.21.15.107:0/1780829094 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f12c40a11e0 msgr2=0x7f12c40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:52.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.007+0000 7f12e2620700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f12c40a11e0 0x7f12c40a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0ffae0 tx=0x7f12cc009670 comp rx=0 tx=0).stop 2023-07-21T22:28:52.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.007+0000 7f12e2620700 1 -- 172.21.15.107:0/1780829094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12dc100610 msgr2=0x7f12dc19be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:52.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.007+0000 7f12e2620700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12dc100610 0x7f12dc19be10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f12d40062a0 tx=0x7f12d4004db0 comp rx=0 tx=0).stop 2023-07-21T22:28:52.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.007+0000 7f12e2620700 1 -- 172.21.15.107:0/1780829094 shutdown_connections 2023-07-21T22:28:52.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.007+0000 7f12e2620700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12dc100610 0x7f12dc19be10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:52.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.007+0000 7f12e2620700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f12c40a11e0 0x7f12c40a3690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:52.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.007+0000 7f12e2620700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12dc0fea80 0x7f12dc194850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:52.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.007+0000 7f12e2620700 1 --2- 172.21.15.107:0/1780829094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12dc0ffc80 0x7f12dc194d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:52.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.007+0000 7f12e2620700 1 -- 172.21.15.107:0/1780829094 >> 172.21.15.107:0/1780829094 conn(0x7f12dc0fa0b0 msgr2=0x7f12dc103120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:52.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.008+0000 7f12e2620700 1 -- 172.21.15.107:0/1780829094 shutdown_connections 2023-07-21T22:28:52.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:52.008+0000 7f12e2620700 1 -- 172.21.15.107:0/1780829094 wait complete. 2023-07-21T22:28:52.061 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2023-07-21T22:28:52.412 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:52 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1780829094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T22:28:52.413 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-21T22:28:52.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:52 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1780829094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T22:28:52.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:52 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1780829094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T22:28:52.965 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:28:53.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:53 smithi107 ceph-mon[96543]: pgmap v287: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s; 120 B/s, 0 objects/s recovering 2023-07-21T22:28:53.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:53 smithi190 ceph-mon[98334]: pgmap v287: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s; 120 B/s, 0 objects/s recovering 2023-07-21T22:28:53.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:53 smithi132 ceph-mon[97863]: pgmap v287: 97 pgs: 3 peering, 94 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s; 120 B/s, 0 objects/s recovering 2023-07-21T22:28:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.098+0000 7fc289f24700 1 -- 172.21.15.107:0/283221160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc284100650 msgr2=0x7fc284102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.098+0000 7fc289f24700 1 --2- 172.21.15.107:0/283221160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc284100650 0x7fc284102a30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc274009990 tx=0x7fc274009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.098+0000 7fc289f24700 1 -- 172.21.15.107:0/283221160 shutdown_connections 2023-07-21T22:28:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.098+0000 7fc289f24700 1 --2- 172.21.15.107:0/283221160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc284105890 0x7fc284107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.098+0000 7fc289f24700 1 --2- 172.21.15.107:0/283221160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc284100650 0x7fc284102a30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.098+0000 7fc289f24700 1 --2- 172.21.15.107:0/283221160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc284102f70 0x7fc284105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.098+0000 7fc289f24700 1 -- 172.21.15.107:0/283221160 >> 172.21.15.107:0/283221160 conn(0x7fc2840fa0b0 msgr2=0x7fc2840fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:54.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.098+0000 7fc289f24700 1 -- 172.21.15.107:0/283221160 shutdown_connections 2023-07-21T22:28:54.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.098+0000 7fc289f24700 1 -- 172.21.15.107:0/283221160 wait complete. 2023-07-21T22:28:54.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.098+0000 7fc289f24700 1 Processor -- start 2023-07-21T22:28:54.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc289f24700 1 -- start start 2023-07-21T22:28:54.148 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc289f24700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc284100650 0x7fc2841958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:54.148 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc289f24700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc284102f70 0x7fc284195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:54.148 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc289f24700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc284105890 0x7fc28419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:54.148 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc289f24700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28419d480 con 0x7fc284102f70 2023-07-21T22:28:54.148 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc289f24700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28419d5c0 con 0x7fc284105890 2023-07-21T22:28:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc289f24700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28419d750 con 0x7fc284100650 2023-07-21T22:28:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc282ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc284102f70 0x7fc284195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc282ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc284102f70 0x7fc284195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47530/0 (socket says 172.21.15.107:47530) 2023-07-21T22:28:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc282ffd700 1 -- 172.21.15.107:0/67392058 learned_addr learned my addr 172.21.15.107:0/67392058 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc283fff700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc284105890 0x7fc28419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc2837fe700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc284100650 0x7fc2841958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc282ffd700 1 -- 172.21.15.107:0/67392058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc284100650 msgr2=0x7fc2841958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc282ffd700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc284100650 0x7fc2841958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc282ffd700 1 -- 172.21.15.107:0/67392058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc284105890 msgr2=0x7fc28419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc282ffd700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc284105890 0x7fc28419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc282ffd700 1 -- 172.21.15.107:0/67392058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2740095f0 con 0x7fc284102f70 2023-07-21T22:28:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.099+0000 7fc283fff700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc284105890 0x7fc28419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:28:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.100+0000 7fc282ffd700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc284102f70 0x7fc284195e30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc27800d9b0 tx=0x7fc27800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.100+0000 7fc280ff9700 1 -- 172.21.15.107:0/67392058 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278004220 con 0x7fc284102f70 2023-07-21T22:28:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.100+0000 7fc289f24700 1 -- 172.21.15.107:0/67392058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28419da30 con 0x7fc284102f70 2023-07-21T22:28:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.100+0000 7fc280ff9700 1 -- 172.21.15.107:0/67392058 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc278007640 con 0x7fc284102f70 2023-07-21T22:28:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.100+0000 7fc280ff9700 1 -- 172.21.15.107:0/67392058 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278005590 con 0x7fc284102f70 2023-07-21T22:28:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.100+0000 7fc289f24700 1 -- 172.21.15.107:0/67392058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28419df30 con 0x7fc284102f70 2023-07-21T22:28:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.102+0000 7fc280ff9700 1 -- 172.21.15.107:0/67392058 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fc278004750 con 0x7fc284102f70 2023-07-21T22:28:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.102+0000 7fc289f24700 1 -- 172.21.15.107:0/67392058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2840fbc20 con 0x7fc284102f70 2023-07-21T22:28:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.102+0000 7fc280ff9700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc26c0a11e0 0x7fc26c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.103+0000 7fc2837fe700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc26c0a11e0 0x7fc26c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.103+0000 7fc280ff9700 1 -- 172.21.15.107:0/67392058 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc2780cc8b0 con 0x7fc284102f70 2023-07-21T22:28:54.153 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.103+0000 7fc2837fe700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc26c0a11e0 0x7fc26c0a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc27400b2e0 tx=0x7fc274005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:54.153 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.107+0000 7fc280ff9700 1 -- 172.21.15.107:0/67392058 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc278095e80 con 0x7fc284102f70 2023-07-21T22:28:54.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.297+0000 7fc289f24700 1 -- 172.21.15.107:0/67392058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fc28402ca50 con 0x7fc284102f70 2023-07-21T22:28:54.344 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.298+0000 7fc280ff9700 1 -- 172.21.15.107:0/67392058 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) v1 ==== 56+0+5367 (secure 0 0 0) 0x7fc278016070 con 0x7fc284102f70 2023-07-21T22:28:54.344 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 84 2023-07-21T22:28:54.345 INFO:teuthology.orchestra.run.smithi107.stdout:fsid af99f5d0-2814-11ee-9b34-001a4aab830c 2023-07-21T22:28:54.345 INFO:teuthology.orchestra.run.smithi107.stdout:created 2023-07-21T22:20:56.799654+0000 2023-07-21T22:28:54.345 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2023-07-21T22:28:47.235779+0000 2023-07-21T22:28:54.345 INFO:teuthology.orchestra.run.smithi107.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-21T22:28:54.345 INFO:teuthology.orchestra.run.smithi107.stdout:crush_version 26 2023-07-21T22:28:54.345 INFO:teuthology.orchestra.run.smithi107.stdout:full_ratio 0.95 2023-07-21T22:28:54.345 INFO:teuthology.orchestra.run.smithi107.stdout:backfillfull_ratio 0.9 2023-07-21T22:28:54.346 INFO:teuthology.orchestra.run.smithi107.stdout:nearfull_ratio 0.85 2023-07-21T22:28:54.346 INFO:teuthology.orchestra.run.smithi107.stdout:require_min_compat_client luminous 2023-07-21T22:28:54.346 INFO:teuthology.orchestra.run.smithi107.stdout:min_compat_client luminous 2023-07-21T22:28:54.346 INFO:teuthology.orchestra.run.smithi107.stdout:require_osd_release reef 2023-07-21T22:28:54.346 INFO:teuthology.orchestra.run.smithi107.stdout:stretch_mode_enabled false 2023-07-21T22:28:54.346 INFO:teuthology.orchestra.run.smithi107.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-21T22:28:54.346 INFO:teuthology.orchestra.run.smithi107.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-07-21T22:28:54.347 INFO:teuthology.orchestra.run.smithi107.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-07-21T22:28:54.347 INFO:teuthology.orchestra.run.smithi107.stdout:max_osd 12 2023-07-21T22:28:54.347 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6802/4235328012,v1:172.21.15.107:6803/4235328012] [v2:172.21.15.107:6804/4235328012,v1:172.21.15.107:6805/4235328012] exists,up dc595a3a-edf0-47ac-b533-25c80518c634 2023-07-21T22:28:54.347 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6800/1473726199,v1:172.21.15.132:6801/1473726199] [v2:172.21.15.132:6802/1473726199,v1:172.21.15.132:6803/1473726199] exists,up 6d5fd94a-6288-42ed-9f8a-413b2e687e26 2023-07-21T22:28:54.347 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6800/1160922995,v1:172.21.15.190:6801/1160922995] [v2:172.21.15.190:6802/1160922995,v1:172.21.15.190:6803/1160922995] exists,up e5ce38db-d9b7-4f49-9945-fe7e1aacf27d 2023-07-21T22:28:54.347 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 up in weight 1 up_from 25 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6810/1375259004,v1:172.21.15.107:6811/1375259004] [v2:172.21.15.107:6812/1375259004,v1:172.21.15.107:6813/1375259004] exists,up eaefb60a-452f-4eb7-a984-5b459aa49d4e 2023-07-21T22:28:54.347 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 up in weight 1 up_from 33 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6808/2021101216,v1:172.21.15.132:6809/2021101216] [v2:172.21.15.132:6810/2021101216,v1:172.21.15.132:6811/2021101216] exists,up 8f114d7a-5079-4c48-a9fa-c477d9447aa7 2023-07-21T22:28:54.348 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6808/1950039533,v1:172.21.15.190:6809/1950039533] [v2:172.21.15.190:6810/1950039533,v1:172.21.15.190:6811/1950039533] exists,up 93ba2f1d-2b22-4199-9cae-05fef817b637 2023-07-21T22:28:54.348 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6 up in weight 1 up_from 45 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6818/3497265368,v1:172.21.15.107:6819/3497265368] [v2:172.21.15.107:6820/3497265368,v1:172.21.15.107:6821/3497265368] exists,up 59ce5822-cb79-41f8-bc1e-de34c37311d9 2023-07-21T22:28:54.348 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 up in weight 1 up_from 50 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6816/2500130834,v1:172.21.15.132:6817/2500130834] [v2:172.21.15.132:6818/2500130834,v1:172.21.15.132:6819/2500130834] exists,up 0013f445-b592-48e8-9620-ca2ff64a8e24 2023-07-21T22:28:54.348 INFO:teuthology.orchestra.run.smithi107.stdout:osd.8 up in weight 1 up_from 57 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6816/3597513159,v1:172.21.15.190:6817/3597513159] [v2:172.21.15.190:6818/3597513159,v1:172.21.15.190:6819/3597513159] exists,up f0beb03d-307c-4f41-8e17-6e0418063c75 2023-07-21T22:28:54.348 INFO:teuthology.orchestra.run.smithi107.stdout:osd.9 up in weight 1 up_from 65 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6826/458109252,v1:172.21.15.107:6827/458109252] [v2:172.21.15.107:6828/458109252,v1:172.21.15.107:6829/458109252] exists,up b6203384-ccdb-419c-aa4a-0f64c3bdcba0 2023-07-21T22:28:54.348 INFO:teuthology.orchestra.run.smithi107.stdout:osd.10 up in weight 1 up_from 70 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6824/2698708367,v1:172.21.15.132:6825/2698708367] [v2:172.21.15.132:6826/2698708367,v1:172.21.15.132:6827/2698708367] exists,up fc031611-c261-490b-ab22-31fd52691970 2023-07-21T22:28:54.348 INFO:teuthology.orchestra.run.smithi107.stdout:osd.11 up in weight 1 up_from 77 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.190:6824/1655561112,v1:172.21.15.190:6825/1655561112] [v2:172.21.15.190:6826/1655561112,v1:172.21.15.190:6827/1655561112] exists,up b9d95a8c-e6d0-49a6-9c50-2adee825168b 2023-07-21T22:28:54.349 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.b [8,11] 2023-07-21T22:28:54.349 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.e [2,4] 2023-07-21T22:28:54.349 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.f [8,11] 2023-07-21T22:28:54.349 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.1b [2,1] 2023-07-21T22:28:54.349 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.0 [5,9] 2023-07-21T22:28:54.349 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.11 [4,6] 2023-07-21T22:28:54.349 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.1d [4,7] 2023-07-21T22:28:54.350 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.21 [4,8] 2023-07-21T22:28:54.350 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.33 [5,9] 2023-07-21T22:28:54.350 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.38 [3,9] 2023-07-21T22:28:54.350 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/3494357734 expires 2023-07-22T22:21:45.714358+0000 2023-07-21T22:28:54.350 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/2305207412 expires 2023-07-22T22:21:45.714358+0000 2023-07-21T22:28:54.350 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6801/659042147 expires 2023-07-22T22:21:45.714358+0000 2023-07-21T22:28:54.350 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/3404483652 expires 2023-07-22T22:21:24.460216+0000 2023-07-21T22:28:54.351 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6800/659042147 expires 2023-07-22T22:21:45.714358+0000 2023-07-21T22:28:54.351 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6800/3802710476 expires 2023-07-22T22:21:24.460216+0000 2023-07-21T22:28:54.351 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/3814985795 expires 2023-07-22T22:21:24.460216+0000 2023-07-21T22:28:54.351 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6801/3802710476 expires 2023-07-22T22:21:24.460216+0000 2023-07-21T22:28:54.351 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/2067809774 expires 2023-07-22T22:21:24.460216+0000 2023-07-21T22:28:54.351 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/2344232960 expires 2023-07-22T22:21:45.714358+0000 2023-07-21T22:28:54.352 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.300+0000 7fc289f24700 1 -- 172.21.15.107:0/67392058 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc26c0a11e0 msgr2=0x7fc26c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:54.352 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.300+0000 7fc289f24700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc26c0a11e0 0x7fc26c0a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc27400b2e0 tx=0x7fc274005fb0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.352 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.300+0000 7fc289f24700 1 -- 172.21.15.107:0/67392058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc284102f70 msgr2=0x7fc284195e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:54.352 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.300+0000 7fc289f24700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc284102f70 0x7fc284195e30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc27800d9b0 tx=0x7fc27800dcc0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.352 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.300+0000 7fc289f24700 1 -- 172.21.15.107:0/67392058 shutdown_connections 2023-07-21T22:28:54.353 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.300+0000 7fc289f24700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc284105890 0x7fc28419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.353 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.300+0000 7fc289f24700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc26c0a11e0 0x7fc26c0a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.353 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.300+0000 7fc289f24700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc284100650 0x7fc2841958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.353 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.300+0000 7fc289f24700 1 --2- 172.21.15.107:0/67392058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc284102f70 0x7fc284195e30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.353 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.300+0000 7fc289f24700 1 -- 172.21.15.107:0/67392058 >> 172.21.15.107:0/67392058 conn(0x7fc2840fa0b0 msgr2=0x7fc2841012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:54.353 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.300+0000 7fc289f24700 1 -- 172.21.15.107:0/67392058 shutdown_connections 2023-07-21T22:28:54.354 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.300+0000 7fc289f24700 1 -- 172.21.15.107:0/67392058 wait complete. 2023-07-21T22:28:54.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:54 smithi190 ceph-mon[98334]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 123 B/s, 0 objects/s recovering 2023-07-21T22:28:54.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:54 smithi190 ceph-mon[98334]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2023-07-21T22:28:54.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:54 smithi190 ceph-mon[98334]: Cluster is now healthy 2023-07-21T22:28:54.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:54 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/67392058' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T22:28:54.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:54 smithi132 ceph-mon[97863]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 123 B/s, 0 objects/s recovering 2023-07-21T22:28:54.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:54 smithi132 ceph-mon[97863]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2023-07-21T22:28:54.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:54 smithi132 ceph-mon[97863]: Cluster is now healthy 2023-07-21T22:28:54.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:54 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/67392058' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T22:28:54.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:54 smithi107 ceph-mon[96543]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 123 B/s, 0 objects/s recovering 2023-07-21T22:28:54.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:54 smithi107 ceph-mon[96543]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2023-07-21T22:28:54.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:54 smithi107 ceph-mon[96543]: Cluster is now healthy 2023-07-21T22:28:54.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:54 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/67392058' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T22:28:54.715 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-21T22:28:54.728 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:28:54.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.806+0000 7f46c80a6700 1 -- 172.21.15.107:0/3784287448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c01017e0 msgr2=0x7f46c01080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:54.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.806+0000 7f46c80a6700 1 --2- 172.21.15.107:0/3784287448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c01017e0 0x7f46c01080a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f46b800b0a0 tx=0x7f46b800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.806+0000 7f46c80a6700 1 -- 172.21.15.107:0/3784287448 shutdown_connections 2023-07-21T22:28:54.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.806+0000 7f46c80a6700 1 --2- 172.21.15.107:0/3784287448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c01017e0 0x7f46c01080a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.806+0000 7f46c80a6700 1 --2- 172.21.15.107:0/3784287448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c00fdde0 0x7f46c00fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.806+0000 7f46c80a6700 1 --2- 172.21.15.107:0/3784287448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c0100e50 0x7f46c01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.806+0000 7f46c80a6700 1 -- 172.21.15.107:0/3784287448 >> 172.21.15.107:0/3784287448 conn(0x7f46c00f97a0 msgr2=0x7f46c00fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:54.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c80a6700 1 -- 172.21.15.107:0/3784287448 shutdown_connections 2023-07-21T22:28:54.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c80a6700 1 -- 172.21.15.107:0/3784287448 wait complete. 2023-07-21T22:28:54.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c80a6700 1 Processor -- start 2023-07-21T22:28:54.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c80a6700 1 -- start start 2023-07-21T22:28:54.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c80a6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c00fdde0 0x7f46c01958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:54.858 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c80a6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c0100e50 0x7f46c0195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:54.858 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c80a6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c01017e0 0x7f46c019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:54.858 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c80a6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c019d420 con 0x7f46c0100e50 2023-07-21T22:28:54.858 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c80a6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c019d560 con 0x7f46c01017e0 2023-07-21T22:28:54.858 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c80a6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c019d6f0 con 0x7f46c00fdde0 2023-07-21T22:28:54.858 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c5641700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c0100e50 0x7f46c0195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:54.858 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c5641700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c0100e50 0x7f46c0195de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47542/0 (socket says 172.21.15.107:47542) 2023-07-21T22:28:54.858 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c5641700 1 -- 172.21.15.107:0/2963135473 learned_addr learned my addr 172.21.15.107:0/2963135473 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.807+0000 7f46c5e42700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c00fdde0 0x7f46c01958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.808+0000 7f46c6643700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c01017e0 0x7f46c019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.808+0000 7f46c5641700 1 -- 172.21.15.107:0/2963135473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c00fdde0 msgr2=0x7f46c01958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.808+0000 7f46c5641700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c00fdde0 0x7f46c01958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.808+0000 7f46c5641700 1 -- 172.21.15.107:0/2963135473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c01017e0 msgr2=0x7f46c019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.808+0000 7f46c5641700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c01017e0 0x7f46c019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:54.859 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.808+0000 7f46c5641700 1 -- 172.21.15.107:0/2963135473 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b8009d00 con 0x7f46c0100e50 2023-07-21T22:28:54.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.808+0000 7f46c5641700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c0100e50 0x7f46c0195de0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f46b000c920 tx=0x7f46b000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:54.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.808+0000 7f46b6ffd700 1 -- 172.21.15.107:0/2963135473 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b0007b00 con 0x7f46c0100e50 2023-07-21T22:28:54.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.808+0000 7f46c80a6700 1 -- 172.21.15.107:0/2963135473 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c019d9d0 con 0x7f46c0100e50 2023-07-21T22:28:54.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.808+0000 7f46b6ffd700 1 -- 172.21.15.107:0/2963135473 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46b000ce70 con 0x7f46c0100e50 2023-07-21T22:28:54.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.808+0000 7f46b6ffd700 1 -- 172.21.15.107:0/2963135473 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b00054e0 con 0x7f46c0100e50 2023-07-21T22:28:54.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.808+0000 7f46c80a6700 1 -- 172.21.15.107:0/2963135473 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c019dee0 con 0x7f46c0100e50 2023-07-21T22:28:54.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.809+0000 7f46c80a6700 1 -- 172.21.15.107:0/2963135473 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c0107100 con 0x7f46c0100e50 2023-07-21T22:28:54.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.810+0000 7f46b6ffd700 1 -- 172.21.15.107:0/2963135473 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f46b0004750 con 0x7f46c0100e50 2023-07-21T22:28:54.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.811+0000 7f46b6ffd700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f46ac0a1220 0x7f46ac0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:54.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.811+0000 7f46c5e42700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f46ac0a1220 0x7f46ac0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:54.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.811+0000 7f46b6ffd700 1 -- 172.21.15.107:0/2963135473 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f46b00cca70 con 0x7f46c0100e50 2023-07-21T22:28:54.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.811+0000 7f46c5e42700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f46ac0a1220 0x7f46ac0a36d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f46c0068630 tx=0x7f46bc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:54.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:54.814+0000 7f46b6ffd700 1 -- 172.21.15.107:0/2963135473 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f46b0095fe0 con 0x7f46c0100e50 2023-07-21T22:28:55.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.002+0000 7f46c80a6700 1 -- 172.21.15.107:0/2963135473 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46c002ca20 con 0x7f46c0100e50 2023-07-21T22:28:55.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.003+0000 7f46b6ffd700 1 -- 172.21.15.107:0/2963135473 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22727 (secure 0 0 0) 0x7f46b001d020 con 0x7f46c0100e50 2023-07-21T22:28:55.050 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:55.051 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":84,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:28:47.235779+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:28:55.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.005+0000 7f46c80a6700 1 -- 172.21.15.107:0/2963135473 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f46ac0a1220 msgr2=0x7f46ac0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.005+0000 7f46c80a6700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f46ac0a1220 0x7f46ac0a36d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f46c0068630 tx=0x7f46bc00b410 comp rx=0 tx=0).stop 2023-07-21T22:28:55.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.005+0000 7f46c80a6700 1 -- 172.21.15.107:0/2963135473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c0100e50 msgr2=0x7f46c0195de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.005+0000 7f46c80a6700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c0100e50 0x7f46c0195de0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f46b000c920 tx=0x7f46b000cc30 comp rx=0 tx=0).stop 2023-07-21T22:28:55.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.005+0000 7f46c80a6700 1 -- 172.21.15.107:0/2963135473 shutdown_connections 2023-07-21T22:28:55.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.005+0000 7f46c80a6700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46c01017e0 0x7f46c019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.005+0000 7f46c80a6700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f46ac0a1220 0x7f46ac0a36d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.005+0000 7f46c80a6700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46c00fdde0 0x7f46c01958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.005+0000 7f46c80a6700 1 --2- 172.21.15.107:0/2963135473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c0100e50 0x7f46c0195de0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.005+0000 7f46c80a6700 1 -- 172.21.15.107:0/2963135473 >> 172.21.15.107:0/2963135473 conn(0x7f46c00f97a0 msgr2=0x7f46c0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:55.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.005+0000 7f46c80a6700 1 -- 172.21.15.107:0/2963135473 shutdown_connections 2023-07-21T22:28:55.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.005+0000 7f46c80a6700 1 -- 172.21.15.107:0/2963135473 wait complete. 2023-07-21T22:28:55.065 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:23:52.031819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:27:39.494209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:27:40.672303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:28:55.067 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:28:55.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.145+0000 7f72d4b94700 1 -- 172.21.15.107:0/3793715535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d00fec50 msgr2=0x7f72d00ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.145+0000 7f72d4b94700 1 --2- 172.21.15.107:0/3793715535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d00fec50 0x7f72d00ff0a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009990 tx=0x7f72c0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.145+0000 7f72d4b94700 1 -- 172.21.15.107:0/3793715535 shutdown_connections 2023-07-21T22:28:55.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.145+0000 7f72d4b94700 1 --2- 172.21.15.107:0/3793715535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d00ff5e0 0x7f72d0106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.145+0000 7f72d4b94700 1 --2- 172.21.15.107:0/3793715535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72d00fffb0 0x7f72d01003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.145+0000 7f72d4b94700 1 --2- 172.21.15.107:0/3793715535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d00fec50 0x7f72d00ff0a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.145+0000 7f72d4b94700 1 -- 172.21.15.107:0/3793715535 >> 172.21.15.107:0/3793715535 conn(0x7f72d00f9770 msgr2=0x7f72d00fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:55.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.145+0000 7f72d4b94700 1 -- 172.21.15.107:0/3793715535 shutdown_connections 2023-07-21T22:28:55.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.145+0000 7f72d4b94700 1 -- 172.21.15.107:0/3793715535 wait complete. 2023-07-21T22:28:55.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.145+0000 7f72d4b94700 1 Processor -- start 2023-07-21T22:28:55.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72d4b94700 1 -- start start 2023-07-21T22:28:55.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72d4b94700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d00fec50 0x7f72d0195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:55.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72d4b94700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d00ff5e0 0x7f72d0195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:55.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72d4b94700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72d00fffb0 0x7f72d019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:55.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72d4b94700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d019d380 con 0x7f72d00fec50 2023-07-21T22:28:55.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72d4b94700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d019d4c0 con 0x7f72d00ff5e0 2023-07-21T22:28:55.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72d4b94700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d019d650 con 0x7f72d00fffb0 2023-07-21T22:28:55.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72ce59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d00fec50 0x7f72d0195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:55.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72ce59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d00fec50 0x7f72d0195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47554/0 (socket says 172.21.15.107:47554) 2023-07-21T22:28:55.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72ce59c700 1 -- 172.21.15.107:0/1875100880 learned_addr learned my addr 172.21.15.107:0/1875100880 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:55.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72ce59c700 1 -- 172.21.15.107:0/1875100880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72d00fffb0 msgr2=0x7f72d019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72ced9d700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72d00fffb0 0x7f72d019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:55.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72cdd9b700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d00ff5e0 0x7f72d0195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:55.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72ce59c700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72d00fffb0 0x7f72d019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72ce59c700 1 -- 172.21.15.107:0/1875100880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d00ff5e0 msgr2=0x7f72d0195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72ce59c700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d00ff5e0 0x7f72d0195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72ce59c700 1 -- 172.21.15.107:0/1875100880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c00095f0 con 0x7f72d00fec50 2023-07-21T22:28:55.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72ced9d700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72d00fffb0 0x7f72d019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:55.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72ce59c700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d00fec50 0x7f72d0195800 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f72c400d980 tx=0x7f72c400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:55.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72bf7fe700 1 -- 172.21.15.107:0/1875100880 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c400de60 con 0x7f72d00fec50 2023-07-21T22:28:55.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72bf7fe700 1 -- 172.21.15.107:0/1875100880 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72c40044e0 con 0x7f72d00fec50 2023-07-21T22:28:55.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72d4b94700 1 -- 172.21.15.107:0/1875100880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d019d930 con 0x7f72d00fec50 2023-07-21T22:28:55.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.146+0000 7f72bf7fe700 1 -- 172.21.15.107:0/1875100880 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4003be0 con 0x7f72d00fec50 2023-07-21T22:28:55.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.147+0000 7f72d4b94700 1 -- 172.21.15.107:0/1875100880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d019de90 con 0x7f72d00fec50 2023-07-21T22:28:55.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.147+0000 7f72d4b94700 1 -- 172.21.15.107:0/1875100880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d010a1a0 con 0x7f72d00fec50 2023-07-21T22:28:55.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.149+0000 7f72bf7fe700 1 -- 172.21.15.107:0/1875100880 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f72c4009bb0 con 0x7f72d00fec50 2023-07-21T22:28:55.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.149+0000 7f72bf7fe700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f72b80a1260 0x7f72b80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:55.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.150+0000 7f72cdd9b700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f72b80a1260 0x7f72b80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:55.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.150+0000 7f72bf7fe700 1 -- 172.21.15.107:0/1875100880 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f72c40ccb60 con 0x7f72d00fec50 2023-07-21T22:28:55.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.150+0000 7f72cdd9b700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f72b80a1260 0x7f72b80a3710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f72c00095c0 tx=0x7f72c0009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:55.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.153+0000 7f72bf7fe700 1 -- 172.21.15.107:0/1875100880 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f72c40961b0 con 0x7f72d00fec50 2023-07-21T22:28:55.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.342+0000 7f72d4b94700 1 -- 172.21.15.107:0/1875100880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72d019dac0 con 0x7f72d00fec50 2023-07-21T22:28:55.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.342+0000 7f72bf7fe700 1 -- 172.21.15.107:0/1875100880 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72d019dac0 con 0x7f72d00fec50 2023-07-21T22:28:55.389 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2023-07-21T22:28:55.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.344+0000 7f72d4b94700 1 -- 172.21.15.107:0/1875100880 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f72b80a1260 msgr2=0x7f72b80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.344+0000 7f72d4b94700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f72b80a1260 0x7f72b80a3710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f72c00095c0 tx=0x7f72c0009530 comp rx=0 tx=0).stop 2023-07-21T22:28:55.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.344+0000 7f72d4b94700 1 -- 172.21.15.107:0/1875100880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d00fec50 msgr2=0x7f72d0195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.344+0000 7f72d4b94700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d00fec50 0x7f72d0195800 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f72c400d980 tx=0x7f72c400dd40 comp rx=0 tx=0).stop 2023-07-21T22:28:55.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.344+0000 7f72d4b94700 1 -- 172.21.15.107:0/1875100880 shutdown_connections 2023-07-21T22:28:55.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.344+0000 7f72d4b94700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f72b80a1260 0x7f72b80a3710 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.344+0000 7f72d4b94700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72d00fffb0 0x7f72d019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.344+0000 7f72d4b94700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d00fec50 0x7f72d0195800 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.393 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.344+0000 7f72d4b94700 1 --2- 172.21.15.107:0/1875100880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d00ff5e0 0x7f72d0195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.393 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.344+0000 7f72d4b94700 1 -- 172.21.15.107:0/1875100880 >> 172.21.15.107:0/1875100880 conn(0x7f72d00f9770 msgr2=0x7f72d00fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:55.393 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.344+0000 7f72d4b94700 1 -- 172.21.15.107:0/1875100880 shutdown_connections 2023-07-21T22:28:55.393 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.344+0000 7f72d4b94700 1 -- 172.21.15.107:0/1875100880 wait complete. 2023-07-21T22:28:55.403 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:28:55.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.481+0000 7f3400425700 1 -- 172.21.15.107:0/1963947427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80fefc0 msgr2=0x7f33f80ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.481+0000 7f3400425700 1 --2- 172.21.15.107:0/1963947427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80fefc0 0x7f33f80ff3d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009990 tx=0x7f33e8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.481+0000 7f3400425700 1 -- 172.21.15.107:0/1963947427 shutdown_connections 2023-07-21T22:28:55.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.481+0000 7f3400425700 1 --2- 172.21.15.107:0/1963947427 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f81003f0 0x7f33f8106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.481+0000 7f3400425700 1 --2- 172.21.15.107:0/1963947427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80fefc0 0x7f33f80ff3d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.481+0000 7f3400425700 1 --2- 172.21.15.107:0/1963947427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33f80ff910 0x7f33f80ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.481+0000 7f3400425700 1 -- 172.21.15.107:0/1963947427 >> 172.21.15.107:0/1963947427 conn(0x7f33f80f9770 msgr2=0x7f33f80fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:55.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.481+0000 7f3400425700 1 -- 172.21.15.107:0/1963947427 shutdown_connections 2023-07-21T22:28:55.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.481+0000 7f3400425700 1 -- 172.21.15.107:0/1963947427 wait complete. 2023-07-21T22:28:55.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.481+0000 7f3400425700 1 Processor -- start 2023-07-21T22:28:55.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.481+0000 7f3400425700 1 -- start start 2023-07-21T22:28:55.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.481+0000 7f3400425700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80fefc0 0x7f33f8194730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:55.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.481+0000 7f3400425700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f80ff910 0x7f33f8194c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:55.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f3400425700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33f81003f0 0x7f33f819bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:55.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f3400425700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f819c2b0 con 0x7f33f80fefc0 2023-07-21T22:28:55.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f3400425700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f819c3f0 con 0x7f33f81003f0 2023-07-21T22:28:55.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f3400425700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f819c580 con 0x7f33f80ff910 2023-07-21T22:28:55.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33fe1c1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80fefc0 0x7f33f8194730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:55.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33fe1c1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80fefc0 0x7f33f8194730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47568/0 (socket says 172.21.15.107:47568) 2023-07-21T22:28:55.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33fe1c1700 1 -- 172.21.15.107:0/39431379 learned_addr learned my addr 172.21.15.107:0/39431379 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:55.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33fd9c0700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f80ff910 0x7f33f8194c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:55.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33fe9c2700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33f81003f0 0x7f33f819bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:55.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33fe1c1700 1 -- 172.21.15.107:0/39431379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f80ff910 msgr2=0x7f33f8194c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33fe1c1700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f80ff910 0x7f33f8194c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33fe1c1700 1 -- 172.21.15.107:0/39431379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33f81003f0 msgr2=0x7f33f819bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33fe1c1700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33f81003f0 0x7f33f819bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33fe1c1700 1 -- 172.21.15.107:0/39431379 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e80095f0 con 0x7f33f80fefc0 2023-07-21T22:28:55.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33fe1c1700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80fefc0 0x7f33f8194730 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f33e8000c00 tx=0x7f33e800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:55.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33ef7fe700 1 -- 172.21.15.107:0/39431379 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e801d070 con 0x7f33f80fefc0 2023-07-21T22:28:55.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33ef7fe700 1 -- 172.21.15.107:0/39431379 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33e800fb50 con 0x7f33f80fefc0 2023-07-21T22:28:55.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f3400425700 1 -- 172.21.15.107:0/39431379 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f80fdc40 con 0x7f33f80fefc0 2023-07-21T22:28:55.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f33ef7fe700 1 -- 172.21.15.107:0/39431379 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e80173f0 con 0x7f33f80fefc0 2023-07-21T22:28:55.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.482+0000 7f3400425700 1 -- 172.21.15.107:0/39431379 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f80fe000 con 0x7f33f80fefc0 2023-07-21T22:28:55.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.484+0000 7f3400425700 1 -- 172.21.15.107:0/39431379 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f80fe9f0 con 0x7f33f80fefc0 2023-07-21T22:28:55.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.484+0000 7f33ef7fe700 1 -- 172.21.15.107:0/39431379 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f33e800fcc0 con 0x7f33f80fefc0 2023-07-21T22:28:55.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.485+0000 7f33ef7fe700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f33e40a11e0 0x7f33e40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:55.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.485+0000 7f33fd9c0700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f33e40a11e0 0x7f33e40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:55.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.485+0000 7f33ef7fe700 1 -- 172.21.15.107:0/39431379 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f33e80cdca0 con 0x7f33f80fefc0 2023-07-21T22:28:55.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.485+0000 7f33fd9c0700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f33e40a11e0 0x7f33e40a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f33f4005950 tx=0x7f33f400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.489+0000 7f33ef7fe700 1 -- 172.21.15.107:0/39431379 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f33e8097300 con 0x7f33f80fefc0 2023-07-21T22:28:55.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:55 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2963135473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:55.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:55 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2963135473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:55.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:55 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2963135473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:55.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.678+0000 7f3400425700 1 -- 172.21.15.107:0/39431379 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33f8106130 con 0x7f33f80fefc0 2023-07-21T22:28:55.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.679+0000 7f33ef7fe700 1 -- 172.21.15.107:0/39431379 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f33e8027020 con 0x7f33f80fefc0 2023-07-21T22:28:55.725 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2023-07-21T22:28:55.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.681+0000 7f3400425700 1 -- 172.21.15.107:0/39431379 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f33e40a11e0 msgr2=0x7f33e40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.681+0000 7f3400425700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f33e40a11e0 0x7f33e40a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f33f4005950 tx=0x7f33f400b410 comp rx=0 tx=0).stop 2023-07-21T22:28:55.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.681+0000 7f3400425700 1 -- 172.21.15.107:0/39431379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80fefc0 msgr2=0x7f33f8194730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.681+0000 7f3400425700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80fefc0 0x7f33f8194730 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f33e8000c00 tx=0x7f33e800f710 comp rx=0 tx=0).stop 2023-07-21T22:28:55.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.681+0000 7f3400425700 1 -- 172.21.15.107:0/39431379 shutdown_connections 2023-07-21T22:28:55.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.681+0000 7f3400425700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33f81003f0 0x7f33f819bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.681+0000 7f3400425700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80fefc0 0x7f33f8194730 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.681+0000 7f3400425700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f33e40a11e0 0x7f33e40a3690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.681+0000 7f3400425700 1 --2- 172.21.15.107:0/39431379 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33f80ff910 0x7f33f8194c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.681+0000 7f3400425700 1 -- 172.21.15.107:0/39431379 >> 172.21.15.107:0/39431379 conn(0x7f33f80f9770 msgr2=0x7f33f8103290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:55.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.681+0000 7f3400425700 1 -- 172.21.15.107:0/39431379 shutdown_connections 2023-07-21T22:28:55.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.681+0000 7f3400425700 1 -- 172.21.15.107:0/39431379 wait complete. 2023-07-21T22:28:55.740 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:28:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.818+0000 7f9f862e6700 1 -- 172.21.15.107:0/4209052139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f800ffae0 msgr2=0x7f9f800ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.818+0000 7f9f862e6700 1 --2- 172.21.15.107:0/4209052139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f800ffae0 0x7f9f800ffef0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009990 tx=0x7f9f68009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.818+0000 7f9f862e6700 1 -- 172.21.15.107:0/4209052139 shutdown_connections 2023-07-21T22:28:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.818+0000 7f9f862e6700 1 --2- 172.21.15.107:0/4209052139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f80101660 0x7f9f80107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.818+0000 7f9f862e6700 1 --2- 172.21.15.107:0/4209052139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f800ffae0 0x7f9f800ffef0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.818+0000 7f9f862e6700 1 --2- 172.21.15.107:0/4209052139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f80100cd0 0x7f9f80101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.818+0000 7f9f862e6700 1 -- 172.21.15.107:0/4209052139 >> 172.21.15.107:0/4209052139 conn(0x7f9f800fb120 msgr2=0x7f9f800fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:55.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.818+0000 7f9f862e6700 1 -- 172.21.15.107:0/4209052139 shutdown_connections 2023-07-21T22:28:55.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.818+0000 7f9f862e6700 1 -- 172.21.15.107:0/4209052139 wait complete. 2023-07-21T22:28:55.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.818+0000 7f9f862e6700 1 Processor -- start 2023-07-21T22:28:55.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.818+0000 7f9f862e6700 1 -- start start 2023-07-21T22:28:55.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f862e6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f800ffae0 0x7f9f800742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:55.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f862e6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f80100cd0 0x7f9f80072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:55.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f862e6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f80101660 0x7f9f80072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:55.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f862e6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f800735e0 con 0x7f9f80100cd0 2023-07-21T22:28:55.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f862e6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f80073750 con 0x7f9f800ffae0 2023-07-21T22:28:55.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f862e6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8019f810 con 0x7f9f80101660 2023-07-21T22:28:55.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f7f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f80100cd0 0x7f9f80072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:55.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f7f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f80100cd0 0x7f9f80072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47584/0 (socket says 172.21.15.107:47584) 2023-07-21T22:28:55.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f7f7fe700 1 -- 172.21.15.107:0/2335909738 learned_addr learned my addr 172.21.15.107:0/2335909738 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:55.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f7ffff700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f800ffae0 0x7f9f800742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:55.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f84883700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f80101660 0x7f9f80072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:55.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f7f7fe700 1 -- 172.21.15.107:0/2335909738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f80101660 msgr2=0x7f9f80072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f7f7fe700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f80101660 0x7f9f80072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f7f7fe700 1 -- 172.21.15.107:0/2335909738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f800ffae0 msgr2=0x7f9f800742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:55.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f7f7fe700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f800ffae0 0x7f9f800742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:55.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f7f7fe700 1 -- 172.21.15.107:0/2335909738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f680095f0 con 0x7f9f80100cd0 2023-07-21T22:28:55.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.819+0000 7f9f7f7fe700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f80100cd0 0x7f9f80072900 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000c9b0 tx=0x7f9f7000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:55.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.820+0000 7f9f7d7fa700 1 -- 172.21.15.107:0/2335909738 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70007b00 con 0x7f9f80100cd0 2023-07-21T22:28:55.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.820+0000 7f9f862e6700 1 -- 172.21.15.107:0/2335909738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8019faf0 con 0x7f9f80100cd0 2023-07-21T22:28:55.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.820+0000 7f9f7d7fa700 1 -- 172.21.15.107:0/2335909738 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f7000ce50 con 0x7f9f80100cd0 2023-07-21T22:28:55.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.820+0000 7f9f7d7fa700 1 -- 172.21.15.107:0/2335909738 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70018660 con 0x7f9f80100cd0 2023-07-21T22:28:55.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.820+0000 7f9f862e6700 1 -- 172.21.15.107:0/2335909738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f801a0000 con 0x7f9f80100cd0 2023-07-21T22:28:55.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.820+0000 7f9f862e6700 1 -- 172.21.15.107:0/2335909738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f801070a0 con 0x7f9f80100cd0 2023-07-21T22:28:55.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.822+0000 7f9f7d7fa700 1 -- 172.21.15.107:0/2335909738 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f9f7000f450 con 0x7f9f80100cd0 2023-07-21T22:28:55.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.822+0000 7f9f7d7fa700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9f6c0a11e0 0x7f9f6c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:55.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.822+0000 7f9f7ffff700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9f6c0a11e0 0x7f9f6c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:55.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.822+0000 7f9f7d7fa700 1 -- 172.21.15.107:0/2335909738 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9f700ccc30 con 0x7f9f80100cd0 2023-07-21T22:28:55.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.822+0000 7f9f7ffff700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9f6c0a11e0 0x7f9f6c0a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800b5c0 tx=0x7f9f68005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:55.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:55.825+0000 7f9f7d7fa700 1 -- 172.21.15.107:0/2335909738 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9f70096290 con 0x7f9f80100cd0 2023-07-21T22:28:56.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.012+0000 7f9f862e6700 1 -- 172.21.15.107:0/2335909738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9f8002c9e0 con 0x7f9f80100cd0 2023-07-21T22:28:56.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.013+0000 7f9f7d7fa700 1 -- 172.21.15.107:0/2335909738 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9f70007e60 con 0x7f9f80100cd0 2023-07-21T22:28:56.060 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2023-07-21T22:28:56.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.015+0000 7f9f862e6700 1 -- 172.21.15.107:0/2335909738 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9f6c0a11e0 msgr2=0x7f9f6c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.015+0000 7f9f862e6700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9f6c0a11e0 0x7f9f6c0a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800b5c0 tx=0x7f9f68005960 comp rx=0 tx=0).stop 2023-07-21T22:28:56.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.015+0000 7f9f862e6700 1 -- 172.21.15.107:0/2335909738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f80100cd0 msgr2=0x7f9f80072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.015+0000 7f9f862e6700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f80100cd0 0x7f9f80072900 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000c9b0 tx=0x7f9f7000ccc0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.015+0000 7f9f862e6700 1 -- 172.21.15.107:0/2335909738 shutdown_connections 2023-07-21T22:28:56.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.015+0000 7f9f862e6700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9f6c0a11e0 0x7f9f6c0a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.015+0000 7f9f862e6700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f80101660 0x7f9f80072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.015+0000 7f9f862e6700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f800ffae0 0x7f9f800742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.015+0000 7f9f862e6700 1 --2- 172.21.15.107:0/2335909738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f80100cd0 0x7f9f80072900 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.015+0000 7f9f862e6700 1 -- 172.21.15.107:0/2335909738 >> 172.21.15.107:0/2335909738 conn(0x7f9f800fb120 msgr2=0x7f9f801041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:56.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.015+0000 7f9f862e6700 1 -- 172.21.15.107:0/2335909738 shutdown_connections 2023-07-21T22:28:56.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.015+0000 7f9f862e6700 1 -- 172.21.15.107:0/2335909738 wait complete. 2023-07-21T22:28:56.074 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:28:56.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.151+0000 7fcb61c77700 1 -- 172.21.15.107:0/52540193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb5c0fec10 msgr2=0x7fcb5c0ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.151+0000 7fcb61c77700 1 --2- 172.21.15.107:0/52540193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb5c0fec10 0x7fcb5c0ff080 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009990 tx=0x7fcb50009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.151+0000 7fcb61c77700 1 -- 172.21.15.107:0/52540193 shutdown_connections 2023-07-21T22:28:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.151+0000 7fcb61c77700 1 --2- 172.21.15.107:0/52540193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb5c0ff5c0 0x7fcb5c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.151+0000 7fcb61c77700 1 --2- 172.21.15.107:0/52540193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb5c100050 0x7fcb5c100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.151+0000 7fcb61c77700 1 --2- 172.21.15.107:0/52540193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb5c0fec10 0x7fcb5c0ff080 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.151+0000 7fcb61c77700 1 -- 172.21.15.107:0/52540193 >> 172.21.15.107:0/52540193 conn(0x7fcb5c0f9770 msgr2=0x7fcb5c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:56.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.151+0000 7fcb61c77700 1 -- 172.21.15.107:0/52540193 shutdown_connections 2023-07-21T22:28:56.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.151+0000 7fcb61c77700 1 -- 172.21.15.107:0/52540193 wait complete. 2023-07-21T22:28:56.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.151+0000 7fcb61c77700 1 Processor -- start 2023-07-21T22:28:56.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.151+0000 7fcb61c77700 1 -- start start 2023-07-21T22:28:56.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb61c77700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb5c0fec10 0x7fcb5c1946b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:56.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb61c77700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb5c0ff5c0 0x7fcb5c194bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:56.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb61c77700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb5c100050 0x7fcb5c19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:56.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb61c77700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c19c230 con 0x7fcb5c0fec10 2023-07-21T22:28:56.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb61c77700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c19c370 con 0x7fcb5c0ff5c0 2023-07-21T22:28:56.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb61c77700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c19c500 con 0x7fcb5c100050 2023-07-21T22:28:56.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb5b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb5c0fec10 0x7fcb5c1946b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:56.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb5b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb5c0fec10 0x7fcb5c1946b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47596/0 (socket says 172.21.15.107:47596) 2023-07-21T22:28:56.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb5b7fe700 1 -- 172.21.15.107:0/3515613357 learned_addr learned my addr 172.21.15.107:0/3515613357 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:56.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb5bfff700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb5c100050 0x7fcb5c19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:56.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb5affd700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb5c0ff5c0 0x7fcb5c194bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:56.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb5b7fe700 1 -- 172.21.15.107:0/3515613357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb5c100050 msgr2=0x7fcb5c19bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb5b7fe700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb5c100050 0x7fcb5c19bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb5b7fe700 1 -- 172.21.15.107:0/3515613357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb5c0ff5c0 msgr2=0x7fcb5c194bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb5b7fe700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb5c0ff5c0 0x7fcb5c194bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb5b7fe700 1 -- 172.21.15.107:0/3515613357 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb500095f0 con 0x7fcb5c0fec10 2023-07-21T22:28:56.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb5affd700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb5c0ff5c0 0x7fcb5c194bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:56.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb5b7fe700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb5c0fec10 0x7fcb5c1946b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00d950 tx=0x7fcb4c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:56.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb58ff9700 1 -- 172.21.15.107:0/3515613357 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c009940 con 0x7fcb5c0fec10 2023-07-21T22:28:56.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb58ff9700 1 -- 172.21.15.107:0/3515613357 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb4c010460 con 0x7fcb5c0fec10 2023-07-21T22:28:56.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb61c77700 1 -- 172.21.15.107:0/3515613357 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c19c7e0 con 0x7fcb5c0fec10 2023-07-21T22:28:56.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb58ff9700 1 -- 172.21.15.107:0/3515613357 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c00f620 con 0x7fcb5c0fec10 2023-07-21T22:28:56.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.152+0000 7fcb61c77700 1 -- 172.21.15.107:0/3515613357 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c19ccf0 con 0x7fcb5c0fec10 2023-07-21T22:28:56.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.153+0000 7fcb61c77700 1 -- 172.21.15.107:0/3515613357 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5c1058c0 con 0x7fcb5c0fec10 2023-07-21T22:28:56.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.155+0000 7fcb58ff9700 1 -- 172.21.15.107:0/3515613357 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fcb4c0108d0 con 0x7fcb5c0fec10 2023-07-21T22:28:56.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.155+0000 7fcb58ff9700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcb440a1220 0x7fcb440a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:56.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.155+0000 7fcb5affd700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcb440a1220 0x7fcb440a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:56.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.155+0000 7fcb58ff9700 1 -- 172.21.15.107:0/3515613357 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcb4c0cc950 con 0x7fcb5c0fec10 2023-07-21T22:28:56.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.155+0000 7fcb5affd700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcb440a1220 0x7fcb440a36d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000c00 tx=0x7fcb50005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:56.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.159+0000 7fcb58ff9700 1 -- 172.21.15.107:0/3515613357 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcb4c095f20 con 0x7fcb5c0fec10 2023-07-21T22:28:56.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.395+0000 7fcb61c77700 1 -- 172.21.15.107:0/3515613357 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb5c02c850 con 0x7fcb5c0fec10 2023-07-21T22:28:56.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.396+0000 7fcb58ff9700 1 -- 172.21.15.107:0/3515613357 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10868 (secure 0 0 0) 0x7fcb4c016090 con 0x7fcb5c0fec10 2023-07-21T22:28:56.443 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:56.443 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:27:49.034257+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:28:56.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.398+0000 7fcb61c77700 1 -- 172.21.15.107:0/3515613357 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcb440a1220 msgr2=0x7fcb440a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.398+0000 7fcb61c77700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcb440a1220 0x7fcb440a36d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000c00 tx=0x7fcb50005fb0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.398+0000 7fcb61c77700 1 -- 172.21.15.107:0/3515613357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb5c0fec10 msgr2=0x7fcb5c1946b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.398+0000 7fcb61c77700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb5c0fec10 0x7fcb5c1946b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00d950 tx=0x7fcb4c00dc60 comp rx=0 tx=0).stop 2023-07-21T22:28:56.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.398+0000 7fcb61c77700 1 -- 172.21.15.107:0/3515613357 shutdown_connections 2023-07-21T22:28:56.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.398+0000 7fcb61c77700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcb440a1220 0x7fcb440a36d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.398+0000 7fcb61c77700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb5c100050 0x7fcb5c19bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.398+0000 7fcb61c77700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb5c0fec10 0x7fcb5c1946b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.398+0000 7fcb61c77700 1 --2- 172.21.15.107:0/3515613357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb5c0ff5c0 0x7fcb5c194bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.398+0000 7fcb61c77700 1 -- 172.21.15.107:0/3515613357 >> 172.21.15.107:0/3515613357 conn(0x7fcb5c0f9770 msgr2=0x7fcb5c0fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:56.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.398+0000 7fcb61c77700 1 -- 172.21.15.107:0/3515613357 shutdown_connections 2023-07-21T22:28:56.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.398+0000 7fcb61c77700 1 -- 172.21.15.107:0/3515613357 wait complete. 2023-07-21T22:28:56.447 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2023-07-21T22:28:56.457 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:28:56.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.532+0000 7fdb7ab41700 1 -- 172.21.15.107:0/2030012258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb74100cf0 msgr2=0x7fdb74101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.532+0000 7fdb7ab41700 1 --2- 172.21.15.107:0/2030012258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb74100cf0 0x7fdb74101140 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009990 tx=0x7fdb60009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.532+0000 7fdb7ab41700 1 -- 172.21.15.107:0/2030012258 shutdown_connections 2023-07-21T22:28:56.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.532+0000 7fdb7ab41700 1 --2- 172.21.15.107:0/2030012258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb74101680 0x7fdb74107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.532+0000 7fdb7ab41700 1 --2- 172.21.15.107:0/2030012258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb740ffb00 0x7fdb740fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.532+0000 7fdb7ab41700 1 --2- 172.21.15.107:0/2030012258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb74100cf0 0x7fdb74101140 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.532+0000 7fdb7ab41700 1 -- 172.21.15.107:0/2030012258 >> 172.21.15.107:0/2030012258 conn(0x7fdb740fb100 msgr2=0x7fdb740fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:56.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.532+0000 7fdb7ab41700 1 -- 172.21.15.107:0/2030012258 shutdown_connections 2023-07-21T22:28:56.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.532+0000 7fdb7ab41700 1 -- 172.21.15.107:0/2030012258 wait complete. 2023-07-21T22:28:56.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb7ab41700 1 Processor -- start 2023-07-21T22:28:56.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb7ab41700 1 -- start start 2023-07-21T22:28:56.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb7ab41700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb740ffb00 0x7fdb741957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:56.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb7ab41700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb74100cf0 0x7fdb74195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:56.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb7ab41700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb74101680 0x7fdb7419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:56.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb7ab41700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7419d320 con 0x7fdb74101680 2023-07-21T22:28:56.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb7ab41700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7419d460 con 0x7fdb74100cf0 2023-07-21T22:28:56.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb7ab41700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7419d5f0 con 0x7fdb740ffb00 2023-07-21T22:28:56.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb790de700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb74101680 0x7fdb7419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:56.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb790de700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb74101680 0x7fdb7419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47614/0 (socket says 172.21.15.107:47614) 2023-07-21T22:28:56.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb790de700 1 -- 172.21.15.107:0/4244973157 learned_addr learned my addr 172.21.15.107:0/4244973157 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:56.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb788dd700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb740ffb00 0x7fdb741957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:56.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb73fff700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb74100cf0 0x7fdb74195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:56.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb790de700 1 -- 172.21.15.107:0/4244973157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb740ffb00 msgr2=0x7fdb741957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb790de700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb740ffb00 0x7fdb741957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb790de700 1 -- 172.21.15.107:0/4244973157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb74100cf0 msgr2=0x7fdb74195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb790de700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb74100cf0 0x7fdb74195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb790de700 1 -- 172.21.15.107:0/4244973157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb600095f0 con 0x7fdb74101680 2023-07-21T22:28:56.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb73fff700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb74100cf0 0x7fdb74195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:56.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.533+0000 7fdb790de700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb74101680 0x7fdb7419cd50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c007ee0 tx=0x7fdb6c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:56.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.534+0000 7fdb71ffb700 1 -- 172.21.15.107:0/4244973157 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c017070 con 0x7fdb74101680 2023-07-21T22:28:56.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.534+0000 7fdb7ab41700 1 -- 172.21.15.107:0/4244973157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7419d8d0 con 0x7fdb74101680 2023-07-21T22:28:56.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.534+0000 7fdb71ffb700 1 -- 172.21.15.107:0/4244973157 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb6c00f040 con 0x7fdb74101680 2023-07-21T22:28:56.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.534+0000 7fdb71ffb700 1 -- 172.21.15.107:0/4244973157 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c005740 con 0x7fdb74101680 2023-07-21T22:28:56.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.534+0000 7fdb7ab41700 1 -- 172.21.15.107:0/4244973157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7419dd90 con 0x7fdb74101680 2023-07-21T22:28:56.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.535+0000 7fdb7ab41700 1 -- 172.21.15.107:0/4244973157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7410b2d0 con 0x7fdb74101680 2023-07-21T22:28:56.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.536+0000 7fdb71ffb700 1 -- 172.21.15.107:0/4244973157 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fdb6c00dc60 con 0x7fdb74101680 2023-07-21T22:28:56.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.536+0000 7fdb71ffb700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdb640a1220 0x7fdb640a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:56.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.536+0000 7fdb71ffb700 1 -- 172.21.15.107:0/4244973157 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdb6c0cc970 con 0x7fdb74101680 2023-07-21T22:28:56.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.536+0000 7fdb788dd700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdb640a1220 0x7fdb640a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:56.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.537+0000 7fdb788dd700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdb640a1220 0x7fdb640a36d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb74100b50 tx=0x7fdb6800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:56.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.540+0000 7fdb71ffb700 1 -- 172.21.15.107:0/4244973157 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdb6c095f40 con 0x7fdb74101680 2023-07-21T22:28:56.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:56 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1875100880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:28:56.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:56 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/39431379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:28:56.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:56 smithi190 ceph-mon[98334]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s; 103 B/s, 0 objects/s recovering 2023-07-21T22:28:56.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:56 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2335909738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:28:56.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:56 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1875100880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:28:56.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:56 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/39431379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:28:56.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:56 smithi132 ceph-mon[97863]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s; 103 B/s, 0 objects/s recovering 2023-07-21T22:28:56.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:56 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2335909738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:28:56.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:56 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1875100880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:28:56.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:56 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/39431379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:28:56.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:56 smithi107 ceph-mon[96543]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 6 op/s; 103 B/s, 0 objects/s recovering 2023-07-21T22:28:56.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:56 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2335909738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:28:56.776 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.730+0000 7fdb7ab41700 1 -- 172.21.15.107:0/4244973157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb741069a0 con 0x7fdb74101680 2023-07-21T22:28:56.777 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.731+0000 7fdb71ffb700 1 -- 172.21.15.107:0/4244973157 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22727 (secure 0 0 0) 0x7fdb6c099560 con 0x7fdb74101680 2023-07-21T22:28:56.778 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:56.778 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":84,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:28:47.235779+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:28:56.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.733+0000 7fdb7ab41700 1 -- 172.21.15.107:0/4244973157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdb640a1220 msgr2=0x7fdb640a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.733+0000 7fdb7ab41700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdb640a1220 0x7fdb640a36d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb74100b50 tx=0x7fdb6800a380 comp rx=0 tx=0).stop 2023-07-21T22:28:56.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.733+0000 7fdb7ab41700 1 -- 172.21.15.107:0/4244973157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb74101680 msgr2=0x7fdb7419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.733+0000 7fdb7ab41700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb74101680 0x7fdb7419cd50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c007ee0 tx=0x7fdb6c00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.733+0000 7fdb7ab41700 1 -- 172.21.15.107:0/4244973157 shutdown_connections 2023-07-21T22:28:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.733+0000 7fdb7ab41700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb74101680 0x7fdb7419cd50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.733+0000 7fdb7ab41700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdb640a1220 0x7fdb640a36d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.733+0000 7fdb7ab41700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb740ffb00 0x7fdb741957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.733+0000 7fdb7ab41700 1 --2- 172.21.15.107:0/4244973157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb74100cf0 0x7fdb74195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.733+0000 7fdb7ab41700 1 -- 172.21.15.107:0/4244973157 >> 172.21.15.107:0/4244973157 conn(0x7fdb740fb100 msgr2=0x7fdb740fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.734+0000 7fdb7ab41700 1 -- 172.21.15.107:0/4244973157 shutdown_connections 2023-07-21T22:28:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.734+0000 7fdb7ab41700 1 -- 172.21.15.107:0/4244973157 wait complete. 2023-07-21T22:28:56.793 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:23:52.031819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:27:39.494209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:27:40.672303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:28:56.793 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:28:56.917 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.870+0000 7fcc1e95d700 1 -- 172.21.15.107:0/4210840674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc18100ce0 msgr2=0x7fcc18101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.917 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.870+0000 7fcc1e95d700 1 --2- 172.21.15.107:0/4210840674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc18100ce0 0x7fcc18101130 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009990 tx=0x7fcc0c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.918 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.870+0000 7fcc1e95d700 1 -- 172.21.15.107:0/4210840674 shutdown_connections 2023-07-21T22:28:56.918 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.870+0000 7fcc1e95d700 1 --2- 172.21.15.107:0/4210840674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc18101670 0x7fcc18107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.918 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.870+0000 7fcc1e95d700 1 --2- 172.21.15.107:0/4210840674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc180ffaf0 0x7fcc180fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.918 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.870+0000 7fcc1e95d700 1 --2- 172.21.15.107:0/4210840674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc18100ce0 0x7fcc18101130 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.918 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.870+0000 7fcc1e95d700 1 -- 172.21.15.107:0/4210840674 >> 172.21.15.107:0/4210840674 conn(0x7fcc180fb130 msgr2=0x7fcc180fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:56.918 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.870+0000 7fcc1e95d700 1 -- 172.21.15.107:0/4210840674 shutdown_connections 2023-07-21T22:28:56.918 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.870+0000 7fcc1e95d700 1 -- 172.21.15.107:0/4210840674 wait complete. 2023-07-21T22:28:56.919 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc1e95d700 1 Processor -- start 2023-07-21T22:28:56.919 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc1e95d700 1 -- start start 2023-07-21T22:28:56.921 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc1e95d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc180ffaf0 0x7fcc18195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:56.921 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc1e95d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc18100ce0 0x7fcc18195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:56.921 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc1e95d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc18101670 0x7fcc1819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:56.921 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc1e95d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1819d3a0 con 0x7fcc180ffaf0 2023-07-21T22:28:56.922 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc1e95d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1819d4e0 con 0x7fcc18101670 2023-07-21T22:28:56.922 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc1e95d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1819d670 con 0x7fcc18100ce0 2023-07-21T22:28:56.922 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc17fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc180ffaf0 0x7fcc18195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:56.922 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc17fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc180ffaf0 0x7fcc18195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47622/0 (socket says 172.21.15.107:47622) 2023-07-21T22:28:56.922 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc17fff700 1 -- 172.21.15.107:0/2015859464 learned_addr learned my addr 172.21.15.107:0/2015859464 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:56.922 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc177fe700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc18100ce0 0x7fcc18195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:56.922 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc1cefa700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc18101670 0x7fcc1819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:56.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc17fff700 1 -- 172.21.15.107:0/2015859464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc18100ce0 msgr2=0x7fcc18195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc17fff700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc18100ce0 0x7fcc18195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc17fff700 1 -- 172.21.15.107:0/2015859464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc18101670 msgr2=0x7fcc1819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:56.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc17fff700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc18101670 0x7fcc1819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:56.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc17fff700 1 -- 172.21.15.107:0/2015859464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c0095f0 con 0x7fcc180ffaf0 2023-07-21T22:28:56.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc177fe700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc18100ce0 0x7fcc18195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:56.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc1cefa700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc18101670 0x7fcc1819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:56.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.871+0000 7fcc17fff700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc180ffaf0 0x7fcc18195820 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800d980 tx=0x7fcc0800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:56.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.872+0000 7fcc157fa700 1 -- 172.21.15.107:0/2015859464 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0800bb60 con 0x7fcc180ffaf0 2023-07-21T22:28:56.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.872+0000 7fcc1e95d700 1 -- 172.21.15.107:0/2015859464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1819d950 con 0x7fcc180ffaf0 2023-07-21T22:28:56.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.872+0000 7fcc157fa700 1 -- 172.21.15.107:0/2015859464 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc08007620 con 0x7fcc180ffaf0 2023-07-21T22:28:56.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.872+0000 7fcc157fa700 1 -- 172.21.15.107:0/2015859464 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0800f620 con 0x7fcc180ffaf0 2023-07-21T22:28:56.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.872+0000 7fcc1e95d700 1 -- 172.21.15.107:0/2015859464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1819de60 con 0x7fcc180ffaf0 2023-07-21T22:28:56.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.872+0000 7fcc1e95d700 1 -- 172.21.15.107:0/2015859464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc18107080 con 0x7fcc180ffaf0 2023-07-21T22:28:56.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.874+0000 7fcc157fa700 1 -- 172.21.15.107:0/2015859464 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fcc08010460 con 0x7fcc180ffaf0 2023-07-21T22:28:56.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.874+0000 7fcc157fa700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc000a11e0 0x7fcc000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:56.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.874+0000 7fcc157fa700 1 -- 172.21.15.107:0/2015859464 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcc080cd350 con 0x7fcc180ffaf0 2023-07-21T22:28:56.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.874+0000 7fcc177fe700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc000a11e0 0x7fcc000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:56.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.875+0000 7fcc177fe700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc000a11e0 0x7fcc000a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c000c00 tx=0x7fcc0c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:56.925 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:56.878+0000 7fcc157fa700 1 -- 172.21.15.107:0/2015859464 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcc08096920 con 0x7fcc180ffaf0 2023-07-21T22:28:57.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.065+0000 7fcc1e95d700 1 -- 172.21.15.107:0/2015859464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc1802cb00 con 0x7fcc180ffaf0 2023-07-21T22:28:57.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.066+0000 7fcc157fa700 1 -- 172.21.15.107:0/2015859464 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc0800de60 con 0x7fcc180ffaf0 2023-07-21T22:28:57.113 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2023-07-21T22:28:57.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.068+0000 7fcc1e95d700 1 -- 172.21.15.107:0/2015859464 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc000a11e0 msgr2=0x7fcc000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.068+0000 7fcc1e95d700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc000a11e0 0x7fcc000a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c000c00 tx=0x7fcc0c005fb0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.068+0000 7fcc1e95d700 1 -- 172.21.15.107:0/2015859464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc180ffaf0 msgr2=0x7fcc18195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.068+0000 7fcc1e95d700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc180ffaf0 0x7fcc18195820 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800d980 tx=0x7fcc0800dd40 comp rx=0 tx=0).stop 2023-07-21T22:28:57.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.068+0000 7fcc1e95d700 1 -- 172.21.15.107:0/2015859464 shutdown_connections 2023-07-21T22:28:57.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.068+0000 7fcc1e95d700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc18101670 0x7fcc1819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.068+0000 7fcc1e95d700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc180ffaf0 0x7fcc18195820 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.068+0000 7fcc1e95d700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcc000a11e0 0x7fcc000a3690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.068+0000 7fcc1e95d700 1 --2- 172.21.15.107:0/2015859464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc18100ce0 0x7fcc18195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.068+0000 7fcc1e95d700 1 -- 172.21.15.107:0/2015859464 >> 172.21.15.107:0/2015859464 conn(0x7fcc180fb130 msgr2=0x7fcc181041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:57.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.068+0000 7fcc1e95d700 1 -- 172.21.15.107:0/2015859464 shutdown_connections 2023-07-21T22:28:57.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.068+0000 7fcc1e95d700 1 -- 172.21.15.107:0/2015859464 wait complete. 2023-07-21T22:28:57.127 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:28:57.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.205+0000 7f8e16d19700 1 -- 172.21.15.107:0/1519489489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e100fff60 msgr2=0x7f8e10100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.205+0000 7f8e16d19700 1 --2- 172.21.15.107:0/1519489489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e100fff60 0x7f8e10100370 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009990 tx=0x7f8e04009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.205+0000 7f8e16d19700 1 -- 172.21.15.107:0/1519489489 shutdown_connections 2023-07-21T22:28:57.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.205+0000 7f8e16d19700 1 --2- 172.21.15.107:0/1519489489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100ff620 0x7f8e10106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.205+0000 7f8e16d19700 1 --2- 172.21.15.107:0/1519489489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e100fff60 0x7f8e10100370 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.205+0000 7f8e16d19700 1 --2- 172.21.15.107:0/1519489489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e100fec90 0x7f8e100ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.205+0000 7f8e16d19700 1 -- 172.21.15.107:0/1519489489 >> 172.21.15.107:0/1519489489 conn(0x7f8e100f9770 msgr2=0x7f8e100fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:57.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e16d19700 1 -- 172.21.15.107:0/1519489489 shutdown_connections 2023-07-21T22:28:57.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e16d19700 1 -- 172.21.15.107:0/1519489489 wait complete. 2023-07-21T22:28:57.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e16d19700 1 Processor -- start 2023-07-21T22:28:57.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e16d19700 1 -- start start 2023-07-21T22:28:57.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e16d19700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e100fec90 0x7f8e100fe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:57.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e16d19700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e100ff620 0x7f8e1019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:57.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e16d19700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100fff60 0x7f8e1019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:57.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e16d19700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1019d370 con 0x7f8e100ff620 2023-07-21T22:28:57.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e16d19700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1019d4e0 con 0x7f8e100fec90 2023-07-21T22:28:57.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e16d19700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1019d6a0 con 0x7f8e100fff60 2023-07-21T22:28:57.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e0ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e100ff620 0x7f8e1019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:57.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e14ab5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e100fec90 0x7f8e100fe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:57.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e0ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e100ff620 0x7f8e1019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47638/0 (socket says 172.21.15.107:47638) 2023-07-21T22:28:57.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.206+0000 7f8e0ffff700 1 -- 172.21.15.107:0/803544294 learned_addr learned my addr 172.21.15.107:0/803544294 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:57.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.207+0000 7f8e0ffff700 1 -- 172.21.15.107:0/803544294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100fff60 msgr2=0x7f8e1019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.207+0000 7f8e152b6700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100fff60 0x7f8e1019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:57.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.207+0000 7f8e0ffff700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100fff60 0x7f8e1019cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.207+0000 7f8e0ffff700 1 -- 172.21.15.107:0/803544294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e100fec90 msgr2=0x7f8e100fe6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.207+0000 7f8e0ffff700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e100fec90 0x7f8e100fe6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.207+0000 7f8e0ffff700 1 -- 172.21.15.107:0/803544294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e040095f0 con 0x7f8e100ff620 2023-07-21T22:28:57.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.207+0000 7f8e152b6700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100fff60 0x7f8e1019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:57.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.207+0000 7f8e0ffff700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e100ff620 0x7f8e1019a590 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00d980 tx=0x7f8dfc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:57.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.207+0000 7f8e0dffb700 1 -- 172.21.15.107:0/803544294 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc004220 con 0x7f8e100ff620 2023-07-21T22:28:57.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.207+0000 7f8e0dffb700 1 -- 172.21.15.107:0/803544294 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dfc007640 con 0x7f8e100ff620 2023-07-21T22:28:57.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.207+0000 7f8e16d19700 1 -- 172.21.15.107:0/803544294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1019d980 con 0x7f8e100ff620 2023-07-21T22:28:57.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.207+0000 7f8e0dffb700 1 -- 172.21.15.107:0/803544294 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc005610 con 0x7f8e100ff620 2023-07-21T22:28:57.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.207+0000 7f8e16d19700 1 -- 172.21.15.107:0/803544294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1019de90 con 0x7f8e100ff620 2023-07-21T22:28:57.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.208+0000 7f8e16d19700 1 -- 172.21.15.107:0/803544294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e10105a30 con 0x7f8e100ff620 2023-07-21T22:28:57.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.210+0000 7f8e0dffb700 1 -- 172.21.15.107:0/803544294 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8dfc004750 con 0x7f8e100ff620 2023-07-21T22:28:57.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.210+0000 7f8e0dffb700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8e000a11e0 0x7f8e000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:57.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.211+0000 7f8e0dffb700 1 -- 172.21.15.107:0/803544294 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8dfc0ccf10 con 0x7f8e100ff620 2023-07-21T22:28:57.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.211+0000 7f8e14ab5700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8e000a11e0 0x7f8e000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:57.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.211+0000 7f8e14ab5700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8e000a11e0 0x7f8e000a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8e04006010 tx=0x7f8e04005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:57.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.214+0000 7f8e0dffb700 1 -- 172.21.15.107:0/803544294 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8dfc096480 con 0x7f8e100ff620 2023-07-21T22:28:57.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:57 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3515613357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:57.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:57 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/4244973157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:57.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:57 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2015859464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:28:57.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.404+0000 7f8e16d19700 1 -- 172.21.15.107:0/803544294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8e1002c9e0 con 0x7f8e100ff620 2023-07-21T22:28:57.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.404+0000 7f8e0dffb700 1 -- 172.21.15.107:0/803544294 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8dfc0160a0 con 0x7f8e100ff620 2023-07-21T22:28:57.451 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2023-07-21T22:28:57.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.406+0000 7f8e16d19700 1 -- 172.21.15.107:0/803544294 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8e000a11e0 msgr2=0x7f8e000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.406+0000 7f8e16d19700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8e000a11e0 0x7f8e000a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8e04006010 tx=0x7f8e04005960 comp rx=0 tx=0).stop 2023-07-21T22:28:57.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.406+0000 7f8e16d19700 1 -- 172.21.15.107:0/803544294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e100ff620 msgr2=0x7f8e1019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.406+0000 7f8e16d19700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e100ff620 0x7f8e1019a590 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00d980 tx=0x7f8dfc00dc90 comp rx=0 tx=0).stop 2023-07-21T22:28:57.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.406+0000 7f8e16d19700 1 -- 172.21.15.107:0/803544294 shutdown_connections 2023-07-21T22:28:57.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.406+0000 7f8e16d19700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8e000a11e0 0x7f8e000a3690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.406+0000 7f8e16d19700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100fff60 0x7f8e1019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.406+0000 7f8e16d19700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e100fec90 0x7f8e100fe6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.406+0000 7f8e16d19700 1 --2- 172.21.15.107:0/803544294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e100ff620 0x7f8e1019a590 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.406+0000 7f8e16d19700 1 -- 172.21.15.107:0/803544294 >> 172.21.15.107:0/803544294 conn(0x7f8e100f9770 msgr2=0x7f8e100fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:57.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.406+0000 7f8e16d19700 1 -- 172.21.15.107:0/803544294 shutdown_connections 2023-07-21T22:28:57.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.406+0000 7f8e16d19700 1 -- 172.21.15.107:0/803544294 wait complete. 2023-07-21T22:28:57.465 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:28:57.590 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.543+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1134231219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f4101790 msgr2=0x7ff3f4108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.543+0000 7ff3fa9d2700 1 --2- 172.21.15.107:0/1134231219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f4101790 0x7ff3f4108050 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00b0a0 tx=0x7ff3ec00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.543+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1134231219 shutdown_connections 2023-07-21T22:28:57.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.543+0000 7ff3fa9d2700 1 --2- 172.21.15.107:0/1134231219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f4101790 0x7ff3f4108050 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.543+0000 7ff3fa9d2700 1 --2- 172.21.15.107:0/1134231219 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3f40fdd90 0x7ff3f40fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.543+0000 7ff3fa9d2700 1 --2- 172.21.15.107:0/1134231219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3f4100e00 0x7ff3f4101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.543+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1134231219 >> 172.21.15.107:0/1134231219 conn(0x7ff3f40f9770 msgr2=0x7ff3f40fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:57.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.543+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1134231219 shutdown_connections 2023-07-21T22:28:57.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.543+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1134231219 wait complete. 2023-07-21T22:28:57.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.544+0000 7ff3fa9d2700 1 Processor -- start 2023-07-21T22:28:57.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.544+0000 7ff3fa9d2700 1 -- start start 2023-07-21T22:28:57.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.544+0000 7ff3fa9d2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3f40fdd90 0x7ff3f4100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:57.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.544+0000 7ff3fa9d2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3f4100e00 0x7ff3f40fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:57.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.544+0000 7ff3fa9d2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f4101790 0x7ff3f40ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:57.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.544+0000 7ff3fa9d2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40ffa90 con 0x7ff3f4101790 2023-07-21T22:28:57.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.544+0000 7ff3fa9d2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40ffc00 con 0x7ff3f4100e00 2023-07-21T22:28:57.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.544+0000 7ff3fa9d2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f419f980 con 0x7ff3f40fdd90 2023-07-21T22:28:57.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.544+0000 7ff3f8f6f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f4101790 0x7ff3f40ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:57.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.544+0000 7ff3f8f6f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f4101790 0x7ff3f40ff2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47654/0 (socket says 172.21.15.107:47654) 2023-07-21T22:28:57.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.544+0000 7ff3f8f6f700 1 -- 172.21.15.107:0/1880578055 learned_addr learned my addr 172.21.15.107:0/1880578055 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:57.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.544+0000 7ff3f37fe700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3f4100e00 0x7ff3f40fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:57.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.544+0000 7ff3f3fff700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3f40fdd90 0x7ff3f4100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:57.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.545+0000 7ff3f8f6f700 1 -- 172.21.15.107:0/1880578055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3f40fdd90 msgr2=0x7ff3f4100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.545+0000 7ff3f8f6f700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3f40fdd90 0x7ff3f4100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.545+0000 7ff3f8f6f700 1 -- 172.21.15.107:0/1880578055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3f4100e00 msgr2=0x7ff3f40fedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.545+0000 7ff3f8f6f700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3f4100e00 0x7ff3f40fedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.545+0000 7ff3f8f6f700 1 -- 172.21.15.107:0/1880578055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec009d00 con 0x7ff3f4101790 2023-07-21T22:28:57.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.545+0000 7ff3f3fff700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3f40fdd90 0x7ff3f4100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:57.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.545+0000 7ff3f8f6f700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f4101790 0x7ff3f40ff2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00b780 tx=0x7ff3ec00bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:57.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.545+0000 7ff3f17fa700 1 -- 172.21.15.107:0/1880578055 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec00e040 con 0x7ff3f4101790 2023-07-21T22:28:57.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.545+0000 7ff3f17fa700 1 -- 172.21.15.107:0/1880578055 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3ec004e60 con 0x7ff3f4101790 2023-07-21T22:28:57.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.545+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1880578055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f419fc00 con 0x7ff3f4101790 2023-07-21T22:28:57.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.545+0000 7ff3f17fa700 1 -- 172.21.15.107:0/1880578055 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec01b740 con 0x7ff3f4101790 2023-07-21T22:28:57.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.545+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1880578055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f41a0080 con 0x7ff3f4101790 2023-07-21T22:28:57.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.546+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1880578055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f4107170 con 0x7ff3f4101790 2023-07-21T22:28:57.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.548+0000 7ff3f17fa700 1 -- 172.21.15.107:0/1880578055 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ff3ec01da40 con 0x7ff3f4101790 2023-07-21T22:28:57.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.548+0000 7ff3f17fa700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff3e00a11e0 0x7ff3e00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:57.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.548+0000 7ff3f3fff700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff3e00a11e0 0x7ff3e00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:57.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.548+0000 7ff3f17fa700 1 -- 172.21.15.107:0/1880578055 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff3ec0cec80 con 0x7ff3f4101790 2023-07-21T22:28:57.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.548+0000 7ff3f3fff700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff3e00a11e0 0x7ff3e00a3690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40673d0 tx=0x7ff3e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:57.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.551+0000 7ff3f17fa700 1 -- 172.21.15.107:0/1880578055 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff3ec0982e0 con 0x7ff3f4101790 2023-07-21T22:28:57.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:57 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3515613357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:57.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:57 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/4244973157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:57.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:57 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2015859464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:28:57.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:57 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3515613357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:57.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:57 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/4244973157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:57.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:57 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2015859464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:28:57.786 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.740+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1880578055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3f402ca60 con 0x7ff3f4101790 2023-07-21T22:28:57.787 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.740+0000 7ff3f17fa700 1 -- 172.21.15.107:0/1880578055 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3ec09b900 con 0x7ff3f4101790 2023-07-21T22:28:57.787 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2023-07-21T22:28:57.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.742+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1880578055 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff3e00a11e0 msgr2=0x7ff3e00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.742+0000 7ff3fa9d2700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff3e00a11e0 0x7ff3e00a3690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40673d0 tx=0x7ff3e4009260 comp rx=0 tx=0).stop 2023-07-21T22:28:57.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.742+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1880578055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f4101790 msgr2=0x7ff3f40ff2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.742+0000 7ff3fa9d2700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f4101790 0x7ff3f40ff2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00b780 tx=0x7ff3ec00bd00 comp rx=0 tx=0).stop 2023-07-21T22:28:57.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.742+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1880578055 shutdown_connections 2023-07-21T22:28:57.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.742+0000 7ff3fa9d2700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3f4101790 0x7ff3f40ff2f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.742+0000 7ff3fa9d2700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff3e00a11e0 0x7ff3e00a3690 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.742+0000 7ff3fa9d2700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3f40fdd90 0x7ff3f4100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.742+0000 7ff3fa9d2700 1 --2- 172.21.15.107:0/1880578055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3f4100e00 0x7ff3f40fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.742+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1880578055 >> 172.21.15.107:0/1880578055 conn(0x7ff3f40f9770 msgr2=0x7ff3f41042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:57.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.742+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1880578055 shutdown_connections 2023-07-21T22:28:57.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.742+0000 7ff3fa9d2700 1 -- 172.21.15.107:0/1880578055 wait complete. 2023-07-21T22:28:57.801 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:28:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.875+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1319011020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa40ffaf0 msgr2=0x7fcfa40fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.875+0000 7fcfab6d5700 1 --2- 172.21.15.107:0/1319011020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa40ffaf0 0x7fcfa40fff00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009990 tx=0x7fcfa0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.875+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1319011020 shutdown_connections 2023-07-21T22:28:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.875+0000 7fcfab6d5700 1 --2- 172.21.15.107:0/1319011020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfa4101670 0x7fcfa4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.875+0000 7fcfab6d5700 1 --2- 172.21.15.107:0/1319011020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa40ffaf0 0x7fcfa40fff00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.875+0000 7fcfab6d5700 1 --2- 172.21.15.107:0/1319011020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4100ce0 0x7fcfa4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.875+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1319011020 >> 172.21.15.107:0/1319011020 conn(0x7fcfa40fb130 msgr2=0x7fcfa40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1319011020 shutdown_connections 2023-07-21T22:28:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1319011020 wait complete. 2023-07-21T22:28:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfab6d5700 1 Processor -- start 2023-07-21T22:28:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfab6d5700 1 -- start start 2023-07-21T22:28:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfab6d5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfa40ffaf0 0x7fcfa4195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfab6d5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa4100ce0 0x7fcfa4195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfab6d5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4101670 0x7fcfa419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfab6d5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa419d310 con 0x7fcfa4100ce0 2023-07-21T22:28:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfab6d5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa419d450 con 0x7fcfa40ffaf0 2023-07-21T22:28:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfab6d5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa419d5e0 con 0x7fcfa4101670 2023-07-21T22:28:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfa8c70700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa4100ce0 0x7fcfa4195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfa8c70700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa4100ce0 0x7fcfa4195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47666/0 (socket says 172.21.15.107:47666) 2023-07-21T22:28:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfa8c70700 1 -- 172.21.15.107:0/1962660709 learned_addr learned my addr 172.21.15.107:0/1962660709 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.876+0000 7fcfa9471700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfa40ffaf0 0x7fcfa4195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.877+0000 7fcfa8c70700 1 -- 172.21.15.107:0/1962660709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4101670 msgr2=0x7fcfa419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.877+0000 7fcfa9c72700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4101670 0x7fcfa419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.877+0000 7fcfa8c70700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4101670 0x7fcfa419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.877+0000 7fcfa8c70700 1 -- 172.21.15.107:0/1962660709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfa40ffaf0 msgr2=0x7fcfa4195790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.877+0000 7fcfa8c70700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfa40ffaf0 0x7fcfa4195790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.877+0000 7fcfa8c70700 1 -- 172.21.15.107:0/1962660709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa00095f0 con 0x7fcfa4100ce0 2023-07-21T22:28:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.877+0000 7fcfa9c72700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4101670 0x7fcfa419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.877+0000 7fcfa8c70700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa4100ce0 0x7fcfa4195cd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400c9b0 tx=0x7fcf9400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.877+0000 7fcf9a7fc700 1 -- 172.21.15.107:0/1962660709 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf94007b00 con 0x7fcfa4100ce0 2023-07-21T22:28:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.877+0000 7fcf9a7fc700 1 -- 172.21.15.107:0/1962660709 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf94007c60 con 0x7fcfa4100ce0 2023-07-21T22:28:57.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.877+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1962660709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa419d8c0 con 0x7fcfa4100ce0 2023-07-21T22:28:57.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.877+0000 7fcf9a7fc700 1 -- 172.21.15.107:0/1962660709 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf94018660 con 0x7fcfa4100ce0 2023-07-21T22:28:57.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.877+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1962660709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa4072920 con 0x7fcfa4100ce0 2023-07-21T22:28:57.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.878+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1962660709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa410b310 con 0x7fcfa4100ce0 2023-07-21T22:28:57.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.879+0000 7fcf9a7fc700 1 -- 172.21.15.107:0/1962660709 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fcf9400f450 con 0x7fcfa4100ce0 2023-07-21T22:28:57.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.879+0000 7fcf9a7fc700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcf900a12a0 0x7fcf900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:57.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.880+0000 7fcfa9471700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcf900a12a0 0x7fcf900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:57.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.880+0000 7fcf9a7fc700 1 -- 172.21.15.107:0/1962660709 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcf940ccb10 con 0x7fcfa4100ce0 2023-07-21T22:28:57.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.880+0000 7fcfa9471700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcf900a12a0 0x7fcf900a3750 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000b5c0 tx=0x7fcfa0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:57.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:57.884+0000 7fcf9a7fc700 1 -- 172.21.15.107:0/1962660709 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcf94096080 con 0x7fcfa4100ce0 2023-07-21T22:28:58.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.113+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1962660709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfa4072fc0 con 0x7fcfa4100ce0 2023-07-21T22:28:58.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.114+0000 7fcf9a7fc700 1 -- 172.21.15.107:0/1962660709 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10868 (secure 0 0 0) 0x7fcf940996a0 con 0x7fcfa4100ce0 2023-07-21T22:28:58.161 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:58.161 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:27:49.034257+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:28:58.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.116+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1962660709 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcf900a12a0 msgr2=0x7fcf900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.116+0000 7fcfab6d5700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcf900a12a0 0x7fcf900a3750 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000b5c0 tx=0x7fcfa0005fd0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.116+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1962660709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa4100ce0 msgr2=0x7fcfa4195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.116+0000 7fcfab6d5700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa4100ce0 0x7fcfa4195cd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400c9b0 tx=0x7fcf9400ccc0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.116+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1962660709 shutdown_connections 2023-07-21T22:28:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.116+0000 7fcfab6d5700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcf900a12a0 0x7fcf900a3750 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.116+0000 7fcfab6d5700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfa4101670 0x7fcfa419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.116+0000 7fcfab6d5700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfa40ffaf0 0x7fcfa4195790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.116+0000 7fcfab6d5700 1 --2- 172.21.15.107:0/1962660709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfa4100ce0 0x7fcfa4195cd0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.116+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1962660709 >> 172.21.15.107:0/1962660709 conn(0x7fcfa40fb130 msgr2=0x7fcfa40fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.116+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1962660709 shutdown_connections 2023-07-21T22:28:58.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.116+0000 7fcfab6d5700 1 -- 172.21.15.107:0/1962660709 wait complete. 2023-07-21T22:28:58.165 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2023-07-21T22:28:58.175 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:28:58.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.248+0000 7fae6a695700 1 -- 172.21.15.107:0/840019593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae641057d0 msgr2=0x7fae64107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.248+0000 7fae6a695700 1 --2- 172.21.15.107:0/840019593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae641057d0 0x7fae64107bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00b0a0 tx=0x7fae5c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.248+0000 7fae6a695700 1 -- 172.21.15.107:0/840019593 shutdown_connections 2023-07-21T22:28:58.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.248+0000 7fae6a695700 1 --2- 172.21.15.107:0/840019593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae641057d0 0x7fae64107bf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.248+0000 7fae6a695700 1 --2- 172.21.15.107:0/840019593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae64100590 0x7fae64102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.248+0000 7fae6a695700 1 --2- 172.21.15.107:0/840019593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae64102eb0 0x7fae64105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.248+0000 7fae6a695700 1 -- 172.21.15.107:0/840019593 >> 172.21.15.107:0/840019593 conn(0x7fae640f9fd0 msgr2=0x7fae640fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.248+0000 7fae6a695700 1 -- 172.21.15.107:0/840019593 shutdown_connections 2023-07-21T22:28:58.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.248+0000 7fae6a695700 1 -- 172.21.15.107:0/840019593 wait complete. 2023-07-21T22:28:58.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.249+0000 7fae6a695700 1 Processor -- start 2023-07-21T22:28:58.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.249+0000 7fae6a695700 1 -- start start 2023-07-21T22:28:58.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.249+0000 7fae6a695700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae64100590 0x7fae641958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.249+0000 7fae6a695700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae64102eb0 0x7fae64195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.249+0000 7fae6a695700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae641057d0 0x7fae6419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.249+0000 7fae6a695700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6419d420 con 0x7fae641057d0 2023-07-21T22:28:58.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.249+0000 7fae6a695700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6419d560 con 0x7fae64102eb0 2023-07-21T22:28:58.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.249+0000 7fae6a695700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6419d6f0 con 0x7fae64100590 2023-07-21T22:28:58.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.249+0000 7fae68c32700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae641057d0 0x7fae6419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.249+0000 7fae68c32700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae641057d0 0x7fae6419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47682/0 (socket says 172.21.15.107:47682) 2023-07-21T22:28:58.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.249+0000 7fae68c32700 1 -- 172.21.15.107:0/3094264248 learned_addr learned my addr 172.21.15.107:0/3094264248 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.249+0000 7fae637fe700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae64102eb0 0x7fae64195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.250+0000 7fae63fff700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae64100590 0x7fae641958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.250+0000 7fae68c32700 1 -- 172.21.15.107:0/3094264248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae64100590 msgr2=0x7fae641958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.250+0000 7fae68c32700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae64100590 0x7fae641958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.250+0000 7fae68c32700 1 -- 172.21.15.107:0/3094264248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae64102eb0 msgr2=0x7fae64195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.250+0000 7fae68c32700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae64102eb0 0x7fae64195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.250+0000 7fae68c32700 1 -- 172.21.15.107:0/3094264248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c009d00 con 0x7fae641057d0 2023-07-21T22:28:58.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.250+0000 7fae68c32700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae641057d0 0x7fae6419ce50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009fd0 tx=0x7fae5c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.250+0000 7fae617fa700 1 -- 172.21.15.107:0/3094264248 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c00e040 con 0x7fae641057d0 2023-07-21T22:28:58.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.250+0000 7fae6a695700 1 -- 172.21.15.107:0/3094264248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6419d970 con 0x7fae641057d0 2023-07-21T22:28:58.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.250+0000 7fae617fa700 1 -- 172.21.15.107:0/3094264248 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5c003e10 con 0x7fae641057d0 2023-07-21T22:28:58.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.250+0000 7fae617fa700 1 -- 172.21.15.107:0/3094264248 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c01b6a0 con 0x7fae641057d0 2023-07-21T22:28:58.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.250+0000 7fae6a695700 1 -- 172.21.15.107:0/3094264248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6419dd40 con 0x7fae641057d0 2023-07-21T22:28:58.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.251+0000 7fae6a695700 1 -- 172.21.15.107:0/3094264248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae640fbb40 con 0x7fae641057d0 2023-07-21T22:28:58.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.252+0000 7fae617fa700 1 -- 172.21.15.107:0/3094264248 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fae5c01d470 con 0x7fae641057d0 2023-07-21T22:28:58.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.253+0000 7fae617fa700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fae4c0a1220 0x7fae4c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.253+0000 7fae63fff700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fae4c0a1220 0x7fae4c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.253+0000 7fae617fa700 1 -- 172.21.15.107:0/3094264248 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fae5c0cde70 con 0x7fae641057d0 2023-07-21T22:28:58.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.253+0000 7fae63fff700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fae4c0a1220 0x7fae4c0a36d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae54005950 tx=0x7fae5400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.257+0000 7fae617fa700 1 -- 172.21.15.107:0/3094264248 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fae5c0974d0 con 0x7fae641057d0 2023-07-21T22:28:58.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:58 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/803544294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:28:58.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:58 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1880578055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:28:58.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:58 smithi107 ceph-mon[96543]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s; 93 B/s, 0 objects/s recovering 2023-07-21T22:28:58.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:58 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1962660709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:58.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.448+0000 7fae6a695700 1 -- 172.21.15.107:0/3094264248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae6402caa0 con 0x7fae641057d0 2023-07-21T22:28:58.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.450+0000 7fae617fa700 1 -- 172.21.15.107:0/3094264248 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22727 (secure 0 0 0) 0x7fae5c09aaf0 con 0x7fae641057d0 2023-07-21T22:28:58.497 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:28:58.497 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":84,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:28:47.235779+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:28:58.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.452+0000 7fae6a695700 1 -- 172.21.15.107:0/3094264248 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fae4c0a1220 msgr2=0x7fae4c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.452+0000 7fae6a695700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fae4c0a1220 0x7fae4c0a36d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae54005950 tx=0x7fae5400b410 comp rx=0 tx=0).stop 2023-07-21T22:28:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.452+0000 7fae6a695700 1 -- 172.21.15.107:0/3094264248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae641057d0 msgr2=0x7fae6419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.452+0000 7fae6a695700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae641057d0 0x7fae6419ce50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009fd0 tx=0x7fae5c00bce0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.452+0000 7fae6a695700 1 -- 172.21.15.107:0/3094264248 shutdown_connections 2023-07-21T22:28:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.452+0000 7fae6a695700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae641057d0 0x7fae6419ce50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.452+0000 7fae6a695700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fae4c0a1220 0x7fae4c0a36d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.452+0000 7fae6a695700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae64100590 0x7fae641958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.452+0000 7fae6a695700 1 --2- 172.21.15.107:0/3094264248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae64102eb0 0x7fae64195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.452+0000 7fae6a695700 1 -- 172.21.15.107:0/3094264248 >> 172.21.15.107:0/3094264248 conn(0x7fae640f9fd0 msgr2=0x7fae641011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.452+0000 7fae6a695700 1 -- 172.21.15.107:0/3094264248 shutdown_connections 2023-07-21T22:28:58.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.452+0000 7fae6a695700 1 -- 172.21.15.107:0/3094264248 wait complete. 2023-07-21T22:28:58.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-21T22:27:41.478391+0000', 'modified': '2023-07-21T22:27:49.034257+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24533, 'mds_1': 24493, 'mds_2': 24466, 'mds_3': 24455, 'mds_4': 24437}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14646': {'gid': 14646, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6839/1956222571', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 1956222571}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 1956222571}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14658': {'gid': 14658, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6841/1878776680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6840', 'nonce': 1878776680}, {'type': 'v1', 'addr': '172.21.15.107:6841', 'nonce': 1878776680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6833/253562772', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 253562772}, {'type': 'v1', 'addr': '172.21.15.190:6833', 'nonce': 253562772}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24437': {'gid': 24437, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.132:6833/3293513462', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6832', 'nonce': 3293513462}, {'type': 'v1', 'addr': '172.21.15.132:6833', 'nonce': 3293513462}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.132:6835/1022724305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6834', 'nonce': 1022724305}, {'type': 'v1', 'addr': '172.21.15.132:6835', 'nonce': 1022724305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.190:6835/2286851849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 2286851849}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 2286851849}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.190:6837/3774629948', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6836', 'nonce': 3774629948}, {'type': 'v1', 'addr': '172.21.15.190:6837', 'nonce': 3774629948}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6837/1492770170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6836', 'nonce': 1492770170}, {'type': 'v1', 'addr': '172.21.15.132:6837', 'nonce': 1492770170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.190:6839/2960500982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6838', 'nonce': 2960500982}, {'type': 'v1', 'addr': '172.21.15.190:6839', 'nonce': 2960500982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24533': {'gid': 24533, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6839/523672176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6838', 'nonce': 523672176}, {'type': 'v1', 'addr': '172.21.15.132:6839', 'nonce': 523672176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-21T22:28:58.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-07-21T22:28:58.512 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-07-21T22:28:58.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.586+0000 7f0d4c063700 1 -- 172.21.15.107:0/185080954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d440fea50 msgr2=0x7f0d440fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.586+0000 7f0d4c063700 1 --2- 172.21.15.107:0/185080954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d440fea50 0x7f0d440fee60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009990 tx=0x7f0d40009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.586+0000 7f0d4c063700 1 -- 172.21.15.107:0/185080954 shutdown_connections 2023-07-21T22:28:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.586+0000 7f0d4c063700 1 --2- 172.21.15.107:0/185080954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d441005d0 0x7f0d44106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.586+0000 7f0d4c063700 1 --2- 172.21.15.107:0/185080954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d440fea50 0x7f0d440fee60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.586+0000 7f0d4c063700 1 --2- 172.21.15.107:0/185080954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d440ffc40 0x7f0d44100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.586+0000 7f0d4c063700 1 -- 172.21.15.107:0/185080954 >> 172.21.15.107:0/185080954 conn(0x7f0d440fa070 msgr2=0x7f0d440fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.587+0000 7f0d4c063700 1 -- 172.21.15.107:0/185080954 shutdown_connections 2023-07-21T22:28:58.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.587+0000 7f0d4c063700 1 -- 172.21.15.107:0/185080954 wait complete. 2023-07-21T22:28:58.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.587+0000 7f0d4c063700 1 Processor -- start 2023-07-21T22:28:58.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.587+0000 7f0d4c063700 1 -- start start 2023-07-21T22:28:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.587+0000 7f0d4c063700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d440fea50 0x7f0d44195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.587+0000 7f0d4c063700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d440ffc40 0x7f0d44195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.587+0000 7f0d4c063700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d441005d0 0x7f0d4419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.587+0000 7f0d4c063700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4419d3f0 con 0x7f0d441005d0 2023-07-21T22:28:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.587+0000 7f0d4c063700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4419d530 con 0x7f0d440fea50 2023-07-21T22:28:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.587+0000 7f0d4c063700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4419d6c0 con 0x7f0d440ffc40 2023-07-21T22:28:58.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.587+0000 7f0d4a600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d441005d0 0x7f0d4419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.587+0000 7f0d4a600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d441005d0 0x7f0d4419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47696/0 (socket says 172.21.15.107:47696) 2023-07-21T22:28:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d4a600700 1 -- 172.21.15.107:0/217556240 learned_addr learned my addr 172.21.15.107:0/217556240 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d49dff700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d440fea50 0x7f0d44195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d495fe700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d440ffc40 0x7f0d44195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d4a600700 1 -- 172.21.15.107:0/217556240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d440ffc40 msgr2=0x7f0d44195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d4a600700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d440ffc40 0x7f0d44195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d4a600700 1 -- 172.21.15.107:0/217556240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d440fea50 msgr2=0x7f0d44195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d4a600700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d440fea50 0x7f0d44195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d4a600700 1 -- 172.21.15.107:0/217556240 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d400095f0 con 0x7f0d441005d0 2023-07-21T22:28:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d4a600700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d441005d0 0x7f0d4419ce20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00ef30 tx=0x7f0d3c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d3affd700 1 -- 172.21.15.107:0/217556240 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c018070 con 0x7f0d441005d0 2023-07-21T22:28:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d3affd700 1 -- 172.21.15.107:0/217556240 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d3c00f040 con 0x7f0d441005d0 2023-07-21T22:28:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d4c063700 1 -- 172.21.15.107:0/217556240 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4419d940 con 0x7f0d441005d0 2023-07-21T22:28:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d3affd700 1 -- 172.21.15.107:0/217556240 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c00cdb0 con 0x7f0d441005d0 2023-07-21T22:28:58.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.588+0000 7f0d4c063700 1 -- 172.21.15.107:0/217556240 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4419de50 con 0x7f0d441005d0 2023-07-21T22:28:58.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.590+0000 7f0d3affd700 1 -- 172.21.15.107:0/217556240 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f0d3c003a00 con 0x7f0d441005d0 2023-07-21T22:28:58.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.591+0000 7f0d3affd700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0d300a1260 0x7f0d300a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.591+0000 7f0d3affd700 1 -- 172.21.15.107:0/217556240 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d3c0cc6d0 con 0x7f0d441005d0 2023-07-21T22:28:58.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.591+0000 7f0d49dff700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0d300a1260 0x7f0d300a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.591+0000 7f0d49dff700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0d300a1260 0x7f0d300a3710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b2e0 tx=0x7f0d40005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d581e8b80 1 Processor -- start 2023-07-21T22:28:58.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d581e8b80 1 -- start start 2023-07-21T22:28:58.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d581e8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68567e0 0x55a2f6856bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d581e8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f6857150 0x55a2f68553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d581e8b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f6855e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d581e8b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f675ac60 con 0x55a2f68559c0 2023-07-21T22:28:58.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d581e8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f67ad9f0 con 0x55a2f6857150 2023-07-21T22:28:58.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d581e8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f67a6e80 con 0x55a2f68567e0 2023-07-21T22:28:58.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d4a600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f6855e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d4a600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f6855e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47704/0 (socket says 172.21.15.107:47704) 2023-07-21T22:28:58.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d4a600700 1 -- 172.21.15.107:0/2072747435 learned_addr learned my addr 172.21.15.107:0/2072747435 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d49dff700 1 --2- 172.21.15.107:0/2072747435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68567e0 0x55a2f6856bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d495fe700 1 --2- 172.21.15.107:0/2072747435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f6857150 0x55a2f68553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d4a600700 1 -- 172.21.15.107:0/2072747435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68567e0 msgr2=0x55a2f6856bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d4a600700 1 --2- 172.21.15.107:0/2072747435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68567e0 0x55a2f6856bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d4a600700 1 -- 172.21.15.107:0/2072747435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f6857150 msgr2=0x55a2f68553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d4a600700 1 --2- 172.21.15.107:0/2072747435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f6857150 0x55a2f68553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d4a600700 1 -- 172.21.15.107:0/2072747435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2f65d4d90 con 0x55a2f68559c0 2023-07-21T22:28:58.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.592+0000 7f0d4a600700 1 --2- 172.21.15.107:0/2072747435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f6855e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0090a0 tx=0x7f0d3c0994d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efda22ef74f26fe5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.593+0000 7f0d2ffff700 1 -- 172.21.15.107:0/2072747435 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c099990 con 0x55a2f68559c0 2023-07-21T22:28:58.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.593+0000 7f0d2ffff700 1 -- 172.21.15.107:0/2072747435 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d3c09d9c0 con 0x55a2f68559c0 2023-07-21T22:28:58.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.593+0000 7f0d2ffff700 1 -- 172.21.15.107:0/2072747435 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c099e50 con 0x55a2f68559c0 2023-07-21T22:28:58.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.593+0000 7f0d581e8b80 1 -- 172.21.15.107:0/2072747435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 msgr2=0x55a2f6855e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.593+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/2072747435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f6855e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0090a0 tx=0x7f0d3c0994d0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.593+0000 7f0d581e8b80 1 -- 172.21.15.107:0/2072747435 shutdown_connections 2023-07-21T22:28:58.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.593+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/2072747435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f6855e40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.593+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/2072747435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68567e0 0x55a2f6856bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.593+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/2072747435 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f6857150 0x55a2f68553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.593+0000 7f0d581e8b80 1 -- 172.21.15.107:0/2072747435 >> 172.21.15.107:0/2072747435 conn(0x55a2f684aea0 msgr2=0x55a2f6850310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.593+0000 7f0d581e8b80 1 -- 172.21.15.107:0/2072747435 shutdown_connections 2023-07-21T22:28:58.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.593+0000 7f0d581e8b80 1 -- 172.21.15.107:0/2072747435 wait complete. 2023-07-21T22:28:58.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d581e8b80 1 -- start start 2023-07-21T22:28:58.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d581e8b80 1 Processor -- start 2023-07-21T22:28:58.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d581e8b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f685f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d581e8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68567e0 0x55a2f685f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d581e8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f6857150 0x55a2f68666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d581e8b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f6746a70 con 0x55a2f68559c0 2023-07-21T22:28:58.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d581e8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f6746770 con 0x55a2f6857150 2023-07-21T22:28:58.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d581e8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f662a8d0 con 0x55a2f68567e0 2023-07-21T22:28:58.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d581e8b80 10 client.0 _ll_register_callbacks cb 0x55a2f6744710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:58.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d49dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f685f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d49dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f685f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47706/0 (socket says 172.21.15.107:47706) 2023-07-21T22:28:58.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d49dff700 1 -- 172.21.15.107:0/429084580 learned_addr learned my addr 172.21.15.107:0/429084580 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d4a600700 1 --2- 172.21.15.107:0/429084580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f6857150 0x55a2f68666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.594+0000 7f0d495fe700 1 --2- 172.21.15.107:0/429084580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68567e0 0x55a2f685f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d49dff700 1 -- 172.21.15.107:0/429084580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68567e0 msgr2=0x55a2f685f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d49dff700 1 --2- 172.21.15.107:0/429084580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68567e0 0x55a2f685f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d49dff700 1 -- 172.21.15.107:0/429084580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f6857150 msgr2=0x55a2f68666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d49dff700 1 --2- 172.21.15.107:0/429084580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f6857150 0x55a2f68666e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d49dff700 1 -- 172.21.15.107:0/429084580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4419de50 con 0x55a2f68559c0 2023-07-21T22:28:58.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d49dff700 1 --2- 172.21.15.107:0/429084580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f685f160 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0d440ffaa0 tx=0x7f0d40009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0cb233b0df3e27 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d2d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:58.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d2d7fa700 1 -- 172.21.15.107:0/429084580 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4000fe20 con 0x55a2f68559c0 2023-07-21T22:28:58.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d2d7fa700 1 -- 172.21.15.107:0/429084580 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d40024620 con 0x55a2f68559c0 2023-07-21T22:28:58.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d581e8b80 1 -- 172.21.15.107:0/429084580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2f67004a0 con 0x55a2f68559c0 2023-07-21T22:28:58.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d2d7fa700 1 -- 172.21.15.107:0/429084580 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d400247e0 con 0x55a2f68559c0 2023-07-21T22:28:58.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d581e8b80 10 client.15270 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:58.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d581e8b80 1 -- 172.21.15.107:0/429084580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a2f65bdc10 con 0x55a2f68559c0 2023-07-21T22:28:58.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d2d7fa700 1 -- 172.21.15.107:0/429084580 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d4003f020 con 0x55a2f68559c0 2023-07-21T22:28:58.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d2d7fa700 1 -- 172.21.15.107:0/429084580 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0d4002f070 con 0x55a2f68559c0 2023-07-21T22:28:58.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d581e8b80 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:58.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d581e8b80 10 client.15270 resolve_mds: resolved name 'k' to daemon mds.k 2023-07-21T22:28:58.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d581e8b80 20 client.15270 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:58.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/429084580 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55a2f686a430 0x55a2f68697e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d581e8b80 4 client.15270 mds_command: new command op to 24437 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:58.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.595+0000 7f0d581e8b80 1 -- 172.21.15.107:0/429084580 --> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2f679fe60 con 0x55a2f686a430 2023-07-21T22:28:58.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.596+0000 7f0d495fe700 1 --2- 172.21.15.107:0/429084580 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55a2f686a430 0x55a2f68697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.596+0000 7f0d495fe700 1 --2- 172.21.15.107:0/429084580 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55a2f686a430 0x55a2f68697e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac989704f965f4ad server_cookie=d5627ed315620d5c in_seq=0 out_seq=0 2023-07-21T22:28:58.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.596+0000 7f0d2d7fa700 10 client.15270 ms_handle_connect on v2:172.21.15.132:6832/3293513462 2023-07-21T22:28:58.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d2d7fa700 1 -- 172.21.15.107:0/429084580 <== mds.4 v2:172.21.15.132:6832/3293513462 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a2f679fe60 con 0x55a2f686a430 2023-07-21T22:28:58.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d2d7fa700 10 client.15270 handle_command_reply: tid=0 2023-07-21T22:28:58.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d581e8b80 1 client.15270 shutdown 2023-07-21T22:28:58.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d581e8b80 1 -- 172.21.15.107:0/429084580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 msgr2=0x55a2f685f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/429084580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f685f160 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0d440ffaa0 tx=0x7f0d40009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d581e8b80 1 -- 172.21.15.107:0/429084580 shutdown_connections 2023-07-21T22:28:58.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/429084580 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55a2f686a430 0x55a2f68697e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/429084580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f6857150 0x55a2f68666e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/429084580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f685f160 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/429084580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68567e0 0x55a2f685f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d2d7fa700 0 client.15270 ms_handle_reset on v2:172.21.15.132:6832/3293513462 2023-07-21T22:28:58.659 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d581e8b80 1 -- 172.21.15.107:0/429084580 >> 172.21.15.107:0/429084580 conn(0x55a2f684aea0 msgr2=0x55a2f684f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.659 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d581e8b80 1 -- 172.21.15.107:0/429084580 shutdown_connections 2023-07-21T22:28:58.659 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d581e8b80 1 -- 172.21.15.107:0/429084580 wait complete. 2023-07-21T22:28:58.659 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.597+0000 7f0d581e8b80 20 client.15270 trim_cache size 0 max 16384 2023-07-21T22:28:58.659 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:58 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/803544294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:28:58.660 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:58 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1880578055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:28:58.660 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:58 smithi190 ceph-mon[98334]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s; 93 B/s, 0 objects/s recovering 2023-07-21T22:28:58.660 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:58 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1962660709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:58.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.614+0000 7f0d581e8b80 1 Processor -- start 2023-07-21T22:28:58.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.614+0000 7f0d581e8b80 1 -- start start 2023-07-21T22:28:58.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d581e8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f684aea0 0x55a2f684ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d581e8b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f685dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d581e8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f68567e0 0x55a2f685e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d581e8b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f6775790 con 0x55a2f68559c0 2023-07-21T22:28:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d581e8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f6636de0 con 0x55a2f68567e0 2023-07-21T22:28:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d581e8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f6742d50 con 0x55a2f684aea0 2023-07-21T22:28:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d495fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f685dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d495fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f685dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47716/0 (socket says 172.21.15.107:47716) 2023-07-21T22:28:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d495fe700 1 -- 172.21.15.107:0/1211642904 learned_addr learned my addr 172.21.15.107:0/1211642904 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d49dff700 1 --2- 172.21.15.107:0/1211642904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f684aea0 0x55a2f684ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d4a600700 1 --2- 172.21.15.107:0/1211642904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f68567e0 0x55a2f685e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d495fe700 1 -- 172.21.15.107:0/1211642904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f684aea0 msgr2=0x55a2f684ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d495fe700 1 --2- 172.21.15.107:0/1211642904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f684aea0 0x55a2f684ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d495fe700 1 -- 172.21.15.107:0/1211642904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f68567e0 msgr2=0x55a2f685e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d495fe700 1 --2- 172.21.15.107:0/1211642904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f68567e0 0x55a2f685e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d495fe700 1 -- 172.21.15.107:0/1211642904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2f65fb200 con 0x55a2f68559c0 2023-07-21T22:28:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d495fe700 1 --2- 172.21.15.107:0/1211642904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f685dd20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400a810 tx=0x7f0d3400a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e58976d9b197094 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d38ff9700 1 -- 172.21.15.107:0/1211642904 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34017070 con 0x55a2f68559c0 2023-07-21T22:28:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d38ff9700 1 -- 172.21.15.107:0/1211642904 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d34018040 con 0x55a2f68559c0 2023-07-21T22:28:58.665 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.615+0000 7f0d38ff9700 1 -- 172.21.15.107:0/1211642904 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34019ac0 con 0x55a2f68559c0 2023-07-21T22:28:58.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.616+0000 7f0d581e8b80 1 -- 172.21.15.107:0/1211642904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 msgr2=0x55a2f685dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.616+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/1211642904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f685dd20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400a810 tx=0x7f0d3400a440 comp rx=0 tx=0).stop 2023-07-21T22:28:58.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.616+0000 7f0d581e8b80 1 -- 172.21.15.107:0/1211642904 shutdown_connections 2023-07-21T22:28:58.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.616+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/1211642904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f68567e0 0x55a2f685e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.616+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/1211642904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f684aea0 0x55a2f684ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.616+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/1211642904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68559c0 0x55a2f685dd20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.616+0000 7f0d581e8b80 1 -- 172.21.15.107:0/1211642904 >> 172.21.15.107:0/1211642904 conn(0x55a2f68555f0 msgr2=0x55a2f6873180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.616+0000 7f0d581e8b80 1 -- 172.21.15.107:0/1211642904 shutdown_connections 2023-07-21T22:28:58.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.616+0000 7f0d581e8b80 1 -- 172.21.15.107:0/1211642904 wait complete. 2023-07-21T22:28:58.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d581e8b80 1 -- start start 2023-07-21T22:28:58.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d581e8b80 1 Processor -- start 2023-07-21T22:28:58.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d581e8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f684aea0 0x55a2f6861f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d581e8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68559c0 0x55a2f6862450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d581e8b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68567e0 0x55a2f6865470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d581e8b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f68659b0 con 0x55a2f68567e0 2023-07-21T22:28:58.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d581e8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f6865b20 con 0x55a2f684aea0 2023-07-21T22:28:58.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d581e8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f6865ce0 con 0x55a2f68559c0 2023-07-21T22:28:58.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d581e8b80 10 client.0 _ll_register_callbacks cb 0x7f0d4000bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:58.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d4a600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68567e0 0x55a2f6865470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d4a600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68567e0 0x55a2f6865470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47724/0 (socket says 172.21.15.107:47724) 2023-07-21T22:28:58.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d4a600700 1 -- 172.21.15.107:0/609394031 learned_addr learned my addr 172.21.15.107:0/609394031 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d495fe700 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68559c0 0x55a2f6862450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d49dff700 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f684aea0 0x55a2f6861f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d4a600700 1 -- 172.21.15.107:0/609394031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68559c0 msgr2=0x55a2f6862450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d4a600700 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68559c0 0x55a2f6862450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d4a600700 1 -- 172.21.15.107:0/609394031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f684aea0 msgr2=0x55a2f6861f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.617+0000 7f0d4a600700 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f684aea0 0x55a2f6861f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d4a600700 1 -- 172.21.15.107:0/609394031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3400a960 con 0x55a2f68567e0 2023-07-21T22:28:58.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d495fe700 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68559c0 0x55a2f6862450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:58.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d4a600700 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68567e0 0x55a2f6865470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c099690 tx=0x7f0d3c095cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ff6145a29b725c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d2d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:58.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d2d7fa700 1 -- 172.21.15.107:0/609394031 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c0c5020 con 0x55a2f68567e0 2023-07-21T22:28:58.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d2d7fa700 1 -- 172.21.15.107:0/609394031 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d3c099990 con 0x55a2f68567e0 2023-07-21T22:28:58.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d581e8b80 1 -- 172.21.15.107:0/609394031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d3c09d9c0 con 0x55a2f68567e0 2023-07-21T22:28:58.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d2d7fa700 1 -- 172.21.15.107:0/609394031 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d3c0a1d50 con 0x55a2f68567e0 2023-07-21T22:28:58.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d581e8b80 10 client.15276 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d581e8b80 1 -- 172.21.15.107:0/609394031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a2f65d3260 con 0x55a2f68567e0 2023-07-21T22:28:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d2d7fa700 1 -- 172.21.15.107:0/609394031 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d3c09d9c0 con 0x55a2f68567e0 2023-07-21T22:28:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d2d7fa700 1 -- 172.21.15.107:0/609394031 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0d3c099d10 con 0x55a2f68567e0 2023-07-21T22:28:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d581e8b80 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d581e8b80 10 client.15276 resolve_mds: resolved name 'k' to daemon mds.k 2023-07-21T22:28:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d581e8b80 20 client.15276 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55a2f6857150 0x55a2f68c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d581e8b80 4 client.15276 mds_command: new command op to 24437 tid=0[{"prefix": "status"}] 2023-07-21T22:28:58.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d581e8b80 1 -- 172.21.15.107:0/609394031 --> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a2f6762000 con 0x55a2f6857150 2023-07-21T22:28:58.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.618+0000 7f0d49dff700 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55a2f6857150 0x55a2f68c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d2d7fa700 10 client.15276 ms_handle_connect on v2:172.21.15.132:6832/3293513462 2023-07-21T22:28:58.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d49dff700 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55a2f6857150 0x55a2f68c05f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fbba181bfe252070 server_cookie=5b5ab72fd3f7e82e in_seq=0 out_seq=0 2023-07-21T22:28:58.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d2d7fa700 1 -- 172.21.15.107:0/609394031 <== mds.4 v2:172.21.15.132:6832/3293513462 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a2f6762000 con 0x55a2f6857150 2023-07-21T22:28:58.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d2d7fa700 10 client.15276 handle_command_reply: tid=0 2023-07-21T22:28:58.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d581e8b80 1 client.15276 shutdown 2023-07-21T22:28:58.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d581e8b80 1 -- 172.21.15.107:0/609394031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68567e0 msgr2=0x55a2f6865470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68567e0 0x55a2f6865470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c099690 tx=0x7f0d3c095cd0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d581e8b80 1 -- 172.21.15.107:0/609394031 shutdown_connections 2023-07-21T22:28:58.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2f68567e0 0x55a2f6865470 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x55a2f6857150 0x55a2f68c05f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a2f684aea0 0x55a2f6861f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d2d7fa700 0 client.15276 ms_handle_reset on v2:172.21.15.132:6832/3293513462 2023-07-21T22:28:58.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d581e8b80 1 --2- 172.21.15.107:0/609394031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a2f68559c0 0x55a2f6862450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d581e8b80 1 -- 172.21.15.107:0/609394031 >> 172.21.15.107:0/609394031 conn(0x55a2f68555f0 msgr2=0x55a2f684ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d581e8b80 1 -- 172.21.15.107:0/609394031 shutdown_connections 2023-07-21T22:28:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d581e8b80 1 -- 172.21.15.107:0/609394031 wait complete. 2023-07-21T22:28:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.619+0000 7f0d581e8b80 20 client.15276 trim_cache size 0 max 16384 2023-07-21T22:28:58.676 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:28:58.677 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "af99f5d0-2814-11ee-9b34-001a4aab830c", 2023-07-21T22:28:58.677 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 4, 2023-07-21T22:28:58.677 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24437, 2023-07-21T22:28:58.677 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2023-07-21T22:28:58.677 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2023-07-21T22:28:58.677 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2023-07-21T22:28:58.677 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 70.588626059999996, 2023-07-21T22:28:58.678 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2023-07-21T22:28:58.678 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 84, 2023-07-21T22:28:58.678 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:28:58.678 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 104.62889618299999 2023-07-21T22:28:58.678 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:28:58.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.620+0000 7f0d4c063700 1 -- 172.21.15.107:0/217556240 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0d300a1260 msgr2=0x7f0d300a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.620+0000 7f0d4c063700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0d300a1260 0x7f0d300a3710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b2e0 tx=0x7f0d40005fb0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.620+0000 7f0d4c063700 1 -- 172.21.15.107:0/217556240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d441005d0 msgr2=0x7f0d4419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.620+0000 7f0d4c063700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d441005d0 0x7f0d4419ce20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00ef30 tx=0x7f0d3c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.620+0000 7f0d4c063700 1 -- 172.21.15.107:0/217556240 shutdown_connections 2023-07-21T22:28:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.620+0000 7f0d4c063700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d441005d0 0x7f0d4419ce20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.620+0000 7f0d4c063700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d440fea50 0x7f0d44195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.620+0000 7f0d4c063700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0d300a1260 0x7f0d300a3710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.620+0000 7f0d4c063700 1 --2- 172.21.15.107:0/217556240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d440ffc40 0x7f0d44195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.620+0000 7f0d4c063700 1 -- 172.21.15.107:0/217556240 >> 172.21.15.107:0/217556240 conn(0x7f0d440fa070 msgr2=0x7f0d440fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.620+0000 7f0d4c063700 1 -- 172.21.15.107:0/217556240 shutdown_connections 2023-07-21T22:28:58.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.620+0000 7f0d4c063700 1 -- 172.21.15.107:0/217556240 wait complete. 2023-07-21T22:28:58.681 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-07-21T22:28:58.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:58 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/803544294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:28:58.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:58 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1880578055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:28:58.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:58 smithi132 ceph-mon[97863]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s; 93 B/s, 0 objects/s recovering 2023-07-21T22:28:58.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:58 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1962660709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:28:58.805 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 -- 172.21.15.107:0/3810936894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a41005d0 msgr2=0x7fa6a4106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.805 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 --2- 172.21.15.107:0/3810936894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a41005d0 0x7fa6a4106e90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00b0a0 tx=0x7fa69c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.806 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 -- 172.21.15.107:0/3810936894 shutdown_connections 2023-07-21T22:28:58.806 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 --2- 172.21.15.107:0/3810936894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a41005d0 0x7fa6a4106e90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.806 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 --2- 172.21.15.107:0/3810936894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6a40fea50 0x7fa6a40fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.806 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 --2- 172.21.15.107:0/3810936894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a40ffc40 0x7fa6a4100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 -- 172.21.15.107:0/3810936894 >> 172.21.15.107:0/3810936894 conn(0x7fa6a40fa070 msgr2=0x7fa6a40fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 -- 172.21.15.107:0/3810936894 shutdown_connections 2023-07-21T22:28:58.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 -- 172.21.15.107:0/3810936894 wait complete. 2023-07-21T22:28:58.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 Processor -- start 2023-07-21T22:28:58.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 -- start start 2023-07-21T22:28:58.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a40fea50 0x7fa6a4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a40ffc40 0x7fa6a41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6a41005d0 0x7fa6a419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a419f6e0 con 0x7fa6a40fea50 2023-07-21T22:28:58.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a419f820 con 0x7fa6a40ffc40 2023-07-21T22:28:58.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.757+0000 7fa6ab1a8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a419f9b0 con 0x7fa6a41005d0 2023-07-21T22:28:58.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a8f44700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a40fea50 0x7fa6a4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a8f44700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a40fea50 0x7fa6a4197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47734/0 (socket says 172.21.15.107:47734) 2023-07-21T22:28:58.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a8f44700 1 -- 172.21.15.107:0/1312040818 learned_addr learned my addr 172.21.15.107:0/1312040818 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a40ffc40 0x7fa6a41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a9745700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6a41005d0 0x7fa6a419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a8f44700 1 -- 172.21.15.107:0/1312040818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6a41005d0 msgr2=0x7fa6a419f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a8f44700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6a41005d0 0x7fa6a419f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a8f44700 1 -- 172.21.15.107:0/1312040818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a40ffc40 msgr2=0x7fa6a41980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a8f44700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a40ffc40 0x7fa6a41980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a8f44700 1 -- 172.21.15.107:0/1312040818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c009d00 con 0x7fa6a40fea50 2023-07-21T22:28:58.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a40ffc40 0x7fa6a41980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:58.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a8f44700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a40fea50 0x7fa6a4197b60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa69800d980 tx=0x7fa69800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a1ffb700 1 -- 172.21.15.107:0/1312040818 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69800de60 con 0x7fa6a40fea50 2023-07-21T22:28:58.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a1ffb700 1 -- 172.21.15.107:0/1312040818 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6980044e0 con 0x7fa6a40fea50 2023-07-21T22:28:58.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6ab1a8700 1 -- 172.21.15.107:0/1312040818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a419fc90 con 0x7fa6a40fea50 2023-07-21T22:28:58.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6a1ffb700 1 -- 172.21.15.107:0/1312040818 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698003be0 con 0x7fa6a40fea50 2023-07-21T22:28:58.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.758+0000 7fa6ab1a8700 1 -- 172.21.15.107:0/1312040818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a41a01a0 con 0x7fa6a40fea50 2023-07-21T22:28:58.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.761+0000 7fa6a1ffb700 1 -- 172.21.15.107:0/1312040818 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fa698009bb0 con 0x7fa6a40fea50 2023-07-21T22:28:58.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.761+0000 7fa6a1ffb700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa6940a11e0 0x7fa6940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.762+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa6940a11e0 0x7fa6940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.762+0000 7fa6a1ffb700 1 -- 172.21.15.107:0/1312040818 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa6980cc8d0 con 0x7fa6a40fea50 2023-07-21T22:28:58.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.762+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa6940a11e0 0x7fa6940a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa690005fd0 tx=0x7fa690005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.762+0000 7fa6b732db80 1 Processor -- start 2023-07-21T22:28:58.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.762+0000 7fa6b732db80 1 -- start start 2023-07-21T22:28:58.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.762+0000 7fa6b732db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc357e0 0x5630ecc35bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.762+0000 7fa6b732db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 0x5630ecc343e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.762+0000 7fa6b732db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc349c0 0x5630ecc34e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.762+0000 7fa6b732db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ecb39c60 con 0x5630ecc36150 2023-07-21T22:28:58.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.762+0000 7fa6b732db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ecb8c9f0 con 0x5630ecc349c0 2023-07-21T22:28:58.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.762+0000 7fa6b732db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ecb85e80 con 0x5630ecc357e0 2023-07-21T22:28:58.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.763+0000 7fa6a3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 0x5630ecc343e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.763+0000 7fa6a3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 0x5630ecc343e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47744/0 (socket says 172.21.15.107:47744) 2023-07-21T22:28:58.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.763+0000 7fa6a3fff700 1 -- 172.21.15.107:0/2608528245 learned_addr learned my addr 172.21.15.107:0/2608528245 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.763+0000 7fa6a8f44700 1 --2- 172.21.15.107:0/2608528245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc357e0 0x5630ecc35bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.763+0000 7fa6a3fff700 1 -- 172.21.15.107:0/2608528245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc357e0 msgr2=0x5630ecc35bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.763+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/2608528245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc357e0 0x5630ecc35bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.763+0000 7fa6a3fff700 1 -- 172.21.15.107:0/2608528245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc349c0 msgr2=0x5630ecc34e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.763+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/2608528245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc349c0 0x5630ecc34e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.763+0000 7fa6a3fff700 1 -- 172.21.15.107:0/2608528245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630ec9b3d90 con 0x5630ecc36150 2023-07-21T22:28:58.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.763+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/2608528245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 0x5630ecc343e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa69000db90 tx=0x7fa690010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974ba927f370c44c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.763+0000 7fa68effd700 1 -- 172.21.15.107:0/2608528245 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69000f8b0 con 0x5630ecc36150 2023-07-21T22:28:58.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.763+0000 7fa68effd700 1 -- 172.21.15.107:0/2608528245 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa69001c450 con 0x5630ecc36150 2023-07-21T22:28:58.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.763+0000 7fa68effd700 1 -- 172.21.15.107:0/2608528245 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69001b610 con 0x5630ecc36150 2023-07-21T22:28:58.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.764+0000 7fa6b732db80 1 -- 172.21.15.107:0/2608528245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 msgr2=0x5630ecc343e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.764+0000 7fa6b732db80 1 --2- 172.21.15.107:0/2608528245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 0x5630ecc343e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa69000db90 tx=0x7fa690010000 comp rx=0 tx=0).stop 2023-07-21T22:28:58.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.764+0000 7fa6b732db80 1 -- 172.21.15.107:0/2608528245 shutdown_connections 2023-07-21T22:28:58.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.764+0000 7fa6b732db80 1 --2- 172.21.15.107:0/2608528245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc349c0 0x5630ecc34e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.764+0000 7fa6b732db80 1 --2- 172.21.15.107:0/2608528245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc357e0 0x5630ecc35bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.764+0000 7fa6b732db80 1 --2- 172.21.15.107:0/2608528245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 0x5630ecc343e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.764+0000 7fa6b732db80 1 -- 172.21.15.107:0/2608528245 >> 172.21.15.107:0/2608528245 conn(0x5630ecc29ea0 msgr2=0x5630ecc2f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.764+0000 7fa6b732db80 1 -- 172.21.15.107:0/2608528245 shutdown_connections 2023-07-21T22:28:58.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.764+0000 7fa6b732db80 1 -- 172.21.15.107:0/2608528245 wait complete. 2023-07-21T22:28:58.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.764+0000 7fa6b732db80 1 -- start start 2023-07-21T22:28:58.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.764+0000 7fa6b732db80 1 Processor -- start 2023-07-21T22:28:58.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6b732db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc349c0 0x5630ecc3e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6b732db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc357e0 0x5630ecc3e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6b732db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc36150 0x5630ecc45600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6b732db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ecb25a70 con 0x5630ecc349c0 2023-07-21T22:28:58.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6b732db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ecb25770 con 0x5630ecc36150 2023-07-21T22:28:58.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6b732db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630eca098d0 con 0x5630ecc357e0 2023-07-21T22:28:58.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6b732db80 10 client.0 _ll_register_callbacks cb 0x5630ecb23710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:58.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6a8f44700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc349c0 0x5630ecc3e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6a8f44700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc349c0 0x5630ecc3e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47748/0 (socket says 172.21.15.107:47748) 2023-07-21T22:28:58.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6a8f44700 1 -- 172.21.15.107:0/3235954916 learned_addr learned my addr 172.21.15.107:0/3235954916 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/3235954916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc357e0 0x5630ecc3e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6a8f44700 1 -- 172.21.15.107:0/3235954916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc357e0 msgr2=0x5630ecc3e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.823 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6a8f44700 1 --2- 172.21.15.107:0/3235954916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc357e0 0x5630ecc3e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.823 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6a8f44700 1 -- 172.21.15.107:0/3235954916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc36150 msgr2=0x5630ecc45600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.823 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6a8f44700 1 --2- 172.21.15.107:0/3235954916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc36150 0x5630ecc45600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.823 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6a8f44700 1 -- 172.21.15.107:0/3235954916 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69000d840 con 0x5630ecc349c0 2023-07-21T22:28:58.823 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/3235954916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc357e0 0x5630ecc3e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:58.823 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6a8f44700 1 --2- 172.21.15.107:0/3235954916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc349c0 0x5630ecc3e080 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa698009fd0 tx=0x7fa69809dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace0bdad7a095b2c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.823 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa683fff700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:58.823 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa683fff700 1 -- 172.21.15.107:0/3235954916 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6980b9040 con 0x5630ecc349c0 2023-07-21T22:28:58.824 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa683fff700 1 -- 172.21.15.107:0/3235954916 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa698098ff0 con 0x5630ecc349c0 2023-07-21T22:28:58.824 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.765+0000 7fa6b732db80 1 -- 172.21.15.107:0/3235954916 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630ecadf4a0 con 0x5630ecc349c0 2023-07-21T22:28:58.824 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.766+0000 7fa683fff700 1 -- 172.21.15.107:0/3235954916 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6980991b0 con 0x5630ecc349c0 2023-07-21T22:28:58.824 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.766+0000 7fa6b732db80 10 client.15288 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:58.824 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.766+0000 7fa6b732db80 1 -- 172.21.15.107:0/3235954916 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5630ec99cc10 con 0x5630ecc349c0 2023-07-21T22:28:58.824 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.766+0000 7fa683fff700 1 -- 172.21.15.107:0/3235954916 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5630ecadf4a0 con 0x5630ecc349c0 2023-07-21T22:28:58.824 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.766+0000 7fa683fff700 1 -- 172.21.15.107:0/3235954916 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa698098b40 con 0x5630ecc349c0 2023-07-21T22:28:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.766+0000 7fa6b732db80 10 client.15288 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.766+0000 7fa6b732db80 10 client.15288 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-21T22:28:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.766+0000 7fa6b732db80 20 client.15288 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.766+0000 7fa6b732db80 1 --2- 172.21.15.107:0/3235954916 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5630ecc492e0 0x5630ecc48700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.766+0000 7fa6b732db80 4 client.15288 mds_command: new command op to 24455 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.766+0000 7fa6b732db80 1 -- 172.21.15.107:0/3235954916 --> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630ecb7ee60 con 0x5630ecc492e0 2023-07-21T22:28:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.766+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/3235954916 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5630ecc492e0 0x5630ecc48700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa683fff700 10 client.15288 ms_handle_connect on v2:172.21.15.132:6834/1022724305 2023-07-21T22:28:58.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/3235954916 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5630ecc492e0 0x5630ecc48700 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=535f93aa0234168 server_cookie=82230c750d83c9da in_seq=0 out_seq=0 2023-07-21T22:28:58.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa683fff700 1 -- 172.21.15.107:0/3235954916 <== mds.3 v2:172.21.15.132:6834/1022724305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5630ecb7ee60 con 0x5630ecc492e0 2023-07-21T22:28:58.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa683fff700 10 client.15288 handle_command_reply: tid=0 2023-07-21T22:28:58.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa6b732db80 1 client.15288 shutdown 2023-07-21T22:28:58.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa6b732db80 1 -- 172.21.15.107:0/3235954916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc349c0 msgr2=0x5630ecc3e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa6b732db80 1 --2- 172.21.15.107:0/3235954916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc349c0 0x5630ecc3e080 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa698009fd0 tx=0x7fa69809dbe0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa6b732db80 1 -- 172.21.15.107:0/3235954916 shutdown_connections 2023-07-21T22:28:58.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa6b732db80 1 --2- 172.21.15.107:0/3235954916 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5630ecc492e0 0x5630ecc48700 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa6b732db80 1 --2- 172.21.15.107:0/3235954916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc36150 0x5630ecc45600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa6b732db80 1 --2- 172.21.15.107:0/3235954916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc349c0 0x5630ecc3e080 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa6b732db80 1 --2- 172.21.15.107:0/3235954916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc357e0 0x5630ecc3e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa683fff700 0 client.15288 ms_handle_reset on v2:172.21.15.132:6834/1022724305 2023-07-21T22:28:58.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.767+0000 7fa6b732db80 1 -- 172.21.15.107:0/3235954916 >> 172.21.15.107:0/3235954916 conn(0x5630ecc29ea0 msgr2=0x5630ecc2e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.768+0000 7fa6b732db80 1 -- 172.21.15.107:0/3235954916 shutdown_connections 2023-07-21T22:28:58.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.768+0000 7fa6b732db80 1 -- 172.21.15.107:0/3235954916 wait complete. 2023-07-21T22:28:58.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.768+0000 7fa6b732db80 20 client.15288 trim_cache size 0 max 16384 2023-07-21T22:28:58.838 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6b732db80 1 Processor -- start 2023-07-21T22:28:58.838 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6b732db80 1 -- start start 2023-07-21T22:28:58.839 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6b732db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc349c0 0x5630ecc38a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.839 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6b732db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc357e0 0x5630ecc455c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.839 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6b732db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 0x5630ecc38f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.839 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6b732db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ecadf4a0 con 0x5630ecc36150 2023-07-21T22:28:58.839 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6b732db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ecb2f470 con 0x5630ecc357e0 2023-07-21T22:28:58.839 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6b732db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630eca15de0 con 0x5630ecc349c0 2023-07-21T22:28:58.839 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6a9745700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 0x5630ecc38f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.840 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6a9745700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 0x5630ecc38f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47760/0 (socket says 172.21.15.107:47760) 2023-07-21T22:28:58.840 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6a9745700 1 -- 172.21.15.107:0/1290902122 learned_addr learned my addr 172.21.15.107:0/1290902122 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.840 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/1290902122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc357e0 0x5630ecc455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.840 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6a8f44700 1 --2- 172.21.15.107:0/1290902122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc349c0 0x5630ecc38a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.840 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6a9745700 1 -- 172.21.15.107:0/1290902122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc349c0 msgr2=0x5630ecc38a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.840 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6a9745700 1 --2- 172.21.15.107:0/1290902122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc349c0 0x5630ecc38a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.841 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.785+0000 7fa6a9745700 1 -- 172.21.15.107:0/1290902122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc357e0 msgr2=0x5630ecc455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.841 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6a9745700 1 --2- 172.21.15.107:0/1290902122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc357e0 0x5630ecc455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.841 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6a9745700 1 -- 172.21.15.107:0/1290902122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630ec9da200 con 0x5630ecc36150 2023-07-21T22:28:58.841 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/1290902122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc357e0 0x5630ecc455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:58.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6a8f44700 1 --2- 172.21.15.107:0/1290902122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc349c0 0x5630ecc38a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:58.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6a9745700 1 --2- 172.21.15.107:0/1290902122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 0x5630ecc38f80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00b070 tx=0x7fa69c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fce431d9e0982b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa68f7fe700 1 -- 172.21.15.107:0/1290902122 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00e050 con 0x5630ecc36150 2023-07-21T22:28:58.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa68f7fe700 1 -- 172.21.15.107:0/1290902122 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa69c008eb0 con 0x5630ecc36150 2023-07-21T22:28:58.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa68f7fe700 1 -- 172.21.15.107:0/1290902122 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c0052a0 con 0x5630ecc36150 2023-07-21T22:28:58.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6b732db80 1 -- 172.21.15.107:0/1290902122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 msgr2=0x5630ecc38f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6b732db80 1 --2- 172.21.15.107:0/1290902122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 0x5630ecc38f80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00b070 tx=0x7fa69c009c00 comp rx=0 tx=0).stop 2023-07-21T22:28:58.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6b732db80 1 -- 172.21.15.107:0/1290902122 shutdown_connections 2023-07-21T22:28:58.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6b732db80 1 --2- 172.21.15.107:0/1290902122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc36150 0x5630ecc38f80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6b732db80 1 --2- 172.21.15.107:0/1290902122 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc349c0 0x5630ecc38a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6b732db80 1 --2- 172.21.15.107:0/1290902122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc357e0 0x5630ecc455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6b732db80 1 -- 172.21.15.107:0/1290902122 >> 172.21.15.107:0/1290902122 conn(0x5630ecc29ea0 msgr2=0x5630ecc48700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6b732db80 1 -- 172.21.15.107:0/1290902122 shutdown_connections 2023-07-21T22:28:58.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.786+0000 7fa6b732db80 1 -- 172.21.15.107:0/1290902122 wait complete. 2023-07-21T22:28:58.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6b732db80 1 -- start start 2023-07-21T22:28:58.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6b732db80 1 Processor -- start 2023-07-21T22:28:58.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6b732db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc349c0 0x5630ecc40b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6b732db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc357e0 0x5630ecc410d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6b732db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc36150 0x5630ecc440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6b732db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ecc44630 con 0x5630ecc357e0 2023-07-21T22:28:58.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6b732db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ecc447a0 con 0x5630ecc349c0 2023-07-21T22:28:58.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6b732db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630ecc44960 con 0x5630ecc36150 2023-07-21T22:28:58.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6b732db80 10 client.0 _ll_register_callbacks cb 0x7fa698009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:58.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6a3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc357e0 0x5630ecc410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6a3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc357e0 0x5630ecc410d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47764/0 (socket says 172.21.15.107:47764) 2023-07-21T22:28:58.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6a3fff700 1 -- 172.21.15.107:0/2966690578 learned_addr learned my addr 172.21.15.107:0/2966690578 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6a8f44700 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc349c0 0x5630ecc40b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6a3fff700 1 -- 172.21.15.107:0/2966690578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc36150 msgr2=0x5630ecc440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6a9745700 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc36150 0x5630ecc440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc36150 0x5630ecc440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6a3fff700 1 -- 172.21.15.107:0/2966690578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc349c0 msgr2=0x5630ecc40b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc349c0 0x5630ecc40b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.787+0000 7fa6a3fff700 1 -- 172.21.15.107:0/2966690578 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a41077c0 con 0x5630ecc357e0 2023-07-21T22:28:58.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa6a8f44700 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc349c0 0x5630ecc40b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:58.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa6a9745700 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc36150 0x5630ecc440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:58.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc357e0 0x5630ecc410d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa69000db30 tx=0x7fa69000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ed0a07d8a9629e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa683fff700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:58.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa683fff700 1 -- 172.21.15.107:0/2966690578 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69001c840 con 0x5630ecc357e0 2023-07-21T22:28:58.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa683fff700 1 -- 172.21.15.107:0/2966690578 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa690030320 con 0x5630ecc357e0 2023-07-21T22:28:58.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa6b732db80 1 -- 172.21.15.107:0/2966690578 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa69001c450 con 0x5630ecc357e0 2023-07-21T22:28:58.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa683fff700 1 -- 172.21.15.107:0/2966690578 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6900304e0 con 0x5630ecc357e0 2023-07-21T22:28:58.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa6b732db80 10 client.15294 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:58.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa6b732db80 1 -- 172.21.15.107:0/2966690578 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5630ec9b2260 con 0x5630ecc357e0 2023-07-21T22:28:58.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa683fff700 1 -- 172.21.15.107:0/2966690578 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa69001c450 con 0x5630ecc357e0 2023-07-21T22:28:58.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa683fff700 1 -- 172.21.15.107:0/2966690578 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa69000b050 con 0x5630ecc357e0 2023-07-21T22:28:58.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa6b732db80 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:58.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa6b732db80 10 client.15294 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-21T22:28:58.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa6b732db80 20 client.15294 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:58.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa6b732db80 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5630ecca2ca0 0x5630ecc9f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa6b732db80 4 client.15294 mds_command: new command op to 24455 tid=0[{"prefix": "status"}] 2023-07-21T22:28:58.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa6b732db80 1 -- 172.21.15.107:0/2966690578 --> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5630ecb41000 con 0x5630ecca2ca0 2023-07-21T22:28:58.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.788+0000 7fa6a8f44700 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5630ecca2ca0 0x5630ecc9f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa683fff700 10 client.15294 ms_handle_connect on v2:172.21.15.132:6834/1022724305 2023-07-21T22:28:58.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa6a8f44700 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5630ecca2ca0 0x5630ecc9f5d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f6772e763b3de0b4 server_cookie=4e1c234bfa336eb0 in_seq=0 out_seq=0 2023-07-21T22:28:58.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa683fff700 1 -- 172.21.15.107:0/2966690578 <== mds.3 v2:172.21.15.132:6834/1022724305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5630ecb41000 con 0x5630ecca2ca0 2023-07-21T22:28:58.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa683fff700 10 client.15294 handle_command_reply: tid=0 2023-07-21T22:28:58.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa6b732db80 1 client.15294 shutdown 2023-07-21T22:28:58.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa6b732db80 1 -- 172.21.15.107:0/2966690578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc357e0 msgr2=0x5630ecc410d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa6b732db80 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc357e0 0x5630ecc410d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa69000db30 tx=0x7fa69000ded0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa6b732db80 1 -- 172.21.15.107:0/2966690578 shutdown_connections 2023-07-21T22:28:58.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa6b732db80 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5630ecc36150 0x5630ecc440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa6b732db80 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5630ecca2ca0 0x5630ecc9f5d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa6b732db80 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5630ecc349c0 0x5630ecc40b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa683fff700 0 client.15294 ms_handle_reset on v2:172.21.15.132:6834/1022724305 2023-07-21T22:28:58.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa6b732db80 1 --2- 172.21.15.107:0/2966690578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630ecc357e0 0x5630ecc410d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.851 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa6b732db80 1 -- 172.21.15.107:0/2966690578 >> 172.21.15.107:0/2966690578 conn(0x5630ecc29ea0 msgr2=0x5630ecc38a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.851 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa6b732db80 1 -- 172.21.15.107:0/2966690578 shutdown_connections 2023-07-21T22:28:58.851 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa6b732db80 1 -- 172.21.15.107:0/2966690578 wait complete. 2023-07-21T22:28:58.851 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.789+0000 7fa6b732db80 20 client.15294 trim_cache size 0 max 16384 2023-07-21T22:28:58.851 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:28:58.852 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "af99f5d0-2814-11ee-9b34-001a4aab830c", 2023-07-21T22:28:58.852 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 3, 2023-07-21T22:28:58.852 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24455, 2023-07-21T22:28:58.852 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2023-07-21T22:28:58.852 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2023-07-21T22:28:58.852 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2023-07-21T22:28:58.852 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 71.768016880999994, 2023-07-21T22:28:58.853 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2023-07-21T22:28:58.853 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 84, 2023-07-21T22:28:58.853 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:28:58.853 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 98.290828281000003 2023-07-21T22:28:58.853 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:28:58.854 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.790+0000 7fa6ab1a8700 1 -- 172.21.15.107:0/1312040818 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa6940a11e0 msgr2=0x7fa6940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.854 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.790+0000 7fa6ab1a8700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa6940a11e0 0x7fa6940a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa690005fd0 tx=0x7fa690005dc0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.790+0000 7fa6ab1a8700 1 -- 172.21.15.107:0/1312040818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a40fea50 msgr2=0x7fa6a4197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.790+0000 7fa6ab1a8700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a40fea50 0x7fa6a4197b60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa69800d980 tx=0x7fa69800dd40 comp rx=0 tx=0).stop 2023-07-21T22:28:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.790+0000 7fa6ab1a8700 1 -- 172.21.15.107:0/1312040818 shutdown_connections 2023-07-21T22:28:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.790+0000 7fa6ab1a8700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa6940a11e0 0x7fa6940a3690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.790+0000 7fa6ab1a8700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6a41005d0 0x7fa6a419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.790+0000 7fa6ab1a8700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a40fea50 0x7fa6a4197b60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.790+0000 7fa6ab1a8700 1 --2- 172.21.15.107:0/1312040818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a40ffc40 0x7fa6a41980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.790+0000 7fa6ab1a8700 1 -- 172.21.15.107:0/1312040818 >> 172.21.15.107:0/1312040818 conn(0x7fa6a40fa070 msgr2=0x7fa6a40688a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.790+0000 7fa6ab1a8700 1 -- 172.21.15.107:0/1312040818 shutdown_connections 2023-07-21T22:28:58.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.790+0000 7fa6ab1a8700 1 -- 172.21.15.107:0/1312040818 wait complete. 2023-07-21T22:28:58.856 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-07-21T22:28:58.978 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.931+0000 7fc19ce90700 1 -- 172.21.15.107:0/269513848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1980fea00 msgr2=0x7fc1980fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.979 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.931+0000 7fc19ce90700 1 --2- 172.21.15.107:0/269513848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1980fea00 0x7fc1980fee10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc188009990 tx=0x7fc188009ca0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.979 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.931+0000 7fc19ce90700 1 -- 172.21.15.107:0/269513848 shutdown_connections 2023-07-21T22:28:58.979 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.931+0000 7fc19ce90700 1 --2- 172.21.15.107:0/269513848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc198100580 0x7fc198106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.979 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.931+0000 7fc19ce90700 1 --2- 172.21.15.107:0/269513848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1980fea00 0x7fc1980fee10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.979 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.931+0000 7fc19ce90700 1 --2- 172.21.15.107:0/269513848 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1980ffbf0 0x7fc198100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.979 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.931+0000 7fc19ce90700 1 -- 172.21.15.107:0/269513848 >> 172.21.15.107:0/269513848 conn(0x7fc1980fa060 msgr2=0x7fc1980fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.979 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.931+0000 7fc19ce90700 1 -- 172.21.15.107:0/269513848 shutdown_connections 2023-07-21T22:28:58.980 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.931+0000 7fc19ce90700 1 -- 172.21.15.107:0/269513848 wait complete. 2023-07-21T22:28:58.980 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.932+0000 7fc19ce90700 1 Processor -- start 2023-07-21T22:28:58.980 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.932+0000 7fc19ce90700 1 -- start start 2023-07-21T22:28:58.981 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.932+0000 7fc19ce90700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1980fea00 0x7fc1980693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.982 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.932+0000 7fc19ce90700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1980ffbf0 0x7fc19819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.982 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.932+0000 7fc19ce90700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc198100580 0x7fc19819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.982 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.932+0000 7fc19ce90700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19819d400 con 0x7fc1980ffbf0 2023-07-21T22:28:58.982 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.932+0000 7fc19ce90700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19819d570 con 0x7fc198100580 2023-07-21T22:28:58.982 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.932+0000 7fc19ce90700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19819d730 con 0x7fc1980fea00 2023-07-21T22:28:58.982 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.932+0000 7fc195d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1980ffbf0 0x7fc19819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.982 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc195d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1980ffbf0 0x7fc19819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47778/0 (socket says 172.21.15.107:47778) 2023-07-21T22:28:58.983 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc195d9b700 1 -- 172.21.15.107:0/951018660 learned_addr learned my addr 172.21.15.107:0/951018660 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.983 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc196d9d700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc198100580 0x7fc19819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.983 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc19659c700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1980fea00 0x7fc1980693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.983 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc195d9b700 1 -- 172.21.15.107:0/951018660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1980fea00 msgr2=0x7fc1980693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.983 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc195d9b700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1980fea00 0x7fc1980693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.983 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc195d9b700 1 -- 172.21.15.107:0/951018660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc198100580 msgr2=0x7fc19819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.983 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc195d9b700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc198100580 0x7fc19819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc195d9b700 1 -- 172.21.15.107:0/951018660 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1880095f0 con 0x7fc1980ffbf0 2023-07-21T22:28:58.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc195d9b700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1980ffbf0 0x7fc19819a5e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00d950 tx=0x7fc18c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc1877fe700 1 -- 172.21.15.107:0/951018660 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c009b40 con 0x7fc1980ffbf0 2023-07-21T22:28:58.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc1877fe700 1 -- 172.21.15.107:0/951018660 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc18c009ca0 con 0x7fc1980ffbf0 2023-07-21T22:28:58.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc19ce90700 1 -- 172.21.15.107:0/951018660 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19819da10 con 0x7fc1980ffbf0 2023-07-21T22:28:58.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc1877fe700 1 -- 172.21.15.107:0/951018660 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00f620 con 0x7fc1980ffbf0 2023-07-21T22:28:58.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.933+0000 7fc19ce90700 1 -- 172.21.15.107:0/951018660 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19819dfa0 con 0x7fc1980ffbf0 2023-07-21T22:28:58.989 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.936+0000 7fc1877fe700 1 -- 172.21.15.107:0/951018660 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fc18c010460 con 0x7fc1980ffbf0 2023-07-21T22:28:58.989 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.936+0000 7fc1877fe700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc1800a11e0 0x7fc1800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.989 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.936+0000 7fc1877fe700 1 -- 172.21.15.107:0/951018660 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc18c0cce40 con 0x7fc1980ffbf0 2023-07-21T22:28:58.989 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.936+0000 7fc19659c700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc1800a11e0 0x7fc1800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.989 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.937+0000 7fc19659c700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc1800a11e0 0x7fc1800a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc18800b5c0 tx=0x7fc188005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.990 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.937+0000 7fc1a9015b80 1 Processor -- start 2023-07-21T22:28:58.990 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.937+0000 7fc1a9015b80 1 -- start start 2023-07-21T22:28:58.990 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.937+0000 7fc1a9015b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a17e0 0x5605d11a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.990 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.937+0000 7fc1a9015b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 0x5605d11a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.990 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.937+0000 7fc1a9015b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a09c0 0x5605d11a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.990 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.937+0000 7fc1a9015b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d10a5c60 con 0x5605d11a2150 2023-07-21T22:28:58.990 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.937+0000 7fc1a9015b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d10f89f0 con 0x5605d11a09c0 2023-07-21T22:28:58.990 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.937+0000 7fc1a9015b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d10f1e80 con 0x5605d11a17e0 2023-07-21T22:28:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.937+0000 7fc195d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 0x5605d11a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.937+0000 7fc195d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 0x5605d11a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47786/0 (socket says 172.21.15.107:47786) 2023-07-21T22:28:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.937+0000 7fc195d9b700 1 -- 172.21.15.107:0/3458210773 learned_addr learned my addr 172.21.15.107:0/3458210773 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.938+0000 7fc196d9d700 1 --2- 172.21.15.107:0/3458210773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a09c0 0x5605d11a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.938+0000 7fc195d9b700 1 -- 172.21.15.107:0/3458210773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a17e0 msgr2=0x5605d11a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.938+0000 7fc19659c700 1 --2- 172.21.15.107:0/3458210773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a17e0 0x5605d11a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.938+0000 7fc195d9b700 1 --2- 172.21.15.107:0/3458210773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a17e0 0x5605d11a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.938+0000 7fc195d9b700 1 -- 172.21.15.107:0/3458210773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a09c0 msgr2=0x5605d11a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.938+0000 7fc195d9b700 1 --2- 172.21.15.107:0/3458210773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a09c0 0x5605d11a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.938+0000 7fc195d9b700 1 -- 172.21.15.107:0/3458210773 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605d0f1fd90 con 0x5605d11a2150 2023-07-21T22:28:58.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.938+0000 7fc196d9d700 1 --2- 172.21.15.107:0/3458210773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a09c0 0x5605d11a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:58.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.938+0000 7fc19659c700 1 --2- 172.21.15.107:0/3458210773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a17e0 0x5605d11a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:58.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.938+0000 7fc195d9b700 1 --2- 172.21.15.107:0/3458210773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 0x5605d11a03e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc18c000c00 tx=0x7fc18c099d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d1da77cf78600d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.938+0000 7fc184ff9700 1 -- 172.21.15.107:0/3458210773 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0a2520 con 0x5605d11a2150 2023-07-21T22:28:58.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.938+0000 7fc184ff9700 1 -- 172.21.15.107:0/3458210773 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc18c0caa20 con 0x5605d11a2150 2023-07-21T22:28:58.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.938+0000 7fc184ff9700 1 -- 172.21.15.107:0/3458210773 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0995c0 con 0x5605d11a2150 2023-07-21T22:28:58.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 -- 172.21.15.107:0/3458210773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 msgr2=0x5605d11a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/3458210773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 0x5605d11a03e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc18c000c00 tx=0x7fc18c099d40 comp rx=0 tx=0).stop 2023-07-21T22:28:58.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 -- 172.21.15.107:0/3458210773 shutdown_connections 2023-07-21T22:28:58.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/3458210773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a09c0 0x5605d11a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/3458210773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a17e0 0x5605d11a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/3458210773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 0x5605d11a03e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 -- 172.21.15.107:0/3458210773 >> 172.21.15.107:0/3458210773 conn(0x5605d1195ea0 msgr2=0x5605d119b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 -- 172.21.15.107:0/3458210773 shutdown_connections 2023-07-21T22:28:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 -- 172.21.15.107:0/3458210773 wait complete. 2023-07-21T22:28:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 -- start start 2023-07-21T22:28:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 Processor -- start 2023-07-21T22:28:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a09c0 0x5605d11aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a17e0 0x5605d11aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a2150 0x5605d11b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d1091a70 con 0x5605d11a17e0 2023-07-21T22:28:58.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d1091770 con 0x5605d11a09c0 2023-07-21T22:28:58.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d0f758d0 con 0x5605d11a2150 2023-07-21T22:28:58.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc1a9015b80 10 client.0 _ll_register_callbacks cb 0x5605d108f710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:58.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.939+0000 7fc195d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a17e0 0x5605d11aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc195d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a17e0 0x5605d11aa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47792/0 (socket says 172.21.15.107:47792) 2023-07-21T22:28:58.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc195d9b700 1 -- 172.21.15.107:0/991001257 learned_addr learned my addr 172.21.15.107:0/991001257 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:58.996 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc19659c700 1 --2- 172.21.15.107:0/991001257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a09c0 0x5605d11aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.996 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc196d9d700 1 --2- 172.21.15.107:0/991001257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a2150 0x5605d11b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.996 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc195d9b700 1 -- 172.21.15.107:0/991001257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a2150 msgr2=0x5605d11b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.996 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc195d9b700 1 --2- 172.21.15.107:0/991001257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a2150 0x5605d11b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.996 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc195d9b700 1 -- 172.21.15.107:0/991001257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a09c0 msgr2=0x5605d11aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:58.996 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc195d9b700 1 --2- 172.21.15.107:0/991001257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a09c0 0x5605d11aa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:58.996 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc195d9b700 1 -- 172.21.15.107:0/991001257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19819dfa0 con 0x5605d11a17e0 2023-07-21T22:28:58.996 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc195d9b700 1 --2- 172.21.15.107:0/991001257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a17e0 0x5605d11aa570 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc18c09a290 tx=0x7fc18c010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276dcbf7151d0624 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:58.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc17dffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:58.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc17dffb700 1 -- 172.21.15.107:0/991001257 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0ca2a0 con 0x5605d11a17e0 2023-07-21T22:28:58.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc17dffb700 1 -- 172.21.15.107:0/991001257 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc18c0ca400 con 0x5605d11a17e0 2023-07-21T22:28:58.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc1a9015b80 1 -- 172.21.15.107:0/991001257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605d104b4a0 con 0x5605d11a17e0 2023-07-21T22:28:58.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc17dffb700 1 -- 172.21.15.107:0/991001257 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc18c0995c0 con 0x5605d11a17e0 2023-07-21T22:28:58.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc1a9015b80 10 client.15306 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:58.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc1a9015b80 1 -- 172.21.15.107:0/991001257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5605d0f08c10 con 0x5605d11a17e0 2023-07-21T22:28:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.940+0000 7fc17dffb700 1 -- 172.21.15.107:0/991001257 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5605d104b4a0 con 0x5605d11a17e0 2023-07-21T22:28:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.941+0000 7fc17dffb700 1 -- 172.21.15.107:0/991001257 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc18c0b2ab0 con 0x5605d11a17e0 2023-07-21T22:28:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.941+0000 7fc1a9015b80 10 client.15306 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.941+0000 7fc1a9015b80 10 client.15306 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-21T22:28:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.941+0000 7fc1a9015b80 20 client.15306 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.941+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/991001257 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5605d11b5360 0x5605d11b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.941+0000 7fc1a9015b80 4 client.15306 mds_command: new command op to 24466 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:58.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.941+0000 7fc1a9015b80 1 -- 172.21.15.107:0/991001257 --> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605d10eae60 con 0x5605d11b5360 2023-07-21T22:28:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.941+0000 7fc19659c700 1 --2- 172.21.15.107:0/991001257 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5605d11b5360 0x5605d11b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.941+0000 7fc17dffb700 10 client.15306 ms_handle_connect on v2:172.21.15.190:6834/2286851849 2023-07-21T22:28:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.941+0000 7fc19659c700 1 --2- 172.21.15.107:0/991001257 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5605d11b5360 0x5605d11b4450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37e0ea453ee60535 server_cookie=7ad90c879428bbec in_seq=0 out_seq=0 2023-07-21T22:28:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc17dffb700 1 -- 172.21.15.107:0/991001257 <== mds.2 v2:172.21.15.190:6834/2286851849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5605d10eae60 con 0x5605d11b5360 2023-07-21T22:28:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc17dffb700 10 client.15306 handle_command_reply: tid=0 2023-07-21T22:28:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc1a9015b80 1 client.15306 shutdown 2023-07-21T22:28:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc1a9015b80 1 -- 172.21.15.107:0/991001257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a17e0 msgr2=0x5605d11aa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:58.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/991001257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a17e0 0x5605d11aa570 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc18c09a290 tx=0x7fc18c010940 comp rx=0 tx=0).stop 2023-07-21T22:28:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc1a9015b80 1 -- 172.21.15.107:0/991001257 shutdown_connections 2023-07-21T22:28:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/991001257 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5605d11b5360 0x5605d11b4450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/991001257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a2150 0x5605d11b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/991001257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a09c0 0x5605d11aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc17dffb700 0 client.15306 ms_handle_reset on v2:172.21.15.190:6834/2286851849 2023-07-21T22:28:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/991001257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a17e0 0x5605d11aa570 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc1a9015b80 1 -- 172.21.15.107:0/991001257 >> 172.21.15.107:0/991001257 conn(0x5605d1195ea0 msgr2=0x5605d119a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.001 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc1a9015b80 1 -- 172.21.15.107:0/991001257 shutdown_connections 2023-07-21T22:28:59.001 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc1a9015b80 1 -- 172.21.15.107:0/991001257 wait complete. 2023-07-21T22:28:59.001 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.942+0000 7fc1a9015b80 20 client.15306 trim_cache size 0 max 16384 2023-07-21T22:28:59.007 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc1a9015b80 1 Processor -- start 2023-07-21T22:28:59.007 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc1a9015b80 1 -- start start 2023-07-21T22:28:59.007 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc1a9015b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a09c0 0x5605d11a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.007 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc1a9015b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a17e0 0x5605d11a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc1a9015b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a2150 0x5605d11b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc1a9015b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d104b4a0 con 0x5605d11a09c0 2023-07-21T22:28:59.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc1a9015b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d11ab220 con 0x5605d11a2150 2023-07-21T22:28:59.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc1a9015b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d109b470 con 0x5605d11a17e0 2023-07-21T22:28:59.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc19659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a09c0 0x5605d11a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc19659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a09c0 0x5605d11a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47798/0 (socket says 172.21.15.107:47798) 2023-07-21T22:28:59.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc19659c700 1 -- 172.21.15.107:0/2369614030 learned_addr learned my addr 172.21.15.107:0/2369614030 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:59.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc195d9b700 1 --2- 172.21.15.107:0/2369614030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a17e0 0x5605d11a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc196d9d700 1 --2- 172.21.15.107:0/2369614030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a2150 0x5605d11b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc19659c700 1 -- 172.21.15.107:0/2369614030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a17e0 msgr2=0x5605d11a4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc19659c700 1 --2- 172.21.15.107:0/2369614030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a17e0 0x5605d11a4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc19659c700 1 -- 172.21.15.107:0/2369614030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a2150 msgr2=0x5605d11b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc19659c700 1 --2- 172.21.15.107:0/2369614030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a2150 0x5605d11b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc19659c700 1 -- 172.21.15.107:0/2369614030 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605d0f46200 con 0x5605d11a09c0 2023-07-21T22:28:59.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.960+0000 7fc19659c700 1 --2- 172.21.15.107:0/2369614030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a09c0 0x5605d11a4a80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc188015000 tx=0x7fc18801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1a415d8d0b77bc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.961+0000 7fc1857fa700 1 -- 172.21.15.107:0/2369614030 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18800bb80 con 0x5605d11a09c0 2023-07-21T22:28:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.961+0000 7fc1857fa700 1 -- 172.21.15.107:0/2369614030 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc18802c040 con 0x5605d11a09c0 2023-07-21T22:28:59.010 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.961+0000 7fc1857fa700 1 -- 172.21.15.107:0/2369614030 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1880049f0 con 0x5605d11a09c0 2023-07-21T22:28:59.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.961+0000 7fc1a9015b80 1 -- 172.21.15.107:0/2369614030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a09c0 msgr2=0x5605d11a4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.961+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/2369614030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a09c0 0x5605d11a4a80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc188015000 tx=0x7fc18801e000 comp rx=0 tx=0).stop 2023-07-21T22:28:59.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.961+0000 7fc1a9015b80 1 -- 172.21.15.107:0/2369614030 shutdown_connections 2023-07-21T22:28:59.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.961+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/2369614030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a2150 0x5605d11b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.013 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.961+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/2369614030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a09c0 0x5605d11a4a80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.961+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/2369614030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a17e0 0x5605d11a4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.961+0000 7fc1a9015b80 1 -- 172.21.15.107:0/2369614030 >> 172.21.15.107:0/2369614030 conn(0x5605d1195ea0 msgr2=0x5605d11a8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.961+0000 7fc1a9015b80 1 -- 172.21.15.107:0/2369614030 shutdown_connections 2023-07-21T22:28:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.961+0000 7fc1a9015b80 1 -- 172.21.15.107:0/2369614030 wait complete. 2023-07-21T22:28:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.961+0000 7fc1a9015b80 1 -- start start 2023-07-21T22:28:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc1a9015b80 1 Processor -- start 2023-07-21T22:28:59.014 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc1a9015b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a09c0 0x5605d11ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc1a9015b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a17e0 0x5605d11acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc1a9015b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 0x5605d11afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc1a9015b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d11b0260 con 0x5605d11a2150 2023-07-21T22:28:59.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc1a9015b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d11b03d0 con 0x5605d11a17e0 2023-07-21T22:28:59.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc1a9015b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605d11b0590 con 0x5605d11a09c0 2023-07-21T22:28:59.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc1a9015b80 10 client.0 _ll_register_callbacks cb 0x7fc18c099c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:59.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc196d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 0x5605d11afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.015 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc196d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 0x5605d11afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47808/0 (socket says 172.21.15.107:47808) 2023-07-21T22:28:59.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc196d9d700 1 -- 172.21.15.107:0/605464342 learned_addr learned my addr 172.21.15.107:0/605464342 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:59.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc195d9b700 1 --2- 172.21.15.107:0/605464342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a17e0 0x5605d11acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc19659c700 1 --2- 172.21.15.107:0/605464342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a09c0 0x5605d11ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc196d9d700 1 -- 172.21.15.107:0/605464342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a09c0 msgr2=0x5605d11ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc196d9d700 1 --2- 172.21.15.107:0/605464342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a09c0 0x5605d11ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc196d9d700 1 -- 172.21.15.107:0/605464342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a17e0 msgr2=0x5605d11acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc196d9d700 1 --2- 172.21.15.107:0/605464342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a17e0 0x5605d11acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc196d9d700 1 -- 172.21.15.107:0/605464342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc198107770 con 0x5605d11a2150 2023-07-21T22:28:59.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc196d9d700 1 --2- 172.21.15.107:0/605464342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 0x5605d11afd20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc19000aaa0 tx=0x7fc19000aad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e398cf80fc6fe86 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc17dffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:59.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc17dffb700 1 -- 172.21.15.107:0/605464342 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190011040 con 0x5605d11a2150 2023-07-21T22:28:59.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc17dffb700 1 -- 172.21.15.107:0/605464342 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc190006e40 con 0x5605d11a2150 2023-07-21T22:28:59.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.962+0000 7fc1a9015b80 1 -- 172.21.15.107:0/605464342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc18c0caa20 con 0x5605d11a2150 2023-07-21T22:28:59.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.963+0000 7fc17dffb700 1 -- 172.21.15.107:0/605464342 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc190017840 con 0x5605d11a2150 2023-07-21T22:28:59.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.963+0000 7fc1a9015b80 10 client.15312 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:59.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.963+0000 7fc1a9015b80 1 -- 172.21.15.107:0/605464342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5605d0f1e260 con 0x5605d11a2150 2023-07-21T22:28:59.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.963+0000 7fc17dffb700 1 -- 172.21.15.107:0/605464342 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc18c0caa20 con 0x5605d11a2150 2023-07-21T22:28:59.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.963+0000 7fc17dffb700 1 -- 172.21.15.107:0/605464342 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc190016ac0 con 0x5605d11a2150 2023-07-21T22:28:59.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.963+0000 7fc1a9015b80 10 client.15312 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:59.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.963+0000 7fc1a9015b80 10 client.15312 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-21T22:28:59.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.963+0000 7fc1a9015b80 20 client.15312 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:59.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.963+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/605464342 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5605d120e470 0x5605d120adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.963+0000 7fc1a9015b80 4 client.15312 mds_command: new command op to 24466 tid=0[{"prefix": "status"}] 2023-07-21T22:28:59.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.963+0000 7fc1a9015b80 1 -- 172.21.15.107:0/605464342 --> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5605d10ad000 con 0x5605d120e470 2023-07-21T22:28:59.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.963+0000 7fc19659c700 1 --2- 172.21.15.107:0/605464342 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5605d120e470 0x5605d120adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc17dffb700 10 client.15312 ms_handle_connect on v2:172.21.15.190:6834/2286851849 2023-07-21T22:28:59.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc19659c700 1 --2- 172.21.15.107:0/605464342 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5605d120e470 0x5605d120adc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aea1eccb56fd55a5 server_cookie=da0c09ecb09295a9 in_seq=0 out_seq=0 2023-07-21T22:28:59.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc17dffb700 1 -- 172.21.15.107:0/605464342 <== mds.2 v2:172.21.15.190:6834/2286851849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5605d10ad000 con 0x5605d120e470 2023-07-21T22:28:59.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc17dffb700 10 client.15312 handle_command_reply: tid=0 2023-07-21T22:28:59.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc1a9015b80 1 client.15312 shutdown 2023-07-21T22:28:59.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc1a9015b80 1 -- 172.21.15.107:0/605464342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 msgr2=0x5605d11afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/605464342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 0x5605d11afd20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc19000aaa0 tx=0x7fc19000aad0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc1a9015b80 1 -- 172.21.15.107:0/605464342 shutdown_connections 2023-07-21T22:28:59.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/605464342 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x5605d120e470 0x5605d120adc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/605464342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605d11a2150 0x5605d11afd20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc17dffb700 0 client.15312 ms_handle_reset on v2:172.21.15.190:6834/2286851849 2023-07-21T22:28:59.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/605464342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5605d11a09c0 0x5605d11ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc1a9015b80 1 --2- 172.21.15.107:0/605464342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605d11a17e0 0x5605d11acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc1a9015b80 1 -- 172.21.15.107:0/605464342 >> 172.21.15.107:0/605464342 conn(0x5605d1195ea0 msgr2=0x5605d11a4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc1a9015b80 1 -- 172.21.15.107:0/605464342 shutdown_connections 2023-07-21T22:28:59.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc1a9015b80 1 -- 172.21.15.107:0/605464342 wait complete. 2023-07-21T22:28:59.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.964+0000 7fc1a9015b80 20 client.15312 trim_cache size 0 max 16384 2023-07-21T22:28:59.021 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:28:59.022 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "af99f5d0-2814-11ee-9b34-001a4aab830c", 2023-07-21T22:28:59.022 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 2, 2023-07-21T22:28:59.022 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24466, 2023-07-21T22:28:59.022 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2023-07-21T22:28:59.022 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2023-07-21T22:28:59.022 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2023-07-21T22:28:59.022 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 72.948928925000004, 2023-07-21T22:28:59.022 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2023-07-21T22:28:59.023 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 84, 2023-07-21T22:28:59.023 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:28:59.023 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 96.372895811000006 2023-07-21T22:28:59.023 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:28:59.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.965+0000 7fc19ce90700 1 -- 172.21.15.107:0/951018660 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc1800a11e0 msgr2=0x7fc1800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.965+0000 7fc19ce90700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc1800a11e0 0x7fc1800a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc18800b5c0 tx=0x7fc188005fb0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.965+0000 7fc19ce90700 1 -- 172.21.15.107:0/951018660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1980ffbf0 msgr2=0x7fc19819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.965+0000 7fc19ce90700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1980ffbf0 0x7fc19819a5e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00d950 tx=0x7fc18c00dd10 comp rx=0 tx=0).stop 2023-07-21T22:28:59.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.965+0000 7fc19ce90700 1 -- 172.21.15.107:0/951018660 shutdown_connections 2023-07-21T22:28:59.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.965+0000 7fc19ce90700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc198100580 0x7fc19819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.965+0000 7fc19ce90700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc1800a11e0 0x7fc1800a3690 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.965+0000 7fc19ce90700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1980fea00 0x7fc1980693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.965+0000 7fc19ce90700 1 --2- 172.21.15.107:0/951018660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1980ffbf0 0x7fc19819a5e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.965+0000 7fc19ce90700 1 -- 172.21.15.107:0/951018660 >> 172.21.15.107:0/951018660 conn(0x7fc1980fa060 msgr2=0x7fc1981030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.965+0000 7fc19ce90700 1 -- 172.21.15.107:0/951018660 shutdown_connections 2023-07-21T22:28:59.026 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:58.965+0000 7fc19ce90700 1 -- 172.21.15.107:0/951018660 wait complete. 2023-07-21T22:28:59.026 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-21T22:28:59.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.102+0000 7fd5689a2700 1 -- 172.21.15.107:0/3941498246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5641005f0 msgr2=0x7fd564106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.102+0000 7fd5689a2700 1 --2- 172.21.15.107:0/3941498246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5641005f0 0x7fd564106e90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00b0a0 tx=0x7fd55c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.102+0000 7fd5689a2700 1 -- 172.21.15.107:0/3941498246 shutdown_connections 2023-07-21T22:28:59.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.102+0000 7fd5689a2700 1 --2- 172.21.15.107:0/3941498246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5641005f0 0x7fd564106e90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.102+0000 7fd5689a2700 1 --2- 172.21.15.107:0/3941498246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5640fef30 0x7fd5640ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.102+0000 7fd5689a2700 1 --2- 172.21.15.107:0/3941498246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5640ffc40 0x7fd5641000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.102+0000 7fd5689a2700 1 -- 172.21.15.107:0/3941498246 >> 172.21.15.107:0/3941498246 conn(0x7fd5640f97b0 msgr2=0x7fd5640fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.102+0000 7fd5689a2700 1 -- 172.21.15.107:0/3941498246 shutdown_connections 2023-07-21T22:28:59.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.102+0000 7fd5689a2700 1 -- 172.21.15.107:0/3941498246 wait complete. 2023-07-21T22:28:59.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd5689a2700 1 Processor -- start 2023-07-21T22:28:59.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd5689a2700 1 -- start start 2023-07-21T22:28:59.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd5689a2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5640fef30 0x7fd564195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.152 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd5689a2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5640ffc40 0x7fd564195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.153 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd5689a2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5641005f0 0x7fd56419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.153 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd5689a2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56419d4a0 con 0x7fd5641005f0 2023-07-21T22:28:59.153 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd5689a2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56419d5e0 con 0x7fd5640fef30 2023-07-21T22:28:59.153 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd5689a2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56419d770 con 0x7fd5640ffc40 2023-07-21T22:28:59.153 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd562d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5641005f0 0x7fd56419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.153 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd562d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5641005f0 0x7fd56419ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47822/0 (socket says 172.21.15.107:47822) 2023-07-21T22:28:59.153 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd562d9d700 1 -- 172.21.15.107:0/1216241116 learned_addr learned my addr 172.21.15.107:0/1216241116 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:59.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd561d9b700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5640ffc40 0x7fd564195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd56259c700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5640fef30 0x7fd564195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd562d9d700 1 -- 172.21.15.107:0/1216241116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5640ffc40 msgr2=0x7fd564195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd562d9d700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5640ffc40 0x7fd564195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd562d9d700 1 -- 172.21.15.107:0/1216241116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5640fef30 msgr2=0x7fd564195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd562d9d700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5640fef30 0x7fd564195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.103+0000 7fd562d9d700 1 -- 172.21.15.107:0/1216241116 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c009d00 con 0x7fd5641005f0 2023-07-21T22:28:59.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.104+0000 7fd562d9d700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5641005f0 0x7fd56419ced0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0062a0 tx=0x7fd55c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.104+0000 7fd5537fe700 1 -- 172.21.15.107:0/1216241116 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c00e040 con 0x7fd5641005f0 2023-07-21T22:28:59.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.104+0000 7fd5689a2700 1 -- 172.21.15.107:0/1216241116 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56419d9f0 con 0x7fd5641005f0 2023-07-21T22:28:59.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.104+0000 7fd5537fe700 1 -- 172.21.15.107:0/1216241116 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd55c00cec0 con 0x7fd5641005f0 2023-07-21T22:28:59.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.104+0000 7fd5537fe700 1 -- 172.21.15.107:0/1216241116 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c01baf0 con 0x7fd5641005f0 2023-07-21T22:28:59.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.104+0000 7fd5689a2700 1 -- 172.21.15.107:0/1216241116 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56419de70 con 0x7fd5641005f0 2023-07-21T22:28:59.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.106+0000 7fd5537fe700 1 -- 172.21.15.107:0/1216241116 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd55c01bc50 con 0x7fd5641005f0 2023-07-21T22:28:59.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.107+0000 7fd5537fe700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd54c0a11e0 0x7fd54c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.107+0000 7fd56259c700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd54c0a11e0 0x7fd54c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.107+0000 7fd5537fe700 1 -- 172.21.15.107:0/1216241116 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd55c0ce5d0 con 0x7fd5641005f0 2023-07-21T22:28:59.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.107+0000 7fd56259c700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd54c0a11e0 0x7fd54c0a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd5640683d0 tx=0x7fd55800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.107+0000 7fd574b27b80 1 Processor -- start 2023-07-21T22:28:59.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd574b27b80 1 -- start start 2023-07-21T22:28:59.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd574b27b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68c7e0 0x55e7ae68cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd574b27b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 0x55e7ae68b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd574b27b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68b9c0 0x55e7ae68be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd574b27b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ae590c60 con 0x55e7ae68c7e0 2023-07-21T22:28:59.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd574b27b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ae5e39f0 con 0x55e7ae68d150 2023-07-21T22:28:59.161 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd574b27b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ae5dce80 con 0x55e7ae68b9c0 2023-07-21T22:28:59.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd56259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68c7e0 0x55e7ae68cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd56259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68c7e0 0x55e7ae68cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47826/0 (socket says 172.21.15.107:47826) 2023-07-21T22:28:59.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd56259c700 1 -- 172.21.15.107:0/3524399006 learned_addr learned my addr 172.21.15.107:0/3524399006 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:59.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd56259c700 1 -- 172.21.15.107:0/3524399006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68b9c0 msgr2=0x55e7ae68be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd56259c700 1 --2- 172.21.15.107:0/3524399006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68b9c0 0x55e7ae68be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd56259c700 1 -- 172.21.15.107:0/3524399006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 msgr2=0x55e7ae68b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd56259c700 1 --2- 172.21.15.107:0/3524399006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 0x55e7ae68b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.162 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd56259c700 1 -- 172.21.15.107:0/3524399006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7ae40ad90 con 0x55e7ae68c7e0 2023-07-21T22:28:59.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd56259c700 1 --2- 172.21.15.107:0/3524399006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68c7e0 0x55e7ae68cbf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd558010f70 tx=0x7fd558013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af14bc30384db6a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.108+0000 7fd550ff9700 1 -- 172.21.15.107:0/3524399006 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55801e620 con 0x55e7ae68c7e0 2023-07-21T22:28:59.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.109+0000 7fd550ff9700 1 -- 172.21.15.107:0/3524399006 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd558015070 con 0x55e7ae68c7e0 2023-07-21T22:28:59.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.109+0000 7fd550ff9700 1 -- 172.21.15.107:0/3524399006 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55801ca50 con 0x55e7ae68c7e0 2023-07-21T22:28:59.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.109+0000 7fd574b27b80 1 -- 172.21.15.107:0/3524399006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68c7e0 msgr2=0x55e7ae68cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.109+0000 7fd574b27b80 1 --2- 172.21.15.107:0/3524399006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68c7e0 0x55e7ae68cbf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd558010f70 tx=0x7fd558013000 comp rx=0 tx=0).stop 2023-07-21T22:28:59.163 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.109+0000 7fd574b27b80 1 -- 172.21.15.107:0/3524399006 shutdown_connections 2023-07-21T22:28:59.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.109+0000 7fd574b27b80 1 --2- 172.21.15.107:0/3524399006 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68b9c0 0x55e7ae68be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.109+0000 7fd574b27b80 1 --2- 172.21.15.107:0/3524399006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68c7e0 0x55e7ae68cbf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.109+0000 7fd574b27b80 1 --2- 172.21.15.107:0/3524399006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 0x55e7ae68b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.109+0000 7fd574b27b80 1 -- 172.21.15.107:0/3524399006 >> 172.21.15.107:0/3524399006 conn(0x55e7ae680ea0 msgr2=0x55e7ae686310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.109+0000 7fd574b27b80 1 -- 172.21.15.107:0/3524399006 shutdown_connections 2023-07-21T22:28:59.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.109+0000 7fd574b27b80 1 -- 172.21.15.107:0/3524399006 wait complete. 2023-07-21T22:28:59.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd574b27b80 1 -- start start 2023-07-21T22:28:59.164 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd574b27b80 1 Processor -- start 2023-07-21T22:28:59.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd574b27b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 0x55e7ae695080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd574b27b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae6955c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd574b27b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 0x55e7ae69c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd574b27b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ae57ca70 con 0x55e7ae68b9c0 2023-07-21T22:28:59.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd574b27b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ae57c770 con 0x55e7ae68d150 2023-07-21T22:28:59.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd574b27b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ae4608d0 con 0x55e7ae68c7e0 2023-07-21T22:28:59.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd574b27b80 10 client.0 _ll_register_callbacks cb 0x55e7ae57a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:59.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd56259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 0x55e7ae695080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd56259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 0x55e7ae695080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47832/0 (socket says 172.21.15.107:47832) 2023-07-21T22:28:59.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd56259c700 1 -- 172.21.15.107:0/297222499 learned_addr learned my addr 172.21.15.107:0/297222499 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:59.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd561d9b700 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae6955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd562d9d700 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 0x55e7ae69c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd56259c700 1 -- 172.21.15.107:0/297222499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 msgr2=0x55e7ae6955c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd56259c700 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae6955c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd56259c700 1 -- 172.21.15.107:0/297222499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 msgr2=0x55e7ae69c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd56259c700 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 0x55e7ae69c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd56259c700 1 -- 172.21.15.107:0/297222499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd558010c50 con 0x55e7ae68b9c0 2023-07-21T22:28:59.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.110+0000 7fd562d9d700 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 0x55e7ae69c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:59.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd56259c700 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 0x55e7ae695080 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd558010fe0 tx=0x7fd55801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00539fe0e3385c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd549ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:59.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd549ffb700 1 -- 172.21.15.107:0/297222499 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55802c070 con 0x55e7ae68b9c0 2023-07-21T22:28:59.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd549ffb700 1 -- 172.21.15.107:0/297222499 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd558017070 con 0x55e7ae68b9c0 2023-07-21T22:28:59.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd574b27b80 1 -- 172.21.15.107:0/297222499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7ae5364a0 con 0x55e7ae68b9c0 2023-07-21T22:28:59.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd549ffb700 1 -- 172.21.15.107:0/297222499 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd55801ce20 con 0x55e7ae68b9c0 2023-07-21T22:28:59.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd574b27b80 10 client.15324 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:59.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd574b27b80 1 -- 172.21.15.107:0/297222499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e7ae3f3c10 con 0x55e7ae68b9c0 2023-07-21T22:28:59.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd549ffb700 1 -- 172.21.15.107:0/297222499 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd558040080 con 0x55e7ae68b9c0 2023-07-21T22:28:59.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd549ffb700 1 -- 172.21.15.107:0/297222499 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd55803a030 con 0x55e7ae68b9c0 2023-07-21T22:28:59.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd574b27b80 10 client.15324 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:59.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd574b27b80 10 client.15324 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T22:28:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd574b27b80 20 client.15324 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd574b27b80 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55e7ae6a01e0 0x55e7ae69f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd574b27b80 4 client.15324 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd574b27b80 1 -- 172.21.15.107:0/297222499 --> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7ae5d5e60 con 0x55e7ae6a01e0 2023-07-21T22:28:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.111+0000 7fd561d9b700 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55e7ae6a01e0 0x55e7ae69f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.112+0000 7fd549ffb700 10 client.15324 ms_handle_connect on v2:172.21.15.190:6838/2960500982 2023-07-21T22:28:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.112+0000 7fd561d9b700 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55e7ae6a01e0 0x55e7ae69f640 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f44f0a43111ccb8 server_cookie=51086ed5bd28e594 in_seq=0 out_seq=0 2023-07-21T22:28:59.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.112+0000 7fd549ffb700 1 -- 172.21.15.107:0/297222499 <== mds.1 v2:172.21.15.190:6838/2960500982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e7ae5d5e60 con 0x55e7ae6a01e0 2023-07-21T22:28:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.112+0000 7fd549ffb700 10 client.15324 handle_command_reply: tid=0 2023-07-21T22:28:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.112+0000 7fd574b27b80 1 client.15324 shutdown 2023-07-21T22:28:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.113+0000 7fd574b27b80 1 -- 172.21.15.107:0/297222499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 msgr2=0x55e7ae695080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.113+0000 7fd574b27b80 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 0x55e7ae695080 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd558010fe0 tx=0x7fd55801ca70 comp rx=0 tx=0).stop 2023-07-21T22:28:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.113+0000 7fd574b27b80 1 -- 172.21.15.107:0/297222499 shutdown_connections 2023-07-21T22:28:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.113+0000 7fd574b27b80 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55e7ae6a01e0 0x55e7ae69f640 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.113+0000 7fd574b27b80 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 0x55e7ae69c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.113+0000 7fd549ffb700 0 client.15324 ms_handle_reset on v2:172.21.15.190:6838/2960500982 2023-07-21T22:28:59.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.113+0000 7fd574b27b80 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 0x55e7ae695080 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.113+0000 7fd574b27b80 1 --2- 172.21.15.107:0/297222499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae6955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.113+0000 7fd574b27b80 1 -- 172.21.15.107:0/297222499 >> 172.21.15.107:0/297222499 conn(0x55e7ae680ea0 msgr2=0x55e7ae6859b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.113+0000 7fd574b27b80 1 -- 172.21.15.107:0/297222499 shutdown_connections 2023-07-21T22:28:59.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.113+0000 7fd574b27b80 1 -- 172.21.15.107:0/297222499 wait complete. 2023-07-21T22:28:59.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.113+0000 7fd574b27b80 20 client.15324 trim_cache size 0 max 16384 2023-07-21T22:28:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd574b27b80 1 Processor -- start 2023-07-21T22:28:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd574b27b80 1 -- start start 2023-07-21T22:28:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd574b27b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68b9c0 0x55e7ae69f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd574b27b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae69c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd574b27b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68d150 0x55e7ae692f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd574b27b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ae586470 con 0x55e7ae68d150 2023-07-21T22:28:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd574b27b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ae5ab790 con 0x55e7ae68b9c0 2023-07-21T22:28:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd574b27b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ae59b480 con 0x55e7ae68c7e0 2023-07-21T22:28:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd562d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68d150 0x55e7ae692f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd562d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68d150 0x55e7ae692f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47844/0 (socket says 172.21.15.107:47844) 2023-07-21T22:28:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd562d9d700 1 -- 172.21.15.107:0/2931106066 learned_addr learned my addr 172.21.15.107:0/2931106066 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd56259c700 1 --2- 172.21.15.107:0/2931106066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68b9c0 0x55e7ae69f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd561d9b700 1 --2- 172.21.15.107:0/2931106066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae69c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd562d9d700 1 -- 172.21.15.107:0/2931106066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 msgr2=0x55e7ae69c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.130+0000 7fd562d9d700 1 --2- 172.21.15.107:0/2931106066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae69c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd562d9d700 1 -- 172.21.15.107:0/2931106066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68b9c0 msgr2=0x55e7ae69f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd562d9d700 1 --2- 172.21.15.107:0/2931106066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68b9c0 0x55e7ae69f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd562d9d700 1 -- 172.21.15.107:0/2931106066 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7ae431200 con 0x55e7ae68d150 2023-07-21T22:28:59.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd561d9b700 1 --2- 172.21.15.107:0/2931106066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae69c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:59.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd562d9d700 1 --2- 172.21.15.107:0/2931106066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68d150 0x55e7ae692f30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009fd0 tx=0x7fd55c01df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90091ca44c89596 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd5517fa700 1 -- 172.21.15.107:0/2931106066 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c09b8f0 con 0x55e7ae68d150 2023-07-21T22:28:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd5517fa700 1 -- 172.21.15.107:0/2931106066 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd55c086040 con 0x55e7ae68d150 2023-07-21T22:28:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd5517fa700 1 -- 172.21.15.107:0/2931106066 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c011c40 con 0x55e7ae68d150 2023-07-21T22:28:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd574b27b80 1 -- 172.21.15.107:0/2931106066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68d150 msgr2=0x55e7ae692f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd574b27b80 1 --2- 172.21.15.107:0/2931106066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68d150 0x55e7ae692f30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009fd0 tx=0x7fd55c01df50 comp rx=0 tx=0).stop 2023-07-21T22:28:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd574b27b80 1 -- 172.21.15.107:0/2931106066 shutdown_connections 2023-07-21T22:28:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd574b27b80 1 --2- 172.21.15.107:0/2931106066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68d150 0x55e7ae692f30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd574b27b80 1 --2- 172.21.15.107:0/2931106066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68b9c0 0x55e7ae69f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd574b27b80 1 --2- 172.21.15.107:0/2931106066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae69c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd574b27b80 1 -- 172.21.15.107:0/2931106066 >> 172.21.15.107:0/2931106066 conn(0x55e7ae680ea0 msgr2=0x55e7ae693d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd574b27b80 1 -- 172.21.15.107:0/2931106066 shutdown_connections 2023-07-21T22:28:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.131+0000 7fd574b27b80 1 -- 172.21.15.107:0/2931106066 wait complete. 2023-07-21T22:28:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd574b27b80 1 -- start start 2023-07-21T22:28:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd574b27b80 1 Processor -- start 2023-07-21T22:28:59.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd574b27b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 0x55e7ae691b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd574b27b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae6920d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd574b27b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 0x55e7ae69abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd574b27b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ae69b100 con 0x55e7ae68b9c0 2023-07-21T22:28:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd574b27b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ae69b270 con 0x55e7ae68d150 2023-07-21T22:28:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd574b27b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ae69b430 con 0x55e7ae68c7e0 2023-07-21T22:28:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd574b27b80 10 client.0 _ll_register_callbacks cb 0x7fd558010e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd56259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 0x55e7ae691b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd56259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 0x55e7ae691b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47846/0 (socket says 172.21.15.107:47846) 2023-07-21T22:28:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd56259c700 1 -- 172.21.15.107:0/688230854 learned_addr learned my addr 172.21.15.107:0/688230854 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd562d9d700 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 0x55e7ae69abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd561d9b700 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae6920d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd56259c700 1 -- 172.21.15.107:0/688230854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 msgr2=0x55e7ae6920d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd56259c700 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae6920d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd56259c700 1 -- 172.21.15.107:0/688230854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 msgr2=0x55e7ae69abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd56259c700 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 0x55e7ae69abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd56259c700 1 -- 172.21.15.107:0/688230854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56419de70 con 0x55e7ae68b9c0 2023-07-21T22:28:59.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd561d9b700 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae6920d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:59.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd56259c700 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 0x55e7ae691b90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd55803c460 tx=0x7fd5580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94c2e04ab94f9c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd549ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:59.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.132+0000 7fd549ffb700 1 -- 172.21.15.107:0/688230854 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55802c070 con 0x55e7ae68b9c0 2023-07-21T22:28:59.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd549ffb700 1 -- 172.21.15.107:0/688230854 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd55803ce40 con 0x55e7ae68b9c0 2023-07-21T22:28:59.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd574b27b80 1 -- 172.21.15.107:0/688230854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd558015070 con 0x55e7ae68b9c0 2023-07-21T22:28:59.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd549ffb700 1 -- 172.21.15.107:0/688230854 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd55800be40 con 0x55e7ae68b9c0 2023-07-21T22:28:59.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd574b27b80 10 client.15330 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:59.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd574b27b80 1 -- 172.21.15.107:0/688230854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e7ae409260 con 0x55e7ae68b9c0 2023-07-21T22:28:59.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd549ffb700 1 -- 172.21.15.107:0/688230854 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd558040020 con 0x55e7ae68b9c0 2023-07-21T22:28:59.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd549ffb700 1 -- 172.21.15.107:0/688230854 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd5580117d0 con 0x55e7ae68b9c0 2023-07-21T22:28:59.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd574b27b80 10 client.15330 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:59.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd574b27b80 10 client.15330 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T22:28:59.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd574b27b80 20 client.15330 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:59.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd574b27b80 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55e7ae6f90f0 0x55e7ae69c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd574b27b80 4 client.15330 mds_command: new command op to 24493 tid=0[{"prefix": "status"}] 2023-07-21T22:28:59.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd574b27b80 1 -- 172.21.15.107:0/688230854 --> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e7ae598000 con 0x55e7ae6f90f0 2023-07-21T22:28:59.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.133+0000 7fd561d9b700 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55e7ae6f90f0 0x55e7ae69c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd549ffb700 10 client.15330 ms_handle_connect on v2:172.21.15.190:6838/2960500982 2023-07-21T22:28:59.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd561d9b700 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55e7ae6f90f0 0x55e7ae69c5d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c43f9f1448ce9192 server_cookie=9441afb0f7e6bfc8 in_seq=0 out_seq=0 2023-07-21T22:28:59.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd549ffb700 1 -- 172.21.15.107:0/688230854 <== mds.1 v2:172.21.15.190:6838/2960500982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55e7ae598000 con 0x55e7ae6f90f0 2023-07-21T22:28:59.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd549ffb700 10 client.15330 handle_command_reply: tid=0 2023-07-21T22:28:59.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd574b27b80 1 client.15330 shutdown 2023-07-21T22:28:59.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd574b27b80 1 -- 172.21.15.107:0/688230854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 msgr2=0x55e7ae691b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd574b27b80 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 0x55e7ae691b90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd55803c460 tx=0x7fd5580047c0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd574b27b80 1 -- 172.21.15.107:0/688230854 shutdown_connections 2023-07-21T22:28:59.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd574b27b80 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55e7ae6f90f0 0x55e7ae69c5d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd574b27b80 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e7ae68d150 0x55e7ae69abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd549ffb700 0 client.15330 ms_handle_reset on v2:172.21.15.190:6838/2960500982 2023-07-21T22:28:59.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd574b27b80 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7ae68b9c0 0x55e7ae691b90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd574b27b80 1 --2- 172.21.15.107:0/688230854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e7ae68c7e0 0x55e7ae6920d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd574b27b80 1 -- 172.21.15.107:0/688230854 >> 172.21.15.107:0/688230854 conn(0x55e7ae680ea0 msgr2=0x55e7ae69f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd574b27b80 1 -- 172.21.15.107:0/688230854 shutdown_connections 2023-07-21T22:28:59.192 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd574b27b80 1 -- 172.21.15.107:0/688230854 wait complete. 2023-07-21T22:28:59.192 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.134+0000 7fd574b27b80 20 client.15330 trim_cache size 0 max 16384 2023-07-21T22:28:59.192 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:28:59.192 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "af99f5d0-2814-11ee-9b34-001a4aab830c", 2023-07-21T22:28:59.192 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 1, 2023-07-21T22:28:59.193 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24493, 2023-07-21T22:28:59.193 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2023-07-21T22:28:59.193 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2023-07-21T22:28:59.193 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2023-07-21T22:28:59.193 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 74.024429815999994, 2023-07-21T22:28:59.193 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2023-07-21T22:28:59.193 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 84, 2023-07-21T22:28:59.193 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:28:59.194 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 89.907000416000002 2023-07-21T22:28:59.194 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:28:59.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.135+0000 7fd5689a2700 1 -- 172.21.15.107:0/1216241116 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd54c0a11e0 msgr2=0x7fd54c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.135+0000 7fd5689a2700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd54c0a11e0 0x7fd54c0a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd5640683d0 tx=0x7fd55800b410 comp rx=0 tx=0).stop 2023-07-21T22:28:59.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.135+0000 7fd5689a2700 1 -- 172.21.15.107:0/1216241116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5641005f0 msgr2=0x7fd56419ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.135+0000 7fd5689a2700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5641005f0 0x7fd56419ced0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0062a0 tx=0x7fd55c004d00 comp rx=0 tx=0).stop 2023-07-21T22:28:59.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.135+0000 7fd5689a2700 1 -- 172.21.15.107:0/1216241116 shutdown_connections 2023-07-21T22:28:59.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.135+0000 7fd5689a2700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5641005f0 0x7fd56419ced0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.135+0000 7fd5689a2700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5640fef30 0x7fd564195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.135+0000 7fd5689a2700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd54c0a11e0 0x7fd54c0a3690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.135+0000 7fd5689a2700 1 --2- 172.21.15.107:0/1216241116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5640ffc40 0x7fd564195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.135+0000 7fd5689a2700 1 -- 172.21.15.107:0/1216241116 >> 172.21.15.107:0/1216241116 conn(0x7fd5640f97b0 msgr2=0x7fd5640fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.135+0000 7fd5689a2700 1 -- 172.21.15.107:0/1216241116 shutdown_connections 2023-07-21T22:28:59.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.135+0000 7fd5689a2700 1 -- 172.21.15.107:0/1216241116 wait complete. 2023-07-21T22:28:59.197 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-21T22:28:59.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.272+0000 7f812cba2700 1 -- 172.21.15.107:0/406766279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128100590 msgr2=0x7f8128106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.272+0000 7f812cba2700 1 --2- 172.21.15.107:0/406766279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128100590 0x7f8128106e50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f812000b0a0 tx=0x7f812000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.272+0000 7f812cba2700 1 -- 172.21.15.107:0/406766279 shutdown_connections 2023-07-21T22:28:59.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.272+0000 7f812cba2700 1 --2- 172.21.15.107:0/406766279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128100590 0x7f8128106e50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.272+0000 7f812cba2700 1 --2- 172.21.15.107:0/406766279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81280fea10 0x7f81280fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.272+0000 7f812cba2700 1 --2- 172.21.15.107:0/406766279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81280ffc00 0x7f8128100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.272+0000 7f812cba2700 1 -- 172.21.15.107:0/406766279 >> 172.21.15.107:0/406766279 conn(0x7f81280fa030 msgr2=0x7f81280fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.272+0000 7f812cba2700 1 -- 172.21.15.107:0/406766279 shutdown_connections 2023-07-21T22:28:59.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.272+0000 7f812cba2700 1 -- 172.21.15.107:0/406766279 wait complete. 2023-07-21T22:28:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f812cba2700 1 Processor -- start 2023-07-21T22:28:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f812cba2700 1 -- start start 2023-07-21T22:28:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f812cba2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81280fea10 0x7f8128195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f812cba2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81280ffc00 0x7f8128195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f812cba2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128100590 0x7f812819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f812cba2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812819d4c0 con 0x7f8128100590 2023-07-21T22:28:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f812cba2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812819d600 con 0x7f81280ffc00 2023-07-21T22:28:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f812cba2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812819d790 con 0x7f81280fea10 2023-07-21T22:28:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f8126d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128100590 0x7f812819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f8126d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128100590 0x7f812819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47864/0 (socket says 172.21.15.107:47864) 2023-07-21T22:28:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f8126d9d700 1 -- 172.21.15.107:0/3749112513 learned_addr learned my addr 172.21.15.107:0/3749112513 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f8125d9b700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81280ffc00 0x7f8128195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f812659c700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81280fea10 0x7f8128195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f8126d9d700 1 -- 172.21.15.107:0/3749112513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81280fea10 msgr2=0x7f8128195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f8126d9d700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81280fea10 0x7f8128195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f8126d9d700 1 -- 172.21.15.107:0/3749112513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81280ffc00 msgr2=0x7f8128195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f8126d9d700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81280ffc00 0x7f8128195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f8126d9d700 1 -- 172.21.15.107:0/3749112513 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8120009d00 con 0x7f8128100590 2023-07-21T22:28:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.273+0000 7f812659c700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81280fea10 0x7f8128195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:59.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.274+0000 7f8125d9b700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81280ffc00 0x7f8128195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:59.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.274+0000 7f8126d9d700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128100590 0x7f812819cef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f812000b830 tx=0x7f8120004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.274+0000 7f811f7fe700 1 -- 172.21.15.107:0/3749112513 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812000e040 con 0x7f8128100590 2023-07-21T22:28:59.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.274+0000 7f811f7fe700 1 -- 172.21.15.107:0/3749112513 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81200094c0 con 0x7f8128100590 2023-07-21T22:28:59.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.274+0000 7f812cba2700 1 -- 172.21.15.107:0/3749112513 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812819da10 con 0x7f8128100590 2023-07-21T22:28:59.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.274+0000 7f811f7fe700 1 -- 172.21.15.107:0/3749112513 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812001bc70 con 0x7f8128100590 2023-07-21T22:28:59.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.274+0000 7f812cba2700 1 -- 172.21.15.107:0/3749112513 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812819de90 con 0x7f8128100590 2023-07-21T22:28:59.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.276+0000 7f811f7fe700 1 -- 172.21.15.107:0/3749112513 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f812001d890 con 0x7f8128100590 2023-07-21T22:28:59.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.277+0000 7f811f7fe700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81140a11e0 0x7f81140a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.277+0000 7f811f7fe700 1 -- 172.21.15.107:0/3749112513 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f81200cea10 con 0x7f8128100590 2023-07-21T22:28:59.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.277+0000 7f812659c700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81140a11e0 0x7f81140a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.277+0000 7f812659c700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81140a11e0 0x7f81140a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8110005950 tx=0x7f81100058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.277+0000 7f8138d27b80 1 Processor -- start 2023-07-21T22:28:59.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8138d27b80 1 -- start start 2023-07-21T22:28:59.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8138d27b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ac7e0 0x561aae2acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8138d27b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ad150 0x561aae2ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8138d27b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ab9c0 0x561aae2abe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8138d27b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aae1b0c60 con 0x561aae2ab9c0 2023-07-21T22:28:59.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8138d27b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aae2039f0 con 0x561aae2ad150 2023-07-21T22:28:59.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8138d27b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aae1fce80 con 0x561aae2ac7e0 2023-07-21T22:28:59.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8126d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ab9c0 0x561aae2abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8126d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ab9c0 0x561aae2abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47872/0 (socket says 172.21.15.107:47872) 2023-07-21T22:28:59.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8126d9d700 1 -- 172.21.15.107:0/4230817658 learned_addr learned my addr 172.21.15.107:0/4230817658 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:59.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f812659c700 1 --2- 172.21.15.107:0/4230817658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ac7e0 0x561aae2acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8126d9d700 1 -- 172.21.15.107:0/4230817658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ac7e0 msgr2=0x561aae2acbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8126d9d700 1 --2- 172.21.15.107:0/4230817658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ac7e0 0x561aae2acbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8126d9d700 1 -- 172.21.15.107:0/4230817658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ad150 msgr2=0x561aae2ab3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8126d9d700 1 --2- 172.21.15.107:0/4230817658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ad150 0x561aae2ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8126d9d700 1 -- 172.21.15.107:0/4230817658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aae02ad90 con 0x561aae2ab9c0 2023-07-21T22:28:59.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f812659c700 1 --2- 172.21.15.107:0/4230817658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ac7e0 0x561aae2acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:59.334 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f8126d9d700 1 --2- 172.21.15.107:0/4230817658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ab9c0 0x561aae2abe40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8120098230 tx=0x7f812000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9376d2386f75e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.334 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.278+0000 7f811cff9700 1 -- 172.21.15.107:0/4230817658 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81200a40f0 con 0x561aae2ab9c0 2023-07-21T22:28:59.334 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.279+0000 7f811cff9700 1 -- 172.21.15.107:0/4230817658 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f812001b450 con 0x561aae2ab9c0 2023-07-21T22:28:59.334 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.279+0000 7f811cff9700 1 -- 172.21.15.107:0/4230817658 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812009bf90 con 0x561aae2ab9c0 2023-07-21T22:28:59.334 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.279+0000 7f8138d27b80 1 -- 172.21.15.107:0/4230817658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ab9c0 msgr2=0x561aae2abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.334 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.279+0000 7f8138d27b80 1 --2- 172.21.15.107:0/4230817658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ab9c0 0x561aae2abe40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8120098230 tx=0x7f812000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.334 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.279+0000 7f8138d27b80 1 -- 172.21.15.107:0/4230817658 shutdown_connections 2023-07-21T22:28:59.335 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.279+0000 7f8138d27b80 1 --2- 172.21.15.107:0/4230817658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ab9c0 0x561aae2abe40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.335 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.279+0000 7f8138d27b80 1 --2- 172.21.15.107:0/4230817658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ac7e0 0x561aae2acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.335 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.279+0000 7f8138d27b80 1 --2- 172.21.15.107:0/4230817658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ad150 0x561aae2ab3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.335 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.279+0000 7f8138d27b80 1 -- 172.21.15.107:0/4230817658 >> 172.21.15.107:0/4230817658 conn(0x561aae2a0ea0 msgr2=0x561aae2a6310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.335 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.279+0000 7f8138d27b80 1 -- 172.21.15.107:0/4230817658 shutdown_connections 2023-07-21T22:28:59.335 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.279+0000 7f8138d27b80 1 -- 172.21.15.107:0/4230817658 wait complete. 2023-07-21T22:28:59.335 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8138d27b80 1 -- start start 2023-07-21T22:28:59.335 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8138d27b80 1 Processor -- start 2023-07-21T22:28:59.336 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8138d27b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ab9c0 0x561aae2b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.336 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8138d27b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2b5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.336 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8138d27b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ad150 0x561aae2bc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.336 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8138d27b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aae19ca70 con 0x561aae2ac7e0 2023-07-21T22:28:59.336 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8138d27b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aae19c770 con 0x561aae2ad150 2023-07-21T22:28:59.336 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8138d27b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aae0808d0 con 0x561aae2ab9c0 2023-07-21T22:28:59.336 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8138d27b80 10 client.0 _ll_register_callbacks cb 0x561aae19a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:59.337 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8125d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.337 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8125d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47876/0 (socket says 172.21.15.107:47876) 2023-07-21T22:28:59.337 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8125d9b700 1 -- 172.21.15.107:0/3837218606 learned_addr learned my addr 172.21.15.107:0/3837218606 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:59.337 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f812659c700 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ab9c0 0x561aae2b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.337 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8126d9d700 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ad150 0x561aae2bc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.337 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8125d9b700 1 -- 172.21.15.107:0/3837218606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ab9c0 msgr2=0x561aae2b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.338 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8125d9b700 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ab9c0 0x561aae2b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.338 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8125d9b700 1 -- 172.21.15.107:0/3837218606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ad150 msgr2=0x561aae2bc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.338 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8125d9b700 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ad150 0x561aae2bc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.338 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8125d9b700 1 -- 172.21.15.107:0/3837218606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812819de90 con 0x561aae2ac7e0 2023-07-21T22:28:59.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.280+0000 7f8126d9d700 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ad150 0x561aae2bc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:28:59.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f8125d9b700 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2b5360 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8118003b30 tx=0x7f8118003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa0b58aa15d6cef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f810dffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:59.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f810dffb700 1 -- 172.21.15.107:0/3837218606 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811800fd20 con 0x561aae2ac7e0 2023-07-21T22:28:59.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f810dffb700 1 -- 172.21.15.107:0/3837218606 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f811800fe80 con 0x561aae2ac7e0 2023-07-21T22:28:59.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f8138d27b80 1 -- 172.21.15.107:0/3837218606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561aae1564a0 con 0x561aae2ac7e0 2023-07-21T22:28:59.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f810dffb700 1 -- 172.21.15.107:0/3837218606 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f811801a460 con 0x561aae2ac7e0 2023-07-21T22:28:59.339 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f8138d27b80 10 client.15342 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:59.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f8138d27b80 1 -- 172.21.15.107:0/3837218606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561aae013c10 con 0x561aae2ac7e0 2023-07-21T22:28:59.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f810dffb700 1 -- 172.21.15.107:0/3837218606 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561aae1564a0 con 0x561aae2ac7e0 2023-07-21T22:28:59.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f810dffb700 1 -- 172.21.15.107:0/3837218606 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8118023050 con 0x561aae2ac7e0 2023-07-21T22:28:59.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f8138d27b80 10 client.15342 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:59.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f8138d27b80 10 client.15342 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:28:59.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f8138d27b80 20 client.15342 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:59.340 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f8138d27b80 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561aae2c0150 0x561aae2bf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f8138d27b80 4 client.15342 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:59.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.281+0000 7f8138d27b80 1 -- 172.21.15.107:0/3837218606 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561aae1f5e60 con 0x561aae2c0150 2023-07-21T22:28:59.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.282+0000 7f812659c700 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561aae2c0150 0x561aae2bf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.282+0000 7f810dffb700 10 client.15342 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:28:59.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.282+0000 7f812659c700 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561aae2c0150 0x561aae2bf550 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b1523efc580a138 server_cookie=ca98cddfc1884168 in_seq=0 out_seq=0 2023-07-21T22:28:59.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f810dffb700 1 -- 172.21.15.107:0/3837218606 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561aae1f5e60 con 0x561aae2c0150 2023-07-21T22:28:59.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f810dffb700 10 client.15342 handle_command_reply: tid=0 2023-07-21T22:28:59.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f8138d27b80 1 client.15342 shutdown 2023-07-21T22:28:59.342 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f8138d27b80 1 -- 172.21.15.107:0/3837218606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 msgr2=0x561aae2b5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.342 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f8138d27b80 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2b5360 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8118003b30 tx=0x7f8118003e40 comp rx=0 tx=0).stop 2023-07-21T22:28:59.342 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f8138d27b80 1 -- 172.21.15.107:0/3837218606 shutdown_connections 2023-07-21T22:28:59.342 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f8138d27b80 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561aae2c0150 0x561aae2bf550 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.342 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f8138d27b80 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ad150 0x561aae2bc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f810dffb700 0 client.15342 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:28:59.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f8138d27b80 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ab9c0 0x561aae2b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f8138d27b80 1 --2- 172.21.15.107:0/3837218606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2b5360 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.344 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f8138d27b80 1 -- 172.21.15.107:0/3837218606 >> 172.21.15.107:0/3837218606 conn(0x561aae2a0ea0 msgr2=0x561aae2a59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.344 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f8138d27b80 1 -- 172.21.15.107:0/3837218606 shutdown_connections 2023-07-21T22:28:59.344 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f8138d27b80 1 -- 172.21.15.107:0/3837218606 wait complete. 2023-07-21T22:28:59.344 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.283+0000 7f8138d27b80 20 client.15342 trim_cache size 0 max 16384 2023-07-21T22:28:59.351 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.300+0000 7f8138d27b80 1 Processor -- start 2023-07-21T22:28:59.351 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.300+0000 7f8138d27b80 1 -- start start 2023-07-21T22:28:59.352 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.300+0000 7f8138d27b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ab9c0 0x561aae2bcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.352 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.300+0000 7f8138d27b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2bc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.352 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.300+0000 7f8138d27b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ad150 0x561aae2b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.352 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.300+0000 7f8138d27b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aae1f5e60 con 0x561aae2ac7e0 2023-07-21T22:28:59.353 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.300+0000 7f8138d27b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aae1564a0 con 0x561aae2ab9c0 2023-07-21T22:28:59.353 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.300+0000 7f8138d27b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aae1cb790 con 0x561aae2ad150 2023-07-21T22:28:59.353 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.300+0000 7f8125d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2bc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.354 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.300+0000 7f8125d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2bc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47884/0 (socket says 172.21.15.107:47884) 2023-07-21T22:28:59.354 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.300+0000 7f8125d9b700 1 -- 172.21.15.107:0/2259458593 learned_addr learned my addr 172.21.15.107:0/2259458593 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:59.354 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f8125d9b700 1 -- 172.21.15.107:0/2259458593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ad150 msgr2=0x561aae2b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.354 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f8125d9b700 1 --2- 172.21.15.107:0/2259458593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ad150 0x561aae2b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.355 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f812659c700 1 --2- 172.21.15.107:0/2259458593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ab9c0 0x561aae2bcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.355 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f8125d9b700 1 -- 172.21.15.107:0/2259458593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ab9c0 msgr2=0x561aae2bcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.355 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f8125d9b700 1 --2- 172.21.15.107:0/2259458593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ab9c0 0x561aae2bcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.356 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f8125d9b700 1 -- 172.21.15.107:0/2259458593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aae051200 con 0x561aae2ac7e0 2023-07-21T22:28:59.356 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f8125d9b700 1 --2- 172.21.15.107:0/2259458593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2bc360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8118003b00 tx=0x7f8118012e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f79087e08569899 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.356 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f811d7fa700 1 -- 172.21.15.107:0/2259458593 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118004370 con 0x561aae2ac7e0 2023-07-21T22:28:59.357 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f811d7fa700 1 -- 172.21.15.107:0/2259458593 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8118025420 con 0x561aae2ac7e0 2023-07-21T22:28:59.357 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f811d7fa700 1 -- 172.21.15.107:0/2259458593 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81180053c0 con 0x561aae2ac7e0 2023-07-21T22:28:59.357 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f8138d27b80 1 -- 172.21.15.107:0/2259458593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 msgr2=0x561aae2bc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.357 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f8138d27b80 1 --2- 172.21.15.107:0/2259458593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2bc360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8118003b00 tx=0x7f8118012e60 comp rx=0 tx=0).stop 2023-07-21T22:28:59.357 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f8138d27b80 1 -- 172.21.15.107:0/2259458593 shutdown_connections 2023-07-21T22:28:59.357 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f8138d27b80 1 --2- 172.21.15.107:0/2259458593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ad150 0x561aae2b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.358 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f8138d27b80 1 --2- 172.21.15.107:0/2259458593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ab9c0 0x561aae2bcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.358 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f8138d27b80 1 --2- 172.21.15.107:0/2259458593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2bc360 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.358 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.301+0000 7f8138d27b80 1 -- 172.21.15.107:0/2259458593 >> 172.21.15.107:0/2259458593 conn(0x561aae2a0ea0 msgr2=0x561aae2c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.358 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8138d27b80 1 -- 172.21.15.107:0/2259458593 shutdown_connections 2023-07-21T22:28:59.358 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8138d27b80 1 -- 172.21.15.107:0/2259458593 wait complete. 2023-07-21T22:28:59.358 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8138d27b80 1 -- start start 2023-07-21T22:28:59.358 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8138d27b80 1 Processor -- start 2023-07-21T22:28:59.358 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8138d27b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ab9c0 0x561aae2b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.359 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8138d27b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.359 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8138d27b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ad150 0x561aae2bb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.359 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8138d27b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aae2bb700 con 0x561aae2ac7e0 2023-07-21T22:28:59.359 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8138d27b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aae2bb870 con 0x561aae2ab9c0 2023-07-21T22:28:59.359 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8138d27b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aae2bba30 con 0x561aae2ad150 2023-07-21T22:28:59.359 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8138d27b80 10 client.0 _ll_register_callbacks cb 0x561aae19a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:59.360 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8125d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.360 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8125d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2b81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47890/0 (socket says 172.21.15.107:47890) 2023-07-21T22:28:59.360 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f8125d9b700 1 -- 172.21.15.107:0/226041711 learned_addr learned my addr 172.21.15.107:0/226041711 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:28:59.360 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.302+0000 7f812659c700 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ab9c0 0x561aae2b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.360 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8125d9b700 1 -- 172.21.15.107:0/226041711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ad150 msgr2=0x561aae2bb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.360 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8126d9d700 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ad150 0x561aae2bb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.361 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8125d9b700 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ad150 0x561aae2bb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.361 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8125d9b700 1 -- 172.21.15.107:0/226041711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ab9c0 msgr2=0x561aae2b7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:59.361 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8125d9b700 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ab9c0 0x561aae2b7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.361 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8125d9b700 1 -- 172.21.15.107:0/226041711 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561aae013c10 con 0x561aae2ac7e0 2023-07-21T22:28:59.361 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f812659c700 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ab9c0 0x561aae2b7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:28:59.361 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8125d9b700 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2b81a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f81180046c0 tx=0x7f8118004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82259baedd8b9cfb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:59.361 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f810dffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:28:59.361 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f810dffb700 1 -- 172.21.15.107:0/226041711 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118005550 con 0x561aae2ac7e0 2023-07-21T22:28:59.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f810dffb700 1 -- 172.21.15.107:0/226041711 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8118000e40 con 0x561aae2ac7e0 2023-07-21T22:28:59.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8138d27b80 1 -- 172.21.15.107:0/226041711 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f812001b450 con 0x561aae2ac7e0 2023-07-21T22:28:59.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f810dffb700 1 -- 172.21.15.107:0/226041711 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81180056b0 con 0x561aae2ac7e0 2023-07-21T22:28:59.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8138d27b80 10 client.15348 fetch_fsmap learned FSMap version 27 2023-07-21T22:28:59.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8138d27b80 1 -- 172.21.15.107:0/226041711 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561aae029260 con 0x561aae2ac7e0 2023-07-21T22:28:59.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f810dffb700 1 -- 172.21.15.107:0/226041711 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8118047020 con 0x561aae2ac7e0 2023-07-21T22:28:59.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f810dffb700 1 -- 172.21.15.107:0/226041711 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f811802c070 con 0x561aae2ac7e0 2023-07-21T22:28:59.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8138d27b80 10 client.15348 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:28:59.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8138d27b80 10 client.15348 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:28:59.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8138d27b80 20 client.15348 populate_metadata read hostname 'smithi107' 2023-07-21T22:28:59.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8138d27b80 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561aae319a10 0x561aae3163d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:59.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8138d27b80 4 client.15348 mds_command: new command op to 24533 tid=0[{"prefix": "status"}] 2023-07-21T22:28:59.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.303+0000 7f8138d27b80 1 -- 172.21.15.107:0/226041711 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561aae1b8000 con 0x561aae319a10 2023-07-21T22:28:59.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.304+0000 7f812659c700 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561aae319a10 0x561aae3163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:59.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.304+0000 7f810dffb700 10 client.15348 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:28:59.364 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.304+0000 7f812659c700 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561aae319a10 0x561aae3163d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa8a3a3027bf76a6 server_cookie=5d48df25414217a0 in_seq=0 out_seq=0 2023-07-21T22:28:59.364 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.304+0000 7f810dffb700 1 -- 172.21.15.107:0/226041711 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x561aae1b8000 con 0x561aae319a10 2023-07-21T22:28:59.364 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.304+0000 7f810dffb700 10 client.15348 handle_command_reply: tid=0 2023-07-21T22:28:59.364 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.304+0000 7f8138d27b80 1 client.15348 shutdown 2023-07-21T22:28:59.364 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f8138d27b80 1 -- 172.21.15.107:0/226041711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 msgr2=0x561aae2b81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.364 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f8138d27b80 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2b81a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f81180046c0 tx=0x7f8118004200 comp rx=0 tx=0).stop 2023-07-21T22:28:59.364 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f8138d27b80 1 -- 172.21.15.107:0/226041711 shutdown_connections 2023-07-21T22:28:59.365 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f8138d27b80 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561aae319a10 0x561aae3163d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.365 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f8138d27b80 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561aae2ad150 0x561aae2bb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.365 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f8138d27b80 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561aae2ab9c0 0x561aae2b7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.365 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f810dffb700 0 client.15348 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:28:59.365 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f8138d27b80 1 --2- 172.21.15.107:0/226041711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561aae2ac7e0 0x561aae2b81a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.365 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f8138d27b80 1 -- 172.21.15.107:0/226041711 >> 172.21.15.107:0/226041711 conn(0x561aae2a0ea0 msgr2=0x561aae2bcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.365 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f8138d27b80 1 -- 172.21.15.107:0/226041711 shutdown_connections 2023-07-21T22:28:59.365 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f8138d27b80 1 -- 172.21.15.107:0/226041711 wait complete. 2023-07-21T22:28:59.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f8138d27b80 20 client.15348 trim_cache size 0 max 16384 2023-07-21T22:28:59.366 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:28:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "af99f5d0-2814-11ee-9b34-001a4aab830c", 2023-07-21T22:28:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2023-07-21T22:28:59.366 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24533, 2023-07-21T22:28:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2023-07-21T22:28:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2023-07-21T22:28:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2023-07-21T22:28:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 77.233343306999998, 2023-07-21T22:28:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2023-07-21T22:28:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 84, 2023-07-21T22:28:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 82, 2023-07-21T22:28:59.367 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 83.341644643999999 2023-07-21T22:28:59.368 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:28:59.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f812cba2700 1 -- 172.21.15.107:0/3749112513 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81140a11e0 msgr2=0x7f81140a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f812cba2700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81140a11e0 0x7f81140a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8110005950 tx=0x7f81100058e0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f812cba2700 1 -- 172.21.15.107:0/3749112513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128100590 msgr2=0x7f812819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:59.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f812cba2700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128100590 0x7f812819cef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f812000b830 tx=0x7f8120004d30 comp rx=0 tx=0).stop 2023-07-21T22:28:59.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f812cba2700 1 -- 172.21.15.107:0/3749112513 shutdown_connections 2023-07-21T22:28:59.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f812cba2700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8128100590 0x7f812819cef0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.372 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f812cba2700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f81140a11e0 0x7f81140a3690 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.372 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f812cba2700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81280fea10 0x7f8128195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.372 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f812cba2700 1 --2- 172.21.15.107:0/3749112513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81280ffc00 0x7f8128195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:59.372 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f812cba2700 1 -- 172.21.15.107:0/3749112513 >> 172.21.15.107:0/3749112513 conn(0x7f81280fa030 msgr2=0x7f81280fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:59.372 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f812cba2700 1 -- 172.21.15.107:0/3749112513 shutdown_connections 2023-07-21T22:28:59.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:28:59.305+0000 7f812cba2700 1 -- 172.21.15.107:0/3749112513 wait complete. 2023-07-21T22:28:59.373 INFO:teuthology.run_tasks:Running task kclient... 2023-07-21T22:28:59.400 INFO:tasks.kclient:Mounting kernel clients... 2023-07-21T22:28:59.400 INFO:tasks.kclient:config is {'client.0': {}} 2023-07-21T22:28:59.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:28:59.401 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-21T22:28:59.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:28:59.418 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2023-07-21T22:28:59.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:28:59 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3094264248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:59.487 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2023-07-21T22:28:59.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:28:59.489 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-07-21T22:28:59.489 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-21T22:28:59.490 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-21T22:28:59.490 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph fs ls 2023-07-21T22:28:59.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:28:59 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3094264248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:28:59.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:28:59 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3094264248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:29:00.077 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:29:00.589 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:00 smithi107 ceph-mon[96543]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 82 B/s, 0 objects/s recovering 2023-07-21T22:29:00.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:00 smithi190 ceph-mon[98334]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 82 B/s, 0 objects/s recovering 2023-07-21T22:29:00.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:00 smithi132 ceph-mon[97863]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 82 B/s, 0 objects/s recovering 2023-07-21T22:29:01.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.248+0000 7f10a1f7d700 1 -- 172.21.15.107:0/2308032770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c0ffe00 msgr2=0x7f109c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:01.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.248+0000 7f10a1f7d700 1 --2- 172.21.15.107:0/2308032770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c0ffe00 0x7f109c1001d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f10900099c0 tx=0x7f1090009cd0 comp rx=0 tx=0).stop 2023-07-21T22:29:01.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.248+0000 7f10a1f7d700 1 -- 172.21.15.107:0/2308032770 shutdown_connections 2023-07-21T22:29:01.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.248+0000 7f10a1f7d700 1 --2- 172.21.15.107:0/2308032770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f109c104ea0 0x7f109c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:01.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.248+0000 7f10a1f7d700 1 --2- 172.21.15.107:0/2308032770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c0ffe00 0x7f109c1001d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:01.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.248+0000 7f10a1f7d700 1 --2- 172.21.15.107:0/2308032770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f109c1007a0 0x7f109c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:01.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.248+0000 7f10a1f7d700 1 -- 172.21.15.107:0/2308032770 >> 172.21.15.107:0/2308032770 conn(0x7f109c0fb660 msgr2=0x7f109c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:01.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.248+0000 7f10a1f7d700 1 -- 172.21.15.107:0/2308032770 shutdown_connections 2023-07-21T22:29:01.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.248+0000 7f10a1f7d700 1 -- 172.21.15.107:0/2308032770 wait complete. 2023-07-21T22:29:01.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.249+0000 7f10a1f7d700 1 Processor -- start 2023-07-21T22:29:01.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.249+0000 7f10a1f7d700 1 -- start start 2023-07-21T22:29:01.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.249+0000 7f10a1f7d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f109c0ffe00 0x7f109c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:01.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.249+0000 7f10a1f7d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1007a0 0x7f109c19bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:01.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.249+0000 7f10a1f7d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f109c104ea0 0x7f109c195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:01.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.249+0000 7f10a1f7d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c196090 con 0x7f109c1007a0 2023-07-21T22:29:01.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.249+0000 7f10a1f7d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c196200 con 0x7f109c104ea0 2023-07-21T22:29:01.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.249+0000 7f10a1f7d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c1963c0 con 0x7f109c0ffe00 2023-07-21T22:29:01.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.249+0000 7f109affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1007a0 0x7f109c19bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:01.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.249+0000 7f109affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1007a0 0x7f109c19bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47904/0 (socket says 172.21.15.107:47904) 2023-07-21T22:29:01.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.249+0000 7f109affd700 1 -- 172.21.15.107:0/3174979815 learned_addr learned my addr 172.21.15.107:0/3174979815 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:01.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.249+0000 7f109bfff700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f109c104ea0 0x7f109c195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:01.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.249+0000 7f109b7fe700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f109c0ffe00 0x7f109c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:01.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.250+0000 7f109affd700 1 -- 172.21.15.107:0/3174979815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f109c0ffe00 msgr2=0x7f109c19b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:01.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.250+0000 7f109affd700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f109c0ffe00 0x7f109c19b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.250+0000 7f109affd700 1 -- 172.21.15.107:0/3174979815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f109c104ea0 msgr2=0x7f109c195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.250+0000 7f109affd700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f109c104ea0 0x7f109c195a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.250+0000 7f109affd700 1 -- 172.21.15.107:0/3174979815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10900095f0 con 0x7f109c1007a0 2023-07-21T22:29:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.250+0000 7f109bfff700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f109c104ea0 0x7f109c195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.250+0000 7f109affd700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1007a0 0x7f109c19bee0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f108c00c9e0 tx=0x7f108c00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.250+0000 7f1098ff9700 1 -- 172.21.15.107:0/3174979815 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c007aa0 con 0x7f109c1007a0 2023-07-21T22:29:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.250+0000 7f10a1f7d700 1 -- 172.21.15.107:0/3174979815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c1966a0 con 0x7f109c1007a0 2023-07-21T22:29:01.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.250+0000 7f1098ff9700 1 -- 172.21.15.107:0/3174979815 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f108c007c00 con 0x7f109c1007a0 2023-07-21T22:29:01.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.250+0000 7f1098ff9700 1 -- 172.21.15.107:0/3174979815 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c018660 con 0x7f109c1007a0 2023-07-21T22:29:01.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.250+0000 7f10a1f7d700 1 -- 172.21.15.107:0/3174979815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c1a2990 con 0x7f109c1007a0 2023-07-21T22:29:01.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.251+0000 7f10a1f7d700 1 -- 172.21.15.107:0/3174979815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c04e210 con 0x7f109c1007a0 2023-07-21T22:29:01.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.252+0000 7f1098ff9700 1 -- 172.21.15.107:0/3174979815 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f108c00f450 con 0x7f109c1007a0 2023-07-21T22:29:01.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.252+0000 7f1098ff9700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10840a11e0 0x7f10840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:01.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.253+0000 7f109b7fe700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10840a11e0 0x7f10840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:01.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.253+0000 7f1098ff9700 1 -- 172.21.15.107:0/3174979815 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f108c0ccc70 con 0x7f109c1007a0 2023-07-21T22:29:01.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.253+0000 7f109b7fe700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10840a11e0 0x7f10840a3690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1090009fd0 tx=0x7f1090005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:01.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.256+0000 7f1098ff9700 1 -- 172.21.15.107:0/3174979815 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f108c096240 con 0x7f109c1007a0 2023-07-21T22:29:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.484+0000 7f10a1f7d700 1 -- 172.21.15.107:0/3174979815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f109c196890 con 0x7f109c1007a0 2023-07-21T22:29:01.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.484+0000 7f1098ff9700 1 -- 172.21.15.107:0/3174979815 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f109c196890 con 0x7f109c1007a0 2023-07-21T22:29:01.531 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-21T22:29:01.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.486+0000 7f10a1f7d700 1 -- 172.21.15.107:0/3174979815 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10840a11e0 msgr2=0x7f10840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:01.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.486+0000 7f10a1f7d700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10840a11e0 0x7f10840a3690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1090009fd0 tx=0x7f1090005fb0 comp rx=0 tx=0).stop 2023-07-21T22:29:01.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.486+0000 7f10a1f7d700 1 -- 172.21.15.107:0/3174979815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1007a0 msgr2=0x7f109c19bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:01.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.486+0000 7f10a1f7d700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1007a0 0x7f109c19bee0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f108c00c9e0 tx=0x7f108c00cda0 comp rx=0 tx=0).stop 2023-07-21T22:29:01.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.486+0000 7f10a1f7d700 1 -- 172.21.15.107:0/3174979815 shutdown_connections 2023-07-21T22:29:01.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.486+0000 7f10a1f7d700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f109c104ea0 0x7f109c195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:01.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.486+0000 7f10a1f7d700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10840a11e0 0x7f10840a3690 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:01.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.486+0000 7f10a1f7d700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f109c0ffe00 0x7f109c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:01.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.486+0000 7f10a1f7d700 1 --2- 172.21.15.107:0/3174979815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f109c1007a0 0x7f109c19bee0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:01.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.486+0000 7f10a1f7d700 1 -- 172.21.15.107:0/3174979815 >> 172.21.15.107:0/3174979815 conn(0x7f109c0fb660 msgr2=0x7f109c103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:01.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.486+0000 7f10a1f7d700 1 -- 172.21.15.107:0/3174979815 shutdown_connections 2023-07-21T22:29:01.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:01.486+0000 7f10a1f7d700 1 -- 172.21.15.107:0/3174979815 wait complete. 2023-07-21T22:29:01.918 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-21T22:29:01.918 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-21T22:29:01.918 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2023-07-21T22:29:01.918 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-21T22:29:01.919 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-21T22:29:01.919 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-21T22:29:01.919 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-21T22:29:01.919 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-21T22:29:01.919 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-21T22:29:01.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:01.920 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2023-07-21T22:29:01.938 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-21T22:29:01.938 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-21T22:29:01.938 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2023-07-21T22:29:01.938 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-21T22:29:01.938 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2023-07-21T22:29:01.938 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-21T22:29:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T22:29:01.939 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2023-07-21T22:29:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T22:29:01.939 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2023-07-21T22:29:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T22:29:01.939 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2023-07-21T22:29:01.939 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-21T22:29:01.940 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2023-07-21T22:29:01.940 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-21T22:29:01.940 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 41696sec preferred_lft 41696sec 2023-07-21T22:29:01.940 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::a7c:b29c:9445:8548/64 scope link noprefixroute 2023-07-21T22:29:01.940 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-21T22:29:01.940 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-21T22:29:01.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:01.941 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-21T22:29:01.941 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-21T22:29:01.942 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2023-07-21T22:29:01.942 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2023-07-21T22:29:01.942 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2023-07-21T22:29:01.942 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-21T22:29:01.942 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-21T22:29:02.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:02.110 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-21T22:29:02.139 INFO:teuthology.orchestra.run.smithi107.stdout:1 2023-07-21T22:29:02.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:02.141 DEBUG:teuthology.orchestra.run.smithi107:> route 2023-07-21T22:29:02.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:02 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3174979815' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T22:29:02.211 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2023-07-21T22:29:02.211 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-21T22:29:02.211 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-21T22:29:02.211 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-21T22:29:02.212 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-21T22:29:02.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:02.212 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-21T22:29:02.213 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-21T22:29:02.213 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-21T22:29:02.213 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-21T22:29:02.213 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-21T22:29:02.214 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-21T22:29:02.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:02 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3174979815' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T22:29:02.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:02 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3174979815' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T22:29:02.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:02.507 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-21T22:29:02.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:02.524 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2023-07-21T22:29:02.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:02.580 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-21T22:29:02.581 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-21T22:29:02.581 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-21T22:29:02.581 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-21T22:29:02.581 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-21T22:29:02.674 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-21T22:29:02.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:02.674 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-21T22:29:02.675 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-21T22:29:02.675 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-21T22:29:02.675 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-21T22:29:02.675 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-21T22:29:02.676 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-21T22:29:02.676 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-21T22:29:02.676 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-21T22:29:02.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:02.873 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-21T22:29:02.873 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-21T22:29:02.874 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2023-07-21T22:29:02.874 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2023-07-21T22:29:02.874 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-21T22:29:02.972 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-21T22:29:02.972 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-21T22:29:02.989 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-21T22:29:02.989 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-21T22:29:02.990 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-21T22:29:03.045 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-07-21T22:29:03.046 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-07-21T22:29:03.046 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2 2023-07-21T22:29:03.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:03.047 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-07-21T22:29:03.140 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-07-21T22:29:03.141 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2023-07-21T22:29:03.175 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:03 smithi107 ceph-mon[96543]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 82 B/s, 0 objects/s recovering 2023-07-21T22:29:03.256 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-07-21T22:29:03.256 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2023-07-21T22:29:03.256 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.107:6789,172.21.15.132:6789,172.21.15.190:6789" 2023-07-21T22:29:03.257 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with new device syntax: 0@af99f5d0-2814-11ee-9b34-001a4aab830c.cephfs=/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2 2023-07-21T22:29:03.257 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.107:6789/172.21.15.132:6789/172.21.15.190:6789" will pass to kernel 2023-07-21T22:29:03.257 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: v2 mount support check returned 2 2023-07-21T22:29:03.258 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-21T22:29:03.258 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.107:6789,172.21.15.132:6789,172.21.15.190:6789:/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2 2023-07-21T22:29:03.258 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mds_namespace=cephfs,fsid=af99f5d0-2814-11ee-9b34-001a4aab830c" will pass to kernel 2023-07-21T22:29:03.260 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-21T22:29:03.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:03.261 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-21T22:29:03.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:03.322 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-21T22:29:03.376 INFO:teuthology.orchestra.run.smithi107.stdout:client15357 2023-07-21T22:29:03.377 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:29:03.377 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:29:03.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:03 smithi190 ceph-mon[98334]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 82 B/s, 0 objects/s recovering 2023-07-21T22:29:03.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:03.396 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-21T22:29:03.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:03 smithi132 ceph-mon[97863]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s; 82 B/s, 0 objects/s recovering 2023-07-21T22:29:03.488 INFO:teuthology.orchestra.run.smithi107.stdout:client15357 2023-07-21T22:29:03.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:03.489 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-21T22:29:03.541 INFO:teuthology.orchestra.run.smithi107.stdout:client15357 2023-07-21T22:29:03.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:29:03.542 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c.client15357/status') 2023-07-21T22:29:03.599 INFO:teuthology.orchestra.run.smithi107.stdout:instance: client.15357 (3)192.168.0.1:0/2182681335 2023-07-21T22:29:03.599 INFO:teuthology.orchestra.run.smithi107.stdout:blocklisted: false 2023-07-21T22:29:03.600 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records in 2023-07-21T22:29:03.600 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records out 2023-07-21T22:29:03.600 INFO:teuthology.orchestra.run.smithi107.stderr:70 bytes copied, 2.2262e-05 s, 3.1 MB/s 2023-07-21T22:29:03.601 INFO:teuthology.run_tasks:Running task check-counter... 2023-07-21T22:29:03.612 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-07-21T22:29:03.623 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:29:03.745 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.699+0000 7f9b9749e700 1 -- 172.21.15.107:0/905181746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b90105790 msgr2=0x7f9b90107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:03.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.699+0000 7f9b9749e700 1 --2- 172.21.15.107:0/905181746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b90105790 0x7f9b90107bf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800b0a0 tx=0x7f9b8800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:03.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.699+0000 7f9b9749e700 1 -- 172.21.15.107:0/905181746 shutdown_connections 2023-07-21T22:29:03.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.699+0000 7f9b9749e700 1 --2- 172.21.15.107:0/905181746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b90105790 0x7f9b90107bf0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:03.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.699+0000 7f9b9749e700 1 --2- 172.21.15.107:0/905181746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b90100550 0x7f9b90102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:03.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.699+0000 7f9b9749e700 1 --2- 172.21.15.107:0/905181746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b90102e70 0x7f9b90105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:03.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.699+0000 7f9b9749e700 1 -- 172.21.15.107:0/905181746 >> 172.21.15.107:0/905181746 conn(0x7f9b900f9fd0 msgr2=0x7f9b900fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:03.747 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.699+0000 7f9b9749e700 1 -- 172.21.15.107:0/905181746 shutdown_connections 2023-07-21T22:29:03.747 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.699+0000 7f9b9749e700 1 -- 172.21.15.107:0/905181746 wait complete. 2023-07-21T22:29:03.748 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.699+0000 7f9b9749e700 1 Processor -- start 2023-07-21T22:29:03.749 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.699+0000 7f9b9749e700 1 -- start start 2023-07-21T22:29:03.749 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9749e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b90100550 0x7f9b90195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:03.749 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9749e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b90102e70 0x7f9b90195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:03.749 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9749e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b90105790 0x7f9b9019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:03.749 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9749e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9019d3e0 con 0x7f9b90100550 2023-07-21T22:29:03.749 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9749e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9019d520 con 0x7f9b90105790 2023-07-21T22:29:03.749 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9749e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9019d6b0 con 0x7f9b90102e70 2023-07-21T22:29:03.750 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9523a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b90100550 0x7f9b90195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:03.750 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9523a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b90100550 0x7f9b90195860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47924/0 (socket says 172.21.15.107:47924) 2023-07-21T22:29:03.750 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9523a700 1 -- 172.21.15.107:0/3098761341 learned_addr learned my addr 172.21.15.107:0/3098761341 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:03.750 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9523a700 1 -- 172.21.15.107:0/3098761341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b90102e70 msgr2=0x7f9b90195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:03.750 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9523a700 1 --2- 172.21.15.107:0/3098761341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b90102e70 0x7f9b90195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:03.750 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9523a700 1 -- 172.21.15.107:0/3098761341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b90105790 msgr2=0x7f9b9019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:03.750 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9523a700 1 --2- 172.21.15.107:0/3098761341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b90105790 0x7f9b9019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:03.750 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9523a700 1 -- 172.21.15.107:0/3098761341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b88009d00 con 0x7f9b90100550 2023-07-21T22:29:03.751 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9523a700 1 --2- 172.21.15.107:0/3098761341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b90100550 0x7f9b90195860 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00c920 tx=0x7f9b8c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:03.751 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b867fc700 1 -- 172.21.15.107:0/3098761341 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c007b00 con 0x7f9b90100550 2023-07-21T22:29:03.751 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b867fc700 1 -- 172.21.15.107:0/3098761341 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b8c004cf0 con 0x7f9b90100550 2023-07-21T22:29:03.751 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9749e700 1 -- 172.21.15.107:0/3098761341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9019d990 con 0x7f9b90100550 2023-07-21T22:29:03.751 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b867fc700 1 -- 172.21.15.107:0/3098761341 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c005610 con 0x7f9b90100550 2023-07-21T22:29:03.751 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.700+0000 7f9b9749e700 1 -- 172.21.15.107:0/3098761341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9019dea0 con 0x7f9b90100550 2023-07-21T22:29:03.751 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.701+0000 7f9b9749e700 1 -- 172.21.15.107:0/3098761341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b900fbb40 con 0x7f9b90100550 2023-07-21T22:29:03.752 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.702+0000 7f9b867fc700 1 -- 172.21.15.107:0/3098761341 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f9b8c004750 con 0x7f9b90100550 2023-07-21T22:29:03.752 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.703+0000 7f9b867fc700 1 --2- 172.21.15.107:0/3098761341 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9b7c0a1220 0x7f9b7c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:03.753 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.703+0000 7f9b94a39700 1 --2- 172.21.15.107:0/3098761341 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9b7c0a1220 0x7f9b7c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:03.753 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.703+0000 7f9b867fc700 1 -- 172.21.15.107:0/3098761341 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9b8c0ccb10 con 0x7f9b90100550 2023-07-21T22:29:03.753 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.703+0000 7f9b94a39700 1 --2- 172.21.15.107:0/3098761341 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9b7c0a1220 0x7f9b7c0a36d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000a990 tx=0x7f9b80005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:03.753 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.707+0000 7f9b867fc700 1 -- 172.21.15.107:0/3098761341 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9b8c096080 con 0x7f9b90100550 2023-07-21T22:29:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.897+0000 7f9b9749e700 1 -- 172.21.15.107:0/3098761341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b9002cb00 con 0x7f9b90100550 2023-07-21T22:29:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.899+0000 7f9b867fc700 1 -- 172.21.15.107:0/3098761341 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22727 (secure 0 0 0) 0x7f9b8c01d020 con 0x7f9b90100550 2023-07-21T22:29:03.946 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:29:03.946 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":84,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:28:47.235779+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:29:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.901+0000 7f9b9749e700 1 -- 172.21.15.107:0/3098761341 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9b7c0a1220 msgr2=0x7f9b7c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.901+0000 7f9b9749e700 1 --2- 172.21.15.107:0/3098761341 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9b7c0a1220 0x7f9b7c0a36d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000a990 tx=0x7f9b80005c80 comp rx=0 tx=0).stop 2023-07-21T22:29:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.901+0000 7f9b9749e700 1 -- 172.21.15.107:0/3098761341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b90100550 msgr2=0x7f9b90195860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.901+0000 7f9b9749e700 1 --2- 172.21.15.107:0/3098761341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b90100550 0x7f9b90195860 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00c920 tx=0x7f9b8c00cce0 comp rx=0 tx=0).stop 2023-07-21T22:29:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.901+0000 7f9b9749e700 1 -- 172.21.15.107:0/3098761341 shutdown_connections 2023-07-21T22:29:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.901+0000 7f9b9749e700 1 --2- 172.21.15.107:0/3098761341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b90105790 0x7f9b9019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.901+0000 7f9b9749e700 1 --2- 172.21.15.107:0/3098761341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b90100550 0x7f9b90195860 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.901+0000 7f9b9749e700 1 --2- 172.21.15.107:0/3098761341 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9b7c0a1220 0x7f9b7c0a36d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.901+0000 7f9b9749e700 1 --2- 172.21.15.107:0/3098761341 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b90102e70 0x7f9b90195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.901+0000 7f9b9749e700 1 -- 172.21.15.107:0/3098761341 >> 172.21.15.107:0/3098761341 conn(0x7f9b900f9fd0 msgr2=0x7f9b901011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.901+0000 7f9b9749e700 1 -- 172.21.15.107:0/3098761341 shutdown_connections 2023-07-21T22:29:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:03.901+0000 7f9b9749e700 1 -- 172.21.15.107:0/3098761341 wait complete. 2023-07-21T22:29:03.960 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:23:52.031819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:27:39.494209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:27:40.672303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:29:03.961 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:29:04.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.038+0000 7f82fa23d700 1 -- 172.21.15.107:0/1596506574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f41005c0 msgr2=0x7f82f41029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.038+0000 7f82fa23d700 1 --2- 172.21.15.107:0/1596506574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f41005c0 0x7f82f41029a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009990 tx=0x7f82e4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.038+0000 7f82fa23d700 1 -- 172.21.15.107:0/1596506574 shutdown_connections 2023-07-21T22:29:04.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.038+0000 7f82fa23d700 1 --2- 172.21.15.107:0/1596506574 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f4105800 0x7f82f4107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.038+0000 7f82fa23d700 1 --2- 172.21.15.107:0/1596506574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f41005c0 0x7f82f41029a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.038+0000 7f82fa23d700 1 --2- 172.21.15.107:0/1596506574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82f4102ee0 0x7f82f41052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.038+0000 7f82fa23d700 1 -- 172.21.15.107:0/1596506574 >> 172.21.15.107:0/1596506574 conn(0x7f82f40fa000 msgr2=0x7f82f40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:04.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.038+0000 7f82fa23d700 1 -- 172.21.15.107:0/1596506574 shutdown_connections 2023-07-21T22:29:04.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.038+0000 7f82fa23d700 1 -- 172.21.15.107:0/1596506574 wait complete. 2023-07-21T22:29:04.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.038+0000 7f82fa23d700 1 Processor -- start 2023-07-21T22:29:04.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82fa23d700 1 -- start start 2023-07-21T22:29:04.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82fa23d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f41005c0 0x7f82f4195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:04.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82fa23d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f4102ee0 0x7f82f4195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:04.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82fa23d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82f4105800 0x7f82f419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:04.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82fa23d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f419d390 con 0x7f82f4102ee0 2023-07-21T22:29:04.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82fa23d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f419d4d0 con 0x7f82f4105800 2023-07-21T22:29:04.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82fa23d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f419d660 con 0x7f82f41005c0 2023-07-21T22:29:04.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f4102ee0 0x7f82f4195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:04.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f4102ee0 0x7f82f4195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47940/0 (socket says 172.21.15.107:47940) 2023-07-21T22:29:04.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f2ffd700 1 -- 172.21.15.107:0/978677130 learned_addr learned my addr 172.21.15.107:0/978677130 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:04.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f3fff700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82f4105800 0x7f82f419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:04.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f37fe700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f41005c0 0x7f82f4195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:04.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f2ffd700 1 -- 172.21.15.107:0/978677130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f41005c0 msgr2=0x7f82f4195810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f2ffd700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f41005c0 0x7f82f4195810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f2ffd700 1 -- 172.21.15.107:0/978677130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82f4105800 msgr2=0x7f82f419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f2ffd700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82f4105800 0x7f82f419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f2ffd700 1 -- 172.21.15.107:0/978677130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e40095f0 con 0x7f82f4102ee0 2023-07-21T22:29:04.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f3fff700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82f4105800 0x7f82f419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:04.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f37fe700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f41005c0 0x7f82f4195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:04.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f2ffd700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f4102ee0 0x7f82f4195d50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f82e800e9a0 tx=0x7f82e800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:04.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.039+0000 7f82f0ff9700 1 -- 172.21.15.107:0/978677130 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8009790 con 0x7f82f4102ee0 2023-07-21T22:29:04.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.040+0000 7f82f0ff9700 1 -- 172.21.15.107:0/978677130 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82e800cbd0 con 0x7f82f4102ee0 2023-07-21T22:29:04.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.040+0000 7f82fa23d700 1 -- 172.21.15.107:0/978677130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f419d940 con 0x7f82f4102ee0 2023-07-21T22:29:04.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.040+0000 7f82f0ff9700 1 -- 172.21.15.107:0/978677130 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8018680 con 0x7f82f4102ee0 2023-07-21T22:29:04.098 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.040+0000 7f82fa23d700 1 -- 172.21.15.107:0/978677130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f419de50 con 0x7f82f4102ee0 2023-07-21T22:29:04.098 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.040+0000 7f82fa23d700 1 -- 172.21.15.107:0/978677130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f40fbb70 con 0x7f82f4102ee0 2023-07-21T22:29:04.099 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.042+0000 7f82f0ff9700 1 -- 172.21.15.107:0/978677130 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f82e800cd40 con 0x7f82f4102ee0 2023-07-21T22:29:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.042+0000 7f82f0ff9700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f82dc0a11e0 0x7f82dc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:04.100 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.042+0000 7f82f37fe700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f82dc0a11e0 0x7f82dc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.042+0000 7f82f0ff9700 1 -- 172.21.15.107:0/978677130 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f82e8014070 con 0x7f82f4102ee0 2023-07-21T22:29:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.043+0000 7f82f37fe700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f82dc0a11e0 0x7f82dc0a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f82e400b2e0 tx=0x7f82e4005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:04.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.048+0000 7f82f0ff9700 1 -- 172.21.15.107:0/978677130 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f82e8096880 con 0x7f82f4102ee0 2023-07-21T22:29:04.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.236+0000 7f82fa23d700 1 -- 172.21.15.107:0/978677130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82f402ca60 con 0x7f82f4102ee0 2023-07-21T22:29:04.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.237+0000 7f82f0ff9700 1 -- 172.21.15.107:0/978677130 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82e8099ea0 con 0x7f82f4102ee0 2023-07-21T22:29:04.283 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2023-07-21T22:29:04.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.238+0000 7f82fa23d700 1 -- 172.21.15.107:0/978677130 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f82dc0a11e0 msgr2=0x7f82dc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.238+0000 7f82fa23d700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f82dc0a11e0 0x7f82dc0a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f82e400b2e0 tx=0x7f82e4005e80 comp rx=0 tx=0).stop 2023-07-21T22:29:04.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.239+0000 7f82fa23d700 1 -- 172.21.15.107:0/978677130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f4102ee0 msgr2=0x7f82f4195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.239+0000 7f82fa23d700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f4102ee0 0x7f82f4195d50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f82e800e9a0 tx=0x7f82e800ed60 comp rx=0 tx=0).stop 2023-07-21T22:29:04.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.239+0000 7f82fa23d700 1 -- 172.21.15.107:0/978677130 shutdown_connections 2023-07-21T22:29:04.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.239+0000 7f82fa23d700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82f4105800 0x7f82f419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.239+0000 7f82fa23d700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f82dc0a11e0 0x7f82dc0a3690 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.239+0000 7f82fa23d700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82f41005c0 0x7f82f4195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.239+0000 7f82fa23d700 1 --2- 172.21.15.107:0/978677130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f4102ee0 0x7f82f4195d50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.239+0000 7f82fa23d700 1 -- 172.21.15.107:0/978677130 >> 172.21.15.107:0/978677130 conn(0x7f82f40fa000 msgr2=0x7f82f4101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:04.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.239+0000 7f82fa23d700 1 -- 172.21.15.107:0/978677130 shutdown_connections 2023-07-21T22:29:04.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.239+0000 7f82fa23d700 1 -- 172.21.15.107:0/978677130 wait complete. 2023-07-21T22:29:04.296 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:04 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3098761341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:29:04.297 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:29:04.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:04 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3098761341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:29:04.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.374+0000 7fd18965e700 1 -- 172.21.15.107:0/346451071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1841016d0 msgr2=0x7fd184107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.374+0000 7fd18965e700 1 --2- 172.21.15.107:0/346451071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1841016d0 0x7fd184107f90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00b0a0 tx=0x7fd17c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.374+0000 7fd18965e700 1 -- 172.21.15.107:0/346451071 shutdown_connections 2023-07-21T22:29:04.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.374+0000 7fd18965e700 1 --2- 172.21.15.107:0/346451071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1841016d0 0x7fd184107f90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.374+0000 7fd18965e700 1 --2- 172.21.15.107:0/346451071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1840ffb50 0x7fd1840fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.374+0000 7fd18965e700 1 --2- 172.21.15.107:0/346451071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd184100d40 0x7fd184101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.374+0000 7fd18965e700 1 -- 172.21.15.107:0/346451071 >> 172.21.15.107:0/346451071 conn(0x7fd1840fb170 msgr2=0x7fd1840fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:04.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.374+0000 7fd18965e700 1 -- 172.21.15.107:0/346451071 shutdown_connections 2023-07-21T22:29:04.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.374+0000 7fd18965e700 1 -- 172.21.15.107:0/346451071 wait complete. 2023-07-21T22:29:04.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.375+0000 7fd18965e700 1 Processor -- start 2023-07-21T22:29:04.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.375+0000 7fd18965e700 1 -- start start 2023-07-21T22:29:04.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.375+0000 7fd18965e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1840ffb50 0x7fd1841958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:04.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.375+0000 7fd18965e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd184100d40 0x7fd184195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:04.425 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.375+0000 7fd18965e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1841016d0 0x7fd18419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:04.425 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.375+0000 7fd18965e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18419d460 con 0x7fd1841016d0 2023-07-21T22:29:04.425 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.375+0000 7fd18965e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18419d5a0 con 0x7fd1840ffb50 2023-07-21T22:29:04.425 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.375+0000 7fd18965e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18419d730 con 0x7fd184100d40 2023-07-21T22:29:04.425 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.375+0000 7fd1837fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1841016d0 0x7fd18419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:04.425 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.375+0000 7fd1837fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1841016d0 0x7fd18419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47956/0 (socket says 172.21.15.107:47956) 2023-07-21T22:29:04.425 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.375+0000 7fd1837fe700 1 -- 172.21.15.107:0/3375234648 learned_addr learned my addr 172.21.15.107:0/3375234648 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:04.425 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.375+0000 7fd182ffd700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1840ffb50 0x7fd1841958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.376+0000 7fd1827fc700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd184100d40 0x7fd184195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.376+0000 7fd1837fe700 1 -- 172.21.15.107:0/3375234648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd184100d40 msgr2=0x7fd184195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.376+0000 7fd1837fe700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd184100d40 0x7fd184195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.376+0000 7fd1837fe700 1 -- 172.21.15.107:0/3375234648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1840ffb50 msgr2=0x7fd1841958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.376+0000 7fd1837fe700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1840ffb50 0x7fd1841958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.376+0000 7fd1837fe700 1 -- 172.21.15.107:0/3375234648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c009d00 con 0x7fd1841016d0 2023-07-21T22:29:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.376+0000 7fd1837fe700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1841016d0 0x7fd18419ce90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00b780 tx=0x7fd17c00bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:04.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.376+0000 7fd16bfff700 1 -- 172.21.15.107:0/3375234648 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00e040 con 0x7fd1841016d0 2023-07-21T22:29:04.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.376+0000 7fd18965e700 1 -- 172.21.15.107:0/3375234648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18419d9b0 con 0x7fd1841016d0 2023-07-21T22:29:04.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.376+0000 7fd16bfff700 1 -- 172.21.15.107:0/3375234648 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd17c004e60 con 0x7fd1841016d0 2023-07-21T22:29:04.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.376+0000 7fd16bfff700 1 -- 172.21.15.107:0/3375234648 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c01b740 con 0x7fd1841016d0 2023-07-21T22:29:04.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.376+0000 7fd18965e700 1 -- 172.21.15.107:0/3375234648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18419de30 con 0x7fd1841016d0 2023-07-21T22:29:04.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.377+0000 7fd18965e700 1 -- 172.21.15.107:0/3375234648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18410b2c0 con 0x7fd1841016d0 2023-07-21T22:29:04.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.378+0000 7fd16bfff700 1 -- 172.21.15.107:0/3375234648 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd17c01da40 con 0x7fd1841016d0 2023-07-21T22:29:04.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.378+0000 7fd16bfff700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd16c0a11e0 0x7fd16c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:04.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.379+0000 7fd16bfff700 1 -- 172.21.15.107:0/3375234648 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd17c0cec90 con 0x7fd1841016d0 2023-07-21T22:29:04.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.379+0000 7fd182ffd700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd16c0a11e0 0x7fd16c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:04.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.379+0000 7fd182ffd700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd16c0a11e0 0x7fd16c0a3690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd184100ba0 tx=0x7fd174009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:04.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.382+0000 7fd16bfff700 1 -- 172.21.15.107:0/3375234648 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd17c0982f0 con 0x7fd1841016d0 2023-07-21T22:29:04.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:04 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3098761341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:29:04.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.569+0000 7fd18965e700 1 -- 172.21.15.107:0/3375234648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd184106980 con 0x7fd1841016d0 2023-07-21T22:29:04.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.569+0000 7fd16bfff700 1 -- 172.21.15.107:0/3375234648 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd17c09b910 con 0x7fd1841016d0 2023-07-21T22:29:04.616 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2023-07-21T22:29:04.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.571+0000 7fd18965e700 1 -- 172.21.15.107:0/3375234648 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd16c0a11e0 msgr2=0x7fd16c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.571+0000 7fd18965e700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd16c0a11e0 0x7fd16c0a3690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd184100ba0 tx=0x7fd174009260 comp rx=0 tx=0).stop 2023-07-21T22:29:04.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.571+0000 7fd18965e700 1 -- 172.21.15.107:0/3375234648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1841016d0 msgr2=0x7fd18419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.571+0000 7fd18965e700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1841016d0 0x7fd18419ce90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00b780 tx=0x7fd17c00bd00 comp rx=0 tx=0).stop 2023-07-21T22:29:04.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.571+0000 7fd18965e700 1 -- 172.21.15.107:0/3375234648 shutdown_connections 2023-07-21T22:29:04.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.571+0000 7fd18965e700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1841016d0 0x7fd18419ce90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.571+0000 7fd18965e700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1840ffb50 0x7fd1841958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.571+0000 7fd18965e700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd16c0a11e0 0x7fd16c0a3690 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.571+0000 7fd18965e700 1 --2- 172.21.15.107:0/3375234648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd184100d40 0x7fd184195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.571+0000 7fd18965e700 1 -- 172.21.15.107:0/3375234648 >> 172.21.15.107:0/3375234648 conn(0x7fd1840fb170 msgr2=0x7fd1840fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:04.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.571+0000 7fd18965e700 1 -- 172.21.15.107:0/3375234648 shutdown_connections 2023-07-21T22:29:04.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.571+0000 7fd18965e700 1 -- 172.21.15.107:0/3375234648 wait complete. 2023-07-21T22:29:04.630 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:29:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.708+0000 7f91d3293700 1 -- 172.21.15.107:0/3628468236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc0fe9e0 msgr2=0x7f91cc0fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.708+0000 7f91d3293700 1 --2- 172.21.15.107:0/3628468236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc0fe9e0 0x7f91cc0fedf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009990 tx=0x7f91bc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.708+0000 7f91d3293700 1 -- 172.21.15.107:0/3628468236 shutdown_connections 2023-07-21T22:29:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.708+0000 7f91d3293700 1 --2- 172.21.15.107:0/3628468236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91cc100560 0x7f91cc106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.708+0000 7f91d3293700 1 --2- 172.21.15.107:0/3628468236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc0fe9e0 0x7f91cc0fedf0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.708+0000 7f91d3293700 1 --2- 172.21.15.107:0/3628468236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91cc0ffbd0 0x7f91cc100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.708+0000 7f91d3293700 1 -- 172.21.15.107:0/3628468236 >> 172.21.15.107:0/3628468236 conn(0x7f91cc0fa000 msgr2=0x7f91cc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.708+0000 7f91d3293700 1 -- 172.21.15.107:0/3628468236 shutdown_connections 2023-07-21T22:29:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.708+0000 7f91d3293700 1 -- 172.21.15.107:0/3628468236 wait complete. 2023-07-21T22:29:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.709+0000 7f91d3293700 1 Processor -- start 2023-07-21T22:29:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.709+0000 7f91d3293700 1 -- start start 2023-07-21T22:29:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.709+0000 7f91d3293700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91cc0fe9e0 0x7f91cc068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.709+0000 7f91d3293700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc0ffbd0 0x7f91cc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.709+0000 7f91d3293700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91cc100560 0x7f91cc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.709+0000 7f91d3293700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc19d430 con 0x7f91cc0ffbd0 2023-07-21T22:29:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.709+0000 7f91d3293700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc19d5a0 con 0x7f91cc0fe9e0 2023-07-21T22:29:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.709+0000 7f91d3293700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc19d760 con 0x7f91cc100560 2023-07-21T22:29:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.709+0000 7f91d082e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc0ffbd0 0x7f91cc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.709+0000 7f91d082e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc0ffbd0 0x7f91cc19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47968/0 (socket says 172.21.15.107:47968) 2023-07-21T22:29:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.709+0000 7f91d082e700 1 -- 172.21.15.107:0/1057380757 learned_addr learned my addr 172.21.15.107:0/1057380757 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.709+0000 7f91d102f700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91cc0fe9e0 0x7f91cc068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.710+0000 7f91d1830700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91cc100560 0x7f91cc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.710+0000 7f91d082e700 1 -- 172.21.15.107:0/1057380757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91cc100560 msgr2=0x7f91cc19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.710+0000 7f91d082e700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91cc100560 0x7f91cc19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.710+0000 7f91d082e700 1 -- 172.21.15.107:0/1057380757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91cc0fe9e0 msgr2=0x7f91cc068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.710+0000 7f91d082e700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91cc0fe9e0 0x7f91cc068200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.710+0000 7f91d082e700 1 -- 172.21.15.107:0/1057380757 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc0095f0 con 0x7f91cc0ffbd0 2023-07-21T22:29:04.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.710+0000 7f91d082e700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc0ffbd0 0x7f91cc19a610 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f91c800b530 tx=0x7f91c800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:04.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.710+0000 7f91c27fc700 1 -- 172.21.15.107:0/1057380757 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8011830 con 0x7f91cc0ffbd0 2023-07-21T22:29:04.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.710+0000 7f91c27fc700 1 -- 172.21.15.107:0/1057380757 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91c8011e50 con 0x7f91cc0ffbd0 2023-07-21T22:29:04.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.710+0000 7f91d3293700 1 -- 172.21.15.107:0/1057380757 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc19da40 con 0x7f91cc0ffbd0 2023-07-21T22:29:04.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.710+0000 7f91c27fc700 1 -- 172.21.15.107:0/1057380757 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800f2e0 con 0x7f91cc0ffbd0 2023-07-21T22:29:04.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.710+0000 7f91d3293700 1 -- 172.21.15.107:0/1057380757 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc19dfa0 con 0x7f91cc0ffbd0 2023-07-21T22:29:04.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.711+0000 7f91d3293700 1 -- 172.21.15.107:0/1057380757 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc105960 con 0x7f91cc0ffbd0 2023-07-21T22:29:04.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.712+0000 7f91c27fc700 1 -- 172.21.15.107:0/1057380757 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f91c8010380 con 0x7f91cc0ffbd0 2023-07-21T22:29:04.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.712+0000 7f91c27fc700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91b80a1260 0x7f91b80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:04.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.713+0000 7f91c27fc700 1 -- 172.21.15.107:0/1057380757 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f91c80cc460 con 0x7f91cc0ffbd0 2023-07-21T22:29:04.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.713+0000 7f91d102f700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91b80a1260 0x7f91b80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:04.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.713+0000 7f91d102f700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91b80a1260 0x7f91b80a3710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00b5c0 tx=0x7f91bc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:04.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.717+0000 7f91c27fc700 1 -- 172.21.15.107:0/1057380757 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f91c8095b40 con 0x7f91cc0ffbd0 2023-07-21T22:29:04.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.906+0000 7f91d3293700 1 -- 172.21.15.107:0/1057380757 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91cc02c850 con 0x7f91cc0ffbd0 2023-07-21T22:29:04.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.906+0000 7f91c27fc700 1 -- 172.21.15.107:0/1057380757 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91c8099160 con 0x7f91cc0ffbd0 2023-07-21T22:29:04.953 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2023-07-21T22:29:04.955 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.908+0000 7f91d3293700 1 -- 172.21.15.107:0/1057380757 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91b80a1260 msgr2=0x7f91b80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.955 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.908+0000 7f91d3293700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91b80a1260 0x7f91b80a3710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00b5c0 tx=0x7f91bc005960 comp rx=0 tx=0).stop 2023-07-21T22:29:04.955 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.908+0000 7f91d3293700 1 -- 172.21.15.107:0/1057380757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc0ffbd0 msgr2=0x7f91cc19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:04.955 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.908+0000 7f91d3293700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc0ffbd0 0x7f91cc19a610 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f91c800b530 tx=0x7f91c800b840 comp rx=0 tx=0).stop 2023-07-21T22:29:04.956 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.908+0000 7f91d3293700 1 -- 172.21.15.107:0/1057380757 shutdown_connections 2023-07-21T22:29:04.956 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.908+0000 7f91d3293700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f91b80a1260 0x7f91b80a3710 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.956 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.908+0000 7f91d3293700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f91cc100560 0x7f91cc19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.956 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.908+0000 7f91d3293700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91cc0fe9e0 0x7f91cc068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.956 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.908+0000 7f91d3293700 1 --2- 172.21.15.107:0/1057380757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc0ffbd0 0x7f91cc19a610 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:04.956 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.908+0000 7f91d3293700 1 -- 172.21.15.107:0/1057380757 >> 172.21.15.107:0/1057380757 conn(0x7f91cc0fa000 msgr2=0x7f91cc0fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:04.956 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.909+0000 7f91d3293700 1 -- 172.21.15.107:0/1057380757 shutdown_connections 2023-07-21T22:29:04.957 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:04.909+0000 7f91d3293700 1 -- 172.21.15.107:0/1057380757 wait complete. 2023-07-21T22:29:04.967 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:29:05.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.042+0000 7fd460ca2700 1 -- 172.21.15.107:0/2648535105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd45c105820 msgr2=0x7fd45c107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.042+0000 7fd460ca2700 1 --2- 172.21.15.107:0/2648535105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd45c105820 0x7fd45c107c20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd45400b0a0 tx=0x7fd45400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.042+0000 7fd460ca2700 1 -- 172.21.15.107:0/2648535105 shutdown_connections 2023-07-21T22:29:05.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.042+0000 7fd460ca2700 1 --2- 172.21.15.107:0/2648535105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd45c105820 0x7fd45c107c20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.042+0000 7fd460ca2700 1 --2- 172.21.15.107:0/2648535105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd45c1005e0 0x7fd45c1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.042+0000 7fd460ca2700 1 --2- 172.21.15.107:0/2648535105 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd45c102f00 0x7fd45c1052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.042+0000 7fd460ca2700 1 -- 172.21.15.107:0/2648535105 >> 172.21.15.107:0/2648535105 conn(0x7fd45c0f9fe0 msgr2=0x7fd45c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:05.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.042+0000 7fd460ca2700 1 -- 172.21.15.107:0/2648535105 shutdown_connections 2023-07-21T22:29:05.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.042+0000 7fd460ca2700 1 -- 172.21.15.107:0/2648535105 wait complete. 2023-07-21T22:29:05.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd460ca2700 1 Processor -- start 2023-07-21T22:29:05.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd460ca2700 1 -- start start 2023-07-21T22:29:05.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd460ca2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd45c1005e0 0x7fd45c0fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:05.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd460ca2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd45c102f00 0x7fd45c0fe590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:05.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd460ca2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd45c105820 0x7fd45c0fead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:05.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd460ca2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c0ff270 con 0x7fd45c1005e0 2023-07-21T22:29:05.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd460ca2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c0ff3e0 con 0x7fd45c102f00 2023-07-21T22:29:05.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd460ca2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c19f8d0 con 0x7fd45c105820 2023-07-21T22:29:05.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd45a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd45c1005e0 0x7fd45c0fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:05.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd45a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd45c1005e0 0x7fd45c0fff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47980/0 (socket says 172.21.15.107:47980) 2023-07-21T22:29:05.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd45a59c700 1 -- 172.21.15.107:0/196634754 learned_addr learned my addr 172.21.15.107:0/196634754 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:05.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd459d9b700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd45c102f00 0x7fd45c0fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:05.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd45ad9d700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd45c105820 0x7fd45c0fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:05.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd45a59c700 1 -- 172.21.15.107:0/196634754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd45c105820 msgr2=0x7fd45c0fead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd45a59c700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd45c105820 0x7fd45c0fead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd45a59c700 1 -- 172.21.15.107:0/196634754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd45c102f00 msgr2=0x7fd45c0fe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd45a59c700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd45c102f00 0x7fd45c0fe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd45a59c700 1 -- 172.21.15.107:0/196634754 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454009d00 con 0x7fd45c1005e0 2023-07-21T22:29:05.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd45a59c700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd45c1005e0 0x7fd45c0fff50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e9a0 tx=0x7fd45000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:05.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd44b7fe700 1 -- 172.21.15.107:0/196634754 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450009790 con 0x7fd45c1005e0 2023-07-21T22:29:05.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd44b7fe700 1 -- 172.21.15.107:0/196634754 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd450010430 con 0x7fd45c1005e0 2023-07-21T22:29:05.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd460ca2700 1 -- 172.21.15.107:0/196634754 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45c19fbb0 con 0x7fd45c1005e0 2023-07-21T22:29:05.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.043+0000 7fd44b7fe700 1 -- 172.21.15.107:0/196634754 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450018680 con 0x7fd45c1005e0 2023-07-21T22:29:05.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.044+0000 7fd460ca2700 1 -- 172.21.15.107:0/196634754 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45c1a00c0 con 0x7fd45c1005e0 2023-07-21T22:29:05.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.045+0000 7fd460ca2700 1 -- 172.21.15.107:0/196634754 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45c0fbb50 con 0x7fd45c1005e0 2023-07-21T22:29:05.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.045+0000 7fd44b7fe700 1 -- 172.21.15.107:0/196634754 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd4500105a0 con 0x7fd45c1005e0 2023-07-21T22:29:05.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.046+0000 7fd44b7fe700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd4440a1220 0x7fd4440a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:05.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.046+0000 7fd44b7fe700 1 -- 172.21.15.107:0/196634754 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd450014070 con 0x7fd45c1005e0 2023-07-21T22:29:05.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.046+0000 7fd459d9b700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd4440a1220 0x7fd4440a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:05.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.046+0000 7fd459d9b700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd4440a1220 0x7fd4440a36d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009c60 tx=0x7fd44c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:05.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.050+0000 7fd44b7fe700 1 -- 172.21.15.107:0/196634754 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd450051770 con 0x7fd45c1005e0 2023-07-21T22:29:05.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.242+0000 7fd460ca2700 1 -- 172.21.15.107:0/196634754 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd45c02ca90 con 0x7fd45c1005e0 2023-07-21T22:29:05.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.243+0000 7fd44b7fe700 1 -- 172.21.15.107:0/196634754 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22727 (secure 0 0 0) 0x7fd450096250 con 0x7fd45c1005e0 2023-07-21T22:29:05.290 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:29:05.291 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":84,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:28:47.235779+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:29:05.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.245+0000 7fd460ca2700 1 -- 172.21.15.107:0/196634754 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd4440a1220 msgr2=0x7fd4440a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.245+0000 7fd460ca2700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd4440a1220 0x7fd4440a36d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009c60 tx=0x7fd44c009260 comp rx=0 tx=0).stop 2023-07-21T22:29:05.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.245+0000 7fd460ca2700 1 -- 172.21.15.107:0/196634754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd45c1005e0 msgr2=0x7fd45c0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.245+0000 7fd460ca2700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd45c1005e0 0x7fd45c0fff50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e9a0 tx=0x7fd45000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.245+0000 7fd460ca2700 1 -- 172.21.15.107:0/196634754 shutdown_connections 2023-07-21T22:29:05.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.245+0000 7fd460ca2700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd4440a1220 0x7fd4440a36d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.245+0000 7fd460ca2700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd45c105820 0x7fd45c0fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.245+0000 7fd460ca2700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd45c1005e0 0x7fd45c0fff50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.245+0000 7fd460ca2700 1 --2- 172.21.15.107:0/196634754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd45c102f00 0x7fd45c0fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.245+0000 7fd460ca2700 1 -- 172.21.15.107:0/196634754 >> 172.21.15.107:0/196634754 conn(0x7fd45c0f9fe0 msgr2=0x7fd45c101240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:05.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.246+0000 7fd460ca2700 1 -- 172.21.15.107:0/196634754 shutdown_connections 2023-07-21T22:29:05.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.246+0000 7fd460ca2700 1 -- 172.21.15.107:0/196634754 wait complete. 2023-07-21T22:29:05.303 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:05 smithi107 ceph-mon[96543]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 1 B/s, 0 objects/s recovering 2023-07-21T22:29:05.303 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:05 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/978677130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:29:05.303 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:05 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3375234648' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:29:05.304 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:05 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1057380757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:29:05.305 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:23:52.031819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:27:39.494209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:27:40.672303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:29:05.305 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:29:05.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:05 smithi190 ceph-mon[98334]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 1 B/s, 0 objects/s recovering 2023-07-21T22:29:05.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:05 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/978677130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:29:05.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:05 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3375234648' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:29:05.392 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:05 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1057380757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:29:05.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.383+0000 7fa276af4700 1 -- 172.21.15.107:0/1923921549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2700fe9e0 msgr2=0x7fa2700fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.383+0000 7fa276af4700 1 --2- 172.21.15.107:0/1923921549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2700fe9e0 0x7fa2700fedf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa264009990 tx=0x7fa264009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.383+0000 7fa276af4700 1 -- 172.21.15.107:0/1923921549 shutdown_connections 2023-07-21T22:29:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.383+0000 7fa276af4700 1 --2- 172.21.15.107:0/1923921549 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa270100560 0x7fa270106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.383+0000 7fa276af4700 1 --2- 172.21.15.107:0/1923921549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2700fe9e0 0x7fa2700fedf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.383+0000 7fa276af4700 1 --2- 172.21.15.107:0/1923921549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2700ffbd0 0x7fa270100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.383+0000 7fa276af4700 1 -- 172.21.15.107:0/1923921549 >> 172.21.15.107:0/1923921549 conn(0x7fa2700fa000 msgr2=0x7fa2700fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:05.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.383+0000 7fa276af4700 1 -- 172.21.15.107:0/1923921549 shutdown_connections 2023-07-21T22:29:05.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.383+0000 7fa276af4700 1 -- 172.21.15.107:0/1923921549 wait complete. 2023-07-21T22:29:05.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa276af4700 1 Processor -- start 2023-07-21T22:29:05.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa276af4700 1 -- start start 2023-07-21T22:29:05.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa276af4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2700fe9e0 0x7fa2701959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:05.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa276af4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2700ffbd0 0x7fa270195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:05.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa276af4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270100560 0x7fa27019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:05.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa276af4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27019d540 con 0x7fa270100560 2023-07-21T22:29:05.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa276af4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27019d680 con 0x7fa2700ffbd0 2023-07-21T22:29:05.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa276af4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27019d810 con 0x7fa2700fe9e0 2023-07-21T22:29:05.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa275091700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270100560 0x7fa27019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:05.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa275091700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270100560 0x7fa27019cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47996/0 (socket says 172.21.15.107:47996) 2023-07-21T22:29:05.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa275091700 1 -- 172.21.15.107:0/3528952243 learned_addr learned my addr 172.21.15.107:0/3528952243 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:05.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa274890700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2700fe9e0 0x7fa2701959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:05.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa26ffff700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2700ffbd0 0x7fa270195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:05.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa275091700 1 -- 172.21.15.107:0/3528952243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2700fe9e0 msgr2=0x7fa2701959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa275091700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2700fe9e0 0x7fa2701959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa275091700 1 -- 172.21.15.107:0/3528952243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2700ffbd0 msgr2=0x7fa270195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa275091700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2700ffbd0 0x7fa270195f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa275091700 1 -- 172.21.15.107:0/3528952243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2640095f0 con 0x7fa270100560 2023-07-21T22:29:05.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.384+0000 7fa275091700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270100560 0x7fa27019cf70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa26800b970 tx=0x7fa26800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:05.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.385+0000 7fa26dffb700 1 -- 172.21.15.107:0/3528952243 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800c9b0 con 0x7fa270100560 2023-07-21T22:29:05.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.385+0000 7fa276af4700 1 -- 172.21.15.107:0/3528952243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27019daf0 con 0x7fa270100560 2023-07-21T22:29:05.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.385+0000 7fa26dffb700 1 -- 172.21.15.107:0/3528952243 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa268014440 con 0x7fa270100560 2023-07-21T22:29:05.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.385+0000 7fa26dffb700 1 -- 172.21.15.107:0/3528952243 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2680075d0 con 0x7fa270100560 2023-07-21T22:29:05.436 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:05 smithi132 ceph-mon[97863]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 1 B/s, 0 objects/s recovering 2023-07-21T22:29:05.436 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:05 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/978677130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:29:05.437 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:05 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3375234648' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:29:05.437 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:05 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1057380757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:29:05.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.385+0000 7fa276af4700 1 -- 172.21.15.107:0/3528952243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27019e030 con 0x7fa270100560 2023-07-21T22:29:05.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.386+0000 7fa276af4700 1 -- 172.21.15.107:0/3528952243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27010a2b0 con 0x7fa270100560 2023-07-21T22:29:05.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.387+0000 7fa26dffb700 1 -- 172.21.15.107:0/3528952243 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fa2680145b0 con 0x7fa270100560 2023-07-21T22:29:05.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.387+0000 7fa26dffb700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa2600a11e0 0x7fa2600a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:05.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.387+0000 7fa26dffb700 1 -- 172.21.15.107:0/3528952243 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa268006e40 con 0x7fa270100560 2023-07-21T22:29:05.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.387+0000 7fa274890700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa2600a11e0 0x7fa2600a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:05.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.388+0000 7fa274890700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa2600a11e0 0x7fa2600a3690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa264006010 tx=0x7fa26400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:05.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.392+0000 7fa26dffb700 1 -- 172.21.15.107:0/3528952243 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa26804e770 con 0x7fa270100560 2023-07-21T22:29:05.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.578+0000 7fa276af4700 1 -- 172.21.15.107:0/3528952243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa27019e6a0 con 0x7fa270100560 2023-07-21T22:29:05.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.579+0000 7fa26dffb700 1 -- 172.21.15.107:0/3528952243 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa268095800 con 0x7fa270100560 2023-07-21T22:29:05.626 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2023-07-21T22:29:05.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.581+0000 7fa276af4700 1 -- 172.21.15.107:0/3528952243 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa2600a11e0 msgr2=0x7fa2600a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.581+0000 7fa276af4700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa2600a11e0 0x7fa2600a3690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa264006010 tx=0x7fa26400b540 comp rx=0 tx=0).stop 2023-07-21T22:29:05.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.581+0000 7fa276af4700 1 -- 172.21.15.107:0/3528952243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270100560 msgr2=0x7fa27019cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.581+0000 7fa276af4700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270100560 0x7fa27019cf70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa26800b970 tx=0x7fa26800bd30 comp rx=0 tx=0).stop 2023-07-21T22:29:05.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.581+0000 7fa276af4700 1 -- 172.21.15.107:0/3528952243 shutdown_connections 2023-07-21T22:29:05.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.581+0000 7fa276af4700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa270100560 0x7fa27019cf70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.581+0000 7fa276af4700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa2600a11e0 0x7fa2600a3690 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.581+0000 7fa276af4700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2700fe9e0 0x7fa2701959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.581+0000 7fa276af4700 1 --2- 172.21.15.107:0/3528952243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2700ffbd0 0x7fa270195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.581+0000 7fa276af4700 1 -- 172.21.15.107:0/3528952243 >> 172.21.15.107:0/3528952243 conn(0x7fa2700fa000 msgr2=0x7fa2700676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.581+0000 7fa276af4700 1 -- 172.21.15.107:0/3528952243 shutdown_connections 2023-07-21T22:29:05.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.581+0000 7fa276af4700 1 -- 172.21.15.107:0/3528952243 wait complete. 2023-07-21T22:29:05.639 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:29:05.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.716+0000 7f7db8b26700 1 -- 172.21.15.107:0/2423338382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7db41005b0 msgr2=0x7f7db4106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.716+0000 7f7db8b26700 1 --2- 172.21.15.107:0/2423338382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7db41005b0 0x7f7db4106e50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00b0a0 tx=0x7f7dac00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.716+0000 7f7db8b26700 1 -- 172.21.15.107:0/2423338382 shutdown_connections 2023-07-21T22:29:05.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.716+0000 7f7db8b26700 1 --2- 172.21.15.107:0/2423338382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7db41005b0 0x7f7db4106e50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.716+0000 7f7db8b26700 1 --2- 172.21.15.107:0/2423338382 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db40feef0 0x7f7db40ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.716+0000 7f7db8b26700 1 --2- 172.21.15.107:0/2423338382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7db40ffc00 0x7f7db4100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.716+0000 7f7db8b26700 1 -- 172.21.15.107:0/2423338382 >> 172.21.15.107:0/2423338382 conn(0x7f7db40f9770 msgr2=0x7f7db40fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:05.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.716+0000 7f7db8b26700 1 -- 172.21.15.107:0/2423338382 shutdown_connections 2023-07-21T22:29:05.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.716+0000 7f7db8b26700 1 -- 172.21.15.107:0/2423338382 wait complete. 2023-07-21T22:29:05.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.717+0000 7f7db8b26700 1 Processor -- start 2023-07-21T22:29:05.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.717+0000 7f7db8b26700 1 -- start start 2023-07-21T22:29:05.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.717+0000 7f7db8b26700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7db40feef0 0x7f7db4194830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:05.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.717+0000 7f7db8b26700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db40ffc00 0x7f7db4194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:05.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.717+0000 7f7db8b26700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7db41005b0 0x7f7db419bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:05.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.717+0000 7f7db8b26700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db419c3b0 con 0x7f7db41005b0 2023-07-21T22:29:05.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.717+0000 7f7db8b26700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db419c4f0 con 0x7f7db40feef0 2023-07-21T22:29:05.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.717+0000 7f7db8b26700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db419c680 con 0x7f7db40ffc00 2023-07-21T22:29:05.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.717+0000 7f7db2d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7db41005b0 0x7f7db419bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:05.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.717+0000 7f7db2d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7db41005b0 0x7f7db419bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48010/0 (socket says 172.21.15.107:48010) 2023-07-21T22:29:05.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.717+0000 7f7db2d9d700 1 -- 172.21.15.107:0/1924744971 learned_addr learned my addr 172.21.15.107:0/1924744971 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:05.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.717+0000 7f7db259c700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7db40feef0 0x7f7db4194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:05.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7db1d9b700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db40ffc00 0x7f7db4194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:05.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7db2d9d700 1 -- 172.21.15.107:0/1924744971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db40ffc00 msgr2=0x7f7db4194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7db2d9d700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db40ffc00 0x7f7db4194d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7db2d9d700 1 -- 172.21.15.107:0/1924744971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7db40feef0 msgr2=0x7f7db4194830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7db2d9d700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7db40feef0 0x7f7db4194830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7db2d9d700 1 -- 172.21.15.107:0/1924744971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac009d00 con 0x7f7db41005b0 2023-07-21T22:29:05.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7db1d9b700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db40ffc00 0x7f7db4194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:05.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7db259c700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7db40feef0 0x7f7db4194830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:05.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7db2d9d700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7db41005b0 0x7f7db419bde0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0062a0 tx=0x7f7dac004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:05.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7da37fe700 1 -- 172.21.15.107:0/1924744971 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac00e040 con 0x7f7db41005b0 2023-07-21T22:29:05.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7da37fe700 1 -- 172.21.15.107:0/1924744971 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dac01db50 con 0x7f7db41005b0 2023-07-21T22:29:05.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7db8b26700 1 -- 172.21.15.107:0/1924744971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db419c900 con 0x7f7db41005b0 2023-07-21T22:29:05.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7da37fe700 1 -- 172.21.15.107:0/1924744971 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac01bc40 con 0x7f7db41005b0 2023-07-21T22:29:05.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.718+0000 7f7db8b26700 1 -- 172.21.15.107:0/1924744971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db419cd30 con 0x7f7db41005b0 2023-07-21T22:29:05.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.719+0000 7f7db8b26700 1 -- 172.21.15.107:0/1924744971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db41058e0 con 0x7f7db41005b0 2023-07-21T22:29:05.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.721+0000 7f7da37fe700 1 -- 172.21.15.107:0/1924744971 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f7dac01d690 con 0x7f7db41005b0 2023-07-21T22:29:05.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.721+0000 7f7da37fe700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7d9c0a11e0 0x7f7d9c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:05.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.721+0000 7f7db259c700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7d9c0a11e0 0x7f7d9c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:05.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.721+0000 7f7da37fe700 1 -- 172.21.15.107:0/1924744971 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7dac0cecd0 con 0x7f7db41005b0 2023-07-21T22:29:05.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.722+0000 7f7db259c700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7d9c0a11e0 0x7f7d9c0a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7da8005950 tx=0x7f7da80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:05.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.724+0000 7f7da37fe700 1 -- 172.21.15.107:0/1924744971 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7dac098330 con 0x7f7db41005b0 2023-07-21T22:29:05.961 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.915+0000 7f7db8b26700 1 -- 172.21.15.107:0/1924744971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7db402cac0 con 0x7f7db41005b0 2023-07-21T22:29:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.915+0000 7f7da37fe700 1 -- 172.21.15.107:0/1924744971 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7dac09b950 con 0x7f7db41005b0 2023-07-21T22:29:05.962 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2023-07-21T22:29:05.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.917+0000 7f7db8b26700 1 -- 172.21.15.107:0/1924744971 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7d9c0a11e0 msgr2=0x7f7d9c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.917+0000 7f7db8b26700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7d9c0a11e0 0x7f7d9c0a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7da8005950 tx=0x7f7da80058e0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.917+0000 7f7db8b26700 1 -- 172.21.15.107:0/1924744971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7db41005b0 msgr2=0x7f7db419bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:05.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.917+0000 7f7db8b26700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7db41005b0 0x7f7db419bde0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0062a0 tx=0x7f7dac004db0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.917+0000 7f7db8b26700 1 -- 172.21.15.107:0/1924744971 shutdown_connections 2023-07-21T22:29:05.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.917+0000 7f7db8b26700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7db41005b0 0x7f7db419bde0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.917+0000 7f7db8b26700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7db40feef0 0x7f7db4194830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.917+0000 7f7db8b26700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7d9c0a11e0 0x7f7d9c0a3690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.917+0000 7f7db8b26700 1 --2- 172.21.15.107:0/1924744971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7db40ffc00 0x7f7db4194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:05.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.917+0000 7f7db8b26700 1 -- 172.21.15.107:0/1924744971 >> 172.21.15.107:0/1924744971 conn(0x7f7db40f9770 msgr2=0x7f7db40fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:05.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.918+0000 7f7db8b26700 1 -- 172.21.15.107:0/1924744971 shutdown_connections 2023-07-21T22:29:05.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:05.918+0000 7f7db8b26700 1 -- 172.21.15.107:0/1924744971 wait complete. 2023-07-21T22:29:05.976 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:29:06.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.056+0000 7f39d64b6700 1 -- 172.21.15.107:0/215169694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d0105800 msgr2=0x7f39d0107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:06.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.056+0000 7f39d64b6700 1 --2- 172.21.15.107:0/215169694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d0105800 0x7f39d0107c00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f39c800b0a0 tx=0x7f39c800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.104 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.056+0000 7f39d64b6700 1 -- 172.21.15.107:0/215169694 shutdown_connections 2023-07-21T22:29:06.104 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.056+0000 7f39d64b6700 1 --2- 172.21.15.107:0/215169694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d0105800 0x7f39d0107c00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.104 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.056+0000 7f39d64b6700 1 --2- 172.21.15.107:0/215169694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39d01005c0 0x7f39d01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.104 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.056+0000 7f39d64b6700 1 --2- 172.21.15.107:0/215169694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d0102ee0 0x7f39d01052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.104 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.056+0000 7f39d64b6700 1 -- 172.21.15.107:0/215169694 >> 172.21.15.107:0/215169694 conn(0x7f39d00fa000 msgr2=0x7f39d00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:06.104 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.057+0000 7f39d64b6700 1 -- 172.21.15.107:0/215169694 shutdown_connections 2023-07-21T22:29:06.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.057+0000 7f39d64b6700 1 -- 172.21.15.107:0/215169694 wait complete. 2023-07-21T22:29:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.057+0000 7f39d64b6700 1 Processor -- start 2023-07-21T22:29:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.057+0000 7f39d64b6700 1 -- start start 2023-07-21T22:29:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39d64b6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d01005c0 0x7f39d0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39d64b6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39d0102ee0 0x7f39d0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:06.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39d64b6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d0105800 0x7f39d019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39d64b6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d019f650 con 0x7f39d01005c0 2023-07-21T22:29:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39d64b6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d019f790 con 0x7f39d0105800 2023-07-21T22:29:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39d64b6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d019f920 con 0x7f39d0102ee0 2023-07-21T22:29:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39cffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d01005c0 0x7f39d0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39cffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d01005c0 0x7f39d0197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48020/0 (socket says 172.21.15.107:48020) 2023-07-21T22:29:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39cffff700 1 -- 172.21.15.107:0/3829748508 learned_addr learned my addr 172.21.15.107:0/3829748508 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:06.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39cf7fe700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39d0102ee0 0x7f39d0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39d4a53700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d0105800 0x7f39d019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39cffff700 1 -- 172.21.15.107:0/3829748508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39d0102ee0 msgr2=0x7f39d0198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39cffff700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39d0102ee0 0x7f39d0198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39cffff700 1 -- 172.21.15.107:0/3829748508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d0105800 msgr2=0x7f39d019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39cffff700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d0105800 0x7f39d019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39cffff700 1 -- 172.21.15.107:0/3829748508 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c8009d00 con 0x7f39d01005c0 2023-07-21T22:29:06.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.058+0000 7f39cffff700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d01005c0 0x7f39d0197ad0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f39c000e9a0 tx=0x7f39c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:06.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.061+0000 7f39cd7fa700 1 -- 172.21.15.107:0/3829748508 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0009790 con 0x7f39d01005c0 2023-07-21T22:29:06.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.061+0000 7f39cd7fa700 1 -- 172.21.15.107:0/3829748508 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39c0010430 con 0x7f39d01005c0 2023-07-21T22:29:06.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.061+0000 7f39d64b6700 1 -- 172.21.15.107:0/3829748508 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d019fc00 con 0x7f39d01005c0 2023-07-21T22:29:06.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.061+0000 7f39cd7fa700 1 -- 172.21.15.107:0/3829748508 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0018680 con 0x7f39d01005c0 2023-07-21T22:29:06.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.061+0000 7f39d64b6700 1 -- 172.21.15.107:0/3829748508 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d01a0110 con 0x7f39d01005c0 2023-07-21T22:29:06.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.062+0000 7f39d64b6700 1 -- 172.21.15.107:0/3829748508 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d00fbb70 con 0x7f39d01005c0 2023-07-21T22:29:06.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.063+0000 7f39cd7fa700 1 -- 172.21.15.107:0/3829748508 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f39c00105a0 con 0x7f39d01005c0 2023-07-21T22:29:06.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.063+0000 7f39cd7fa700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f39b80a11e0 0x7f39b80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:06.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.063+0000 7f39cf7fe700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f39b80a11e0 0x7f39b80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:06.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.063+0000 7f39cd7fa700 1 -- 172.21.15.107:0/3829748508 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f39c0014070 con 0x7f39d01005c0 2023-07-21T22:29:06.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.064+0000 7f39cf7fe700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f39b80a11e0 0x7f39b80a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009c60 tx=0x7f39c4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:06.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.068+0000 7f39cd7fa700 1 -- 172.21.15.107:0/3829748508 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f39c0051770 con 0x7f39d01005c0 2023-07-21T22:29:06.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.256+0000 7f39d64b6700 1 -- 172.21.15.107:0/3829748508 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39d002cae0 con 0x7f39d01005c0 2023-07-21T22:29:06.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.256+0000 7f39cd7fa700 1 -- 172.21.15.107:0/3829748508 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f39c0096250 con 0x7f39d01005c0 2023-07-21T22:29:06.303 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2023-07-21T22:29:06.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.258+0000 7f39d64b6700 1 -- 172.21.15.107:0/3829748508 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f39b80a11e0 msgr2=0x7f39b80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:06.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.258+0000 7f39d64b6700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f39b80a11e0 0x7f39b80a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009c60 tx=0x7f39c4009260 comp rx=0 tx=0).stop 2023-07-21T22:29:06.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.258+0000 7f39d64b6700 1 -- 172.21.15.107:0/3829748508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d01005c0 msgr2=0x7f39d0197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:06.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.259+0000 7f39d64b6700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d01005c0 0x7f39d0197ad0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f39c000e9a0 tx=0x7f39c000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.259+0000 7f39d64b6700 1 -- 172.21.15.107:0/3829748508 shutdown_connections 2023-07-21T22:29:06.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.259+0000 7f39d64b6700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d0105800 0x7f39d019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.259+0000 7f39d64b6700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39d01005c0 0x7f39d0197ad0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.259+0000 7f39d64b6700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f39b80a11e0 0x7f39b80a3690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.259+0000 7f39d64b6700 1 --2- 172.21.15.107:0/3829748508 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39d0102ee0 0x7f39d0198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.259+0000 7f39d64b6700 1 -- 172.21.15.107:0/3829748508 >> 172.21.15.107:0/3829748508 conn(0x7f39d00fa000 msgr2=0x7f39d0101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:06.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.259+0000 7f39d64b6700 1 -- 172.21.15.107:0/3829748508 shutdown_connections 2023-07-21T22:29:06.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.259+0000 7f39d64b6700 1 -- 172.21.15.107:0/3829748508 wait complete. 2023-07-21T22:29:06.316 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:06 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/196634754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:29:06.317 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:06 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3528952243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:29:06.317 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:06 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1924744971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:29:06.318 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-07-21T22:29:06.318 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:29:06.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:06 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/196634754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:29:06.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:06 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3528952243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:29:06.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:06 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1924744971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:29:06.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:06 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/196634754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:29:06.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:06 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3528952243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:29:06.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:06 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1924744971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:29:06.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.392+0000 7fac21d8a700 1 -- 172.21.15.107:0/247039294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac1c0febd0 msgr2=0x7fac1c0ff040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:06.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.392+0000 7fac21d8a700 1 --2- 172.21.15.107:0/247039294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac1c0febd0 0x7fac1c0ff040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009990 tx=0x7fac0c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.392+0000 7fac21d8a700 1 -- 172.21.15.107:0/247039294 shutdown_connections 2023-07-21T22:29:06.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.392+0000 7fac21d8a700 1 --2- 172.21.15.107:0/247039294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac1c0ff580 0x7fac1c106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.392+0000 7fac21d8a700 1 --2- 172.21.15.107:0/247039294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac1c0fff80 0x7fac1c100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.392+0000 7fac21d8a700 1 --2- 172.21.15.107:0/247039294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac1c0febd0 0x7fac1c0ff040 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.392+0000 7fac21d8a700 1 -- 172.21.15.107:0/247039294 >> 172.21.15.107:0/247039294 conn(0x7fac1c0f9770 msgr2=0x7fac1c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:06.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.392+0000 7fac21d8a700 1 -- 172.21.15.107:0/247039294 shutdown_connections 2023-07-21T22:29:06.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.392+0000 7fac21d8a700 1 -- 172.21.15.107:0/247039294 wait complete. 2023-07-21T22:29:06.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac21d8a700 1 Processor -- start 2023-07-21T22:29:06.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac21d8a700 1 -- start start 2023-07-21T22:29:06.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac21d8a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac1c0febd0 0x7fac1c0fe650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:06.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac21d8a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac1c0ff580 0x7fac1c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:06.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac21d8a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac1c0fff80 0x7fac1c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:06.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac21d8a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c19f530 con 0x7fac1c0ff580 2023-07-21T22:29:06.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac21d8a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c19f6a0 con 0x7fac1c0fff80 2023-07-21T22:29:06.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac21d8a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c19f860 con 0x7fac1c0febd0 2023-07-21T22:29:06.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac1affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac1c0ff580 0x7fac1c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:06.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac1affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac1c0ff580 0x7fac1c19c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48036/0 (socket says 172.21.15.107:48036) 2023-07-21T22:29:06.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac1affd700 1 -- 172.21.15.107:0/3349380704 learned_addr learned my addr 172.21.15.107:0/3349380704 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:06.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac1b7fe700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac1c0febd0 0x7fac1c0fe650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:06.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac1bfff700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac1c0fff80 0x7fac1c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac1affd700 1 -- 172.21.15.107:0/3349380704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac1c0febd0 msgr2=0x7fac1c0fe650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac1affd700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac1c0febd0 0x7fac1c0fe650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac1affd700 1 -- 172.21.15.107:0/3349380704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac1c0fff80 msgr2=0x7fac1c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac1affd700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac1c0fff80 0x7fac1c19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.393+0000 7fac1affd700 1 -- 172.21.15.107:0/3349380704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c0095f0 con 0x7fac1c0ff580 2023-07-21T22:29:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.394+0000 7fac1affd700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac1c0ff580 0x7fac1c19c900 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fac0c004730 tx=0x7fac0c004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:06.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.394+0000 7fac18ff9700 1 -- 172.21.15.107:0/3349380704 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c01d070 con 0x7fac1c0ff580 2023-07-21T22:29:06.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.394+0000 7fac18ff9700 1 -- 172.21.15.107:0/3349380704 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac0c004c90 con 0x7fac1c0ff580 2023-07-21T22:29:06.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.394+0000 7fac18ff9700 1 -- 172.21.15.107:0/3349380704 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0054b0 con 0x7fac1c0ff580 2023-07-21T22:29:06.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.394+0000 7fac21d8a700 1 -- 172.21.15.107:0/3349380704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c19fae0 con 0x7fac1c0ff580 2023-07-21T22:29:06.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.394+0000 7fac21d8a700 1 -- 172.21.15.107:0/3349380704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c19fe50 con 0x7fac1c0ff580 2023-07-21T22:29:06.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.395+0000 7fac21d8a700 1 -- 172.21.15.107:0/3349380704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1c105f30 con 0x7fac1c0ff580 2023-07-21T22:29:06.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.396+0000 7fac18ff9700 1 -- 172.21.15.107:0/3349380704 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fac0c00fbc0 con 0x7fac1c0ff580 2023-07-21T22:29:06.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.396+0000 7fac18ff9700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fac040a1220 0x7fac040a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:06.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.396+0000 7fac1b7fe700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fac040a1220 0x7fac040a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:06.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.396+0000 7fac18ff9700 1 -- 172.21.15.107:0/3349380704 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fac0c0cdd10 con 0x7fac1c0ff580 2023-07-21T22:29:06.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.397+0000 7fac1b7fe700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fac040a1220 0x7fac040a36d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fac10007be0 tx=0x7fac100073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:06.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.401+0000 7fac18ff9700 1 -- 172.21.15.107:0/3349380704 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fac0c05a770 con 0x7fac1c0ff580 2023-07-21T22:29:06.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.631+0000 7fac21d8a700 1 -- 172.21.15.107:0/3349380704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac1c02cc70 con 0x7fac1c0ff580 2023-07-21T22:29:06.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.632+0000 7fac18ff9700 1 -- 172.21.15.107:0/3349380704 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10868 (secure 0 0 0) 0x7fac0c097370 con 0x7fac1c0ff580 2023-07-21T22:29:06.679 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:29:06.679 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:27:49.034257+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:29:06.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.634+0000 7fac21d8a700 1 -- 172.21.15.107:0/3349380704 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fac040a1220 msgr2=0x7fac040a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:06.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.634+0000 7fac21d8a700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fac040a1220 0x7fac040a36d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fac10007be0 tx=0x7fac100073d0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.634+0000 7fac21d8a700 1 -- 172.21.15.107:0/3349380704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac1c0ff580 msgr2=0x7fac1c19c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:06.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.634+0000 7fac21d8a700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac1c0ff580 0x7fac1c19c900 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fac0c004730 tx=0x7fac0c004760 comp rx=0 tx=0).stop 2023-07-21T22:29:06.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.634+0000 7fac21d8a700 1 -- 172.21.15.107:0/3349380704 shutdown_connections 2023-07-21T22:29:06.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.634+0000 7fac21d8a700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac1c0fff80 0x7fac1c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.634+0000 7fac21d8a700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fac040a1220 0x7fac040a36d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.634+0000 7fac21d8a700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac1c0febd0 0x7fac1c0fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.634+0000 7fac21d8a700 1 --2- 172.21.15.107:0/3349380704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac1c0ff580 0x7fac1c19c900 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.634+0000 7fac21d8a700 1 -- 172.21.15.107:0/3349380704 >> 172.21.15.107:0/3349380704 conn(0x7fac1c0f9770 msgr2=0x7fac1c103070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:06.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.634+0000 7fac21d8a700 1 -- 172.21.15.107:0/3349380704 shutdown_connections 2023-07-21T22:29:06.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.634+0000 7fac21d8a700 1 -- 172.21.15.107:0/3349380704 wait complete. 2023-07-21T22:29:06.682 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2023-07-21T22:29:06.693 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-07-21T22:29:06.694 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-07-21T22:29:06.694 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:29:06.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.769+0000 7f1477c48700 1 -- 172.21.15.107:0/3202868629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14700fec90 msgr2=0x7f14700ff0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:06.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.769+0000 7f1477c48700 1 --2- 172.21.15.107:0/3202868629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14700fec90 0x7f14700ff0e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f146c009990 tx=0x7f146c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.769+0000 7f1477c48700 1 -- 172.21.15.107:0/3202868629 shutdown_connections 2023-07-21T22:29:06.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.769+0000 7f1477c48700 1 --2- 172.21.15.107:0/3202868629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14700ff620 0x7f1470106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.769+0000 7f1477c48700 1 --2- 172.21.15.107:0/3202868629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14700fff60 0x7f1470100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.769+0000 7f1477c48700 1 --2- 172.21.15.107:0/3202868629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14700fec90 0x7f14700ff0e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.769+0000 7f1477c48700 1 -- 172.21.15.107:0/3202868629 >> 172.21.15.107:0/3202868629 conn(0x7f14700f9770 msgr2=0x7f14700fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:06.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.769+0000 7f1477c48700 1 -- 172.21.15.107:0/3202868629 shutdown_connections 2023-07-21T22:29:06.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.769+0000 7f1477c48700 1 -- 172.21.15.107:0/3202868629 wait complete. 2023-07-21T22:29:06.818 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.769+0000 7f1477c48700 1 Processor -- start 2023-07-21T22:29:06.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.769+0000 7f1477c48700 1 -- start start 2023-07-21T22:29:06.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f1477c48700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14700fec90 0x7f1470102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:06.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f1477c48700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14700ff620 0x7f1470101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:06.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f1477c48700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14700fff60 0x7f14701015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:06.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f1477c48700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1470101d60 con 0x7f14700fff60 2023-07-21T22:29:06.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f1477c48700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1470101ed0 con 0x7f14700fec90 2023-07-21T22:29:06.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f1477c48700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147019f8d0 con 0x7f14700ff620 2023-07-21T22:29:06.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f14761e5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14700fff60 0x7f14701015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:06.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f14761e5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14700fff60 0x7f14701015c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48052/0 (socket says 172.21.15.107:48052) 2023-07-21T22:29:06.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f14761e5700 1 -- 172.21.15.107:0/1734857820 learned_addr learned my addr 172.21.15.107:0/1734857820 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:06.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f14751e3700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14700ff620 0x7f1470101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:06.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f14759e4700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14700fec90 0x7f1470102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:06.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f14761e5700 1 -- 172.21.15.107:0/1734857820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14700ff620 msgr2=0x7f1470101080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:06.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f14761e5700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14700ff620 0x7f1470101080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f14761e5700 1 -- 172.21.15.107:0/1734857820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14700fec90 msgr2=0x7f1470102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:06.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f14761e5700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14700fec90 0x7f1470102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:06.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f14761e5700 1 -- 172.21.15.107:0/1734857820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0095f0 con 0x7f14700fff60 2023-07-21T22:29:06.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f14759e4700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14700fec90 0x7f1470102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:06.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f14761e5700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14700fff60 0x7f14701015c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f146800ef30 tx=0x7f146800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:06.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f1466ffd700 1 -- 172.21.15.107:0/1734857820 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468018070 con 0x7f14700fff60 2023-07-21T22:29:06.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f1466ffd700 1 -- 172.21.15.107:0/1734857820 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f146800f040 con 0x7f14700fff60 2023-07-21T22:29:06.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f1466ffd700 1 -- 172.21.15.107:0/1734857820 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146800cdb0 con 0x7f14700fff60 2023-07-21T22:29:06.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f1477c48700 1 -- 172.21.15.107:0/1734857820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147019fb50 con 0x7f14700fff60 2023-07-21T22:29:06.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.770+0000 7f1477c48700 1 -- 172.21.15.107:0/1734857820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14701a0060 con 0x7f14700fff60 2023-07-21T22:29:06.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.771+0000 7f1477c48700 1 -- 172.21.15.107:0/1734857820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147010a1e0 con 0x7f14700fff60 2023-07-21T22:29:06.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.772+0000 7f1466ffd700 1 -- 172.21.15.107:0/1734857820 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f1468003a00 con 0x7f14700fff60 2023-07-21T22:29:06.823 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.773+0000 7f1466ffd700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f145c0a12a0 0x7f145c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:06.823 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.773+0000 7f14759e4700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f145c0a12a0 0x7f145c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:06.823 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.773+0000 7f1466ffd700 1 -- 172.21.15.107:0/1734857820 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f14680cc650 con 0x7f14700fff60 2023-07-21T22:29:06.823 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.773+0000 7f14759e4700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f145c0a12a0 0x7f145c0a3750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1460005fd0 tx=0x7f1460005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:06.824 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.777+0000 7f1466ffd700 1 -- 172.21.15.107:0/1734857820 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1468095c50 con 0x7f14700fff60 2023-07-21T22:29:07.010 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.964+0000 7f1477c48700 1 -- 172.21.15.107:0/1734857820 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14700fe2f0 con 0x7f145c0a12a0 2023-07-21T22:29:07.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.974+0000 7f1466ffd700 1 -- 172.21.15.107:0/1734857820 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343797 (secure 0 0 0) 0x7f14700fe2f0 con 0x7f145c0a12a0 2023-07-21T22:29:07.025 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:29:07.027 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.978+0000 7f1477c48700 1 -- 172.21.15.107:0/1734857820 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f145c0a12a0 msgr2=0x7f145c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.027 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.978+0000 7f1477c48700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f145c0a12a0 0x7f145c0a3750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1460005fd0 tx=0x7f1460005d40 comp rx=0 tx=0).stop 2023-07-21T22:29:07.027 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.978+0000 7f1477c48700 1 -- 172.21.15.107:0/1734857820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14700fff60 msgr2=0x7f14701015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.027 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.978+0000 7f1477c48700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14700fff60 0x7f14701015c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f146800ef30 tx=0x7f146800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.027 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.978+0000 7f1477c48700 1 -- 172.21.15.107:0/1734857820 shutdown_connections 2023-07-21T22:29:07.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.978+0000 7f1477c48700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14700fff60 0x7f14701015c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.978+0000 7f1477c48700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14700fec90 0x7f1470102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.978+0000 7f1477c48700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f145c0a12a0 0x7f145c0a3750 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.978+0000 7f1477c48700 1 --2- 172.21.15.107:0/1734857820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14700ff620 0x7f1470101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.978+0000 7f1477c48700 1 -- 172.21.15.107:0/1734857820 >> 172.21.15.107:0/1734857820 conn(0x7f14700f9770 msgr2=0x7f14700fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:07.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.978+0000 7f1477c48700 1 -- 172.21.15.107:0/1734857820 shutdown_connections 2023-07-21T22:29:07.029 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:06.978+0000 7f1477c48700 1 -- 172.21.15.107:0/1734857820 wait complete. 2023-07-21T22:29:07.029 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2023-07-21T22:29:07.037 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2023-07-21T22:29:05.808726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652602,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":905,"num_read_kb":565,"num_write":413,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181722,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336372,"kb_used_data":12948,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605452,"statfs":{"total":1151940427776,"available":1151595982848,"internally_reserved":0,"allocated":13258752,"data_stored":7220211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":318,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":56,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003089"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244157+0000","last_change":"2023-07-21T22:27:42.091397+0000","last_active":"2023-07-21T22:28:47.244157+0000","last_peered":"2023-07-21T22:28:47.244157+0000","last_clean":"2023-07-21T22:28:47.244157+0000","last_became_active":"2023-07-21T22:27:42.090836+0000","last_became_peered":"2023-07-21T22:27:42.090836+0000","last_unstale":"2023-07-21T22:28:47.244157+0000","last_undegraded":"2023-07-21T22:28:47.244157+0000","last_fullsized":"2023-07-21T22:28:47.244157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:21:03.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382220+0000","last_change":"2023-07-21T22:27:42.097003+0000","last_active":"2023-07-21T22:28:47.382220+0000","last_peered":"2023-07-21T22:28:47.382220+0000","last_clean":"2023-07-21T22:28:47.382220+0000","last_became_active":"2023-07-21T22:27:42.096652+0000","last_became_peered":"2023-07-21T22:27:42.096652+0000","last_unstale":"2023-07-21T22:28:47.382220+0000","last_undegraded":"2023-07-21T22:28:47.382220+0000","last_fullsized":"2023-07-21T22:28:47.382220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:00:27.467585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382162+0000","last_change":"2023-07-21T22:27:42.090090+0000","last_active":"2023-07-21T22:28:47.382162+0000","last_peered":"2023-07-21T22:28:47.382162+0000","last_clean":"2023-07-21T22:28:47.382162+0000","last_became_active":"2023-07-21T22:27:42.089802+0000","last_became_peered":"2023-07-21T22:27:42.089802+0000","last_unstale":"2023-07-21T22:28:47.382162+0000","last_undegraded":"2023-07-21T22:28:47.382162+0000","last_fullsized":"2023-07-21T22:28:47.382162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:15:20.510508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246255+0000","last_change":"2023-07-21T22:27:42.096291+0000","last_active":"2023-07-21T22:28:47.246255+0000","last_peered":"2023-07-21T22:28:47.246255+0000","last_clean":"2023-07-21T22:28:47.246255+0000","last_became_active":"2023-07-21T22:27:42.095982+0000","last_became_peered":"2023-07-21T22:27:42.095982+0000","last_unstale":"2023-07-21T22:28:47.246255+0000","last_undegraded":"2023-07-21T22:28:47.246255+0000","last_fullsized":"2023-07-21T22:28:47.246255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:14:38.335068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246671+0000","last_change":"2023-07-21T22:27:42.097495+0000","last_active":"2023-07-21T22:28:47.246671+0000","last_peered":"2023-07-21T22:28:47.246671+0000","last_clean":"2023-07-21T22:28:47.246671+0000","last_became_active":"2023-07-21T22:27:42.095112+0000","last_became_peered":"2023-07-21T22:27:42.095112+0000","last_unstale":"2023-07-21T22:28:47.246671+0000","last_undegraded":"2023-07-21T22:28:47.246671+0000","last_fullsized":"2023-07-21T22:28:47.246671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:14:17.323583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242851+0000","last_change":"2023-07-21T22:27:42.089880+0000","last_active":"2023-07-21T22:28:47.242851+0000","last_peered":"2023-07-21T22:28:47.242851+0000","last_clean":"2023-07-21T22:28:47.242851+0000","last_became_active":"2023-07-21T22:27:42.088614+0000","last_became_peered":"2023-07-21T22:27:42.088614+0000","last_unstale":"2023-07-21T22:28:47.242851+0000","last_undegraded":"2023-07-21T22:28:47.242851+0000","last_fullsized":"2023-07-21T22:28:47.242851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:06:38.785570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245114+0000","last_change":"2023-07-21T22:27:42.085457+0000","last_active":"2023-07-21T22:28:47.245114+0000","last_peered":"2023-07-21T22:28:47.245114+0000","last_clean":"2023-07-21T22:28:47.245114+0000","last_became_active":"2023-07-21T22:27:42.085146+0000","last_became_peered":"2023-07-21T22:27:42.085146+0000","last_unstale":"2023-07-21T22:28:47.245114+0000","last_undegraded":"2023-07-21T22:28:47.245114+0000","last_fullsized":"2023-07-21T22:28:47.245114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:06:59.371781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.247038+0000","last_change":"2023-07-21T22:28:47.246617+0000","last_active":"2023-07-21T22:28:47.247038+0000","last_peered":"2023-07-21T22:28:47.247038+0000","last_clean":"2023-07-21T22:28:47.247038+0000","last_became_active":"2023-07-21T22:28:47.246322+0000","last_became_peered":"2023-07-21T22:28:47.246322+0000","last_unstale":"2023-07-21T22:28:47.247038+0000","last_undegraded":"2023-07-21T22:28:47.247038+0000","last_fullsized":"2023-07-21T22:28:47.247038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:22.991062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383406+0000","last_change":"2023-07-21T22:27:42.094223+0000","last_active":"2023-07-21T22:28:47.383406+0000","last_peered":"2023-07-21T22:28:47.383406+0000","last_clean":"2023-07-21T22:28:47.383406+0000","last_became_active":"2023-07-21T22:27:42.093875+0000","last_became_peered":"2023-07-21T22:27:42.093875+0000","last_unstale":"2023-07-21T22:28:47.383406+0000","last_undegraded":"2023-07-21T22:28:47.383406+0000","last_fullsized":"2023-07-21T22:28:47.383406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:45:28.313407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243145+0000","last_change":"2023-07-21T22:27:42.090442+0000","last_active":"2023-07-21T22:28:47.243145+0000","last_peered":"2023-07-21T22:28:47.243145+0000","last_clean":"2023-07-21T22:28:47.243145+0000","last_became_active":"2023-07-21T22:27:42.090103+0000","last_became_peered":"2023-07-21T22:27:42.090103+0000","last_unstale":"2023-07-21T22:28:47.243145+0000","last_undegraded":"2023-07-21T22:28:47.243145+0000","last_fullsized":"2023-07-21T22:28:47.243145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:27:20.825119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245155+0000","last_change":"2023-07-21T22:27:42.098364+0000","last_active":"2023-07-21T22:28:47.245155+0000","last_peered":"2023-07-21T22:28:47.245155+0000","last_clean":"2023-07-21T22:28:47.245155+0000","last_became_active":"2023-07-21T22:27:42.097378+0000","last_became_peered":"2023-07-21T22:27:42.097378+0000","last_unstale":"2023-07-21T22:28:47.245155+0000","last_undegraded":"2023-07-21T22:28:47.245155+0000","last_fullsized":"2023-07-21T22:28:47.245155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:40:58.595476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246312+0000","last_change":"2023-07-21T22:27:42.103110+0000","last_active":"2023-07-21T22:28:47.246312+0000","last_peered":"2023-07-21T22:28:47.246312+0000","last_clean":"2023-07-21T22:28:47.246312+0000","last_became_active":"2023-07-21T22:27:42.102790+0000","last_became_peered":"2023-07-21T22:27:42.102790+0000","last_unstale":"2023-07-21T22:28:47.246312+0000","last_undegraded":"2023-07-21T22:28:47.246312+0000","last_fullsized":"2023-07-21T22:28:47.246312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:20:23.914945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244930+0000","last_change":"2023-07-21T22:28:47.244930+0000","last_active":"2023-07-21T22:28:47.244930+0000","last_peered":"2023-07-21T22:28:47.244930+0000","last_clean":"2023-07-21T22:28:47.244930+0000","last_became_active":"2023-07-21T22:28:47.244613+0000","last_became_peered":"2023-07-21T22:28:47.244613+0000","last_unstale":"2023-07-21T22:28:47.244930+0000","last_undegraded":"2023-07-21T22:28:47.244930+0000","last_fullsized":"2023-07-21T22:28:47.244930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:43:03.707499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382008+0000","last_change":"2023-07-21T22:27:42.084423+0000","last_active":"2023-07-21T22:28:47.382008+0000","last_peered":"2023-07-21T22:28:47.382008+0000","last_clean":"2023-07-21T22:28:47.382008+0000","last_became_active":"2023-07-21T22:27:42.084074+0000","last_became_peered":"2023-07-21T22:27:42.084074+0000","last_unstale":"2023-07-21T22:28:47.382008+0000","last_undegraded":"2023-07-21T22:28:47.382008+0000","last_fullsized":"2023-07-21T22:28:47.382008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:48.499214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.381932+0000","last_change":"2023-07-21T22:27:42.091542+0000","last_active":"2023-07-21T22:28:47.381932+0000","last_peered":"2023-07-21T22:28:47.381932+0000","last_clean":"2023-07-21T22:28:47.381932+0000","last_became_active":"2023-07-21T22:27:42.091217+0000","last_became_peered":"2023-07-21T22:27:42.091217+0000","last_unstale":"2023-07-21T22:28:47.381932+0000","last_undegraded":"2023-07-21T22:28:47.381932+0000","last_fullsized":"2023-07-21T22:28:47.381932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:31:21.491062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382557+0000","last_change":"2023-07-21T22:27:42.087505+0000","last_active":"2023-07-21T22:28:47.382557+0000","last_peered":"2023-07-21T22:28:47.382557+0000","last_clean":"2023-07-21T22:28:47.382557+0000","last_became_active":"2023-07-21T22:27:42.087177+0000","last_became_peered":"2023-07-21T22:27:42.087177+0000","last_unstale":"2023-07-21T22:28:47.382557+0000","last_undegraded":"2023-07-21T22:28:47.382557+0000","last_fullsized":"2023-07-21T22:28:47.382557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:06:51.361107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243856+0000","last_change":"2023-07-21T22:27:42.091490+0000","last_active":"2023-07-21T22:28:47.243856+0000","last_peered":"2023-07-21T22:28:47.243856+0000","last_clean":"2023-07-21T22:28:47.243856+0000","last_became_active":"2023-07-21T22:27:42.090782+0000","last_became_peered":"2023-07-21T22:27:42.090782+0000","last_unstale":"2023-07-21T22:28:47.243856+0000","last_undegraded":"2023-07-21T22:28:47.243856+0000","last_fullsized":"2023-07-21T22:28:47.243856+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:34:52.980195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243991+0000","last_change":"2023-07-21T22:27:42.092362+0000","last_active":"2023-07-21T22:28:47.243991+0000","last_peered":"2023-07-21T22:28:47.243991+0000","last_clean":"2023-07-21T22:28:47.243991+0000","last_became_active":"2023-07-21T22:27:42.091915+0000","last_became_peered":"2023-07-21T22:27:42.091915+0000","last_unstale":"2023-07-21T22:28:47.243991+0000","last_undegraded":"2023-07-21T22:28:47.243991+0000","last_fullsized":"2023-07-21T22:28:47.243991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:06:24.497745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242584+0000","last_change":"2023-07-21T22:27:42.098959+0000","last_active":"2023-07-21T22:28:47.242584+0000","last_peered":"2023-07-21T22:28:47.242584+0000","last_clean":"2023-07-21T22:28:47.242584+0000","last_became_active":"2023-07-21T22:27:42.094463+0000","last_became_peered":"2023-07-21T22:27:42.094463+0000","last_unstale":"2023-07-21T22:28:47.242584+0000","last_undegraded":"2023-07-21T22:28:47.242584+0000","last_fullsized":"2023-07-21T22:28:47.242584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:17:23.627490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":108,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:03.141677+0000","last_change":"2023-07-21T22:27:41.074299+0000","last_active":"2023-07-21T22:29:03.141677+0000","last_peered":"2023-07-21T22:29:03.141677+0000","last_clean":"2023-07-21T22:29:03.141677+0000","last_became_active":"2023-07-21T22:27:41.072895+0000","last_became_peered":"2023-07-21T22:27:41.072895+0000","last_unstale":"2023-07-21T22:29:03.141677+0000","last_undegraded":"2023-07-21T22:29:03.141677+0000","last_fullsized":"2023-07-21T22:29:03.141677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:24:58.146931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241511+0000","last_change":"2023-07-21T22:27:42.090477+0000","last_active":"2023-07-21T22:28:47.241511+0000","last_peered":"2023-07-21T22:28:47.241511+0000","last_clean":"2023-07-21T22:28:47.241511+0000","last_became_active":"2023-07-21T22:27:42.090144+0000","last_became_peered":"2023-07-21T22:27:42.090144+0000","last_unstale":"2023-07-21T22:28:47.241511+0000","last_undegraded":"2023-07-21T22:28:47.241511+0000","last_fullsized":"2023-07-21T22:28:47.241511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:42:38.964084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241632+0000","last_change":"2023-07-21T22:27:41.075178+0000","last_active":"2023-07-21T22:28:47.241632+0000","last_peered":"2023-07-21T22:28:47.241632+0000","last_clean":"2023-07-21T22:28:47.241632+0000","last_became_active":"2023-07-21T22:27:41.074861+0000","last_became_peered":"2023-07-21T22:27:41.074861+0000","last_unstale":"2023-07-21T22:28:47.241632+0000","last_undegraded":"2023-07-21T22:28:47.241632+0000","last_fullsized":"2023-07-21T22:28:47.241632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:09:39.913585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382382+0000","last_change":"2023-07-21T22:27:42.093135+0000","last_active":"2023-07-21T22:28:47.382382+0000","last_peered":"2023-07-21T22:28:47.382382+0000","last_clean":"2023-07-21T22:28:47.382382+0000","last_became_active":"2023-07-21T22:27:42.092457+0000","last_became_peered":"2023-07-21T22:27:42.092457+0000","last_unstale":"2023-07-21T22:28:47.382382+0000","last_undegraded":"2023-07-21T22:28:47.382382+0000","last_fullsized":"2023-07-21T22:28:47.382382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:37.129524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.249069+0000","last_change":"2023-07-21T22:27:41.076759+0000","last_active":"2023-07-21T22:28:47.249069+0000","last_peered":"2023-07-21T22:28:47.249069+0000","last_clean":"2023-07-21T22:28:47.249069+0000","last_became_active":"2023-07-21T22:27:41.076376+0000","last_became_peered":"2023-07-21T22:27:41.076376+0000","last_unstale":"2023-07-21T22:28:47.249069+0000","last_undegraded":"2023-07-21T22:28:47.249069+0000","last_fullsized":"2023-07-21T22:28:47.249069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:10:03.951373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382877+0000","last_change":"2023-07-21T22:27:42.093643+0000","last_active":"2023-07-21T22:28:47.382877+0000","last_peered":"2023-07-21T22:28:47.382877+0000","last_clean":"2023-07-21T22:28:47.382877+0000","last_became_active":"2023-07-21T22:27:42.093227+0000","last_became_peered":"2023-07-21T22:27:42.093227+0000","last_unstale":"2023-07-21T22:28:47.382877+0000","last_undegraded":"2023-07-21T22:28:47.382877+0000","last_fullsized":"2023-07-21T22:28:47.382877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:07:08.243533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.248547+0000","last_change":"2023-07-21T22:27:41.073119+0000","last_active":"2023-07-21T22:28:47.248547+0000","last_peered":"2023-07-21T22:28:47.248547+0000","last_clean":"2023-07-21T22:28:47.248547+0000","last_became_active":"2023-07-21T22:27:41.072813+0000","last_became_peered":"2023-07-21T22:27:41.072813+0000","last_unstale":"2023-07-21T22:28:47.248547+0000","last_undegraded":"2023-07-21T22:28:47.248547+0000","last_fullsized":"2023-07-21T22:28:47.248547+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:02.412440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383479+0000","last_change":"2023-07-21T22:28:47.383479+0000","last_active":"2023-07-21T22:28:47.383479+0000","last_peered":"2023-07-21T22:28:47.383479+0000","last_clean":"2023-07-21T22:28:47.383479+0000","last_became_active":"2023-07-21T22:28:47.383074+0000","last_became_peered":"2023-07-21T22:28:47.383074+0000","last_unstale":"2023-07-21T22:28:47.383479+0000","last_undegraded":"2023-07-21T22:28:47.383479+0000","last_fullsized":"2023-07-21T22:28:47.383479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:01:40.184725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":118,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:03.407481+0000","last_change":"2023-07-21T22:28:47.388489+0000","last_active":"2023-07-21T22:29:03.407481+0000","last_peered":"2023-07-21T22:29:03.407481+0000","last_clean":"2023-07-21T22:29:03.407481+0000","last_became_active":"2023-07-21T22:28:47.383903+0000","last_became_peered":"2023-07-21T22:28:47.383903+0000","last_unstale":"2023-07-21T22:29:03.407481+0000","last_undegraded":"2023-07-21T22:29:03.407481+0000","last_fullsized":"2023-07-21T22:29:03.407481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:25:48.113759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246961+0000","last_change":"2023-07-21T22:27:42.082970+0000","last_active":"2023-07-21T22:28:47.246961+0000","last_peered":"2023-07-21T22:28:47.246961+0000","last_clean":"2023-07-21T22:28:47.246961+0000","last_became_active":"2023-07-21T22:27:42.082676+0000","last_became_peered":"2023-07-21T22:27:42.082676+0000","last_unstale":"2023-07-21T22:28:47.246961+0000","last_undegraded":"2023-07-21T22:28:47.246961+0000","last_fullsized":"2023-07-21T22:28:47.246961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:47:24.831459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242691+0000","last_change":"2023-07-21T22:27:41.076844+0000","last_active":"2023-07-21T22:28:47.242691+0000","last_peered":"2023-07-21T22:28:47.242691+0000","last_clean":"2023-07-21T22:28:47.242691+0000","last_became_active":"2023-07-21T22:27:41.076487+0000","last_became_peered":"2023-07-21T22:27:41.076487+0000","last_unstale":"2023-07-21T22:28:47.242691+0000","last_undegraded":"2023-07-21T22:28:47.242691+0000","last_fullsized":"2023-07-21T22:28:47.242691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:03:37.023974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240958+0000","last_change":"2023-07-21T22:27:42.089996+0000","last_active":"2023-07-21T22:28:47.240958+0000","last_peered":"2023-07-21T22:28:47.240958+0000","last_clean":"2023-07-21T22:28:47.240958+0000","last_became_active":"2023-07-21T22:27:42.089668+0000","last_became_peered":"2023-07-21T22:27:42.089668+0000","last_unstale":"2023-07-21T22:28:47.240958+0000","last_undegraded":"2023-07-21T22:28:47.240958+0000","last_fullsized":"2023-07-21T22:28:47.240958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:55:01.151479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239183+0000","last_change":"2023-07-21T22:27:41.075864+0000","last_active":"2023-07-21T22:28:47.239183+0000","last_peered":"2023-07-21T22:28:47.239183+0000","last_clean":"2023-07-21T22:28:47.239183+0000","last_became_active":"2023-07-21T22:27:41.075577+0000","last_became_peered":"2023-07-21T22:27:41.075577+0000","last_unstale":"2023-07-21T22:28:47.239183+0000","last_undegraded":"2023-07-21T22:28:47.239183+0000","last_fullsized":"2023-07-21T22:28:47.239183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:41:53.383706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241389+0000","last_change":"2023-07-21T22:27:42.091730+0000","last_active":"2023-07-21T22:28:47.241389+0000","last_peered":"2023-07-21T22:28:47.241389+0000","last_clean":"2023-07-21T22:28:47.241389+0000","last_became_active":"2023-07-21T22:27:42.091394+0000","last_became_peered":"2023-07-21T22:27:42.091394+0000","last_unstale":"2023-07-21T22:28:47.241389+0000","last_undegraded":"2023-07-21T22:28:47.241389+0000","last_fullsized":"2023-07-21T22:28:47.241389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:21:14.595368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240215+0000","last_change":"2023-07-21T22:27:41.074822+0000","last_active":"2023-07-21T22:28:47.240215+0000","last_peered":"2023-07-21T22:28:47.240215+0000","last_clean":"2023-07-21T22:28:47.240215+0000","last_became_active":"2023-07-21T22:27:41.074542+0000","last_became_peered":"2023-07-21T22:27:41.074542+0000","last_unstale":"2023-07-21T22:28:47.240215+0000","last_undegraded":"2023-07-21T22:28:47.240215+0000","last_fullsized":"2023-07-21T22:28:47.240215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:56:37.030275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243691+0000","last_change":"2023-07-21T22:27:42.094351+0000","last_active":"2023-07-21T22:28:47.243691+0000","last_peered":"2023-07-21T22:28:47.243691+0000","last_clean":"2023-07-21T22:28:47.243691+0000","last_became_active":"2023-07-21T22:27:42.094068+0000","last_became_peered":"2023-07-21T22:27:42.094068+0000","last_unstale":"2023-07-21T22:28:47.243691+0000","last_undegraded":"2023-07-21T22:28:47.243691+0000","last_fullsized":"2023-07-21T22:28:47.243691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:55:43.799214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":170,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:00.402812+0000","last_change":"2023-07-21T22:27:41.079188+0000","last_active":"2023-07-21T22:29:00.402812+0000","last_peered":"2023-07-21T22:29:00.402812+0000","last_clean":"2023-07-21T22:29:00.402812+0000","last_became_active":"2023-07-21T22:27:41.078899+0000","last_became_peered":"2023-07-21T22:27:41.078899+0000","last_unstale":"2023-07-21T22:29:00.402812+0000","last_undegraded":"2023-07-21T22:29:00.402812+0000","last_fullsized":"2023-07-21T22:29:00.402812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:04:07.942421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383198+0000","last_change":"2023-07-21T22:27:42.091194+0000","last_active":"2023-07-21T22:28:47.383198+0000","last_peered":"2023-07-21T22:28:47.383198+0000","last_clean":"2023-07-21T22:28:47.383198+0000","last_became_active":"2023-07-21T22:27:42.090869+0000","last_became_peered":"2023-07-21T22:27:42.090869+0000","last_unstale":"2023-07-21T22:28:47.383198+0000","last_undegraded":"2023-07-21T22:28:47.383198+0000","last_fullsized":"2023-07-21T22:28:47.383198+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:46:43.577329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245092+0000","last_change":"2023-07-21T22:27:41.072701+0000","last_active":"2023-07-21T22:28:47.245092+0000","last_peered":"2023-07-21T22:28:47.245092+0000","last_clean":"2023-07-21T22:28:47.245092+0000","last_became_active":"2023-07-21T22:27:41.072358+0000","last_became_peered":"2023-07-21T22:27:41.072358+0000","last_unstale":"2023-07-21T22:28:47.245092+0000","last_undegraded":"2023-07-21T22:28:47.245092+0000","last_fullsized":"2023-07-21T22:28:47.245092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:11:20.209729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382347+0000","last_change":"2023-07-21T22:27:42.090808+0000","last_active":"2023-07-21T22:28:47.382347+0000","last_peered":"2023-07-21T22:28:47.382347+0000","last_clean":"2023-07-21T22:28:47.382347+0000","last_became_active":"2023-07-21T22:27:42.090506+0000","last_became_peered":"2023-07-21T22:27:42.090506+0000","last_unstale":"2023-07-21T22:28:47.382347+0000","last_undegraded":"2023-07-21T22:28:47.382347+0000","last_fullsized":"2023-07-21T22:28:47.382347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:15.825961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240934+0000","last_change":"2023-07-21T22:27:41.075643+0000","last_active":"2023-07-21T22:28:47.240934+0000","last_peered":"2023-07-21T22:28:47.240934+0000","last_clean":"2023-07-21T22:28:47.240934+0000","last_became_active":"2023-07-21T22:27:41.075336+0000","last_became_peered":"2023-07-21T22:27:41.075336+0000","last_unstale":"2023-07-21T22:28:47.240934+0000","last_undegraded":"2023-07-21T22:28:47.240934+0000","last_fullsized":"2023-07-21T22:28:47.240934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:25:15.271905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383144+0000","last_change":"2023-07-21T22:27:42.096722+0000","last_active":"2023-07-21T22:28:47.383144+0000","last_peered":"2023-07-21T22:28:47.383144+0000","last_clean":"2023-07-21T22:28:47.383144+0000","last_became_active":"2023-07-21T22:27:42.096472+0000","last_became_peered":"2023-07-21T22:27:42.096472+0000","last_unstale":"2023-07-21T22:28:47.383144+0000","last_undegraded":"2023-07-21T22:28:47.383144+0000","last_fullsized":"2023-07-21T22:28:47.383144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:27:25.223020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'28","reported_seq":202,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:00.402637+0000","last_change":"2023-07-21T22:27:41.072297+0000","last_active":"2023-07-21T22:29:00.402637+0000","last_peered":"2023-07-21T22:29:00.402637+0000","last_clean":"2023-07-21T22:29:00.402637+0000","last_became_active":"2023-07-21T22:27:41.071971+0000","last_became_peered":"2023-07-21T22:27:41.071971+0000","last_unstale":"2023-07-21T22:29:00.402637+0000","last_undegraded":"2023-07-21T22:29:00.402637+0000","last_fullsized":"2023-07-21T22:29:00.402637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:28:20.855102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54660,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":156,"num_read_kb":131,"num_write":30,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.248127+0000","last_change":"2023-07-21T22:27:42.097130+0000","last_active":"2023-07-21T22:28:47.248127+0000","last_peered":"2023-07-21T22:28:47.248127+0000","last_clean":"2023-07-21T22:28:47.248127+0000","last_became_active":"2023-07-21T22:27:42.096853+0000","last_became_peered":"2023-07-21T22:27:42.096853+0000","last_unstale":"2023-07-21T22:28:47.248127+0000","last_undegraded":"2023-07-21T22:28:47.248127+0000","last_fullsized":"2023-07-21T22:28:47.248127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:42:15.440580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242791+0000","last_change":"2023-07-21T22:27:41.071755+0000","last_active":"2023-07-21T22:28:47.242791+0000","last_peered":"2023-07-21T22:28:47.242791+0000","last_clean":"2023-07-21T22:28:47.242791+0000","last_became_active":"2023-07-21T22:27:41.071245+0000","last_became_peered":"2023-07-21T22:27:41.071245+0000","last_unstale":"2023-07-21T22:28:47.242791+0000","last_undegraded":"2023-07-21T22:28:47.242791+0000","last_fullsized":"2023-07-21T22:28:47.242791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:17:33.183524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239177+0000","last_change":"2023-07-21T22:27:42.097433+0000","last_active":"2023-07-21T22:28:47.239177+0000","last_peered":"2023-07-21T22:28:47.239177+0000","last_clean":"2023-07-21T22:28:47.239177+0000","last_became_active":"2023-07-21T22:27:42.097100+0000","last_became_peered":"2023-07-21T22:27:42.097100+0000","last_unstale":"2023-07-21T22:28:47.239177+0000","last_undegraded":"2023-07-21T22:28:47.239177+0000","last_fullsized":"2023-07-21T22:28:47.239177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:11:09.018578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240863+0000","last_change":"2023-07-21T22:27:41.074019+0000","last_active":"2023-07-21T22:28:47.240863+0000","last_peered":"2023-07-21T22:28:47.240863+0000","last_clean":"2023-07-21T22:28:47.240863+0000","last_became_active":"2023-07-21T22:27:41.073699+0000","last_became_peered":"2023-07-21T22:27:41.073699+0000","last_unstale":"2023-07-21T22:28:47.240863+0000","last_undegraded":"2023-07-21T22:28:47.240863+0000","last_fullsized":"2023-07-21T22:28:47.240863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:16:39.616177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245536+0000","last_change":"2023-07-21T22:27:42.098010+0000","last_active":"2023-07-21T22:28:47.245536+0000","last_peered":"2023-07-21T22:28:47.245536+0000","last_clean":"2023-07-21T22:28:47.245536+0000","last_became_active":"2023-07-21T22:27:42.097227+0000","last_became_peered":"2023-07-21T22:27:42.097227+0000","last_unstale":"2023-07-21T22:28:47.245536+0000","last_undegraded":"2023-07-21T22:28:47.245536+0000","last_fullsized":"2023-07-21T22:28:47.245536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:29:36.120927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":103,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:04.163532+0000","last_change":"2023-07-21T22:27:41.072298+0000","last_active":"2023-07-21T22:29:04.163532+0000","last_peered":"2023-07-21T22:29:04.163532+0000","last_clean":"2023-07-21T22:29:04.163532+0000","last_became_active":"2023-07-21T22:27:41.071963+0000","last_became_peered":"2023-07-21T22:27:41.071963+0000","last_unstale":"2023-07-21T22:29:04.163532+0000","last_undegraded":"2023-07-21T22:29:04.163532+0000","last_fullsized":"2023-07-21T22:29:04.163532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:14:19.498409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":858,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244894+0000","last_change":"2023-07-21T22:27:42.091276+0000","last_active":"2023-07-21T22:28:47.244894+0000","last_peered":"2023-07-21T22:28:47.244894+0000","last_clean":"2023-07-21T22:28:47.244894+0000","last_became_active":"2023-07-21T22:27:42.090918+0000","last_became_peered":"2023-07-21T22:27:42.090918+0000","last_unstale":"2023-07-21T22:28:47.244894+0000","last_undegraded":"2023-07-21T22:28:47.244894+0000","last_fullsized":"2023-07-21T22:28:47.244894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:15.527348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246226+0000","last_change":"2023-07-21T22:27:41.075633+0000","last_active":"2023-07-21T22:28:47.246226+0000","last_peered":"2023-07-21T22:28:47.246226+0000","last_clean":"2023-07-21T22:28:47.246226+0000","last_became_active":"2023-07-21T22:27:41.075331+0000","last_became_peered":"2023-07-21T22:27:41.075331+0000","last_unstale":"2023-07-21T22:28:47.246226+0000","last_undegraded":"2023-07-21T22:28:47.246226+0000","last_fullsized":"2023-07-21T22:28:47.246226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:26:08.447672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383325+0000","last_change":"2023-07-21T22:28:47.383325+0000","last_active":"2023-07-21T22:28:47.383325+0000","last_peered":"2023-07-21T22:28:47.383325+0000","last_clean":"2023-07-21T22:28:47.383325+0000","last_became_active":"2023-07-21T22:28:47.382987+0000","last_became_peered":"2023-07-21T22:28:47.382987+0000","last_unstale":"2023-07-21T22:28:47.383325+0000","last_undegraded":"2023-07-21T22:28:47.383325+0000","last_fullsized":"2023-07-21T22:28:47.383325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:51:52.190795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.255975+0000","last_change":"2023-07-21T22:28:47.255975+0000","last_active":"2023-07-21T22:28:47.255975+0000","last_peered":"2023-07-21T22:28:47.255975+0000","last_clean":"2023-07-21T22:28:47.255975+0000","last_became_active":"2023-07-21T22:28:47.248730+0000","last_became_peered":"2023-07-21T22:28:47.248730+0000","last_unstale":"2023-07-21T22:28:47.255975+0000","last_undegraded":"2023-07-21T22:28:47.255975+0000","last_fullsized":"2023-07-21T22:28:47.255975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:25:29.022962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243276+0000","last_change":"2023-07-21T22:27:42.092046+0000","last_active":"2023-07-21T22:28:47.243276+0000","last_peered":"2023-07-21T22:28:47.243276+0000","last_clean":"2023-07-21T22:28:47.243276+0000","last_became_active":"2023-07-21T22:27:42.091069+0000","last_became_peered":"2023-07-21T22:27:42.091069+0000","last_unstale":"2023-07-21T22:28:47.243276+0000","last_undegraded":"2023-07-21T22:28:47.243276+0000","last_fullsized":"2023-07-21T22:28:47.243276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:07:37.855640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243883+0000","last_change":"2023-07-21T22:27:41.074847+0000","last_active":"2023-07-21T22:28:47.243883+0000","last_peered":"2023-07-21T22:28:47.243883+0000","last_clean":"2023-07-21T22:28:47.243883+0000","last_became_active":"2023-07-21T22:27:41.074530+0000","last_became_peered":"2023-07-21T22:27:41.074530+0000","last_unstale":"2023-07-21T22:28:47.243883+0000","last_undegraded":"2023-07-21T22:28:47.243883+0000","last_fullsized":"2023-07-21T22:28:47.243883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:28:53.033226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242814+0000","last_change":"2023-07-21T22:27:42.089285+0000","last_active":"2023-07-21T22:28:47.242814+0000","last_peered":"2023-07-21T22:28:47.242814+0000","last_clean":"2023-07-21T22:28:47.242814+0000","last_became_active":"2023-07-21T22:27:42.088970+0000","last_became_peered":"2023-07-21T22:27:42.088970+0000","last_unstale":"2023-07-21T22:28:47.242814+0000","last_undegraded":"2023-07-21T22:28:47.242814+0000","last_fullsized":"2023-07-21T22:28:47.242814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:53:26.517730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:59.137194+0000","last_change":"2023-07-21T22:27:41.077433+0000","last_active":"2023-07-21T22:28:59.137194+0000","last_peered":"2023-07-21T22:28:59.137194+0000","last_clean":"2023-07-21T22:28:59.137194+0000","last_became_active":"2023-07-21T22:27:41.077133+0000","last_became_peered":"2023-07-21T22:27:41.077133+0000","last_unstale":"2023-07-21T22:28:59.137194+0000","last_undegraded":"2023-07-21T22:28:59.137194+0000","last_fullsized":"2023-07-21T22:28:59.137194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:09.473094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243171+0000","last_change":"2023-07-21T22:27:42.089833+0000","last_active":"2023-07-21T22:28:47.243171+0000","last_peered":"2023-07-21T22:28:47.243171+0000","last_clean":"2023-07-21T22:28:47.243171+0000","last_became_active":"2023-07-21T22:27:42.089502+0000","last_became_peered":"2023-07-21T22:27:42.089502+0000","last_unstale":"2023-07-21T22:28:47.243171+0000","last_undegraded":"2023-07-21T22:28:47.243171+0000","last_fullsized":"2023-07-21T22:28:47.243171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:33:04.121577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.248492+0000","last_change":"2023-07-21T22:28:47.248117+0000","last_active":"2023-07-21T22:28:47.248492+0000","last_peered":"2023-07-21T22:28:47.248492+0000","last_clean":"2023-07-21T22:28:47.248492+0000","last_became_active":"2023-07-21T22:28:47.247668+0000","last_became_peered":"2023-07-21T22:28:47.247668+0000","last_unstale":"2023-07-21T22:28:47.248492+0000","last_undegraded":"2023-07-21T22:28:47.248492+0000","last_fullsized":"2023-07-21T22:28:47.248492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:57:06.878014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:59.155853+0000","last_change":"2023-07-21T22:27:41.076784+0000","last_active":"2023-07-21T22:28:59.155853+0000","last_peered":"2023-07-21T22:28:59.155853+0000","last_clean":"2023-07-21T22:28:59.155853+0000","last_became_active":"2023-07-21T22:27:41.076453+0000","last_became_peered":"2023-07-21T22:27:41.076453+0000","last_unstale":"2023-07-21T22:28:59.155853+0000","last_undegraded":"2023-07-21T22:28:59.155853+0000","last_fullsized":"2023-07-21T22:28:59.155853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:32:01.149003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242778+0000","last_change":"2023-07-21T22:27:42.091286+0000","last_active":"2023-07-21T22:28:47.242778+0000","last_peered":"2023-07-21T22:28:47.242778+0000","last_clean":"2023-07-21T22:28:47.242778+0000","last_became_active":"2023-07-21T22:27:42.090986+0000","last_became_peered":"2023-07-21T22:27:42.090986+0000","last_unstale":"2023-07-21T22:28:47.242778+0000","last_undegraded":"2023-07-21T22:28:47.242778+0000","last_fullsized":"2023-07-21T22:28:47.242778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:19.449945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245874+0000","last_change":"2023-07-21T22:27:41.074075+0000","last_active":"2023-07-21T22:28:47.245874+0000","last_peered":"2023-07-21T22:28:47.245874+0000","last_clean":"2023-07-21T22:28:47.245874+0000","last_became_active":"2023-07-21T22:27:41.073757+0000","last_became_peered":"2023-07-21T22:27:41.073757+0000","last_unstale":"2023-07-21T22:28:47.245874+0000","last_undegraded":"2023-07-21T22:28:47.245874+0000","last_fullsized":"2023-07-21T22:28:47.245874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:53.883172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245145+0000","last_change":"2023-07-21T22:26:43.405788+0000","last_active":"2023-07-21T22:28:47.245145+0000","last_peered":"2023-07-21T22:28:47.245145+0000","last_clean":"2023-07-21T22:28:47.245145+0000","last_became_active":"2023-07-21T22:26:43.405351+0000","last_became_peered":"2023-07-21T22:26:43.405351+0000","last_unstale":"2023-07-21T22:28:47.245145+0000","last_undegraded":"2023-07-21T22:28:47.245145+0000","last_fullsized":"2023-07-21T22:28:47.245145+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_clean_scrub_stamp":"2023-07-21T22:23:52.032210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:20:43.690726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239038+0000","last_change":"2023-07-21T22:27:42.097967+0000","last_active":"2023-07-21T22:28:47.239038+0000","last_peered":"2023-07-21T22:28:47.239038+0000","last_clean":"2023-07-21T22:28:47.239038+0000","last_became_active":"2023-07-21T22:27:42.097708+0000","last_became_peered":"2023-07-21T22:27:42.097708+0000","last_unstale":"2023-07-21T22:28:47.239038+0000","last_undegraded":"2023-07-21T22:28:47.239038+0000","last_fullsized":"2023-07-21T22:28:47.239038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:47:00.150179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245422+0000","last_change":"2023-07-21T22:27:41.077853+0000","last_active":"2023-07-21T22:28:47.245422+0000","last_peered":"2023-07-21T22:28:47.245422+0000","last_clean":"2023-07-21T22:28:47.245422+0000","last_became_active":"2023-07-21T22:27:41.077539+0000","last_became_peered":"2023-07-21T22:27:41.077539+0000","last_unstale":"2023-07-21T22:28:47.245422+0000","last_undegraded":"2023-07-21T22:28:47.245422+0000","last_fullsized":"2023-07-21T22:28:47.245422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:11:54.875605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.248834+0000","last_change":"2023-07-21T22:27:41.076529+0000","last_active":"2023-07-21T22:28:47.248834+0000","last_peered":"2023-07-21T22:28:47.248834+0000","last_clean":"2023-07-21T22:28:47.248834+0000","last_became_active":"2023-07-21T22:27:41.076208+0000","last_became_peered":"2023-07-21T22:27:41.076208+0000","last_unstale":"2023-07-21T22:28:47.248834+0000","last_undegraded":"2023-07-21T22:28:47.248834+0000","last_fullsized":"2023-07-21T22:28:47.248834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:31:37.704902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239276+0000","last_change":"2023-07-21T22:27:42.082587+0000","last_active":"2023-07-21T22:28:47.239276+0000","last_peered":"2023-07-21T22:28:47.239276+0000","last_clean":"2023-07-21T22:28:47.239276+0000","last_became_active":"2023-07-21T22:27:42.082212+0000","last_became_peered":"2023-07-21T22:27:42.082212+0000","last_unstale":"2023-07-21T22:28:47.239276+0000","last_undegraded":"2023-07-21T22:28:47.239276+0000","last_fullsized":"2023-07-21T22:28:47.239276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:12:19.542426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244140+0000","last_change":"2023-07-21T22:27:41.073736+0000","last_active":"2023-07-21T22:28:47.244140+0000","last_peered":"2023-07-21T22:28:47.244140+0000","last_clean":"2023-07-21T22:28:47.244140+0000","last_became_active":"2023-07-21T22:27:41.073213+0000","last_became_peered":"2023-07-21T22:27:41.073213+0000","last_unstale":"2023-07-21T22:28:47.244140+0000","last_undegraded":"2023-07-21T22:28:47.244140+0000","last_fullsized":"2023-07-21T22:28:47.244140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:44:51.762687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242235+0000","last_change":"2023-07-21T22:27:42.091313+0000","last_active":"2023-07-21T22:28:47.242235+0000","last_peered":"2023-07-21T22:28:47.242235+0000","last_clean":"2023-07-21T22:28:47.242235+0000","last_became_active":"2023-07-21T22:27:42.091012+0000","last_became_peered":"2023-07-21T22:27:42.091012+0000","last_unstale":"2023-07-21T22:28:47.242235+0000","last_undegraded":"2023-07-21T22:28:47.242235+0000","last_fullsized":"2023-07-21T22:28:47.242235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:53:00.317577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245842+0000","last_change":"2023-07-21T22:27:41.074644+0000","last_active":"2023-07-21T22:28:47.245842+0000","last_peered":"2023-07-21T22:28:47.245842+0000","last_clean":"2023-07-21T22:28:47.245842+0000","last_became_active":"2023-07-21T22:27:41.074364+0000","last_became_peered":"2023-07-21T22:27:41.074364+0000","last_unstale":"2023-07-21T22:28:47.245842+0000","last_undegraded":"2023-07-21T22:28:47.245842+0000","last_fullsized":"2023-07-21T22:28:47.245842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:23:08.316538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382240+0000","last_change":"2023-07-21T22:27:42.089452+0000","last_active":"2023-07-21T22:28:47.382240+0000","last_peered":"2023-07-21T22:28:47.382240+0000","last_clean":"2023-07-21T22:28:47.382240+0000","last_became_active":"2023-07-21T22:27:42.088859+0000","last_became_peered":"2023-07-21T22:27:42.088859+0000","last_unstale":"2023-07-21T22:28:47.382240+0000","last_undegraded":"2023-07-21T22:28:47.382240+0000","last_fullsized":"2023-07-21T22:28:47.382240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:38:51.287342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382549+0000","last_change":"2023-07-21T22:27:41.077067+0000","last_active":"2023-07-21T22:28:47.382549+0000","last_peered":"2023-07-21T22:28:47.382549+0000","last_clean":"2023-07-21T22:28:47.382549+0000","last_became_active":"2023-07-21T22:27:41.076768+0000","last_became_peered":"2023-07-21T22:27:41.076768+0000","last_unstale":"2023-07-21T22:28:47.382549+0000","last_undegraded":"2023-07-21T22:28:47.382549+0000","last_fullsized":"2023-07-21T22:28:47.382549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:07:35.434120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.247883+0000","last_change":"2023-07-21T22:27:42.088789+0000","last_active":"2023-07-21T22:28:47.247883+0000","last_peered":"2023-07-21T22:28:47.247883+0000","last_clean":"2023-07-21T22:28:47.247883+0000","last_became_active":"2023-07-21T22:27:42.088361+0000","last_became_peered":"2023-07-21T22:27:42.088361+0000","last_unstale":"2023-07-21T22:28:47.247883+0000","last_undegraded":"2023-07-21T22:28:47.247883+0000","last_fullsized":"2023-07-21T22:28:47.247883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:51:10.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241552+0000","last_change":"2023-07-21T22:27:41.069595+0000","last_active":"2023-07-21T22:28:47.241552+0000","last_peered":"2023-07-21T22:28:47.241552+0000","last_clean":"2023-07-21T22:28:47.241552+0000","last_became_active":"2023-07-21T22:27:41.068733+0000","last_became_peered":"2023-07-21T22:27:41.068733+0000","last_unstale":"2023-07-21T22:28:47.241552+0000","last_undegraded":"2023-07-21T22:28:47.241552+0000","last_fullsized":"2023-07-21T22:28:47.241552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:49.910593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243049+0000","last_change":"2023-07-21T22:27:42.085701+0000","last_active":"2023-07-21T22:28:47.243049+0000","last_peered":"2023-07-21T22:28:47.243049+0000","last_clean":"2023-07-21T22:28:47.243049+0000","last_became_active":"2023-07-21T22:27:42.085429+0000","last_became_peered":"2023-07-21T22:27:42.085429+0000","last_unstale":"2023-07-21T22:28:47.243049+0000","last_undegraded":"2023-07-21T22:28:47.243049+0000","last_fullsized":"2023-07-21T22:28:47.243049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:34:44.915735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243657+0000","last_change":"2023-07-21T22:27:41.073233+0000","last_active":"2023-07-21T22:28:47.243657+0000","last_peered":"2023-07-21T22:28:47.243657+0000","last_clean":"2023-07-21T22:28:47.243657+0000","last_became_active":"2023-07-21T22:27:41.072866+0000","last_became_peered":"2023-07-21T22:27:41.072866+0000","last_unstale":"2023-07-21T22:28:47.243657+0000","last_undegraded":"2023-07-21T22:28:47.243657+0000","last_fullsized":"2023-07-21T22:28:47.243657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:34:44.838691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241957+0000","last_change":"2023-07-21T22:27:42.100093+0000","last_active":"2023-07-21T22:28:47.241957+0000","last_peered":"2023-07-21T22:28:47.241957+0000","last_clean":"2023-07-21T22:28:47.241957+0000","last_became_active":"2023-07-21T22:27:42.099813+0000","last_became_peered":"2023-07-21T22:27:42.099813+0000","last_unstale":"2023-07-21T22:28:47.241957+0000","last_undegraded":"2023-07-21T22:28:47.241957+0000","last_fullsized":"2023-07-21T22:28:47.241957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:52:06.785113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.259961+0000","last_change":"2023-07-21T22:28:47.259961+0000","last_active":"2023-07-21T22:28:47.259961+0000","last_peered":"2023-07-21T22:28:47.259961+0000","last_clean":"2023-07-21T22:28:47.259961+0000","last_became_active":"2023-07-21T22:28:47.248979+0000","last_became_peered":"2023-07-21T22:28:47.248979+0000","last_unstale":"2023-07-21T22:28:47.259961+0000","last_undegraded":"2023-07-21T22:28:47.259961+0000","last_fullsized":"2023-07-21T22:28:47.259961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:12:44.807097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245648+0000","last_change":"2023-07-21T22:27:42.097736+0000","last_active":"2023-07-21T22:28:47.245648+0000","last_peered":"2023-07-21T22:28:47.245648+0000","last_clean":"2023-07-21T22:28:47.245648+0000","last_became_active":"2023-07-21T22:27:42.097419+0000","last_became_peered":"2023-07-21T22:27:42.097419+0000","last_unstale":"2023-07-21T22:28:47.245648+0000","last_undegraded":"2023-07-21T22:28:47.245648+0000","last_fullsized":"2023-07-21T22:28:47.245648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:07:09.477979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":99,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:00.157924+0000","last_change":"2023-07-21T22:27:41.074922+0000","last_active":"2023-07-21T22:29:00.157924+0000","last_peered":"2023-07-21T22:29:00.157924+0000","last_clean":"2023-07-21T22:29:00.157924+0000","last_became_active":"2023-07-21T22:27:41.074610+0000","last_became_peered":"2023-07-21T22:27:41.074610+0000","last_unstale":"2023-07-21T22:29:00.157924+0000","last_undegraded":"2023-07-21T22:29:00.157924+0000","last_fullsized":"2023-07-21T22:29:00.157924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:19.892702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245234+0000","last_change":"2023-07-21T22:27:42.097312+0000","last_active":"2023-07-21T22:28:47.245234+0000","last_peered":"2023-07-21T22:28:47.245234+0000","last_clean":"2023-07-21T22:28:47.245234+0000","last_became_active":"2023-07-21T22:27:42.096835+0000","last_became_peered":"2023-07-21T22:27:42.096835+0000","last_unstale":"2023-07-21T22:28:47.245234+0000","last_undegraded":"2023-07-21T22:28:47.245234+0000","last_fullsized":"2023-07-21T22:28:47.245234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:34:28.949507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240533+0000","last_change":"2023-07-21T22:27:41.071971+0000","last_active":"2023-07-21T22:28:47.240533+0000","last_peered":"2023-07-21T22:28:47.240533+0000","last_clean":"2023-07-21T22:28:47.240533+0000","last_became_active":"2023-07-21T22:27:41.071550+0000","last_became_peered":"2023-07-21T22:27:41.071550+0000","last_unstale":"2023-07-21T22:28:47.240533+0000","last_undegraded":"2023-07-21T22:28:47.240533+0000","last_fullsized":"2023-07-21T22:28:47.240533+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:35:28.749230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244847+0000","last_change":"2023-07-21T22:27:42.098039+0000","last_active":"2023-07-21T22:28:47.244847+0000","last_peered":"2023-07-21T22:28:47.244847+0000","last_clean":"2023-07-21T22:28:47.244847+0000","last_became_active":"2023-07-21T22:27:42.097737+0000","last_became_peered":"2023-07-21T22:27:42.097737+0000","last_unstale":"2023-07-21T22:28:47.244847+0000","last_undegraded":"2023-07-21T22:28:47.244847+0000","last_fullsized":"2023-07-21T22:28:47.244847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:41:06.382770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.251946+0000","last_change":"2023-07-21T22:28:47.251946+0000","last_active":"2023-07-21T22:28:47.251946+0000","last_peered":"2023-07-21T22:28:47.251946+0000","last_clean":"2023-07-21T22:28:47.251946+0000","last_became_active":"2023-07-21T22:28:47.247973+0000","last_became_peered":"2023-07-21T22:28:47.247973+0000","last_unstale":"2023-07-21T22:28:47.251946+0000","last_undegraded":"2023-07-21T22:28:47.251946+0000","last_fullsized":"2023-07-21T22:28:47.251946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:18.015306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245217+0000","last_change":"2023-07-21T22:27:42.097891+0000","last_active":"2023-07-21T22:28:47.245217+0000","last_peered":"2023-07-21T22:28:47.245217+0000","last_clean":"2023-07-21T22:28:47.245217+0000","last_became_active":"2023-07-21T22:27:42.097022+0000","last_became_peered":"2023-07-21T22:27:42.097022+0000","last_unstale":"2023-07-21T22:28:47.245217+0000","last_undegraded":"2023-07-21T22:28:47.245217+0000","last_fullsized":"2023-07-21T22:28:47.245217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:54:17.707405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243207+0000","last_change":"2023-07-21T22:27:42.085774+0000","last_active":"2023-07-21T22:28:47.243207+0000","last_peered":"2023-07-21T22:28:47.243207+0000","last_clean":"2023-07-21T22:28:47.243207+0000","last_became_active":"2023-07-21T22:27:42.085460+0000","last_became_peered":"2023-07-21T22:27:42.085460+0000","last_unstale":"2023-07-21T22:28:47.243207+0000","last_undegraded":"2023-07-21T22:28:47.243207+0000","last_fullsized":"2023-07-21T22:28:47.243207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:26:42.082026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.251186+0000","last_change":"2023-07-21T22:28:47.251186+0000","last_active":"2023-07-21T22:28:47.251186+0000","last_peered":"2023-07-21T22:28:47.251186+0000","last_clean":"2023-07-21T22:28:47.251186+0000","last_became_active":"2023-07-21T22:28:47.250765+0000","last_became_peered":"2023-07-21T22:28:47.250765+0000","last_unstale":"2023-07-21T22:28:47.251186+0000","last_undegraded":"2023-07-21T22:28:47.251186+0000","last_fullsized":"2023-07-21T22:28:47.251186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:12:57.007813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242433+0000","last_change":"2023-07-21T22:27:42.092918+0000","last_active":"2023-07-21T22:28:47.242433+0000","last_peered":"2023-07-21T22:28:47.242433+0000","last_clean":"2023-07-21T22:28:47.242433+0000","last_became_active":"2023-07-21T22:27:42.092484+0000","last_became_peered":"2023-07-21T22:27:42.092484+0000","last_unstale":"2023-07-21T22:28:47.242433+0000","last_undegraded":"2023-07-21T22:28:47.242433+0000","last_fullsized":"2023-07-21T22:28:47.242433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:43:03.935206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382257+0000","last_change":"2023-07-21T22:27:42.096306+0000","last_active":"2023-07-21T22:28:47.382257+0000","last_peered":"2023-07-21T22:28:47.382257+0000","last_clean":"2023-07-21T22:28:47.382257+0000","last_became_active":"2023-07-21T22:27:42.096010+0000","last_became_peered":"2023-07-21T22:27:42.096010+0000","last_unstale":"2023-07-21T22:28:47.382257+0000","last_undegraded":"2023-07-21T22:28:47.382257+0000","last_fullsized":"2023-07-21T22:28:47.382257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:20:04.344089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239283+0000","last_change":"2023-07-21T22:27:42.097964+0000","last_active":"2023-07-21T22:28:47.239283+0000","last_peered":"2023-07-21T22:28:47.239283+0000","last_clean":"2023-07-21T22:28:47.239283+0000","last_became_active":"2023-07-21T22:27:42.097666+0000","last_became_peered":"2023-07-21T22:27:42.097666+0000","last_unstale":"2023-07-21T22:28:47.239283+0000","last_undegraded":"2023-07-21T22:28:47.239283+0000","last_fullsized":"2023-07-21T22:28:47.239283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:48:04.636292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243609+0000","last_change":"2023-07-21T22:27:42.083225+0000","last_active":"2023-07-21T22:28:47.243609+0000","last_peered":"2023-07-21T22:28:47.243609+0000","last_clean":"2023-07-21T22:28:47.243609+0000","last_became_active":"2023-07-21T22:27:42.082892+0000","last_became_peered":"2023-07-21T22:27:42.082892+0000","last_unstale":"2023-07-21T22:28:47.243609+0000","last_undegraded":"2023-07-21T22:28:47.243609+0000","last_fullsized":"2023-07-21T22:28:47.243609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:57:37.483988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243881+0000","last_change":"2023-07-21T22:27:42.090854+0000","last_active":"2023-07-21T22:28:47.243881+0000","last_peered":"2023-07-21T22:28:47.243881+0000","last_clean":"2023-07-21T22:28:47.243881+0000","last_became_active":"2023-07-21T22:27:42.090501+0000","last_became_peered":"2023-07-21T22:27:42.090501+0000","last_unstale":"2023-07-21T22:28:47.243881+0000","last_undegraded":"2023-07-21T22:28:47.243881+0000","last_fullsized":"2023-07-21T22:28:47.243881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:43:35.575019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242109+0000","last_change":"2023-07-21T22:27:42.094986+0000","last_active":"2023-07-21T22:28:47.242109+0000","last_peered":"2023-07-21T22:28:47.242109+0000","last_clean":"2023-07-21T22:28:47.242109+0000","last_became_active":"2023-07-21T22:27:42.094668+0000","last_became_peered":"2023-07-21T22:27:42.094668+0000","last_unstale":"2023-07-21T22:28:47.242109+0000","last_undegraded":"2023-07-21T22:28:47.242109+0000","last_fullsized":"2023-07-21T22:28:47.242109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:17:57.739239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240373+0000","last_change":"2023-07-21T22:27:42.098033+0000","last_active":"2023-07-21T22:28:47.240373+0000","last_peered":"2023-07-21T22:28:47.240373+0000","last_clean":"2023-07-21T22:28:47.240373+0000","last_became_active":"2023-07-21T22:27:42.097219+0000","last_became_peered":"2023-07-21T22:27:42.097219+0000","last_unstale":"2023-07-21T22:28:47.240373+0000","last_undegraded":"2023-07-21T22:28:47.240373+0000","last_fullsized":"2023-07-21T22:28:47.240373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:58:21.227651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241687+0000","last_change":"2023-07-21T22:27:42.092136+0000","last_active":"2023-07-21T22:28:47.241687+0000","last_peered":"2023-07-21T22:28:47.241687+0000","last_clean":"2023-07-21T22:28:47.241687+0000","last_became_active":"2023-07-21T22:27:42.091559+0000","last_became_peered":"2023-07-21T22:27:42.091559+0000","last_unstale":"2023-07-21T22:28:47.241687+0000","last_undegraded":"2023-07-21T22:28:47.241687+0000","last_fullsized":"2023-07-21T22:28:47.241687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:56.512118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382811+0000","last_change":"2023-07-21T22:27:42.093227+0000","last_active":"2023-07-21T22:28:47.382811+0000","last_peered":"2023-07-21T22:28:47.382811+0000","last_clean":"2023-07-21T22:28:47.382811+0000","last_became_active":"2023-07-21T22:27:42.092918+0000","last_became_peered":"2023-07-21T22:27:42.092918+0000","last_unstale":"2023-07-21T22:28:47.382811+0000","last_undegraded":"2023-07-21T22:28:47.382811+0000","last_fullsized":"2023-07-21T22:28:47.382811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:00:34.144455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239365+0000","last_change":"2023-07-21T22:27:42.086494+0000","last_active":"2023-07-21T22:28:47.239365+0000","last_peered":"2023-07-21T22:28:47.239365+0000","last_clean":"2023-07-21T22:28:47.239365+0000","last_became_active":"2023-07-21T22:27:42.086189+0000","last_became_peered":"2023-07-21T22:27:42.086189+0000","last_unstale":"2023-07-21T22:28:47.239365+0000","last_undegraded":"2023-07-21T22:28:47.239365+0000","last_fullsized":"2023-07-21T22:28:47.239365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:01:11.731302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242968+0000","last_change":"2023-07-21T22:27:42.090874+0000","last_active":"2023-07-21T22:28:47.242968+0000","last_peered":"2023-07-21T22:28:47.242968+0000","last_clean":"2023-07-21T22:28:47.242968+0000","last_became_active":"2023-07-21T22:27:42.090597+0000","last_became_peered":"2023-07-21T22:27:42.090597+0000","last_unstale":"2023-07-21T22:28:47.242968+0000","last_undegraded":"2023-07-21T22:28:47.242968+0000","last_fullsized":"2023-07-21T22:28:47.242968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:16:50.131217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61988,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":779,"num_read_kb":456,"num_write":230,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":986,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":185646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.629},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.52000000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.501},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.80900000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.82499999999999996}]},{"osd":9,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.78900000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.70399999999999996}]}]},{"osd":10,"up_from":70,"seq":300647710752,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1613824,"data_stored":1085151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":8.7710000000000008,"5min":8.7710000000000008,"15min":8.7710000000000008},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":8.8610000000000007,"5min":8.8610000000000007,"15min":8.8610000000000007},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.47299999999999998,"15min":0.46999999999999997},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70399999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.505,"15min":0.501},"min":{"1min":0.40100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.38100000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.495,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80100000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.73899999999999999,"15min":0.67700000000000005},"min":{"1min":0.45100000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":8.5139999999999993,"5min":8.5139999999999993,"15min":8.5139999999999993},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":0.75800000000000001,"15min":0.67800000000000005},"min":{"1min":0.47699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":8.5440000000000005,"5min":8.5440000000000005,"15min":8.5440000000000005},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.36499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75800000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.499}]},{"osd":5,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67900000000000005,"15min":0.67400000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.47499999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.98399999999999999,"5min":1.008,"15min":1.008},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.76400000000000001,"15min":0.72399999999999998},"min":{"1min":0.56899999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":3.972,"5min":3.972,"15min":3.972},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.087,"5min":0.76900000000000002,"15min":0.71599999999999997},"min":{"1min":0.51100000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.8570000000000002,"5min":3.8570000000000002,"15min":3.8570000000000002},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0089999999999999,"5min":1.073,"15min":1.073},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.998,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.69399999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":0.90100000000000002,"15min":0.82199999999999995},"min":{"1min":0.52900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":8.5899999999999999,"5min":8.5899999999999999,"15min":8.5899999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.82299999999999995,"15min":0.81200000000000006},"min":{"1min":0.57499999999999996,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.1469999999999998,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.67000000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.76000000000000001}]}]},{"osd":9,"up_from":65,"seq":279172874277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":439765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.438,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.0379999999999998,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.129,"5min":3.129,"15min":3.129},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.51500000000000001,"15min":0.53000000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.436}]},{"osd":4,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.66500000000000004,"15min":0.68500000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":2.9710000000000001,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0469999999999999,"5min":1.089,"15min":1.089},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.66400000000000003,"15min":0.68600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93700000000000006,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.749}]},{"osd":8,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.73499999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:28:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.72599999999999998,"15min":0.73799999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.745,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.78900000000000003}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":440121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.62},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.35999999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.752}]},{"osd":2,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.45400000000000001,"15min":0.438},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.48099999999999998,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60099999999999998,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.71799999999999997}]},{"osd":4,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.434,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61599999999999999,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.873},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69999999999999996,"15min":0.69699999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.997,"5min":1.097,"15min":1.097},"last":1.004}]},{"osd":7,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.55800000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.054,"5min":1.171,"15min":1.171},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.68999999999999995,"15min":0.68000000000000005},"min":{"1min":0.51000000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1830000000000001,"5min":1.232,"15min":1.232},"last":0.89300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69799999999999995,"15min":0.68400000000000005},"min":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0680000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.94799999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.746,"15min":0.71899999999999997},"min":{"1min":0.67500000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0800000000000001,"5min":1.21,"15min":1.21},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.76900000000000002,"15min":0.745},"min":{"1min":0.68500000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.97699999999999998}]},{"osd":11,"last update":"Fri Jul 21 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.96199999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28456,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716696,"statfs":{"total":95995035648,"available":95965896704,"internally_reserved":0,"allocated":1540096,"data_stored":1032522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53800000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.436,"15min":0.437},"min":{"1min":0.32700000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46400000000000002,"15min":0.46500000000000002},"min":{"1min":0.32900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.34599999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83599999999999997,"5min":1.548,"15min":1.548},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.80700000000000005,"15min":0.88700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82599999999999996,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.77600000000000002,"15min":0.82699999999999996},"min":{"1min":0.434,"5min":0.374,"15min":0.374},"max":{"1min":2.169,"5min":5.7480000000000002,"15min":5.7480000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.78000000000000003,"15min":0.85399999999999998},"min":{"1min":0.45000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.871,"5min":5.7649999999999997,"15min":5.7649999999999997},"last":0.753}]},{"osd":4,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.42999999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.41499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.80300000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.44400000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67700000000000005,"15min":0.66900000000000004},"min":{"1min":0.51600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91800000000000004,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90600000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.69399999999999995,"15min":0.68300000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89800000000000002,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.73599999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.91000000000000003,"15min":0.98399999999999999},"min":{"1min":0.57999999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.0009999999999999,"5min":5.7800000000000002,"15min":5.7800000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78300000000000003,"15min":0.79900000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1060000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.54200000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.79100000000000004,"15min":0.80700000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85799999999999998,"5min":1.819,"15min":1.819},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.753,"15min":0.76100000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.88300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.56499999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.97999999999999998,"5min":1.839,"15min":1.839},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.57599999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96599999999999997,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.76900000000000002}]},{"osd":11,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.70199999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528368,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":439440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.36499999999999999},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.624,"15min":0.56299999999999994},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":6.899,"5min":6.899,"15min":6.899},"last":0.628},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.45600000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54000000000000004,"15min":0.52000000000000002},"min":{"1min":0.44800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.64500000000000002,"15min":0.57499999999999996},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":7.0359999999999996,"5min":7.0359999999999996,"15min":7.0359999999999996},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.496,"15min":0.437},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":7.1289999999999996,"5min":7.1289999999999996,"15min":7.1289999999999996},"last":0.315},{"interface":"front","average":{"1min":1.071,"5min":0.58399999999999996,"15min":0.48599999999999999},"min":{"1min":0.32000000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":12.414999999999999,"5min":12.414999999999999,"15min":12.414999999999999},"last":0.42099999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.64900000000000002,"15min":0.58699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.9690000000000003,"5min":6.9690000000000003,"15min":6.9690000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.48499999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.44700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.60199999999999998,"15min":0.58799999999999997},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59299999999999997,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.70099999999999996,"15min":0.64000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":7.0049999999999999,"5min":7.0049999999999999,"15min":7.0049999999999999},"last":0.55600000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.747,"15min":0.69899999999999995},"min":{"1min":0.55000000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":7.0670000000000002,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70099999999999996,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.60899999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.68000000000000005,"15min":0.66700000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.516,"5min":4.516,"15min":4.516},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.78000000000000003,"15min":0.75600000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":7.1619999999999999,"5min":7.1619999999999999,"15min":7.1619999999999999},"last":0.496}]},{"osd":10,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.83599999999999997,"15min":0.80800000000000005},"min":{"1min":0.52300000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":7.0979999999999999,"5min":7.0979999999999999,"15min":7.0979999999999999},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":496289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51500000000000001,"15min":0.48499999999999999},"min":{"1min":0.41299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.48699999999999999},"min":{"1min":0.40999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.80000000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.505}]},{"osd":2,"last update":"Fri Jul 21 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.57799999999999996,"15min":0.52100000000000002},"min":{"1min":0.46800000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.79900000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56699999999999995,"15min":0.54000000000000004},"min":{"1min":0.48699999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.38300000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.872,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60199999999999998,"15min":0.58999999999999997},"min":{"1min":0.49299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.46400000000000002,"15min":0.42799999999999999},"min":{"1min":0.35999999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.46800000000000003,"15min":0.438},"min":{"1min":0.35399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.60999999999999999,"15min":0.54800000000000004},"min":{"1min":0.48999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86099999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.63400000000000001,"15min":0.57399999999999995},"min":{"1min":0.54000000000000004,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri Jul 21 22:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56299999999999994,"15min":0.53200000000000003},"min":{"1min":0.45300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.61899999999999999,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.752}]},{"osd":7,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.59699999999999998,"15min":0.55200000000000005},"min":{"1min":0.48299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.93500000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.64600000000000002,"15min":0.61199999999999999},"min":{"1min":0.437,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96799999999999997,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82999999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.70099999999999996,"15min":0.67000000000000004},"min":{"1min":0.46300000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92700000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.72499999999999998,"15min":0.69499999999999995},"min":{"1min":0.64800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77400000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.626,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79000000000000004,"15min":0.78500000000000003},"min":{"1min":0.58299999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.80600000000000005}]},{"osd":11,"last update":"Fri Jul 21 22:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.78400000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":441163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.48899999999999999},"min":{"1min":0.372,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.746,"5min":0.873,"15min":0.873},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.501},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68700000000000006,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78800000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55000000000000004,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70099999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.621}]},{"osd":3,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75700000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.441},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83199999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52400000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78500000000000003,"5min":0.879,"15min":0.879},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78500000000000003,"5min":1.002,"15min":1.002},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48299999999999998,"15min":0.49199999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58499999999999996,"5min":0.749,"15min":0.749},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79100000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.40799999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66500000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85499999999999998,"5min":1.04,"15min":1.04},"last":0.48499999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.68400000000000005,"15min":0.69099999999999995},"min":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.61899999999999999,"15min":0.63500000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88600000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.68200000000000005,"15min":0.70399999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89800000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46500000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93700000000000006,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88400000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.501}]},{"osd":11,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69599999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":494995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.501,"15min":0.47699999999999998},"min":{"1min":0.38300000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52000000000000002,"15min":0.47099999999999997},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.68500000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.56399999999999995,"15min":0.498},"min":{"1min":0.43099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.85999999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.57299999999999995,"15min":0.52300000000000002},"min":{"1min":0.45800000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.874,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57799999999999996,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.58199999999999996,"15min":0.54900000000000004},"min":{"1min":0.44600000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.501,"15min":0.46899999999999997},"min":{"1min":0.375,"5min":0.25,"15min":0.25},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.501,"15min":0.47499999999999998},"min":{"1min":0.41199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.435,"5min":0.432,"15min":0.432},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46899999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.58899999999999997,"15min":0.55300000000000005},"min":{"1min":0.46600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.625,"15min":0.59699999999999998},"min":{"1min":0.51900000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.51100000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.63700000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.73099999999999998,"15min":0.71299999999999997},"min":{"1min":0.68300000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.98499999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.92100000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.85299999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":438988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35599999999999998,"15min":0.33500000000000002},"min":{"1min":0.24399999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.629,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.26100000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.496,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.29799999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71499999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.503}]},{"osd":2,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76000000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.495},{"interface":"front","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.57999999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82999999999999996,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70599999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.61399999999999999,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.749,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.50800000000000001,"15min":0.501},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.439}]},{"osd":7,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.40200000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79000000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.65700000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85499999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.56999999999999995}]},{"osd":8,"last update":"Fri Jul 21 22:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.64300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.996,"5min":1.038,"15min":1.038},"last":0.47199999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77300000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.84899999999999998,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.64700000000000002}]},{"osd":11,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.60799999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":439613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.47899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.49299999999999999},"min":{"1min":0.41799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41799999999999998,"15min":0.39700000000000002},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41799999999999998,"15min":0.39200000000000002},"min":{"1min":0.34200000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59099999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.44900000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.443,"5min":0.371,"15min":0.371},"max":{"1min":0.82999999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.83399999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.60899999999999999,"15min":0.54300000000000004},"min":{"1min":0.40100000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.6559999999999997,"5min":5.6559999999999997,"15min":5.6559999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.56699999999999995,"15min":0.53300000000000003},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.5430000000000001,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Jul 21 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.61699999999999999,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.435,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79600000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.78100000000000003}]},{"osd":6,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.65900000000000003,"15min":0.60699999999999998},"min":{"1min":0.45300000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":5.6310000000000002,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.68899999999999995,"15min":0.629},"min":{"1min":0.53100000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61199999999999999,"15min":0.59299999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97299999999999998,"5min":0.995,"15min":0.995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60399999999999998,"15min":0.59299999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72299999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Jul 21 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.54400000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89000000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86699999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80100000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":0.81299999999999994,"15min":0.755},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":7.6639999999999997,"5min":7.6639999999999997,"15min":7.6639999999999997},"last":0.754},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.81699999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78000000000000003,"15min":0.78700000000000003},"min":{"1min":0.53200000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.94599999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77500000000000002,"15min":0.77800000000000002},"min":{"1min":0.496,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94799999999999995,"5min":1.103,"15min":1.103},"last":0.86799999999999999}]},{"osd":11,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.85199999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1512,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1548288,"data_stored":1031965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52000000000000002},"min":{"1min":0.315,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80200000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.51500000000000001,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54600000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.45700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81899999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58799999999999997,"15min":0.54400000000000004},"min":{"1min":0.53000000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89000000000000001,"5min":1.036,"15min":1.036},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.42599999999999999,"15min":0.39100000000000001},"min":{"1min":0.32000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67700000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.43099999999999999,"15min":0.40300000000000002},"min":{"1min":0.32000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.64800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.39900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.436},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58499999999999996,"15min":0.59499999999999997},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76900000000000002,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61299999999999999,"15min":0.56899999999999995},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83599999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.48099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.874,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.629},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63600000000000001,"15min":0.61299999999999999},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89400000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.61499999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90000000000000002,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.67400000000000004,"15min":0.64400000000000002},"min":{"1min":0.58099999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92800000000000005,"5min":1.054,"15min":1.054},"last":0.72899999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.628},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89100000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.749},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.59699999999999998},"min":{"1min":0.45300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81699999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69099999999999995,"15min":0.68700000000000006},"min":{"1min":0.53200000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.54900000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77300000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.80800000000000005,"15min":0.80600000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.99099999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77200000000000002,"15min":0.76800000000000002},"min":{"1min":0.58499999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91600000000000004,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.81299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":57646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:29:07.046 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:29:07.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.153+0000 7f773b65f700 1 -- 172.21.15.107:0/2859315648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77340ffb10 msgr2=0x7f77340fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.153+0000 7f773b65f700 1 --2- 172.21.15.107:0/2859315648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77340ffb10 0x7f77340fff20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7724009990 tx=0x7f7724009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.153+0000 7f773b65f700 1 -- 172.21.15.107:0/2859315648 shutdown_connections 2023-07-21T22:29:07.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.153+0000 7f773b65f700 1 --2- 172.21.15.107:0/2859315648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7734101690 0x7f7734107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.153+0000 7f773b65f700 1 --2- 172.21.15.107:0/2859315648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77340ffb10 0x7f77340fff20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.153+0000 7f773b65f700 1 --2- 172.21.15.107:0/2859315648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7734100d00 0x7f7734101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.153+0000 7f773b65f700 1 -- 172.21.15.107:0/2859315648 >> 172.21.15.107:0/2859315648 conn(0x7f77340fb110 msgr2=0x7f77340fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:07.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.153+0000 7f773b65f700 1 -- 172.21.15.107:0/2859315648 shutdown_connections 2023-07-21T22:29:07.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.153+0000 7f773b65f700 1 -- 172.21.15.107:0/2859315648 wait complete. 2023-07-21T22:29:07.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.153+0000 7f773b65f700 1 Processor -- start 2023-07-21T22:29:07.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.153+0000 7f773b65f700 1 -- start start 2023-07-21T22:29:07.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f773b65f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77340ffb10 0x7f77341957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:07.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f773b65f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7734100d00 0x7f7734195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:07.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f773b65f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7734101690 0x7f773419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:07.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f773b65f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773419d370 con 0x7f7734101690 2023-07-21T22:29:07.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f773b65f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773419d4b0 con 0x7f77340ffb10 2023-07-21T22:29:07.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f773b65f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773419d640 con 0x7f7734100d00 2023-07-21T22:29:07.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f7739bfc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7734101690 0x7f773419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:07.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f7739bfc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7734101690 0x7f773419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48064/0 (socket says 172.21.15.107:48064) 2023-07-21T22:29:07.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f7739bfc700 1 -- 172.21.15.107:0/279732040 learned_addr learned my addr 172.21.15.107:0/279732040 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:07.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f77393fb700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77340ffb10 0x7f77341957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:07.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f7738bfa700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7734100d00 0x7f7734195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:07.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f7739bfc700 1 -- 172.21.15.107:0/279732040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7734100d00 msgr2=0x7f7734195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f7739bfc700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7734100d00 0x7f7734195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f7739bfc700 1 -- 172.21.15.107:0/279732040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77340ffb10 msgr2=0x7f77341957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f7739bfc700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77340ffb10 0x7f77341957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f7739bfc700 1 -- 172.21.15.107:0/279732040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77240095f0 con 0x7f7734101690 2023-07-21T22:29:07.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f7739bfc700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7734101690 0x7f773419cda0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f772c00ef30 tx=0x7f772c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:07.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f772a7fc700 1 -- 172.21.15.107:0/279732040 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c018070 con 0x7f7734101690 2023-07-21T22:29:07.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f772a7fc700 1 -- 172.21.15.107:0/279732040 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f772c00f040 con 0x7f7734101690 2023-07-21T22:29:07.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f773b65f700 1 -- 172.21.15.107:0/279732040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773419d8c0 con 0x7f7734101690 2023-07-21T22:29:07.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f772a7fc700 1 -- 172.21.15.107:0/279732040 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c00cdb0 con 0x7f7734101690 2023-07-21T22:29:07.206 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.154+0000 7f773b65f700 1 -- 172.21.15.107:0/279732040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773419dd80 con 0x7f7734101690 2023-07-21T22:29:07.206 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.155+0000 7f773b65f700 1 -- 172.21.15.107:0/279732040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773410b2e0 con 0x7f7734101690 2023-07-21T22:29:07.206 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.157+0000 7f772a7fc700 1 -- 172.21.15.107:0/279732040 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f772c003a00 con 0x7f7734101690 2023-07-21T22:29:07.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.158+0000 7f772a7fc700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f77200a1220 0x7f77200a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:07.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.158+0000 7f77393fb700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f77200a1220 0x7f77200a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:07.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.158+0000 7f772a7fc700 1 -- 172.21.15.107:0/279732040 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f772c0cd060 con 0x7f7734101690 2023-07-21T22:29:07.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.158+0000 7f77393fb700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f77200a1220 0x7f77200a36d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7724006010 tx=0x7f7724005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:07.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.161+0000 7f772a7fc700 1 -- 172.21.15.107:0/279732040 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f772c0966c0 con 0x7f7734101690 2023-07-21T22:29:07.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:07 smithi190 ceph-mon[98334]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2023-07-21T22:29:07.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:07 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3829748508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:29:07.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:07 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3349380704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:29:07.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.346+0000 7f773b65f700 1 -- 172.21.15.107:0/279732040 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7734060950 con 0x7f77200a1220 2023-07-21T22:29:07.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.356+0000 7f772a7fc700 1 -- 172.21.15.107:0/279732040 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343797 (secure 0 0 0) 0x7f7734060950 con 0x7f77200a1220 2023-07-21T22:29:07.406 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:29:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.359+0000 7f773b65f700 1 -- 172.21.15.107:0/279732040 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f77200a1220 msgr2=0x7f77200a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.359+0000 7f773b65f700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f77200a1220 0x7f77200a36d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7724006010 tx=0x7f7724005960 comp rx=0 tx=0).stop 2023-07-21T22:29:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.359+0000 7f773b65f700 1 -- 172.21.15.107:0/279732040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7734101690 msgr2=0x7f773419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.359+0000 7f773b65f700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7734101690 0x7f773419cda0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f772c00ef30 tx=0x7f772c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.359+0000 7f773b65f700 1 -- 172.21.15.107:0/279732040 shutdown_connections 2023-07-21T22:29:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.359+0000 7f773b65f700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7734101690 0x7f773419cda0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.359+0000 7f773b65f700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77340ffb10 0x7f77341957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.359+0000 7f773b65f700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f77200a1220 0x7f77200a36d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.359+0000 7f773b65f700 1 --2- 172.21.15.107:0/279732040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7734100d00 0x7f7734195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.359+0000 7f773b65f700 1 -- 172.21.15.107:0/279732040 >> 172.21.15.107:0/279732040 conn(0x7f77340fb110 msgr2=0x7f77340fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.359+0000 7f773b65f700 1 -- 172.21.15.107:0/279732040 shutdown_connections 2023-07-21T22:29:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.359+0000 7f773b65f700 1 -- 172.21.15.107:0/279732040 wait complete. 2023-07-21T22:29:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2023-07-21T22:29:07.417 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:07 smithi107 ceph-mon[96543]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2023-07-21T22:29:07.417 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:07 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3829748508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:29:07.417 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:07 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3349380704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:29:07.418 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2023-07-21T22:29:05.808726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652602,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":905,"num_read_kb":565,"num_write":413,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181722,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336372,"kb_used_data":12948,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605452,"statfs":{"total":1151940427776,"available":1151595982848,"internally_reserved":0,"allocated":13258752,"data_stored":7220211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":318,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":56,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003089"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244157+0000","last_change":"2023-07-21T22:27:42.091397+0000","last_active":"2023-07-21T22:28:47.244157+0000","last_peered":"2023-07-21T22:28:47.244157+0000","last_clean":"2023-07-21T22:28:47.244157+0000","last_became_active":"2023-07-21T22:27:42.090836+0000","last_became_peered":"2023-07-21T22:27:42.090836+0000","last_unstale":"2023-07-21T22:28:47.244157+0000","last_undegraded":"2023-07-21T22:28:47.244157+0000","last_fullsized":"2023-07-21T22:28:47.244157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:21:03.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382220+0000","last_change":"2023-07-21T22:27:42.097003+0000","last_active":"2023-07-21T22:28:47.382220+0000","last_peered":"2023-07-21T22:28:47.382220+0000","last_clean":"2023-07-21T22:28:47.382220+0000","last_became_active":"2023-07-21T22:27:42.096652+0000","last_became_peered":"2023-07-21T22:27:42.096652+0000","last_unstale":"2023-07-21T22:28:47.382220+0000","last_undegraded":"2023-07-21T22:28:47.382220+0000","last_fullsized":"2023-07-21T22:28:47.382220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:00:27.467585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382162+0000","last_change":"2023-07-21T22:27:42.090090+0000","last_active":"2023-07-21T22:28:47.382162+0000","last_peered":"2023-07-21T22:28:47.382162+0000","last_clean":"2023-07-21T22:28:47.382162+0000","last_became_active":"2023-07-21T22:27:42.089802+0000","last_became_peered":"2023-07-21T22:27:42.089802+0000","last_unstale":"2023-07-21T22:28:47.382162+0000","last_undegraded":"2023-07-21T22:28:47.382162+0000","last_fullsized":"2023-07-21T22:28:47.382162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:15:20.510508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246255+0000","last_change":"2023-07-21T22:27:42.096291+0000","last_active":"2023-07-21T22:28:47.246255+0000","last_peered":"2023-07-21T22:28:47.246255+0000","last_clean":"2023-07-21T22:28:47.246255+0000","last_became_active":"2023-07-21T22:27:42.095982+0000","last_became_peered":"2023-07-21T22:27:42.095982+0000","last_unstale":"2023-07-21T22:28:47.246255+0000","last_undegraded":"2023-07-21T22:28:47.246255+0000","last_fullsized":"2023-07-21T22:28:47.246255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:14:38.335068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246671+0000","last_change":"2023-07-21T22:27:42.097495+0000","last_active":"2023-07-21T22:28:47.246671+0000","last_peered":"2023-07-21T22:28:47.246671+0000","last_clean":"2023-07-21T22:28:47.246671+0000","last_became_active":"2023-07-21T22:27:42.095112+0000","last_became_peered":"2023-07-21T22:27:42.095112+0000","last_unstale":"2023-07-21T22:28:47.246671+0000","last_undegraded":"2023-07-21T22:28:47.246671+0000","last_fullsized":"2023-07-21T22:28:47.246671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:14:17.323583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242851+0000","last_change":"2023-07-21T22:27:42.089880+0000","last_active":"2023-07-21T22:28:47.242851+0000","last_peered":"2023-07-21T22:28:47.242851+0000","last_clean":"2023-07-21T22:28:47.242851+0000","last_became_active":"2023-07-21T22:27:42.088614+0000","last_became_peered":"2023-07-21T22:27:42.088614+0000","last_unstale":"2023-07-21T22:28:47.242851+0000","last_undegraded":"2023-07-21T22:28:47.242851+0000","last_fullsized":"2023-07-21T22:28:47.242851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:06:38.785570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245114+0000","last_change":"2023-07-21T22:27:42.085457+0000","last_active":"2023-07-21T22:28:47.245114+0000","last_peered":"2023-07-21T22:28:47.245114+0000","last_clean":"2023-07-21T22:28:47.245114+0000","last_became_active":"2023-07-21T22:27:42.085146+0000","last_became_peered":"2023-07-21T22:27:42.085146+0000","last_unstale":"2023-07-21T22:28:47.245114+0000","last_undegraded":"2023-07-21T22:28:47.245114+0000","last_fullsized":"2023-07-21T22:28:47.245114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:06:59.371781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.247038+0000","last_change":"2023-07-21T22:28:47.246617+0000","last_active":"2023-07-21T22:28:47.247038+0000","last_peered":"2023-07-21T22:28:47.247038+0000","last_clean":"2023-07-21T22:28:47.247038+0000","last_became_active":"2023-07-21T22:28:47.246322+0000","last_became_peered":"2023-07-21T22:28:47.246322+0000","last_unstale":"2023-07-21T22:28:47.247038+0000","last_undegraded":"2023-07-21T22:28:47.247038+0000","last_fullsized":"2023-07-21T22:28:47.247038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:22.991062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383406+0000","last_change":"2023-07-21T22:27:42.094223+0000","last_active":"2023-07-21T22:28:47.383406+0000","last_peered":"2023-07-21T22:28:47.383406+0000","last_clean":"2023-07-21T22:28:47.383406+0000","last_became_active":"2023-07-21T22:27:42.093875+0000","last_became_peered":"2023-07-21T22:27:42.093875+0000","last_unstale":"2023-07-21T22:28:47.383406+0000","last_undegraded":"2023-07-21T22:28:47.383406+0000","last_fullsized":"2023-07-21T22:28:47.383406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:45:28.313407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243145+0000","last_change":"2023-07-21T22:27:42.090442+0000","last_active":"2023-07-21T22:28:47.243145+0000","last_peered":"2023-07-21T22:28:47.243145+0000","last_clean":"2023-07-21T22:28:47.243145+0000","last_became_active":"2023-07-21T22:27:42.090103+0000","last_became_peered":"2023-07-21T22:27:42.090103+0000","last_unstale":"2023-07-21T22:28:47.243145+0000","last_undegraded":"2023-07-21T22:28:47.243145+0000","last_fullsized":"2023-07-21T22:28:47.243145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:27:20.825119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245155+0000","last_change":"2023-07-21T22:27:42.098364+0000","last_active":"2023-07-21T22:28:47.245155+0000","last_peered":"2023-07-21T22:28:47.245155+0000","last_clean":"2023-07-21T22:28:47.245155+0000","last_became_active":"2023-07-21T22:27:42.097378+0000","last_became_peered":"2023-07-21T22:27:42.097378+0000","last_unstale":"2023-07-21T22:28:47.245155+0000","last_undegraded":"2023-07-21T22:28:47.245155+0000","last_fullsized":"2023-07-21T22:28:47.245155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:40:58.595476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246312+0000","last_change":"2023-07-21T22:27:42.103110+0000","last_active":"2023-07-21T22:28:47.246312+0000","last_peered":"2023-07-21T22:28:47.246312+0000","last_clean":"2023-07-21T22:28:47.246312+0000","last_became_active":"2023-07-21T22:27:42.102790+0000","last_became_peered":"2023-07-21T22:27:42.102790+0000","last_unstale":"2023-07-21T22:28:47.246312+0000","last_undegraded":"2023-07-21T22:28:47.246312+0000","last_fullsized":"2023-07-21T22:28:47.246312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:20:23.914945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244930+0000","last_change":"2023-07-21T22:28:47.244930+0000","last_active":"2023-07-21T22:28:47.244930+0000","last_peered":"2023-07-21T22:28:47.244930+0000","last_clean":"2023-07-21T22:28:47.244930+0000","last_became_active":"2023-07-21T22:28:47.244613+0000","last_became_peered":"2023-07-21T22:28:47.244613+0000","last_unstale":"2023-07-21T22:28:47.244930+0000","last_undegraded":"2023-07-21T22:28:47.244930+0000","last_fullsized":"2023-07-21T22:28:47.244930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:43:03.707499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382008+0000","last_change":"2023-07-21T22:27:42.084423+0000","last_active":"2023-07-21T22:28:47.382008+0000","last_peered":"2023-07-21T22:28:47.382008+0000","last_clean":"2023-07-21T22:28:47.382008+0000","last_became_active":"2023-07-21T22:27:42.084074+0000","last_became_peered":"2023-07-21T22:27:42.084074+0000","last_unstale":"2023-07-21T22:28:47.382008+0000","last_undegraded":"2023-07-21T22:28:47.382008+0000","last_fullsized":"2023-07-21T22:28:47.382008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:48.499214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.381932+0000","last_change":"2023-07-21T22:27:42.091542+0000","last_active":"2023-07-21T22:28:47.381932+0000","last_peered":"2023-07-21T22:28:47.381932+0000","last_clean":"2023-07-21T22:28:47.381932+0000","last_became_active":"2023-07-21T22:27:42.091217+0000","last_became_peered":"2023-07-21T22:27:42.091217+0000","last_unstale":"2023-07-21T22:28:47.381932+0000","last_undegraded":"2023-07-21T22:28:47.381932+0000","last_fullsized":"2023-07-21T22:28:47.381932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:31:21.491062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382557+0000","last_change":"2023-07-21T22:27:42.087505+0000","last_active":"2023-07-21T22:28:47.382557+0000","last_peered":"2023-07-21T22:28:47.382557+0000","last_clean":"2023-07-21T22:28:47.382557+0000","last_became_active":"2023-07-21T22:27:42.087177+0000","last_became_peered":"2023-07-21T22:27:42.087177+0000","last_unstale":"2023-07-21T22:28:47.382557+0000","last_undegraded":"2023-07-21T22:28:47.382557+0000","last_fullsized":"2023-07-21T22:28:47.382557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:06:51.361107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243856+0000","last_change":"2023-07-21T22:27:42.091490+0000","last_active":"2023-07-21T22:28:47.243856+0000","last_peered":"2023-07-21T22:28:47.243856+0000","last_clean":"2023-07-21T22:28:47.243856+0000","last_became_active":"2023-07-21T22:27:42.090782+0000","last_became_peered":"2023-07-21T22:27:42.090782+0000","last_unstale":"2023-07-21T22:28:47.243856+0000","last_undegraded":"2023-07-21T22:28:47.243856+0000","last_fullsized":"2023-07-21T22:28:47.243856+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:34:52.980195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243991+0000","last_change":"2023-07-21T22:27:42.092362+0000","last_active":"2023-07-21T22:28:47.243991+0000","last_peered":"2023-07-21T22:28:47.243991+0000","last_clean":"2023-07-21T22:28:47.243991+0000","last_became_active":"2023-07-21T22:27:42.091915+0000","last_became_peered":"2023-07-21T22:27:42.091915+0000","last_unstale":"2023-07-21T22:28:47.243991+0000","last_undegraded":"2023-07-21T22:28:47.243991+0000","last_fullsized":"2023-07-21T22:28:47.243991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:06:24.497745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242584+0000","last_change":"2023-07-21T22:27:42.098959+0000","last_active":"2023-07-21T22:28:47.242584+0000","last_peered":"2023-07-21T22:28:47.242584+0000","last_clean":"2023-07-21T22:28:47.242584+0000","last_became_active":"2023-07-21T22:27:42.094463+0000","last_became_peered":"2023-07-21T22:27:42.094463+0000","last_unstale":"2023-07-21T22:28:47.242584+0000","last_undegraded":"2023-07-21T22:28:47.242584+0000","last_fullsized":"2023-07-21T22:28:47.242584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:17:23.627490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":108,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:03.141677+0000","last_change":"2023-07-21T22:27:41.074299+0000","last_active":"2023-07-21T22:29:03.141677+0000","last_peered":"2023-07-21T22:29:03.141677+0000","last_clean":"2023-07-21T22:29:03.141677+0000","last_became_active":"2023-07-21T22:27:41.072895+0000","last_became_peered":"2023-07-21T22:27:41.072895+0000","last_unstale":"2023-07-21T22:29:03.141677+0000","last_undegraded":"2023-07-21T22:29:03.141677+0000","last_fullsized":"2023-07-21T22:29:03.141677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:24:58.146931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241511+0000","last_change":"2023-07-21T22:27:42.090477+0000","last_active":"2023-07-21T22:28:47.241511+0000","last_peered":"2023-07-21T22:28:47.241511+0000","last_clean":"2023-07-21T22:28:47.241511+0000","last_became_active":"2023-07-21T22:27:42.090144+0000","last_became_peered":"2023-07-21T22:27:42.090144+0000","last_unstale":"2023-07-21T22:28:47.241511+0000","last_undegraded":"2023-07-21T22:28:47.241511+0000","last_fullsized":"2023-07-21T22:28:47.241511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:42:38.964084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241632+0000","last_change":"2023-07-21T22:27:41.075178+0000","last_active":"2023-07-21T22:28:47.241632+0000","last_peered":"2023-07-21T22:28:47.241632+0000","last_clean":"2023-07-21T22:28:47.241632+0000","last_became_active":"2023-07-21T22:27:41.074861+0000","last_became_peered":"2023-07-21T22:27:41.074861+0000","last_unstale":"2023-07-21T22:28:47.241632+0000","last_undegraded":"2023-07-21T22:28:47.241632+0000","last_fullsized":"2023-07-21T22:28:47.241632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:09:39.913585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382382+0000","last_change":"2023-07-21T22:27:42.093135+0000","last_active":"2023-07-21T22:28:47.382382+0000","last_peered":"2023-07-21T22:28:47.382382+0000","last_clean":"2023-07-21T22:28:47.382382+0000","last_became_active":"2023-07-21T22:27:42.092457+0000","last_became_peered":"2023-07-21T22:27:42.092457+0000","last_unstale":"2023-07-21T22:28:47.382382+0000","last_undegraded":"2023-07-21T22:28:47.382382+0000","last_fullsized":"2023-07-21T22:28:47.382382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:37.129524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.249069+0000","last_change":"2023-07-21T22:27:41.076759+0000","last_active":"2023-07-21T22:28:47.249069+0000","last_peered":"2023-07-21T22:28:47.249069+0000","last_clean":"2023-07-21T22:28:47.249069+0000","last_became_active":"2023-07-21T22:27:41.076376+0000","last_became_peered":"2023-07-21T22:27:41.076376+0000","last_unstale":"2023-07-21T22:28:47.249069+0000","last_undegraded":"2023-07-21T22:28:47.249069+0000","last_fullsized":"2023-07-21T22:28:47.249069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:10:03.951373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382877+0000","last_change":"2023-07-21T22:27:42.093643+0000","last_active":"2023-07-21T22:28:47.382877+0000","last_peered":"2023-07-21T22:28:47.382877+0000","last_clean":"2023-07-21T22:28:47.382877+0000","last_became_active":"2023-07-21T22:27:42.093227+0000","last_became_peered":"2023-07-21T22:27:42.093227+0000","last_unstale":"2023-07-21T22:28:47.382877+0000","last_undegraded":"2023-07-21T22:28:47.382877+0000","last_fullsized":"2023-07-21T22:28:47.382877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:07:08.243533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.248547+0000","last_change":"2023-07-21T22:27:41.073119+0000","last_active":"2023-07-21T22:28:47.248547+0000","last_peered":"2023-07-21T22:28:47.248547+0000","last_clean":"2023-07-21T22:28:47.248547+0000","last_became_active":"2023-07-21T22:27:41.072813+0000","last_became_peered":"2023-07-21T22:27:41.072813+0000","last_unstale":"2023-07-21T22:28:47.248547+0000","last_undegraded":"2023-07-21T22:28:47.248547+0000","last_fullsized":"2023-07-21T22:28:47.248547+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:02.412440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383479+0000","last_change":"2023-07-21T22:28:47.383479+0000","last_active":"2023-07-21T22:28:47.383479+0000","last_peered":"2023-07-21T22:28:47.383479+0000","last_clean":"2023-07-21T22:28:47.383479+0000","last_became_active":"2023-07-21T22:28:47.383074+0000","last_became_peered":"2023-07-21T22:28:47.383074+0000","last_unstale":"2023-07-21T22:28:47.383479+0000","last_undegraded":"2023-07-21T22:28:47.383479+0000","last_fullsized":"2023-07-21T22:28:47.383479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:01:40.184725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":118,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:03.407481+0000","last_change":"2023-07-21T22:28:47.388489+0000","last_active":"2023-07-21T22:29:03.407481+0000","last_peered":"2023-07-21T22:29:03.407481+0000","last_clean":"2023-07-21T22:29:03.407481+0000","last_became_active":"2023-07-21T22:28:47.383903+0000","last_became_peered":"2023-07-21T22:28:47.383903+0000","last_unstale":"2023-07-21T22:29:03.407481+0000","last_undegraded":"2023-07-21T22:29:03.407481+0000","last_fullsized":"2023-07-21T22:29:03.407481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:25:48.113759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246961+0000","last_change":"2023-07-21T22:27:42.082970+0000","last_active":"2023-07-21T22:28:47.246961+0000","last_peered":"2023-07-21T22:28:47.246961+0000","last_clean":"2023-07-21T22:28:47.246961+0000","last_became_active":"2023-07-21T22:27:42.082676+0000","last_became_peered":"2023-07-21T22:27:42.082676+0000","last_unstale":"2023-07-21T22:28:47.246961+0000","last_undegraded":"2023-07-21T22:28:47.246961+0000","last_fullsized":"2023-07-21T22:28:47.246961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:47:24.831459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242691+0000","last_change":"2023-07-21T22:27:41.076844+0000","last_active":"2023-07-21T22:28:47.242691+0000","last_peered":"2023-07-21T22:28:47.242691+0000","last_clean":"2023-07-21T22:28:47.242691+0000","last_became_active":"2023-07-21T22:27:41.076487+0000","last_became_peered":"2023-07-21T22:27:41.076487+0000","last_unstale":"2023-07-21T22:28:47.242691+0000","last_undegraded":"2023-07-21T22:28:47.242691+0000","last_fullsized":"2023-07-21T22:28:47.242691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:03:37.023974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240958+0000","last_change":"2023-07-21T22:27:42.089996+0000","last_active":"2023-07-21T22:28:47.240958+0000","last_peered":"2023-07-21T22:28:47.240958+0000","last_clean":"2023-07-21T22:28:47.240958+0000","last_became_active":"2023-07-21T22:27:42.089668+0000","last_became_peered":"2023-07-21T22:27:42.089668+0000","last_unstale":"2023-07-21T22:28:47.240958+0000","last_undegraded":"2023-07-21T22:28:47.240958+0000","last_fullsized":"2023-07-21T22:28:47.240958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:55:01.151479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239183+0000","last_change":"2023-07-21T22:27:41.075864+0000","last_active":"2023-07-21T22:28:47.239183+0000","last_peered":"2023-07-21T22:28:47.239183+0000","last_clean":"2023-07-21T22:28:47.239183+0000","last_became_active":"2023-07-21T22:27:41.075577+0000","last_became_peered":"2023-07-21T22:27:41.075577+0000","last_unstale":"2023-07-21T22:28:47.239183+0000","last_undegraded":"2023-07-21T22:28:47.239183+0000","last_fullsized":"2023-07-21T22:28:47.239183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:41:53.383706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241389+0000","last_change":"2023-07-21T22:27:42.091730+0000","last_active":"2023-07-21T22:28:47.241389+0000","last_peered":"2023-07-21T22:28:47.241389+0000","last_clean":"2023-07-21T22:28:47.241389+0000","last_became_active":"2023-07-21T22:27:42.091394+0000","last_became_peered":"2023-07-21T22:27:42.091394+0000","last_unstale":"2023-07-21T22:28:47.241389+0000","last_undegraded":"2023-07-21T22:28:47.241389+0000","last_fullsized":"2023-07-21T22:28:47.241389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:21:14.595368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240215+0000","last_change":"2023-07-21T22:27:41.074822+0000","last_active":"2023-07-21T22:28:47.240215+0000","last_peered":"2023-07-21T22:28:47.240215+0000","last_clean":"2023-07-21T22:28:47.240215+0000","last_became_active":"2023-07-21T22:27:41.074542+0000","last_became_peered":"2023-07-21T22:27:41.074542+0000","last_unstale":"2023-07-21T22:28:47.240215+0000","last_undegraded":"2023-07-21T22:28:47.240215+0000","last_fullsized":"2023-07-21T22:28:47.240215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:56:37.030275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243691+0000","last_change":"2023-07-21T22:27:42.094351+0000","last_active":"2023-07-21T22:28:47.243691+0000","last_peered":"2023-07-21T22:28:47.243691+0000","last_clean":"2023-07-21T22:28:47.243691+0000","last_became_active":"2023-07-21T22:27:42.094068+0000","last_became_peered":"2023-07-21T22:27:42.094068+0000","last_unstale":"2023-07-21T22:28:47.243691+0000","last_undegraded":"2023-07-21T22:28:47.243691+0000","last_fullsized":"2023-07-21T22:28:47.243691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:55:43.799214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":170,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:00.402812+0000","last_change":"2023-07-21T22:27:41.079188+0000","last_active":"2023-07-21T22:29:00.402812+0000","last_peered":"2023-07-21T22:29:00.402812+0000","last_clean":"2023-07-21T22:29:00.402812+0000","last_became_active":"2023-07-21T22:27:41.078899+0000","last_became_peered":"2023-07-21T22:27:41.078899+0000","last_unstale":"2023-07-21T22:29:00.402812+0000","last_undegraded":"2023-07-21T22:29:00.402812+0000","last_fullsized":"2023-07-21T22:29:00.402812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:04:07.942421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383198+0000","last_change":"2023-07-21T22:27:42.091194+0000","last_active":"2023-07-21T22:28:47.383198+0000","last_peered":"2023-07-21T22:28:47.383198+0000","last_clean":"2023-07-21T22:28:47.383198+0000","last_became_active":"2023-07-21T22:27:42.090869+0000","last_became_peered":"2023-07-21T22:27:42.090869+0000","last_unstale":"2023-07-21T22:28:47.383198+0000","last_undegraded":"2023-07-21T22:28:47.383198+0000","last_fullsized":"2023-07-21T22:28:47.383198+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:46:43.577329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245092+0000","last_change":"2023-07-21T22:27:41.072701+0000","last_active":"2023-07-21T22:28:47.245092+0000","last_peered":"2023-07-21T22:28:47.245092+0000","last_clean":"2023-07-21T22:28:47.245092+0000","last_became_active":"2023-07-21T22:27:41.072358+0000","last_became_peered":"2023-07-21T22:27:41.072358+0000","last_unstale":"2023-07-21T22:28:47.245092+0000","last_undegraded":"2023-07-21T22:28:47.245092+0000","last_fullsized":"2023-07-21T22:28:47.245092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:11:20.209729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382347+0000","last_change":"2023-07-21T22:27:42.090808+0000","last_active":"2023-07-21T22:28:47.382347+0000","last_peered":"2023-07-21T22:28:47.382347+0000","last_clean":"2023-07-21T22:28:47.382347+0000","last_became_active":"2023-07-21T22:27:42.090506+0000","last_became_peered":"2023-07-21T22:27:42.090506+0000","last_unstale":"2023-07-21T22:28:47.382347+0000","last_undegraded":"2023-07-21T22:28:47.382347+0000","last_fullsized":"2023-07-21T22:28:47.382347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:15.825961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240934+0000","last_change":"2023-07-21T22:27:41.075643+0000","last_active":"2023-07-21T22:28:47.240934+0000","last_peered":"2023-07-21T22:28:47.240934+0000","last_clean":"2023-07-21T22:28:47.240934+0000","last_became_active":"2023-07-21T22:27:41.075336+0000","last_became_peered":"2023-07-21T22:27:41.075336+0000","last_unstale":"2023-07-21T22:28:47.240934+0000","last_undegraded":"2023-07-21T22:28:47.240934+0000","last_fullsized":"2023-07-21T22:28:47.240934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:25:15.271905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383144+0000","last_change":"2023-07-21T22:27:42.096722+0000","last_active":"2023-07-21T22:28:47.383144+0000","last_peered":"2023-07-21T22:28:47.383144+0000","last_clean":"2023-07-21T22:28:47.383144+0000","last_became_active":"2023-07-21T22:27:42.096472+0000","last_became_peered":"2023-07-21T22:27:42.096472+0000","last_unstale":"2023-07-21T22:28:47.383144+0000","last_undegraded":"2023-07-21T22:28:47.383144+0000","last_fullsized":"2023-07-21T22:28:47.383144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:27:25.223020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'28","reported_seq":202,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:00.402637+0000","last_change":"2023-07-21T22:27:41.072297+0000","last_active":"2023-07-21T22:29:00.402637+0000","last_peered":"2023-07-21T22:29:00.402637+0000","last_clean":"2023-07-21T22:29:00.402637+0000","last_became_active":"2023-07-21T22:27:41.071971+0000","last_became_peered":"2023-07-21T22:27:41.071971+0000","last_unstale":"2023-07-21T22:29:00.402637+0000","last_undegraded":"2023-07-21T22:29:00.402637+0000","last_fullsized":"2023-07-21T22:29:00.402637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:28:20.855102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54660,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":156,"num_read_kb":131,"num_write":30,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.248127+0000","last_change":"2023-07-21T22:27:42.097130+0000","last_active":"2023-07-21T22:28:47.248127+0000","last_peered":"2023-07-21T22:28:47.248127+0000","last_clean":"2023-07-21T22:28:47.248127+0000","last_became_active":"2023-07-21T22:27:42.096853+0000","last_became_peered":"2023-07-21T22:27:42.096853+0000","last_unstale":"2023-07-21T22:28:47.248127+0000","last_undegraded":"2023-07-21T22:28:47.248127+0000","last_fullsized":"2023-07-21T22:28:47.248127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:42:15.440580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242791+0000","last_change":"2023-07-21T22:27:41.071755+0000","last_active":"2023-07-21T22:28:47.242791+0000","last_peered":"2023-07-21T22:28:47.242791+0000","last_clean":"2023-07-21T22:28:47.242791+0000","last_became_active":"2023-07-21T22:27:41.071245+0000","last_became_peered":"2023-07-21T22:27:41.071245+0000","last_unstale":"2023-07-21T22:28:47.242791+0000","last_undegraded":"2023-07-21T22:28:47.242791+0000","last_fullsized":"2023-07-21T22:28:47.242791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:17:33.183524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239177+0000","last_change":"2023-07-21T22:27:42.097433+0000","last_active":"2023-07-21T22:28:47.239177+0000","last_peered":"2023-07-21T22:28:47.239177+0000","last_clean":"2023-07-21T22:28:47.239177+0000","last_became_active":"2023-07-21T22:27:42.097100+0000","last_became_peered":"2023-07-21T22:27:42.097100+0000","last_unstale":"2023-07-21T22:28:47.239177+0000","last_undegraded":"2023-07-21T22:28:47.239177+0000","last_fullsized":"2023-07-21T22:28:47.239177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:11:09.018578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240863+0000","last_change":"2023-07-21T22:27:41.074019+0000","last_active":"2023-07-21T22:28:47.240863+0000","last_peered":"2023-07-21T22:28:47.240863+0000","last_clean":"2023-07-21T22:28:47.240863+0000","last_became_active":"2023-07-21T22:27:41.073699+0000","last_became_peered":"2023-07-21T22:27:41.073699+0000","last_unstale":"2023-07-21T22:28:47.240863+0000","last_undegraded":"2023-07-21T22:28:47.240863+0000","last_fullsized":"2023-07-21T22:28:47.240863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:16:39.616177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245536+0000","last_change":"2023-07-21T22:27:42.098010+0000","last_active":"2023-07-21T22:28:47.245536+0000","last_peered":"2023-07-21T22:28:47.245536+0000","last_clean":"2023-07-21T22:28:47.245536+0000","last_became_active":"2023-07-21T22:27:42.097227+0000","last_became_peered":"2023-07-21T22:27:42.097227+0000","last_unstale":"2023-07-21T22:28:47.245536+0000","last_undegraded":"2023-07-21T22:28:47.245536+0000","last_fullsized":"2023-07-21T22:28:47.245536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:29:36.120927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":103,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:04.163532+0000","last_change":"2023-07-21T22:27:41.072298+0000","last_active":"2023-07-21T22:29:04.163532+0000","last_peered":"2023-07-21T22:29:04.163532+0000","last_clean":"2023-07-21T22:29:04.163532+0000","last_became_active":"2023-07-21T22:27:41.071963+0000","last_became_peered":"2023-07-21T22:27:41.071963+0000","last_unstale":"2023-07-21T22:29:04.163532+0000","last_undegraded":"2023-07-21T22:29:04.163532+0000","last_fullsized":"2023-07-21T22:29:04.163532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:14:19.498409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":858,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244894+0000","last_change":"2023-07-21T22:27:42.091276+0000","last_active":"2023-07-21T22:28:47.244894+0000","last_peered":"2023-07-21T22:28:47.244894+0000","last_clean":"2023-07-21T22:28:47.244894+0000","last_became_active":"2023-07-21T22:27:42.090918+0000","last_became_peered":"2023-07-21T22:27:42.090918+0000","last_unstale":"2023-07-21T22:28:47.244894+0000","last_undegraded":"2023-07-21T22:28:47.244894+0000","last_fullsized":"2023-07-21T22:28:47.244894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:15.527348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246226+0000","last_change":"2023-07-21T22:27:41.075633+0000","last_active":"2023-07-21T22:28:47.246226+0000","last_peered":"2023-07-21T22:28:47.246226+0000","last_clean":"2023-07-21T22:28:47.246226+0000","last_became_active":"2023-07-21T22:27:41.075331+0000","last_became_peered":"2023-07-21T22:27:41.075331+0000","last_unstale":"2023-07-21T22:28:47.246226+0000","last_undegraded":"2023-07-21T22:28:47.246226+0000","last_fullsized":"2023-07-21T22:28:47.246226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:26:08.447672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383325+0000","last_change":"2023-07-21T22:28:47.383325+0000","last_active":"2023-07-21T22:28:47.383325+0000","last_peered":"2023-07-21T22:28:47.383325+0000","last_clean":"2023-07-21T22:28:47.383325+0000","last_became_active":"2023-07-21T22:28:47.382987+0000","last_became_peered":"2023-07-21T22:28:47.382987+0000","last_unstale":"2023-07-21T22:28:47.383325+0000","last_undegraded":"2023-07-21T22:28:47.383325+0000","last_fullsized":"2023-07-21T22:28:47.383325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:51:52.190795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.255975+0000","last_change":"2023-07-21T22:28:47.255975+0000","last_active":"2023-07-21T22:28:47.255975+0000","last_peered":"2023-07-21T22:28:47.255975+0000","last_clean":"2023-07-21T22:28:47.255975+0000","last_became_active":"2023-07-21T22:28:47.248730+0000","last_became_peered":"2023-07-21T22:28:47.248730+0000","last_unstale":"2023-07-21T22:28:47.255975+0000","last_undegraded":"2023-07-21T22:28:47.255975+0000","last_fullsized":"2023-07-21T22:28:47.255975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:25:29.022962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243276+0000","last_change":"2023-07-21T22:27:42.092046+0000","last_active":"2023-07-21T22:28:47.243276+0000","last_peered":"2023-07-21T22:28:47.243276+0000","last_clean":"2023-07-21T22:28:47.243276+0000","last_became_active":"2023-07-21T22:27:42.091069+0000","last_became_peered":"2023-07-21T22:27:42.091069+0000","last_unstale":"2023-07-21T22:28:47.243276+0000","last_undegraded":"2023-07-21T22:28:47.243276+0000","last_fullsized":"2023-07-21T22:28:47.243276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:07:37.855640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243883+0000","last_change":"2023-07-21T22:27:41.074847+0000","last_active":"2023-07-21T22:28:47.243883+0000","last_peered":"2023-07-21T22:28:47.243883+0000","last_clean":"2023-07-21T22:28:47.243883+0000","last_became_active":"2023-07-21T22:27:41.074530+0000","last_became_peered":"2023-07-21T22:27:41.074530+0000","last_unstale":"2023-07-21T22:28:47.243883+0000","last_undegraded":"2023-07-21T22:28:47.243883+0000","last_fullsized":"2023-07-21T22:28:47.243883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:28:53.033226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242814+0000","last_change":"2023-07-21T22:27:42.089285+0000","last_active":"2023-07-21T22:28:47.242814+0000","last_peered":"2023-07-21T22:28:47.242814+0000","last_clean":"2023-07-21T22:28:47.242814+0000","last_became_active":"2023-07-21T22:27:42.088970+0000","last_became_peered":"2023-07-21T22:27:42.088970+0000","last_unstale":"2023-07-21T22:28:47.242814+0000","last_undegraded":"2023-07-21T22:28:47.242814+0000","last_fullsized":"2023-07-21T22:28:47.242814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:53:26.517730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:59.137194+0000","last_change":"2023-07-21T22:27:41.077433+0000","last_active":"2023-07-21T22:28:59.137194+0000","last_peered":"2023-07-21T22:28:59.137194+0000","last_clean":"2023-07-21T22:28:59.137194+0000","last_became_active":"2023-07-21T22:27:41.077133+0000","last_became_peered":"2023-07-21T22:27:41.077133+0000","last_unstale":"2023-07-21T22:28:59.137194+0000","last_undegraded":"2023-07-21T22:28:59.137194+0000","last_fullsized":"2023-07-21T22:28:59.137194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:09.473094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243171+0000","last_change":"2023-07-21T22:27:42.089833+0000","last_active":"2023-07-21T22:28:47.243171+0000","last_peered":"2023-07-21T22:28:47.243171+0000","last_clean":"2023-07-21T22:28:47.243171+0000","last_became_active":"2023-07-21T22:27:42.089502+0000","last_became_peered":"2023-07-21T22:27:42.089502+0000","last_unstale":"2023-07-21T22:28:47.243171+0000","last_undegraded":"2023-07-21T22:28:47.243171+0000","last_fullsized":"2023-07-21T22:28:47.243171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:33:04.121577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.248492+0000","last_change":"2023-07-21T22:28:47.248117+0000","last_active":"2023-07-21T22:28:47.248492+0000","last_peered":"2023-07-21T22:28:47.248492+0000","last_clean":"2023-07-21T22:28:47.248492+0000","last_became_active":"2023-07-21T22:28:47.247668+0000","last_became_peered":"2023-07-21T22:28:47.247668+0000","last_unstale":"2023-07-21T22:28:47.248492+0000","last_undegraded":"2023-07-21T22:28:47.248492+0000","last_fullsized":"2023-07-21T22:28:47.248492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:57:06.878014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:59.155853+0000","last_change":"2023-07-21T22:27:41.076784+0000","last_active":"2023-07-21T22:28:59.155853+0000","last_peered":"2023-07-21T22:28:59.155853+0000","last_clean":"2023-07-21T22:28:59.155853+0000","last_became_active":"2023-07-21T22:27:41.076453+0000","last_became_peered":"2023-07-21T22:27:41.076453+0000","last_unstale":"2023-07-21T22:28:59.155853+0000","last_undegraded":"2023-07-21T22:28:59.155853+0000","last_fullsized":"2023-07-21T22:28:59.155853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:32:01.149003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242778+0000","last_change":"2023-07-21T22:27:42.091286+0000","last_active":"2023-07-21T22:28:47.242778+0000","last_peered":"2023-07-21T22:28:47.242778+0000","last_clean":"2023-07-21T22:28:47.242778+0000","last_became_active":"2023-07-21T22:27:42.090986+0000","last_became_peered":"2023-07-21T22:27:42.090986+0000","last_unstale":"2023-07-21T22:28:47.242778+0000","last_undegraded":"2023-07-21T22:28:47.242778+0000","last_fullsized":"2023-07-21T22:28:47.242778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:19.449945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245874+0000","last_change":"2023-07-21T22:27:41.074075+0000","last_active":"2023-07-21T22:28:47.245874+0000","last_peered":"2023-07-21T22:28:47.245874+0000","last_clean":"2023-07-21T22:28:47.245874+0000","last_became_active":"2023-07-21T22:27:41.073757+0000","last_became_peered":"2023-07-21T22:27:41.073757+0000","last_unstale":"2023-07-21T22:28:47.245874+0000","last_undegraded":"2023-07-21T22:28:47.245874+0000","last_fullsized":"2023-07-21T22:28:47.245874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:53.883172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245145+0000","last_change":"2023-07-21T22:26:43.405788+0000","last_active":"2023-07-21T22:28:47.245145+0000","last_peered":"2023-07-21T22:28:47.245145+0000","last_clean":"2023-07-21T22:28:47.245145+0000","last_became_active":"2023-07-21T22:26:43.405351+0000","last_became_peered":"2023-07-21T22:26:43.405351+0000","last_unstale":"2023-07-21T22:28:47.245145+0000","last_undegraded":"2023-07-21T22:28:47.245145+0000","last_fullsized":"2023-07-21T22:28:47.245145+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_clean_scrub_stamp":"2023-07-21T22:23:52.032210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:20:43.690726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239038+0000","last_change":"2023-07-21T22:27:42.097967+0000","last_active":"2023-07-21T22:28:47.239038+0000","last_peered":"2023-07-21T22:28:47.239038+0000","last_clean":"2023-07-21T22:28:47.239038+0000","last_became_active":"2023-07-21T22:27:42.097708+0000","last_became_peered":"2023-07-21T22:27:42.097708+0000","last_unstale":"2023-07-21T22:28:47.239038+0000","last_undegraded":"2023-07-21T22:28:47.239038+0000","last_fullsized":"2023-07-21T22:28:47.239038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:47:00.150179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245422+0000","last_change":"2023-07-21T22:27:41.077853+0000","last_active":"2023-07-21T22:28:47.245422+0000","last_peered":"2023-07-21T22:28:47.245422+0000","last_clean":"2023-07-21T22:28:47.245422+0000","last_became_active":"2023-07-21T22:27:41.077539+0000","last_became_peered":"2023-07-21T22:27:41.077539+0000","last_unstale":"2023-07-21T22:28:47.245422+0000","last_undegraded":"2023-07-21T22:28:47.245422+0000","last_fullsized":"2023-07-21T22:28:47.245422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:11:54.875605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.248834+0000","last_change":"2023-07-21T22:27:41.076529+0000","last_active":"2023-07-21T22:28:47.248834+0000","last_peered":"2023-07-21T22:28:47.248834+0000","last_clean":"2023-07-21T22:28:47.248834+0000","last_became_active":"2023-07-21T22:27:41.076208+0000","last_became_peered":"2023-07-21T22:27:41.076208+0000","last_unstale":"2023-07-21T22:28:47.248834+0000","last_undegraded":"2023-07-21T22:28:47.248834+0000","last_fullsized":"2023-07-21T22:28:47.248834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:31:37.704902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239276+0000","last_change":"2023-07-21T22:27:42.082587+0000","last_active":"2023-07-21T22:28:47.239276+0000","last_peered":"2023-07-21T22:28:47.239276+0000","last_clean":"2023-07-21T22:28:47.239276+0000","last_became_active":"2023-07-21T22:27:42.082212+0000","last_became_peered":"2023-07-21T22:27:42.082212+0000","last_unstale":"2023-07-21T22:28:47.239276+0000","last_undegraded":"2023-07-21T22:28:47.239276+0000","last_fullsized":"2023-07-21T22:28:47.239276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:12:19.542426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244140+0000","last_change":"2023-07-21T22:27:41.073736+0000","last_active":"2023-07-21T22:28:47.244140+0000","last_peered":"2023-07-21T22:28:47.244140+0000","last_clean":"2023-07-21T22:28:47.244140+0000","last_became_active":"2023-07-21T22:27:41.073213+0000","last_became_peered":"2023-07-21T22:27:41.073213+0000","last_unstale":"2023-07-21T22:28:47.244140+0000","last_undegraded":"2023-07-21T22:28:47.244140+0000","last_fullsized":"2023-07-21T22:28:47.244140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:44:51.762687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242235+0000","last_change":"2023-07-21T22:27:42.091313+0000","last_active":"2023-07-21T22:28:47.242235+0000","last_peered":"2023-07-21T22:28:47.242235+0000","last_clean":"2023-07-21T22:28:47.242235+0000","last_became_active":"2023-07-21T22:27:42.091012+0000","last_became_peered":"2023-07-21T22:27:42.091012+0000","last_unstale":"2023-07-21T22:28:47.242235+0000","last_undegraded":"2023-07-21T22:28:47.242235+0000","last_fullsized":"2023-07-21T22:28:47.242235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:53:00.317577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245842+0000","last_change":"2023-07-21T22:27:41.074644+0000","last_active":"2023-07-21T22:28:47.245842+0000","last_peered":"2023-07-21T22:28:47.245842+0000","last_clean":"2023-07-21T22:28:47.245842+0000","last_became_active":"2023-07-21T22:27:41.074364+0000","last_became_peered":"2023-07-21T22:27:41.074364+0000","last_unstale":"2023-07-21T22:28:47.245842+0000","last_undegraded":"2023-07-21T22:28:47.245842+0000","last_fullsized":"2023-07-21T22:28:47.245842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:23:08.316538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382240+0000","last_change":"2023-07-21T22:27:42.089452+0000","last_active":"2023-07-21T22:28:47.382240+0000","last_peered":"2023-07-21T22:28:47.382240+0000","last_clean":"2023-07-21T22:28:47.382240+0000","last_became_active":"2023-07-21T22:27:42.088859+0000","last_became_peered":"2023-07-21T22:27:42.088859+0000","last_unstale":"2023-07-21T22:28:47.382240+0000","last_undegraded":"2023-07-21T22:28:47.382240+0000","last_fullsized":"2023-07-21T22:28:47.382240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:38:51.287342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382549+0000","last_change":"2023-07-21T22:27:41.077067+0000","last_active":"2023-07-21T22:28:47.382549+0000","last_peered":"2023-07-21T22:28:47.382549+0000","last_clean":"2023-07-21T22:28:47.382549+0000","last_became_active":"2023-07-21T22:27:41.076768+0000","last_became_peered":"2023-07-21T22:27:41.076768+0000","last_unstale":"2023-07-21T22:28:47.382549+0000","last_undegraded":"2023-07-21T22:28:47.382549+0000","last_fullsized":"2023-07-21T22:28:47.382549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:07:35.434120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.247883+0000","last_change":"2023-07-21T22:27:42.088789+0000","last_active":"2023-07-21T22:28:47.247883+0000","last_peered":"2023-07-21T22:28:47.247883+0000","last_clean":"2023-07-21T22:28:47.247883+0000","last_became_active":"2023-07-21T22:27:42.088361+0000","last_became_peered":"2023-07-21T22:27:42.088361+0000","last_unstale":"2023-07-21T22:28:47.247883+0000","last_undegraded":"2023-07-21T22:28:47.247883+0000","last_fullsized":"2023-07-21T22:28:47.247883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:51:10.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241552+0000","last_change":"2023-07-21T22:27:41.069595+0000","last_active":"2023-07-21T22:28:47.241552+0000","last_peered":"2023-07-21T22:28:47.241552+0000","last_clean":"2023-07-21T22:28:47.241552+0000","last_became_active":"2023-07-21T22:27:41.068733+0000","last_became_peered":"2023-07-21T22:27:41.068733+0000","last_unstale":"2023-07-21T22:28:47.241552+0000","last_undegraded":"2023-07-21T22:28:47.241552+0000","last_fullsized":"2023-07-21T22:28:47.241552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:49.910593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243049+0000","last_change":"2023-07-21T22:27:42.085701+0000","last_active":"2023-07-21T22:28:47.243049+0000","last_peered":"2023-07-21T22:28:47.243049+0000","last_clean":"2023-07-21T22:28:47.243049+0000","last_became_active":"2023-07-21T22:27:42.085429+0000","last_became_peered":"2023-07-21T22:27:42.085429+0000","last_unstale":"2023-07-21T22:28:47.243049+0000","last_undegraded":"2023-07-21T22:28:47.243049+0000","last_fullsized":"2023-07-21T22:28:47.243049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:34:44.915735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243657+0000","last_change":"2023-07-21T22:27:41.073233+0000","last_active":"2023-07-21T22:28:47.243657+0000","last_peered":"2023-07-21T22:28:47.243657+0000","last_clean":"2023-07-21T22:28:47.243657+0000","last_became_active":"2023-07-21T22:27:41.072866+0000","last_became_peered":"2023-07-21T22:27:41.072866+0000","last_unstale":"2023-07-21T22:28:47.243657+0000","last_undegraded":"2023-07-21T22:28:47.243657+0000","last_fullsized":"2023-07-21T22:28:47.243657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:34:44.838691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241957+0000","last_change":"2023-07-21T22:27:42.100093+0000","last_active":"2023-07-21T22:28:47.241957+0000","last_peered":"2023-07-21T22:28:47.241957+0000","last_clean":"2023-07-21T22:28:47.241957+0000","last_became_active":"2023-07-21T22:27:42.099813+0000","last_became_peered":"2023-07-21T22:27:42.099813+0000","last_unstale":"2023-07-21T22:28:47.241957+0000","last_undegraded":"2023-07-21T22:28:47.241957+0000","last_fullsized":"2023-07-21T22:28:47.241957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:52:06.785113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.259961+0000","last_change":"2023-07-21T22:28:47.259961+0000","last_active":"2023-07-21T22:28:47.259961+0000","last_peered":"2023-07-21T22:28:47.259961+0000","last_clean":"2023-07-21T22:28:47.259961+0000","last_became_active":"2023-07-21T22:28:47.248979+0000","last_became_peered":"2023-07-21T22:28:47.248979+0000","last_unstale":"2023-07-21T22:28:47.259961+0000","last_undegraded":"2023-07-21T22:28:47.259961+0000","last_fullsized":"2023-07-21T22:28:47.259961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:12:44.807097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245648+0000","last_change":"2023-07-21T22:27:42.097736+0000","last_active":"2023-07-21T22:28:47.245648+0000","last_peered":"2023-07-21T22:28:47.245648+0000","last_clean":"2023-07-21T22:28:47.245648+0000","last_became_active":"2023-07-21T22:27:42.097419+0000","last_became_peered":"2023-07-21T22:27:42.097419+0000","last_unstale":"2023-07-21T22:28:47.245648+0000","last_undegraded":"2023-07-21T22:28:47.245648+0000","last_fullsized":"2023-07-21T22:28:47.245648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:07:09.477979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":99,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:00.157924+0000","last_change":"2023-07-21T22:27:41.074922+0000","last_active":"2023-07-21T22:29:00.157924+0000","last_peered":"2023-07-21T22:29:00.157924+0000","last_clean":"2023-07-21T22:29:00.157924+0000","last_became_active":"2023-07-21T22:27:41.074610+0000","last_became_peered":"2023-07-21T22:27:41.074610+0000","last_unstale":"2023-07-21T22:29:00.157924+0000","last_undegraded":"2023-07-21T22:29:00.157924+0000","last_fullsized":"2023-07-21T22:29:00.157924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:19.892702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245234+0000","last_change":"2023-07-21T22:27:42.097312+0000","last_active":"2023-07-21T22:28:47.245234+0000","last_peered":"2023-07-21T22:28:47.245234+0000","last_clean":"2023-07-21T22:28:47.245234+0000","last_became_active":"2023-07-21T22:27:42.096835+0000","last_became_peered":"2023-07-21T22:27:42.096835+0000","last_unstale":"2023-07-21T22:28:47.245234+0000","last_undegraded":"2023-07-21T22:28:47.245234+0000","last_fullsized":"2023-07-21T22:28:47.245234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:34:28.949507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240533+0000","last_change":"2023-07-21T22:27:41.071971+0000","last_active":"2023-07-21T22:28:47.240533+0000","last_peered":"2023-07-21T22:28:47.240533+0000","last_clean":"2023-07-21T22:28:47.240533+0000","last_became_active":"2023-07-21T22:27:41.071550+0000","last_became_peered":"2023-07-21T22:27:41.071550+0000","last_unstale":"2023-07-21T22:28:47.240533+0000","last_undegraded":"2023-07-21T22:28:47.240533+0000","last_fullsized":"2023-07-21T22:28:47.240533+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:35:28.749230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244847+0000","last_change":"2023-07-21T22:27:42.098039+0000","last_active":"2023-07-21T22:28:47.244847+0000","last_peered":"2023-07-21T22:28:47.244847+0000","last_clean":"2023-07-21T22:28:47.244847+0000","last_became_active":"2023-07-21T22:27:42.097737+0000","last_became_peered":"2023-07-21T22:27:42.097737+0000","last_unstale":"2023-07-21T22:28:47.244847+0000","last_undegraded":"2023-07-21T22:28:47.244847+0000","last_fullsized":"2023-07-21T22:28:47.244847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:41:06.382770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.251946+0000","last_change":"2023-07-21T22:28:47.251946+0000","last_active":"2023-07-21T22:28:47.251946+0000","last_peered":"2023-07-21T22:28:47.251946+0000","last_clean":"2023-07-21T22:28:47.251946+0000","last_became_active":"2023-07-21T22:28:47.247973+0000","last_became_peered":"2023-07-21T22:28:47.247973+0000","last_unstale":"2023-07-21T22:28:47.251946+0000","last_undegraded":"2023-07-21T22:28:47.251946+0000","last_fullsized":"2023-07-21T22:28:47.251946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:18.015306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245217+0000","last_change":"2023-07-21T22:27:42.097891+0000","last_active":"2023-07-21T22:28:47.245217+0000","last_peered":"2023-07-21T22:28:47.245217+0000","last_clean":"2023-07-21T22:28:47.245217+0000","last_became_active":"2023-07-21T22:27:42.097022+0000","last_became_peered":"2023-07-21T22:27:42.097022+0000","last_unstale":"2023-07-21T22:28:47.245217+0000","last_undegraded":"2023-07-21T22:28:47.245217+0000","last_fullsized":"2023-07-21T22:28:47.245217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:54:17.707405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243207+0000","last_change":"2023-07-21T22:27:42.085774+0000","last_active":"2023-07-21T22:28:47.243207+0000","last_peered":"2023-07-21T22:28:47.243207+0000","last_clean":"2023-07-21T22:28:47.243207+0000","last_became_active":"2023-07-21T22:27:42.085460+0000","last_became_peered":"2023-07-21T22:27:42.085460+0000","last_unstale":"2023-07-21T22:28:47.243207+0000","last_undegraded":"2023-07-21T22:28:47.243207+0000","last_fullsized":"2023-07-21T22:28:47.243207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:26:42.082026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.251186+0000","last_change":"2023-07-21T22:28:47.251186+0000","last_active":"2023-07-21T22:28:47.251186+0000","last_peered":"2023-07-21T22:28:47.251186+0000","last_clean":"2023-07-21T22:28:47.251186+0000","last_became_active":"2023-07-21T22:28:47.250765+0000","last_became_peered":"2023-07-21T22:28:47.250765+0000","last_unstale":"2023-07-21T22:28:47.251186+0000","last_undegraded":"2023-07-21T22:28:47.251186+0000","last_fullsized":"2023-07-21T22:28:47.251186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:12:57.007813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242433+0000","last_change":"2023-07-21T22:27:42.092918+0000","last_active":"2023-07-21T22:28:47.242433+0000","last_peered":"2023-07-21T22:28:47.242433+0000","last_clean":"2023-07-21T22:28:47.242433+0000","last_became_active":"2023-07-21T22:27:42.092484+0000","last_became_peered":"2023-07-21T22:27:42.092484+0000","last_unstale":"2023-07-21T22:28:47.242433+0000","last_undegraded":"2023-07-21T22:28:47.242433+0000","last_fullsized":"2023-07-21T22:28:47.242433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:43:03.935206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382257+0000","last_change":"2023-07-21T22:27:42.096306+0000","last_active":"2023-07-21T22:28:47.382257+0000","last_peered":"2023-07-21T22:28:47.382257+0000","last_clean":"2023-07-21T22:28:47.382257+0000","last_became_active":"2023-07-21T22:27:42.096010+0000","last_became_peered":"2023-07-21T22:27:42.096010+0000","last_unstale":"2023-07-21T22:28:47.382257+0000","last_undegraded":"2023-07-21T22:28:47.382257+0000","last_fullsized":"2023-07-21T22:28:47.382257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:20:04.344089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239283+0000","last_change":"2023-07-21T22:27:42.097964+0000","last_active":"2023-07-21T22:28:47.239283+0000","last_peered":"2023-07-21T22:28:47.239283+0000","last_clean":"2023-07-21T22:28:47.239283+0000","last_became_active":"2023-07-21T22:27:42.097666+0000","last_became_peered":"2023-07-21T22:27:42.097666+0000","last_unstale":"2023-07-21T22:28:47.239283+0000","last_undegraded":"2023-07-21T22:28:47.239283+0000","last_fullsized":"2023-07-21T22:28:47.239283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:48:04.636292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243609+0000","last_change":"2023-07-21T22:27:42.083225+0000","last_active":"2023-07-21T22:28:47.243609+0000","last_peered":"2023-07-21T22:28:47.243609+0000","last_clean":"2023-07-21T22:28:47.243609+0000","last_became_active":"2023-07-21T22:27:42.082892+0000","last_became_peered":"2023-07-21T22:27:42.082892+0000","last_unstale":"2023-07-21T22:28:47.243609+0000","last_undegraded":"2023-07-21T22:28:47.243609+0000","last_fullsized":"2023-07-21T22:28:47.243609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:57:37.483988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243881+0000","last_change":"2023-07-21T22:27:42.090854+0000","last_active":"2023-07-21T22:28:47.243881+0000","last_peered":"2023-07-21T22:28:47.243881+0000","last_clean":"2023-07-21T22:28:47.243881+0000","last_became_active":"2023-07-21T22:27:42.090501+0000","last_became_peered":"2023-07-21T22:27:42.090501+0000","last_unstale":"2023-07-21T22:28:47.243881+0000","last_undegraded":"2023-07-21T22:28:47.243881+0000","last_fullsized":"2023-07-21T22:28:47.243881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:43:35.575019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242109+0000","last_change":"2023-07-21T22:27:42.094986+0000","last_active":"2023-07-21T22:28:47.242109+0000","last_peered":"2023-07-21T22:28:47.242109+0000","last_clean":"2023-07-21T22:28:47.242109+0000","last_became_active":"2023-07-21T22:27:42.094668+0000","last_became_peered":"2023-07-21T22:27:42.094668+0000","last_unstale":"2023-07-21T22:28:47.242109+0000","last_undegraded":"2023-07-21T22:28:47.242109+0000","last_fullsized":"2023-07-21T22:28:47.242109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:17:57.739239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240373+0000","last_change":"2023-07-21T22:27:42.098033+0000","last_active":"2023-07-21T22:28:47.240373+0000","last_peered":"2023-07-21T22:28:47.240373+0000","last_clean":"2023-07-21T22:28:47.240373+0000","last_became_active":"2023-07-21T22:27:42.097219+0000","last_became_peered":"2023-07-21T22:27:42.097219+0000","last_unstale":"2023-07-21T22:28:47.240373+0000","last_undegraded":"2023-07-21T22:28:47.240373+0000","last_fullsized":"2023-07-21T22:28:47.240373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:58:21.227651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241687+0000","last_change":"2023-07-21T22:27:42.092136+0000","last_active":"2023-07-21T22:28:47.241687+0000","last_peered":"2023-07-21T22:28:47.241687+0000","last_clean":"2023-07-21T22:28:47.241687+0000","last_became_active":"2023-07-21T22:27:42.091559+0000","last_became_peered":"2023-07-21T22:27:42.091559+0000","last_unstale":"2023-07-21T22:28:47.241687+0000","last_undegraded":"2023-07-21T22:28:47.241687+0000","last_fullsized":"2023-07-21T22:28:47.241687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:56.512118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382811+0000","last_change":"2023-07-21T22:27:42.093227+0000","last_active":"2023-07-21T22:28:47.382811+0000","last_peered":"2023-07-21T22:28:47.382811+0000","last_clean":"2023-07-21T22:28:47.382811+0000","last_became_active":"2023-07-21T22:27:42.092918+0000","last_became_peered":"2023-07-21T22:27:42.092918+0000","last_unstale":"2023-07-21T22:28:47.382811+0000","last_undegraded":"2023-07-21T22:28:47.382811+0000","last_fullsized":"2023-07-21T22:28:47.382811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:00:34.144455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239365+0000","last_change":"2023-07-21T22:27:42.086494+0000","last_active":"2023-07-21T22:28:47.239365+0000","last_peered":"2023-07-21T22:28:47.239365+0000","last_clean":"2023-07-21T22:28:47.239365+0000","last_became_active":"2023-07-21T22:27:42.086189+0000","last_became_peered":"2023-07-21T22:27:42.086189+0000","last_unstale":"2023-07-21T22:28:47.239365+0000","last_undegraded":"2023-07-21T22:28:47.239365+0000","last_fullsized":"2023-07-21T22:28:47.239365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:01:11.731302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242968+0000","last_change":"2023-07-21T22:27:42.090874+0000","last_active":"2023-07-21T22:28:47.242968+0000","last_peered":"2023-07-21T22:28:47.242968+0000","last_clean":"2023-07-21T22:28:47.242968+0000","last_became_active":"2023-07-21T22:27:42.090597+0000","last_became_peered":"2023-07-21T22:27:42.090597+0000","last_unstale":"2023-07-21T22:28:47.242968+0000","last_undegraded":"2023-07-21T22:28:47.242968+0000","last_fullsized":"2023-07-21T22:28:47.242968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:16:50.131217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61988,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":779,"num_read_kb":456,"num_write":230,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":986,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":185646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.629},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.52000000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.501},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.80900000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.82499999999999996}]},{"osd":9,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.78900000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.70399999999999996}]}]},{"osd":10,"up_from":70,"seq":300647710752,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1613824,"data_stored":1085151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":8.7710000000000008,"5min":8.7710000000000008,"15min":8.7710000000000008},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":8.8610000000000007,"5min":8.8610000000000007,"15min":8.8610000000000007},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.47299999999999998,"15min":0.46999999999999997},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70399999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.505,"15min":0.501},"min":{"1min":0.40100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.38100000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.495,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80100000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.73899999999999999,"15min":0.67700000000000005},"min":{"1min":0.45100000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":8.5139999999999993,"5min":8.5139999999999993,"15min":8.5139999999999993},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":0.75800000000000001,"15min":0.67800000000000005},"min":{"1min":0.47699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":8.5440000000000005,"5min":8.5440000000000005,"15min":8.5440000000000005},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.36499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75800000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.499}]},{"osd":5,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67900000000000005,"15min":0.67400000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.47499999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.98399999999999999,"5min":1.008,"15min":1.008},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.76400000000000001,"15min":0.72399999999999998},"min":{"1min":0.56899999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":3.972,"5min":3.972,"15min":3.972},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.087,"5min":0.76900000000000002,"15min":0.71599999999999997},"min":{"1min":0.51100000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.8570000000000002,"5min":3.8570000000000002,"15min":3.8570000000000002},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0089999999999999,"5min":1.073,"15min":1.073},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.998,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.69399999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":0.90100000000000002,"15min":0.82199999999999995},"min":{"1min":0.52900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":8.5899999999999999,"5min":8.5899999999999999,"15min":8.5899999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.82299999999999995,"15min":0.81200000000000006},"min":{"1min":0.57499999999999996,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.1469999999999998,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.67000000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.76000000000000001}]}]},{"osd":9,"up_from":65,"seq":279172874277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":439765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.438,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.0379999999999998,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.129,"5min":3.129,"15min":3.129},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.51500000000000001,"15min":0.53000000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.436}]},{"osd":4,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.66500000000000004,"15min":0.68500000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":2.9710000000000001,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0469999999999999,"5min":1.089,"15min":1.089},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.66400000000000003,"15min":0.68600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93700000000000006,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.749}]},{"osd":8,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.73499999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:28:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.72599999999999998,"15min":0.73799999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.745,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.78900000000000003}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":440121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.62},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.35999999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.752}]},{"osd":2,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.45400000000000001,"15min":0.438},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.48099999999999998,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60099999999999998,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.71799999999999997}]},{"osd":4,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.434,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61599999999999999,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.873},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69999999999999996,"15min":0.69699999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.997,"5min":1.097,"15min":1.097},"last":1.004}]},{"osd":7,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.55800000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.054,"5min":1.171,"15min":1.171},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.68999999999999995,"15min":0.68000000000000005},"min":{"1min":0.51000000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1830000000000001,"5min":1.232,"15min":1.232},"last":0.89300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69799999999999995,"15min":0.68400000000000005},"min":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0680000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.94799999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.746,"15min":0.71899999999999997},"min":{"1min":0.67500000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0800000000000001,"5min":1.21,"15min":1.21},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.76900000000000002,"15min":0.745},"min":{"1min":0.68500000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.97699999999999998}]},{"osd":11,"last update":"Fri Jul 21 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.96199999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28456,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716696,"statfs":{"total":95995035648,"available":95965896704,"internally_reserved":0,"allocated":1540096,"data_stored":1032522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53800000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.436,"15min":0.437},"min":{"1min":0.32700000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46400000000000002,"15min":0.46500000000000002},"min":{"1min":0.32900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.34599999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83599999999999997,"5min":1.548,"15min":1.548},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.80700000000000005,"15min":0.88700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82599999999999996,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.77600000000000002,"15min":0.82699999999999996},"min":{"1min":0.434,"5min":0.374,"15min":0.374},"max":{"1min":2.169,"5min":5.7480000000000002,"15min":5.7480000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.78000000000000003,"15min":0.85399999999999998},"min":{"1min":0.45000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.871,"5min":5.7649999999999997,"15min":5.7649999999999997},"last":0.753}]},{"osd":4,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.42999999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.41499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.80300000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.44400000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67700000000000005,"15min":0.66900000000000004},"min":{"1min":0.51600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91800000000000004,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90600000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.69399999999999995,"15min":0.68300000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89800000000000002,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.73599999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.91000000000000003,"15min":0.98399999999999999},"min":{"1min":0.57999999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.0009999999999999,"5min":5.7800000000000002,"15min":5.7800000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78300000000000003,"15min":0.79900000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1060000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.54200000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.79100000000000004,"15min":0.80700000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85799999999999998,"5min":1.819,"15min":1.819},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.753,"15min":0.76100000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.88300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.56499999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.97999999999999998,"5min":1.839,"15min":1.839},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.57599999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96599999999999997,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.76900000000000002}]},{"osd":11,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.70199999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528368,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":439440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.36499999999999999},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.624,"15min":0.56299999999999994},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":6.899,"5min":6.899,"15min":6.899},"last":0.628},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.45600000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54000000000000004,"15min":0.52000000000000002},"min":{"1min":0.44800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.64500000000000002,"15min":0.57499999999999996},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":7.0359999999999996,"5min":7.0359999999999996,"15min":7.0359999999999996},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.496,"15min":0.437},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":7.1289999999999996,"5min":7.1289999999999996,"15min":7.1289999999999996},"last":0.315},{"interface":"front","average":{"1min":1.071,"5min":0.58399999999999996,"15min":0.48599999999999999},"min":{"1min":0.32000000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":12.414999999999999,"5min":12.414999999999999,"15min":12.414999999999999},"last":0.42099999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.64900000000000002,"15min":0.58699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.9690000000000003,"5min":6.9690000000000003,"15min":6.9690000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.48499999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.44700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.60199999999999998,"15min":0.58799999999999997},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59299999999999997,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.70099999999999996,"15min":0.64000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":7.0049999999999999,"5min":7.0049999999999999,"15min":7.0049999999999999},"last":0.55600000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.747,"15min":0.69899999999999995},"min":{"1min":0.55000000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":7.0670000000000002,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70099999999999996,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.60899999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.68000000000000005,"15min":0.66700000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.516,"5min":4.516,"15min":4.516},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.78000000000000003,"15min":0.75600000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":7.1619999999999999,"5min":7.1619999999999999,"15min":7.1619999999999999},"last":0.496}]},{"osd":10,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.83599999999999997,"15min":0.80800000000000005},"min":{"1min":0.52300000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":7.0979999999999999,"5min":7.0979999999999999,"15min":7.0979999999999999},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":496289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51500000000000001,"15min":0.48499999999999999},"min":{"1min":0.41299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.48699999999999999},"min":{"1min":0.40999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.80000000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.505}]},{"osd":2,"last update":"Fri Jul 21 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.57799999999999996,"15min":0.52100000000000002},"min":{"1min":0.46800000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.79900000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56699999999999995,"15min":0.54000000000000004},"min":{"1min":0.48699999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.38300000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.872,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60199999999999998,"15min":0.58999999999999997},"min":{"1min":0.49299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.46400000000000002,"15min":0.42799999999999999},"min":{"1min":0.35999999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.46800000000000003,"15min":0.438},"min":{"1min":0.35399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.60999999999999999,"15min":0.54800000000000004},"min":{"1min":0.48999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86099999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.63400000000000001,"15min":0.57399999999999995},"min":{"1min":0.54000000000000004,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri Jul 21 22:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56299999999999994,"15min":0.53200000000000003},"min":{"1min":0.45300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.61899999999999999,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.752}]},{"osd":7,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.59699999999999998,"15min":0.55200000000000005},"min":{"1min":0.48299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.93500000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.64600000000000002,"15min":0.61199999999999999},"min":{"1min":0.437,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96799999999999997,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82999999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.70099999999999996,"15min":0.67000000000000004},"min":{"1min":0.46300000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92700000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.72499999999999998,"15min":0.69499999999999995},"min":{"1min":0.64800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77400000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.626,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79000000000000004,"15min":0.78500000000000003},"min":{"1min":0.58299999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.80600000000000005}]},{"osd":11,"last update":"Fri Jul 21 22:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.78400000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":441163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.48899999999999999},"min":{"1min":0.372,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.746,"5min":0.873,"15min":0.873},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.501},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68700000000000006,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78800000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55000000000000004,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70099999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.621}]},{"osd":3,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75700000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.441},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83199999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52400000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78500000000000003,"5min":0.879,"15min":0.879},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78500000000000003,"5min":1.002,"15min":1.002},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48299999999999998,"15min":0.49199999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58499999999999996,"5min":0.749,"15min":0.749},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79100000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.40799999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66500000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85499999999999998,"5min":1.04,"15min":1.04},"last":0.48499999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.68400000000000005,"15min":0.69099999999999995},"min":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.61899999999999999,"15min":0.63500000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88600000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.68200000000000005,"15min":0.70399999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89800000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46500000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93700000000000006,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88400000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.501}]},{"osd":11,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69599999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":494995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.501,"15min":0.47699999999999998},"min":{"1min":0.38300000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52000000000000002,"15min":0.47099999999999997},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.68500000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.56399999999999995,"15min":0.498},"min":{"1min":0.43099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.85999999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.57299999999999995,"15min":0.52300000000000002},"min":{"1min":0.45800000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.874,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57799999999999996,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.58199999999999996,"15min":0.54900000000000004},"min":{"1min":0.44600000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.501,"15min":0.46899999999999997},"min":{"1min":0.375,"5min":0.25,"15min":0.25},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.501,"15min":0.47499999999999998},"min":{"1min":0.41199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.435,"5min":0.432,"15min":0.432},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46899999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.58899999999999997,"15min":0.55300000000000005},"min":{"1min":0.46600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.625,"15min":0.59699999999999998},"min":{"1min":0.51900000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.51100000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.63700000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.73099999999999998,"15min":0.71299999999999997},"min":{"1min":0.68300000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.98499999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.92100000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.85299999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":438988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35599999999999998,"15min":0.33500000000000002},"min":{"1min":0.24399999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.629,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.26100000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.496,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.29799999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71499999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.503}]},{"osd":2,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76000000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.495},{"interface":"front","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.57999999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82999999999999996,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70599999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.61399999999999999,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.749,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.50800000000000001,"15min":0.501},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.439}]},{"osd":7,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.40200000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79000000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.65700000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85499999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.56999999999999995}]},{"osd":8,"last update":"Fri Jul 21 22:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.64300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.996,"5min":1.038,"15min":1.038},"last":0.47199999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77300000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.84899999999999998,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.64700000000000002}]},{"osd":11,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.60799999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":439613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.47899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.49299999999999999},"min":{"1min":0.41799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41799999999999998,"15min":0.39700000000000002},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41799999999999998,"15min":0.39200000000000002},"min":{"1min":0.34200000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59099999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.44900000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.443,"5min":0.371,"15min":0.371},"max":{"1min":0.82999999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.83399999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.60899999999999999,"15min":0.54300000000000004},"min":{"1min":0.40100000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.6559999999999997,"5min":5.6559999999999997,"15min":5.6559999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.56699999999999995,"15min":0.53300000000000003},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.5430000000000001,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Jul 21 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.61699999999999999,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.435,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79600000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.78100000000000003}]},{"osd":6,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.65900000000000003,"15min":0.60699999999999998},"min":{"1min":0.45300000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":5.6310000000000002,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.68899999999999995,"15min":0.629},"min":{"1min":0.53100000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61199999999999999,"15min":0.59299999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97299999999999998,"5min":0.995,"15min":0.995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60399999999999998,"15min":0.59299999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72299999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Jul 21 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.54400000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89000000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86699999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80100000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":0.81299999999999994,"15min":0.755},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":7.6639999999999997,"5min":7.6639999999999997,"15min":7.6639999999999997},"last":0.754},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.81699999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78000000000000003,"15min":0.78700000000000003},"min":{"1min":0.53200000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.94599999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77500000000000002,"15min":0.77800000000000002},"min":{"1min":0.496,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94799999999999995,"5min":1.103,"15min":1.103},"last":0.86799999999999999}]},{"osd":11,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.85199999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1512,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1548288,"data_stored":1031965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52000000000000002},"min":{"1min":0.315,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80200000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.51500000000000001,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54600000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.45700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81899999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58799999999999997,"15min":0.54400000000000004},"min":{"1min":0.53000000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89000000000000001,"5min":1.036,"15min":1.036},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.42599999999999999,"15min":0.39100000000000001},"min":{"1min":0.32000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67700000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.43099999999999999,"15min":0.40300000000000002},"min":{"1min":0.32000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.64800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.39900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.436},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58499999999999996,"15min":0.59499999999999997},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76900000000000002,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61299999999999999,"15min":0.56899999999999995},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83599999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.48099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.874,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.629},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63600000000000001,"15min":0.61299999999999999},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89400000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.61499999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90000000000000002,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.67400000000000004,"15min":0.64400000000000002},"min":{"1min":0.58099999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92800000000000005,"5min":1.054,"15min":1.054},"last":0.72899999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.628},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89100000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.749},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.59699999999999998},"min":{"1min":0.45300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81699999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69099999999999995,"15min":0.68700000000000006},"min":{"1min":0.53200000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.54900000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77300000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.80800000000000005,"15min":0.80600000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.99099999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77200000000000002,"15min":0.76800000000000002},"min":{"1min":0.58499999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91600000000000004,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.81299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":57646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:29:07.426 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-07-21T22:29:07.427 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:29:07.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:07 smithi132 ceph-mon[97863]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2023-07-21T22:29:07.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:07 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3829748508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:29:07.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:07 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3349380704' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:29:07.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.533+0000 7f8c94ac5700 1 -- 172.21.15.107:0/4147749375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c900ffbc0 msgr2=0x7f8c90100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.533+0000 7f8c94ac5700 1 --2- 172.21.15.107:0/4147749375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c900ffbc0 0x7f8c90100010 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009990 tx=0x7f8c80009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.533+0000 7f8c94ac5700 1 -- 172.21.15.107:0/4147749375 shutdown_connections 2023-07-21T22:29:07.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.533+0000 7f8c94ac5700 1 --2- 172.21.15.107:0/4147749375 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c90100550 0x7f8c90106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.533+0000 7f8c94ac5700 1 --2- 172.21.15.107:0/4147749375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c900fe9d0 0x7f8c900fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.533+0000 7f8c94ac5700 1 --2- 172.21.15.107:0/4147749375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c900ffbc0 0x7f8c90100010 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.533+0000 7f8c94ac5700 1 -- 172.21.15.107:0/4147749375 >> 172.21.15.107:0/4147749375 conn(0x7f8c900f9fd0 msgr2=0x7f8c900fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:07.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.533+0000 7f8c94ac5700 1 -- 172.21.15.107:0/4147749375 shutdown_connections 2023-07-21T22:29:07.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.533+0000 7f8c94ac5700 1 -- 172.21.15.107:0/4147749375 wait complete. 2023-07-21T22:29:07.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c94ac5700 1 Processor -- start 2023-07-21T22:29:07.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c94ac5700 1 -- start start 2023-07-21T22:29:07.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c94ac5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c900fe9d0 0x7f8c90194690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:07.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c94ac5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c900ffbc0 0x7f8c90194bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c94ac5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c90100550 0x7f8c9019bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c94ac5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9019c210 con 0x7f8c900fe9d0 2023-07-21T22:29:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c94ac5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9019c350 con 0x7f8c90100550 2023-07-21T22:29:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c94ac5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9019c4e0 con 0x7f8c900ffbc0 2023-07-21T22:29:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c8e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c900fe9d0 0x7f8c90194690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c8e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c900fe9d0 0x7f8c90194690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48072/0 (socket says 172.21.15.107:48072) 2023-07-21T22:29:07.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c8e59c700 1 -- 172.21.15.107:0/292834931 learned_addr learned my addr 172.21.15.107:0/292834931 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c8dd9b700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c900ffbc0 0x7f8c90194bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c8e59c700 1 -- 172.21.15.107:0/292834931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c900ffbc0 msgr2=0x7f8c90194bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c8e59c700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c900ffbc0 0x7f8c90194bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c8ed9d700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c90100550 0x7f8c9019bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c8e59c700 1 -- 172.21.15.107:0/292834931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c90100550 msgr2=0x7f8c9019bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c8e59c700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c90100550 0x7f8c9019bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c8e59c700 1 -- 172.21.15.107:0/292834931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c800095f0 con 0x7f8c900fe9d0 2023-07-21T22:29:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c8dd9b700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c900ffbc0 0x7f8c90194bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.534+0000 7f8c8ed9d700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c90100550 0x7f8c9019bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.535+0000 7f8c8e59c700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c900fe9d0 0x7f8c90194690 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400c970 tx=0x7f8c8400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.535+0000 7f8c7f7fe700 1 -- 172.21.15.107:0/292834931 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84004220 con 0x7f8c900fe9d0 2023-07-21T22:29:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.535+0000 7f8c7f7fe700 1 -- 172.21.15.107:0/292834931 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c84007d60 con 0x7f8c900fe9d0 2023-07-21T22:29:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.535+0000 7f8c7f7fe700 1 -- 172.21.15.107:0/292834931 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84003d00 con 0x7f8c900fe9d0 2023-07-21T22:29:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.535+0000 7f8c94ac5700 1 -- 172.21.15.107:0/292834931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9019c7c0 con 0x7f8c900fe9d0 2023-07-21T22:29:07.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.535+0000 7f8c94ac5700 1 -- 172.21.15.107:0/292834931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9019cc80 con 0x7f8c900fe9d0 2023-07-21T22:29:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.536+0000 7f8c94ac5700 1 -- 172.21.15.107:0/292834931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c90105f30 con 0x7f8c900fe9d0 2023-07-21T22:29:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.537+0000 7f8c7f7fe700 1 -- 172.21.15.107:0/292834931 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8c84004380 con 0x7f8c900fe9d0 2023-07-21T22:29:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.537+0000 7f8c7f7fe700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8c780a1220 0x7f8c780a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.538+0000 7f8c8dd9b700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8c780a1220 0x7f8c780a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.538+0000 7f8c7f7fe700 1 -- 172.21.15.107:0/292834931 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8c840ccb00 con 0x7f8c900fe9d0 2023-07-21T22:29:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.538+0000 7f8c8dd9b700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8c780a1220 0x7f8c780a36d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8c80006010 tx=0x7f8c8000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.541+0000 7f8c7f7fe700 1 -- 172.21.15.107:0/292834931 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8c84096070 con 0x7f8c900fe9d0 2023-07-21T22:29:07.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.734+0000 7f8c94ac5700 1 -- 172.21.15.107:0/292834931 --> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c90060950 con 0x7f8c780a1220 2023-07-21T22:29:07.790 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.743+0000 7f8c7f7fe700 1 -- 172.21.15.107:0/292834931 <== mgr.14150 v2:172.21.15.107:6800/1657248266 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343797 (secure 0 0 0) 0x7f8c90060950 con 0x7f8c780a1220 2023-07-21T22:29:07.793 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:29:07.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.747+0000 7f8c94ac5700 1 -- 172.21.15.107:0/292834931 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8c780a1220 msgr2=0x7f8c780a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.747+0000 7f8c94ac5700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8c780a1220 0x7f8c780a36d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8c80006010 tx=0x7f8c8000b540 comp rx=0 tx=0).stop 2023-07-21T22:29:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.747+0000 7f8c94ac5700 1 -- 172.21.15.107:0/292834931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c900fe9d0 msgr2=0x7f8c90194690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.747+0000 7f8c94ac5700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c900fe9d0 0x7f8c90194690 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400c970 tx=0x7f8c8400cd30 comp rx=0 tx=0).stop 2023-07-21T22:29:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.747+0000 7f8c94ac5700 1 -- 172.21.15.107:0/292834931 shutdown_connections 2023-07-21T22:29:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.747+0000 7f8c94ac5700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c90100550 0x7f8c9019bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.747+0000 7f8c94ac5700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c900fe9d0 0x7f8c90194690 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.747+0000 7f8c94ac5700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8c780a1220 0x7f8c780a36d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.747+0000 7f8c94ac5700 1 --2- 172.21.15.107:0/292834931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c900ffbc0 0x7f8c90194bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.747+0000 7f8c94ac5700 1 -- 172.21.15.107:0/292834931 >> 172.21.15.107:0/292834931 conn(0x7f8c900f9fd0 msgr2=0x7f8c90103070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:07.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.747+0000 7f8c94ac5700 1 -- 172.21.15.107:0/292834931 shutdown_connections 2023-07-21T22:29:07.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.747+0000 7f8c94ac5700 1 -- 172.21.15.107:0/292834931 wait complete. 2023-07-21T22:29:07.797 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2023-07-21T22:29:07.807 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2023-07-21T22:29:05.808726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652602,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":905,"num_read_kb":565,"num_write":413,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181722,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336372,"kb_used_data":12948,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605452,"statfs":{"total":1151940427776,"available":1151595982848,"internally_reserved":0,"allocated":13258752,"data_stored":7220211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":318,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":56,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003089"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244157+0000","last_change":"2023-07-21T22:27:42.091397+0000","last_active":"2023-07-21T22:28:47.244157+0000","last_peered":"2023-07-21T22:28:47.244157+0000","last_clean":"2023-07-21T22:28:47.244157+0000","last_became_active":"2023-07-21T22:27:42.090836+0000","last_became_peered":"2023-07-21T22:27:42.090836+0000","last_unstale":"2023-07-21T22:28:47.244157+0000","last_undegraded":"2023-07-21T22:28:47.244157+0000","last_fullsized":"2023-07-21T22:28:47.244157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:21:03.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382220+0000","last_change":"2023-07-21T22:27:42.097003+0000","last_active":"2023-07-21T22:28:47.382220+0000","last_peered":"2023-07-21T22:28:47.382220+0000","last_clean":"2023-07-21T22:28:47.382220+0000","last_became_active":"2023-07-21T22:27:42.096652+0000","last_became_peered":"2023-07-21T22:27:42.096652+0000","last_unstale":"2023-07-21T22:28:47.382220+0000","last_undegraded":"2023-07-21T22:28:47.382220+0000","last_fullsized":"2023-07-21T22:28:47.382220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:00:27.467585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382162+0000","last_change":"2023-07-21T22:27:42.090090+0000","last_active":"2023-07-21T22:28:47.382162+0000","last_peered":"2023-07-21T22:28:47.382162+0000","last_clean":"2023-07-21T22:28:47.382162+0000","last_became_active":"2023-07-21T22:27:42.089802+0000","last_became_peered":"2023-07-21T22:27:42.089802+0000","last_unstale":"2023-07-21T22:28:47.382162+0000","last_undegraded":"2023-07-21T22:28:47.382162+0000","last_fullsized":"2023-07-21T22:28:47.382162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:15:20.510508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246255+0000","last_change":"2023-07-21T22:27:42.096291+0000","last_active":"2023-07-21T22:28:47.246255+0000","last_peered":"2023-07-21T22:28:47.246255+0000","last_clean":"2023-07-21T22:28:47.246255+0000","last_became_active":"2023-07-21T22:27:42.095982+0000","last_became_peered":"2023-07-21T22:27:42.095982+0000","last_unstale":"2023-07-21T22:28:47.246255+0000","last_undegraded":"2023-07-21T22:28:47.246255+0000","last_fullsized":"2023-07-21T22:28:47.246255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:14:38.335068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246671+0000","last_change":"2023-07-21T22:27:42.097495+0000","last_active":"2023-07-21T22:28:47.246671+0000","last_peered":"2023-07-21T22:28:47.246671+0000","last_clean":"2023-07-21T22:28:47.246671+0000","last_became_active":"2023-07-21T22:27:42.095112+0000","last_became_peered":"2023-07-21T22:27:42.095112+0000","last_unstale":"2023-07-21T22:28:47.246671+0000","last_undegraded":"2023-07-21T22:28:47.246671+0000","last_fullsized":"2023-07-21T22:28:47.246671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:14:17.323583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242851+0000","last_change":"2023-07-21T22:27:42.089880+0000","last_active":"2023-07-21T22:28:47.242851+0000","last_peered":"2023-07-21T22:28:47.242851+0000","last_clean":"2023-07-21T22:28:47.242851+0000","last_became_active":"2023-07-21T22:27:42.088614+0000","last_became_peered":"2023-07-21T22:27:42.088614+0000","last_unstale":"2023-07-21T22:28:47.242851+0000","last_undegraded":"2023-07-21T22:28:47.242851+0000","last_fullsized":"2023-07-21T22:28:47.242851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:06:38.785570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245114+0000","last_change":"2023-07-21T22:27:42.085457+0000","last_active":"2023-07-21T22:28:47.245114+0000","last_peered":"2023-07-21T22:28:47.245114+0000","last_clean":"2023-07-21T22:28:47.245114+0000","last_became_active":"2023-07-21T22:27:42.085146+0000","last_became_peered":"2023-07-21T22:27:42.085146+0000","last_unstale":"2023-07-21T22:28:47.245114+0000","last_undegraded":"2023-07-21T22:28:47.245114+0000","last_fullsized":"2023-07-21T22:28:47.245114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:06:59.371781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.247038+0000","last_change":"2023-07-21T22:28:47.246617+0000","last_active":"2023-07-21T22:28:47.247038+0000","last_peered":"2023-07-21T22:28:47.247038+0000","last_clean":"2023-07-21T22:28:47.247038+0000","last_became_active":"2023-07-21T22:28:47.246322+0000","last_became_peered":"2023-07-21T22:28:47.246322+0000","last_unstale":"2023-07-21T22:28:47.247038+0000","last_undegraded":"2023-07-21T22:28:47.247038+0000","last_fullsized":"2023-07-21T22:28:47.247038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:22.991062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383406+0000","last_change":"2023-07-21T22:27:42.094223+0000","last_active":"2023-07-21T22:28:47.383406+0000","last_peered":"2023-07-21T22:28:47.383406+0000","last_clean":"2023-07-21T22:28:47.383406+0000","last_became_active":"2023-07-21T22:27:42.093875+0000","last_became_peered":"2023-07-21T22:27:42.093875+0000","last_unstale":"2023-07-21T22:28:47.383406+0000","last_undegraded":"2023-07-21T22:28:47.383406+0000","last_fullsized":"2023-07-21T22:28:47.383406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:45:28.313407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243145+0000","last_change":"2023-07-21T22:27:42.090442+0000","last_active":"2023-07-21T22:28:47.243145+0000","last_peered":"2023-07-21T22:28:47.243145+0000","last_clean":"2023-07-21T22:28:47.243145+0000","last_became_active":"2023-07-21T22:27:42.090103+0000","last_became_peered":"2023-07-21T22:27:42.090103+0000","last_unstale":"2023-07-21T22:28:47.243145+0000","last_undegraded":"2023-07-21T22:28:47.243145+0000","last_fullsized":"2023-07-21T22:28:47.243145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:27:20.825119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245155+0000","last_change":"2023-07-21T22:27:42.098364+0000","last_active":"2023-07-21T22:28:47.245155+0000","last_peered":"2023-07-21T22:28:47.245155+0000","last_clean":"2023-07-21T22:28:47.245155+0000","last_became_active":"2023-07-21T22:27:42.097378+0000","last_became_peered":"2023-07-21T22:27:42.097378+0000","last_unstale":"2023-07-21T22:28:47.245155+0000","last_undegraded":"2023-07-21T22:28:47.245155+0000","last_fullsized":"2023-07-21T22:28:47.245155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:40:58.595476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246312+0000","last_change":"2023-07-21T22:27:42.103110+0000","last_active":"2023-07-21T22:28:47.246312+0000","last_peered":"2023-07-21T22:28:47.246312+0000","last_clean":"2023-07-21T22:28:47.246312+0000","last_became_active":"2023-07-21T22:27:42.102790+0000","last_became_peered":"2023-07-21T22:27:42.102790+0000","last_unstale":"2023-07-21T22:28:47.246312+0000","last_undegraded":"2023-07-21T22:28:47.246312+0000","last_fullsized":"2023-07-21T22:28:47.246312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:20:23.914945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244930+0000","last_change":"2023-07-21T22:28:47.244930+0000","last_active":"2023-07-21T22:28:47.244930+0000","last_peered":"2023-07-21T22:28:47.244930+0000","last_clean":"2023-07-21T22:28:47.244930+0000","last_became_active":"2023-07-21T22:28:47.244613+0000","last_became_peered":"2023-07-21T22:28:47.244613+0000","last_unstale":"2023-07-21T22:28:47.244930+0000","last_undegraded":"2023-07-21T22:28:47.244930+0000","last_fullsized":"2023-07-21T22:28:47.244930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:43:03.707499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382008+0000","last_change":"2023-07-21T22:27:42.084423+0000","last_active":"2023-07-21T22:28:47.382008+0000","last_peered":"2023-07-21T22:28:47.382008+0000","last_clean":"2023-07-21T22:28:47.382008+0000","last_became_active":"2023-07-21T22:27:42.084074+0000","last_became_peered":"2023-07-21T22:27:42.084074+0000","last_unstale":"2023-07-21T22:28:47.382008+0000","last_undegraded":"2023-07-21T22:28:47.382008+0000","last_fullsized":"2023-07-21T22:28:47.382008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:48.499214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.381932+0000","last_change":"2023-07-21T22:27:42.091542+0000","last_active":"2023-07-21T22:28:47.381932+0000","last_peered":"2023-07-21T22:28:47.381932+0000","last_clean":"2023-07-21T22:28:47.381932+0000","last_became_active":"2023-07-21T22:27:42.091217+0000","last_became_peered":"2023-07-21T22:27:42.091217+0000","last_unstale":"2023-07-21T22:28:47.381932+0000","last_undegraded":"2023-07-21T22:28:47.381932+0000","last_fullsized":"2023-07-21T22:28:47.381932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:31:21.491062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382557+0000","last_change":"2023-07-21T22:27:42.087505+0000","last_active":"2023-07-21T22:28:47.382557+0000","last_peered":"2023-07-21T22:28:47.382557+0000","last_clean":"2023-07-21T22:28:47.382557+0000","last_became_active":"2023-07-21T22:27:42.087177+0000","last_became_peered":"2023-07-21T22:27:42.087177+0000","last_unstale":"2023-07-21T22:28:47.382557+0000","last_undegraded":"2023-07-21T22:28:47.382557+0000","last_fullsized":"2023-07-21T22:28:47.382557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:06:51.361107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243856+0000","last_change":"2023-07-21T22:27:42.091490+0000","last_active":"2023-07-21T22:28:47.243856+0000","last_peered":"2023-07-21T22:28:47.243856+0000","last_clean":"2023-07-21T22:28:47.243856+0000","last_became_active":"2023-07-21T22:27:42.090782+0000","last_became_peered":"2023-07-21T22:27:42.090782+0000","last_unstale":"2023-07-21T22:28:47.243856+0000","last_undegraded":"2023-07-21T22:28:47.243856+0000","last_fullsized":"2023-07-21T22:28:47.243856+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:34:52.980195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243991+0000","last_change":"2023-07-21T22:27:42.092362+0000","last_active":"2023-07-21T22:28:47.243991+0000","last_peered":"2023-07-21T22:28:47.243991+0000","last_clean":"2023-07-21T22:28:47.243991+0000","last_became_active":"2023-07-21T22:27:42.091915+0000","last_became_peered":"2023-07-21T22:27:42.091915+0000","last_unstale":"2023-07-21T22:28:47.243991+0000","last_undegraded":"2023-07-21T22:28:47.243991+0000","last_fullsized":"2023-07-21T22:28:47.243991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:06:24.497745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242584+0000","last_change":"2023-07-21T22:27:42.098959+0000","last_active":"2023-07-21T22:28:47.242584+0000","last_peered":"2023-07-21T22:28:47.242584+0000","last_clean":"2023-07-21T22:28:47.242584+0000","last_became_active":"2023-07-21T22:27:42.094463+0000","last_became_peered":"2023-07-21T22:27:42.094463+0000","last_unstale":"2023-07-21T22:28:47.242584+0000","last_undegraded":"2023-07-21T22:28:47.242584+0000","last_fullsized":"2023-07-21T22:28:47.242584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:17:23.627490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":108,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:03.141677+0000","last_change":"2023-07-21T22:27:41.074299+0000","last_active":"2023-07-21T22:29:03.141677+0000","last_peered":"2023-07-21T22:29:03.141677+0000","last_clean":"2023-07-21T22:29:03.141677+0000","last_became_active":"2023-07-21T22:27:41.072895+0000","last_became_peered":"2023-07-21T22:27:41.072895+0000","last_unstale":"2023-07-21T22:29:03.141677+0000","last_undegraded":"2023-07-21T22:29:03.141677+0000","last_fullsized":"2023-07-21T22:29:03.141677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:24:58.146931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241511+0000","last_change":"2023-07-21T22:27:42.090477+0000","last_active":"2023-07-21T22:28:47.241511+0000","last_peered":"2023-07-21T22:28:47.241511+0000","last_clean":"2023-07-21T22:28:47.241511+0000","last_became_active":"2023-07-21T22:27:42.090144+0000","last_became_peered":"2023-07-21T22:27:42.090144+0000","last_unstale":"2023-07-21T22:28:47.241511+0000","last_undegraded":"2023-07-21T22:28:47.241511+0000","last_fullsized":"2023-07-21T22:28:47.241511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:42:38.964084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241632+0000","last_change":"2023-07-21T22:27:41.075178+0000","last_active":"2023-07-21T22:28:47.241632+0000","last_peered":"2023-07-21T22:28:47.241632+0000","last_clean":"2023-07-21T22:28:47.241632+0000","last_became_active":"2023-07-21T22:27:41.074861+0000","last_became_peered":"2023-07-21T22:27:41.074861+0000","last_unstale":"2023-07-21T22:28:47.241632+0000","last_undegraded":"2023-07-21T22:28:47.241632+0000","last_fullsized":"2023-07-21T22:28:47.241632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:09:39.913585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382382+0000","last_change":"2023-07-21T22:27:42.093135+0000","last_active":"2023-07-21T22:28:47.382382+0000","last_peered":"2023-07-21T22:28:47.382382+0000","last_clean":"2023-07-21T22:28:47.382382+0000","last_became_active":"2023-07-21T22:27:42.092457+0000","last_became_peered":"2023-07-21T22:27:42.092457+0000","last_unstale":"2023-07-21T22:28:47.382382+0000","last_undegraded":"2023-07-21T22:28:47.382382+0000","last_fullsized":"2023-07-21T22:28:47.382382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:37.129524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.249069+0000","last_change":"2023-07-21T22:27:41.076759+0000","last_active":"2023-07-21T22:28:47.249069+0000","last_peered":"2023-07-21T22:28:47.249069+0000","last_clean":"2023-07-21T22:28:47.249069+0000","last_became_active":"2023-07-21T22:27:41.076376+0000","last_became_peered":"2023-07-21T22:27:41.076376+0000","last_unstale":"2023-07-21T22:28:47.249069+0000","last_undegraded":"2023-07-21T22:28:47.249069+0000","last_fullsized":"2023-07-21T22:28:47.249069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:10:03.951373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382877+0000","last_change":"2023-07-21T22:27:42.093643+0000","last_active":"2023-07-21T22:28:47.382877+0000","last_peered":"2023-07-21T22:28:47.382877+0000","last_clean":"2023-07-21T22:28:47.382877+0000","last_became_active":"2023-07-21T22:27:42.093227+0000","last_became_peered":"2023-07-21T22:27:42.093227+0000","last_unstale":"2023-07-21T22:28:47.382877+0000","last_undegraded":"2023-07-21T22:28:47.382877+0000","last_fullsized":"2023-07-21T22:28:47.382877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:07:08.243533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.248547+0000","last_change":"2023-07-21T22:27:41.073119+0000","last_active":"2023-07-21T22:28:47.248547+0000","last_peered":"2023-07-21T22:28:47.248547+0000","last_clean":"2023-07-21T22:28:47.248547+0000","last_became_active":"2023-07-21T22:27:41.072813+0000","last_became_peered":"2023-07-21T22:27:41.072813+0000","last_unstale":"2023-07-21T22:28:47.248547+0000","last_undegraded":"2023-07-21T22:28:47.248547+0000","last_fullsized":"2023-07-21T22:28:47.248547+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:02.412440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383479+0000","last_change":"2023-07-21T22:28:47.383479+0000","last_active":"2023-07-21T22:28:47.383479+0000","last_peered":"2023-07-21T22:28:47.383479+0000","last_clean":"2023-07-21T22:28:47.383479+0000","last_became_active":"2023-07-21T22:28:47.383074+0000","last_became_peered":"2023-07-21T22:28:47.383074+0000","last_unstale":"2023-07-21T22:28:47.383479+0000","last_undegraded":"2023-07-21T22:28:47.383479+0000","last_fullsized":"2023-07-21T22:28:47.383479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:01:40.184725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":118,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:03.407481+0000","last_change":"2023-07-21T22:28:47.388489+0000","last_active":"2023-07-21T22:29:03.407481+0000","last_peered":"2023-07-21T22:29:03.407481+0000","last_clean":"2023-07-21T22:29:03.407481+0000","last_became_active":"2023-07-21T22:28:47.383903+0000","last_became_peered":"2023-07-21T22:28:47.383903+0000","last_unstale":"2023-07-21T22:29:03.407481+0000","last_undegraded":"2023-07-21T22:29:03.407481+0000","last_fullsized":"2023-07-21T22:29:03.407481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:25:48.113759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246961+0000","last_change":"2023-07-21T22:27:42.082970+0000","last_active":"2023-07-21T22:28:47.246961+0000","last_peered":"2023-07-21T22:28:47.246961+0000","last_clean":"2023-07-21T22:28:47.246961+0000","last_became_active":"2023-07-21T22:27:42.082676+0000","last_became_peered":"2023-07-21T22:27:42.082676+0000","last_unstale":"2023-07-21T22:28:47.246961+0000","last_undegraded":"2023-07-21T22:28:47.246961+0000","last_fullsized":"2023-07-21T22:28:47.246961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:47:24.831459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242691+0000","last_change":"2023-07-21T22:27:41.076844+0000","last_active":"2023-07-21T22:28:47.242691+0000","last_peered":"2023-07-21T22:28:47.242691+0000","last_clean":"2023-07-21T22:28:47.242691+0000","last_became_active":"2023-07-21T22:27:41.076487+0000","last_became_peered":"2023-07-21T22:27:41.076487+0000","last_unstale":"2023-07-21T22:28:47.242691+0000","last_undegraded":"2023-07-21T22:28:47.242691+0000","last_fullsized":"2023-07-21T22:28:47.242691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:03:37.023974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240958+0000","last_change":"2023-07-21T22:27:42.089996+0000","last_active":"2023-07-21T22:28:47.240958+0000","last_peered":"2023-07-21T22:28:47.240958+0000","last_clean":"2023-07-21T22:28:47.240958+0000","last_became_active":"2023-07-21T22:27:42.089668+0000","last_became_peered":"2023-07-21T22:27:42.089668+0000","last_unstale":"2023-07-21T22:28:47.240958+0000","last_undegraded":"2023-07-21T22:28:47.240958+0000","last_fullsized":"2023-07-21T22:28:47.240958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:55:01.151479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239183+0000","last_change":"2023-07-21T22:27:41.075864+0000","last_active":"2023-07-21T22:28:47.239183+0000","last_peered":"2023-07-21T22:28:47.239183+0000","last_clean":"2023-07-21T22:28:47.239183+0000","last_became_active":"2023-07-21T22:27:41.075577+0000","last_became_peered":"2023-07-21T22:27:41.075577+0000","last_unstale":"2023-07-21T22:28:47.239183+0000","last_undegraded":"2023-07-21T22:28:47.239183+0000","last_fullsized":"2023-07-21T22:28:47.239183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:41:53.383706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241389+0000","last_change":"2023-07-21T22:27:42.091730+0000","last_active":"2023-07-21T22:28:47.241389+0000","last_peered":"2023-07-21T22:28:47.241389+0000","last_clean":"2023-07-21T22:28:47.241389+0000","last_became_active":"2023-07-21T22:27:42.091394+0000","last_became_peered":"2023-07-21T22:27:42.091394+0000","last_unstale":"2023-07-21T22:28:47.241389+0000","last_undegraded":"2023-07-21T22:28:47.241389+0000","last_fullsized":"2023-07-21T22:28:47.241389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:21:14.595368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240215+0000","last_change":"2023-07-21T22:27:41.074822+0000","last_active":"2023-07-21T22:28:47.240215+0000","last_peered":"2023-07-21T22:28:47.240215+0000","last_clean":"2023-07-21T22:28:47.240215+0000","last_became_active":"2023-07-21T22:27:41.074542+0000","last_became_peered":"2023-07-21T22:27:41.074542+0000","last_unstale":"2023-07-21T22:28:47.240215+0000","last_undegraded":"2023-07-21T22:28:47.240215+0000","last_fullsized":"2023-07-21T22:28:47.240215+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:56:37.030275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243691+0000","last_change":"2023-07-21T22:27:42.094351+0000","last_active":"2023-07-21T22:28:47.243691+0000","last_peered":"2023-07-21T22:28:47.243691+0000","last_clean":"2023-07-21T22:28:47.243691+0000","last_became_active":"2023-07-21T22:27:42.094068+0000","last_became_peered":"2023-07-21T22:27:42.094068+0000","last_unstale":"2023-07-21T22:28:47.243691+0000","last_undegraded":"2023-07-21T22:28:47.243691+0000","last_fullsized":"2023-07-21T22:28:47.243691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:55:43.799214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":170,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:00.402812+0000","last_change":"2023-07-21T22:27:41.079188+0000","last_active":"2023-07-21T22:29:00.402812+0000","last_peered":"2023-07-21T22:29:00.402812+0000","last_clean":"2023-07-21T22:29:00.402812+0000","last_became_active":"2023-07-21T22:27:41.078899+0000","last_became_peered":"2023-07-21T22:27:41.078899+0000","last_unstale":"2023-07-21T22:29:00.402812+0000","last_undegraded":"2023-07-21T22:29:00.402812+0000","last_fullsized":"2023-07-21T22:29:00.402812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:04:07.942421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383198+0000","last_change":"2023-07-21T22:27:42.091194+0000","last_active":"2023-07-21T22:28:47.383198+0000","last_peered":"2023-07-21T22:28:47.383198+0000","last_clean":"2023-07-21T22:28:47.383198+0000","last_became_active":"2023-07-21T22:27:42.090869+0000","last_became_peered":"2023-07-21T22:27:42.090869+0000","last_unstale":"2023-07-21T22:28:47.383198+0000","last_undegraded":"2023-07-21T22:28:47.383198+0000","last_fullsized":"2023-07-21T22:28:47.383198+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:46:43.577329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245092+0000","last_change":"2023-07-21T22:27:41.072701+0000","last_active":"2023-07-21T22:28:47.245092+0000","last_peered":"2023-07-21T22:28:47.245092+0000","last_clean":"2023-07-21T22:28:47.245092+0000","last_became_active":"2023-07-21T22:27:41.072358+0000","last_became_peered":"2023-07-21T22:27:41.072358+0000","last_unstale":"2023-07-21T22:28:47.245092+0000","last_undegraded":"2023-07-21T22:28:47.245092+0000","last_fullsized":"2023-07-21T22:28:47.245092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:11:20.209729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382347+0000","last_change":"2023-07-21T22:27:42.090808+0000","last_active":"2023-07-21T22:28:47.382347+0000","last_peered":"2023-07-21T22:28:47.382347+0000","last_clean":"2023-07-21T22:28:47.382347+0000","last_became_active":"2023-07-21T22:27:42.090506+0000","last_became_peered":"2023-07-21T22:27:42.090506+0000","last_unstale":"2023-07-21T22:28:47.382347+0000","last_undegraded":"2023-07-21T22:28:47.382347+0000","last_fullsized":"2023-07-21T22:28:47.382347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:15.825961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240934+0000","last_change":"2023-07-21T22:27:41.075643+0000","last_active":"2023-07-21T22:28:47.240934+0000","last_peered":"2023-07-21T22:28:47.240934+0000","last_clean":"2023-07-21T22:28:47.240934+0000","last_became_active":"2023-07-21T22:27:41.075336+0000","last_became_peered":"2023-07-21T22:27:41.075336+0000","last_unstale":"2023-07-21T22:28:47.240934+0000","last_undegraded":"2023-07-21T22:28:47.240934+0000","last_fullsized":"2023-07-21T22:28:47.240934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:25:15.271905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383144+0000","last_change":"2023-07-21T22:27:42.096722+0000","last_active":"2023-07-21T22:28:47.383144+0000","last_peered":"2023-07-21T22:28:47.383144+0000","last_clean":"2023-07-21T22:28:47.383144+0000","last_became_active":"2023-07-21T22:27:42.096472+0000","last_became_peered":"2023-07-21T22:27:42.096472+0000","last_unstale":"2023-07-21T22:28:47.383144+0000","last_undegraded":"2023-07-21T22:28:47.383144+0000","last_fullsized":"2023-07-21T22:28:47.383144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:27:25.223020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'28","reported_seq":202,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:00.402637+0000","last_change":"2023-07-21T22:27:41.072297+0000","last_active":"2023-07-21T22:29:00.402637+0000","last_peered":"2023-07-21T22:29:00.402637+0000","last_clean":"2023-07-21T22:29:00.402637+0000","last_became_active":"2023-07-21T22:27:41.071971+0000","last_became_peered":"2023-07-21T22:27:41.071971+0000","last_unstale":"2023-07-21T22:29:00.402637+0000","last_undegraded":"2023-07-21T22:29:00.402637+0000","last_fullsized":"2023-07-21T22:29:00.402637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:28:20.855102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54660,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":156,"num_read_kb":131,"num_write":30,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.248127+0000","last_change":"2023-07-21T22:27:42.097130+0000","last_active":"2023-07-21T22:28:47.248127+0000","last_peered":"2023-07-21T22:28:47.248127+0000","last_clean":"2023-07-21T22:28:47.248127+0000","last_became_active":"2023-07-21T22:27:42.096853+0000","last_became_peered":"2023-07-21T22:27:42.096853+0000","last_unstale":"2023-07-21T22:28:47.248127+0000","last_undegraded":"2023-07-21T22:28:47.248127+0000","last_fullsized":"2023-07-21T22:28:47.248127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:42:15.440580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242791+0000","last_change":"2023-07-21T22:27:41.071755+0000","last_active":"2023-07-21T22:28:47.242791+0000","last_peered":"2023-07-21T22:28:47.242791+0000","last_clean":"2023-07-21T22:28:47.242791+0000","last_became_active":"2023-07-21T22:27:41.071245+0000","last_became_peered":"2023-07-21T22:27:41.071245+0000","last_unstale":"2023-07-21T22:28:47.242791+0000","last_undegraded":"2023-07-21T22:28:47.242791+0000","last_fullsized":"2023-07-21T22:28:47.242791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:17:33.183524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239177+0000","last_change":"2023-07-21T22:27:42.097433+0000","last_active":"2023-07-21T22:28:47.239177+0000","last_peered":"2023-07-21T22:28:47.239177+0000","last_clean":"2023-07-21T22:28:47.239177+0000","last_became_active":"2023-07-21T22:27:42.097100+0000","last_became_peered":"2023-07-21T22:27:42.097100+0000","last_unstale":"2023-07-21T22:28:47.239177+0000","last_undegraded":"2023-07-21T22:28:47.239177+0000","last_fullsized":"2023-07-21T22:28:47.239177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:11:09.018578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240863+0000","last_change":"2023-07-21T22:27:41.074019+0000","last_active":"2023-07-21T22:28:47.240863+0000","last_peered":"2023-07-21T22:28:47.240863+0000","last_clean":"2023-07-21T22:28:47.240863+0000","last_became_active":"2023-07-21T22:27:41.073699+0000","last_became_peered":"2023-07-21T22:27:41.073699+0000","last_unstale":"2023-07-21T22:28:47.240863+0000","last_undegraded":"2023-07-21T22:28:47.240863+0000","last_fullsized":"2023-07-21T22:28:47.240863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:16:39.616177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245536+0000","last_change":"2023-07-21T22:27:42.098010+0000","last_active":"2023-07-21T22:28:47.245536+0000","last_peered":"2023-07-21T22:28:47.245536+0000","last_clean":"2023-07-21T22:28:47.245536+0000","last_became_active":"2023-07-21T22:27:42.097227+0000","last_became_peered":"2023-07-21T22:27:42.097227+0000","last_unstale":"2023-07-21T22:28:47.245536+0000","last_undegraded":"2023-07-21T22:28:47.245536+0000","last_fullsized":"2023-07-21T22:28:47.245536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:29:36.120927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":103,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:04.163532+0000","last_change":"2023-07-21T22:27:41.072298+0000","last_active":"2023-07-21T22:29:04.163532+0000","last_peered":"2023-07-21T22:29:04.163532+0000","last_clean":"2023-07-21T22:29:04.163532+0000","last_became_active":"2023-07-21T22:27:41.071963+0000","last_became_peered":"2023-07-21T22:27:41.071963+0000","last_unstale":"2023-07-21T22:29:04.163532+0000","last_undegraded":"2023-07-21T22:29:04.163532+0000","last_fullsized":"2023-07-21T22:29:04.163532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:14:19.498409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":858,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244894+0000","last_change":"2023-07-21T22:27:42.091276+0000","last_active":"2023-07-21T22:28:47.244894+0000","last_peered":"2023-07-21T22:28:47.244894+0000","last_clean":"2023-07-21T22:28:47.244894+0000","last_became_active":"2023-07-21T22:27:42.090918+0000","last_became_peered":"2023-07-21T22:27:42.090918+0000","last_unstale":"2023-07-21T22:28:47.244894+0000","last_undegraded":"2023-07-21T22:28:47.244894+0000","last_fullsized":"2023-07-21T22:28:47.244894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:15.527348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.246226+0000","last_change":"2023-07-21T22:27:41.075633+0000","last_active":"2023-07-21T22:28:47.246226+0000","last_peered":"2023-07-21T22:28:47.246226+0000","last_clean":"2023-07-21T22:28:47.246226+0000","last_became_active":"2023-07-21T22:27:41.075331+0000","last_became_peered":"2023-07-21T22:27:41.075331+0000","last_unstale":"2023-07-21T22:28:47.246226+0000","last_undegraded":"2023-07-21T22:28:47.246226+0000","last_fullsized":"2023-07-21T22:28:47.246226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:26:08.447672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.383325+0000","last_change":"2023-07-21T22:28:47.383325+0000","last_active":"2023-07-21T22:28:47.383325+0000","last_peered":"2023-07-21T22:28:47.383325+0000","last_clean":"2023-07-21T22:28:47.383325+0000","last_became_active":"2023-07-21T22:28:47.382987+0000","last_became_peered":"2023-07-21T22:28:47.382987+0000","last_unstale":"2023-07-21T22:28:47.383325+0000","last_undegraded":"2023-07-21T22:28:47.383325+0000","last_fullsized":"2023-07-21T22:28:47.383325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:51:52.190795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.255975+0000","last_change":"2023-07-21T22:28:47.255975+0000","last_active":"2023-07-21T22:28:47.255975+0000","last_peered":"2023-07-21T22:28:47.255975+0000","last_clean":"2023-07-21T22:28:47.255975+0000","last_became_active":"2023-07-21T22:28:47.248730+0000","last_became_peered":"2023-07-21T22:28:47.248730+0000","last_unstale":"2023-07-21T22:28:47.255975+0000","last_undegraded":"2023-07-21T22:28:47.255975+0000","last_fullsized":"2023-07-21T22:28:47.255975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:25:29.022962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243276+0000","last_change":"2023-07-21T22:27:42.092046+0000","last_active":"2023-07-21T22:28:47.243276+0000","last_peered":"2023-07-21T22:28:47.243276+0000","last_clean":"2023-07-21T22:28:47.243276+0000","last_became_active":"2023-07-21T22:27:42.091069+0000","last_became_peered":"2023-07-21T22:27:42.091069+0000","last_unstale":"2023-07-21T22:28:47.243276+0000","last_undegraded":"2023-07-21T22:28:47.243276+0000","last_fullsized":"2023-07-21T22:28:47.243276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:07:37.855640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243883+0000","last_change":"2023-07-21T22:27:41.074847+0000","last_active":"2023-07-21T22:28:47.243883+0000","last_peered":"2023-07-21T22:28:47.243883+0000","last_clean":"2023-07-21T22:28:47.243883+0000","last_became_active":"2023-07-21T22:27:41.074530+0000","last_became_peered":"2023-07-21T22:27:41.074530+0000","last_unstale":"2023-07-21T22:28:47.243883+0000","last_undegraded":"2023-07-21T22:28:47.243883+0000","last_fullsized":"2023-07-21T22:28:47.243883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:28:53.033226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242814+0000","last_change":"2023-07-21T22:27:42.089285+0000","last_active":"2023-07-21T22:28:47.242814+0000","last_peered":"2023-07-21T22:28:47.242814+0000","last_clean":"2023-07-21T22:28:47.242814+0000","last_became_active":"2023-07-21T22:27:42.088970+0000","last_became_peered":"2023-07-21T22:27:42.088970+0000","last_unstale":"2023-07-21T22:28:47.242814+0000","last_undegraded":"2023-07-21T22:28:47.242814+0000","last_fullsized":"2023-07-21T22:28:47.242814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:53:26.517730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:59.137194+0000","last_change":"2023-07-21T22:27:41.077433+0000","last_active":"2023-07-21T22:28:59.137194+0000","last_peered":"2023-07-21T22:28:59.137194+0000","last_clean":"2023-07-21T22:28:59.137194+0000","last_became_active":"2023-07-21T22:27:41.077133+0000","last_became_peered":"2023-07-21T22:27:41.077133+0000","last_unstale":"2023-07-21T22:28:59.137194+0000","last_undegraded":"2023-07-21T22:28:59.137194+0000","last_fullsized":"2023-07-21T22:28:59.137194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:09.473094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243171+0000","last_change":"2023-07-21T22:27:42.089833+0000","last_active":"2023-07-21T22:28:47.243171+0000","last_peered":"2023-07-21T22:28:47.243171+0000","last_clean":"2023-07-21T22:28:47.243171+0000","last_became_active":"2023-07-21T22:27:42.089502+0000","last_became_peered":"2023-07-21T22:27:42.089502+0000","last_unstale":"2023-07-21T22:28:47.243171+0000","last_undegraded":"2023-07-21T22:28:47.243171+0000","last_fullsized":"2023-07-21T22:28:47.243171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:33:04.121577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.248492+0000","last_change":"2023-07-21T22:28:47.248117+0000","last_active":"2023-07-21T22:28:47.248492+0000","last_peered":"2023-07-21T22:28:47.248492+0000","last_clean":"2023-07-21T22:28:47.248492+0000","last_became_active":"2023-07-21T22:28:47.247668+0000","last_became_peered":"2023-07-21T22:28:47.247668+0000","last_unstale":"2023-07-21T22:28:47.248492+0000","last_undegraded":"2023-07-21T22:28:47.248492+0000","last_fullsized":"2023-07-21T22:28:47.248492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:57:06.878014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:59.155853+0000","last_change":"2023-07-21T22:27:41.076784+0000","last_active":"2023-07-21T22:28:59.155853+0000","last_peered":"2023-07-21T22:28:59.155853+0000","last_clean":"2023-07-21T22:28:59.155853+0000","last_became_active":"2023-07-21T22:27:41.076453+0000","last_became_peered":"2023-07-21T22:27:41.076453+0000","last_unstale":"2023-07-21T22:28:59.155853+0000","last_undegraded":"2023-07-21T22:28:59.155853+0000","last_fullsized":"2023-07-21T22:28:59.155853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:32:01.149003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242778+0000","last_change":"2023-07-21T22:27:42.091286+0000","last_active":"2023-07-21T22:28:47.242778+0000","last_peered":"2023-07-21T22:28:47.242778+0000","last_clean":"2023-07-21T22:28:47.242778+0000","last_became_active":"2023-07-21T22:27:42.090986+0000","last_became_peered":"2023-07-21T22:27:42.090986+0000","last_unstale":"2023-07-21T22:28:47.242778+0000","last_undegraded":"2023-07-21T22:28:47.242778+0000","last_fullsized":"2023-07-21T22:28:47.242778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:19.449945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245874+0000","last_change":"2023-07-21T22:27:41.074075+0000","last_active":"2023-07-21T22:28:47.245874+0000","last_peered":"2023-07-21T22:28:47.245874+0000","last_clean":"2023-07-21T22:28:47.245874+0000","last_became_active":"2023-07-21T22:27:41.073757+0000","last_became_peered":"2023-07-21T22:27:41.073757+0000","last_unstale":"2023-07-21T22:28:47.245874+0000","last_undegraded":"2023-07-21T22:28:47.245874+0000","last_fullsized":"2023-07-21T22:28:47.245874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:51:53.883172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245145+0000","last_change":"2023-07-21T22:26:43.405788+0000","last_active":"2023-07-21T22:28:47.245145+0000","last_peered":"2023-07-21T22:28:47.245145+0000","last_clean":"2023-07-21T22:28:47.245145+0000","last_became_active":"2023-07-21T22:26:43.405351+0000","last_became_peered":"2023-07-21T22:26:43.405351+0000","last_unstale":"2023-07-21T22:28:47.245145+0000","last_undegraded":"2023-07-21T22:28:47.245145+0000","last_fullsized":"2023-07-21T22:28:47.245145+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:23:52.032210+0000","last_clean_scrub_stamp":"2023-07-21T22:23:52.032210+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:20:43.690726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239038+0000","last_change":"2023-07-21T22:27:42.097967+0000","last_active":"2023-07-21T22:28:47.239038+0000","last_peered":"2023-07-21T22:28:47.239038+0000","last_clean":"2023-07-21T22:28:47.239038+0000","last_became_active":"2023-07-21T22:27:42.097708+0000","last_became_peered":"2023-07-21T22:27:42.097708+0000","last_unstale":"2023-07-21T22:28:47.239038+0000","last_undegraded":"2023-07-21T22:28:47.239038+0000","last_fullsized":"2023-07-21T22:28:47.239038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:47:00.150179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245422+0000","last_change":"2023-07-21T22:27:41.077853+0000","last_active":"2023-07-21T22:28:47.245422+0000","last_peered":"2023-07-21T22:28:47.245422+0000","last_clean":"2023-07-21T22:28:47.245422+0000","last_became_active":"2023-07-21T22:27:41.077539+0000","last_became_peered":"2023-07-21T22:27:41.077539+0000","last_unstale":"2023-07-21T22:28:47.245422+0000","last_undegraded":"2023-07-21T22:28:47.245422+0000","last_fullsized":"2023-07-21T22:28:47.245422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:11:54.875605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.248834+0000","last_change":"2023-07-21T22:27:41.076529+0000","last_active":"2023-07-21T22:28:47.248834+0000","last_peered":"2023-07-21T22:28:47.248834+0000","last_clean":"2023-07-21T22:28:47.248834+0000","last_became_active":"2023-07-21T22:27:41.076208+0000","last_became_peered":"2023-07-21T22:27:41.076208+0000","last_unstale":"2023-07-21T22:28:47.248834+0000","last_undegraded":"2023-07-21T22:28:47.248834+0000","last_fullsized":"2023-07-21T22:28:47.248834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:31:37.704902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239276+0000","last_change":"2023-07-21T22:27:42.082587+0000","last_active":"2023-07-21T22:28:47.239276+0000","last_peered":"2023-07-21T22:28:47.239276+0000","last_clean":"2023-07-21T22:28:47.239276+0000","last_became_active":"2023-07-21T22:27:42.082212+0000","last_became_peered":"2023-07-21T22:27:42.082212+0000","last_unstale":"2023-07-21T22:28:47.239276+0000","last_undegraded":"2023-07-21T22:28:47.239276+0000","last_fullsized":"2023-07-21T22:28:47.239276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:12:19.542426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244140+0000","last_change":"2023-07-21T22:27:41.073736+0000","last_active":"2023-07-21T22:28:47.244140+0000","last_peered":"2023-07-21T22:28:47.244140+0000","last_clean":"2023-07-21T22:28:47.244140+0000","last_became_active":"2023-07-21T22:27:41.073213+0000","last_became_peered":"2023-07-21T22:27:41.073213+0000","last_unstale":"2023-07-21T22:28:47.244140+0000","last_undegraded":"2023-07-21T22:28:47.244140+0000","last_fullsized":"2023-07-21T22:28:47.244140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:44:51.762687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242235+0000","last_change":"2023-07-21T22:27:42.091313+0000","last_active":"2023-07-21T22:28:47.242235+0000","last_peered":"2023-07-21T22:28:47.242235+0000","last_clean":"2023-07-21T22:28:47.242235+0000","last_became_active":"2023-07-21T22:27:42.091012+0000","last_became_peered":"2023-07-21T22:27:42.091012+0000","last_unstale":"2023-07-21T22:28:47.242235+0000","last_undegraded":"2023-07-21T22:28:47.242235+0000","last_fullsized":"2023-07-21T22:28:47.242235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:53:00.317577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245842+0000","last_change":"2023-07-21T22:27:41.074644+0000","last_active":"2023-07-21T22:28:47.245842+0000","last_peered":"2023-07-21T22:28:47.245842+0000","last_clean":"2023-07-21T22:28:47.245842+0000","last_became_active":"2023-07-21T22:27:41.074364+0000","last_became_peered":"2023-07-21T22:27:41.074364+0000","last_unstale":"2023-07-21T22:28:47.245842+0000","last_undegraded":"2023-07-21T22:28:47.245842+0000","last_fullsized":"2023-07-21T22:28:47.245842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:23:08.316538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382240+0000","last_change":"2023-07-21T22:27:42.089452+0000","last_active":"2023-07-21T22:28:47.382240+0000","last_peered":"2023-07-21T22:28:47.382240+0000","last_clean":"2023-07-21T22:28:47.382240+0000","last_became_active":"2023-07-21T22:27:42.088859+0000","last_became_peered":"2023-07-21T22:27:42.088859+0000","last_unstale":"2023-07-21T22:28:47.382240+0000","last_undegraded":"2023-07-21T22:28:47.382240+0000","last_fullsized":"2023-07-21T22:28:47.382240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:38:51.287342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382549+0000","last_change":"2023-07-21T22:27:41.077067+0000","last_active":"2023-07-21T22:28:47.382549+0000","last_peered":"2023-07-21T22:28:47.382549+0000","last_clean":"2023-07-21T22:28:47.382549+0000","last_became_active":"2023-07-21T22:27:41.076768+0000","last_became_peered":"2023-07-21T22:27:41.076768+0000","last_unstale":"2023-07-21T22:28:47.382549+0000","last_undegraded":"2023-07-21T22:28:47.382549+0000","last_fullsized":"2023-07-21T22:28:47.382549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:07:35.434120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.247883+0000","last_change":"2023-07-21T22:27:42.088789+0000","last_active":"2023-07-21T22:28:47.247883+0000","last_peered":"2023-07-21T22:28:47.247883+0000","last_clean":"2023-07-21T22:28:47.247883+0000","last_became_active":"2023-07-21T22:27:42.088361+0000","last_became_peered":"2023-07-21T22:27:42.088361+0000","last_unstale":"2023-07-21T22:28:47.247883+0000","last_undegraded":"2023-07-21T22:28:47.247883+0000","last_fullsized":"2023-07-21T22:28:47.247883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:51:10.850405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241552+0000","last_change":"2023-07-21T22:27:41.069595+0000","last_active":"2023-07-21T22:28:47.241552+0000","last_peered":"2023-07-21T22:28:47.241552+0000","last_clean":"2023-07-21T22:28:47.241552+0000","last_became_active":"2023-07-21T22:27:41.068733+0000","last_became_peered":"2023-07-21T22:27:41.068733+0000","last_unstale":"2023-07-21T22:28:47.241552+0000","last_undegraded":"2023-07-21T22:28:47.241552+0000","last_fullsized":"2023-07-21T22:28:47.241552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:49.910593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243049+0000","last_change":"2023-07-21T22:27:42.085701+0000","last_active":"2023-07-21T22:28:47.243049+0000","last_peered":"2023-07-21T22:28:47.243049+0000","last_clean":"2023-07-21T22:28:47.243049+0000","last_became_active":"2023-07-21T22:27:42.085429+0000","last_became_peered":"2023-07-21T22:27:42.085429+0000","last_unstale":"2023-07-21T22:28:47.243049+0000","last_undegraded":"2023-07-21T22:28:47.243049+0000","last_fullsized":"2023-07-21T22:28:47.243049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:34:44.915735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243657+0000","last_change":"2023-07-21T22:27:41.073233+0000","last_active":"2023-07-21T22:28:47.243657+0000","last_peered":"2023-07-21T22:28:47.243657+0000","last_clean":"2023-07-21T22:28:47.243657+0000","last_became_active":"2023-07-21T22:27:41.072866+0000","last_became_peered":"2023-07-21T22:27:41.072866+0000","last_unstale":"2023-07-21T22:28:47.243657+0000","last_undegraded":"2023-07-21T22:28:47.243657+0000","last_fullsized":"2023-07-21T22:28:47.243657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:34:44.838691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241957+0000","last_change":"2023-07-21T22:27:42.100093+0000","last_active":"2023-07-21T22:28:47.241957+0000","last_peered":"2023-07-21T22:28:47.241957+0000","last_clean":"2023-07-21T22:28:47.241957+0000","last_became_active":"2023-07-21T22:27:42.099813+0000","last_became_peered":"2023-07-21T22:27:42.099813+0000","last_unstale":"2023-07-21T22:28:47.241957+0000","last_undegraded":"2023-07-21T22:28:47.241957+0000","last_fullsized":"2023-07-21T22:28:47.241957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:52:06.785113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.259961+0000","last_change":"2023-07-21T22:28:47.259961+0000","last_active":"2023-07-21T22:28:47.259961+0000","last_peered":"2023-07-21T22:28:47.259961+0000","last_clean":"2023-07-21T22:28:47.259961+0000","last_became_active":"2023-07-21T22:28:47.248979+0000","last_became_peered":"2023-07-21T22:28:47.248979+0000","last_unstale":"2023-07-21T22:28:47.259961+0000","last_undegraded":"2023-07-21T22:28:47.259961+0000","last_fullsized":"2023-07-21T22:28:47.259961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:12:44.807097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245648+0000","last_change":"2023-07-21T22:27:42.097736+0000","last_active":"2023-07-21T22:28:47.245648+0000","last_peered":"2023-07-21T22:28:47.245648+0000","last_clean":"2023-07-21T22:28:47.245648+0000","last_became_active":"2023-07-21T22:27:42.097419+0000","last_became_peered":"2023-07-21T22:27:42.097419+0000","last_unstale":"2023-07-21T22:28:47.245648+0000","last_undegraded":"2023-07-21T22:28:47.245648+0000","last_fullsized":"2023-07-21T22:28:47.245648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:07:09.477979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":99,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:29:00.157924+0000","last_change":"2023-07-21T22:27:41.074922+0000","last_active":"2023-07-21T22:29:00.157924+0000","last_peered":"2023-07-21T22:29:00.157924+0000","last_clean":"2023-07-21T22:29:00.157924+0000","last_became_active":"2023-07-21T22:27:41.074610+0000","last_became_peered":"2023-07-21T22:27:41.074610+0000","last_unstale":"2023-07-21T22:29:00.157924+0000","last_undegraded":"2023-07-21T22:29:00.157924+0000","last_fullsized":"2023-07-21T22:29:00.157924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:06:19.892702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245234+0000","last_change":"2023-07-21T22:27:42.097312+0000","last_active":"2023-07-21T22:28:47.245234+0000","last_peered":"2023-07-21T22:28:47.245234+0000","last_clean":"2023-07-21T22:28:47.245234+0000","last_became_active":"2023-07-21T22:27:42.096835+0000","last_became_peered":"2023-07-21T22:27:42.096835+0000","last_unstale":"2023-07-21T22:28:47.245234+0000","last_undegraded":"2023-07-21T22:28:47.245234+0000","last_fullsized":"2023-07-21T22:28:47.245234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:34:28.949507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240533+0000","last_change":"2023-07-21T22:27:41.071971+0000","last_active":"2023-07-21T22:28:47.240533+0000","last_peered":"2023-07-21T22:28:47.240533+0000","last_clean":"2023-07-21T22:28:47.240533+0000","last_became_active":"2023-07-21T22:27:41.071550+0000","last_became_peered":"2023-07-21T22:27:41.071550+0000","last_unstale":"2023-07-21T22:28:47.240533+0000","last_undegraded":"2023-07-21T22:28:47.240533+0000","last_fullsized":"2023-07-21T22:28:47.240533+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:35:28.749230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.244847+0000","last_change":"2023-07-21T22:27:42.098039+0000","last_active":"2023-07-21T22:28:47.244847+0000","last_peered":"2023-07-21T22:28:47.244847+0000","last_clean":"2023-07-21T22:28:47.244847+0000","last_became_active":"2023-07-21T22:27:42.097737+0000","last_became_peered":"2023-07-21T22:27:42.097737+0000","last_unstale":"2023-07-21T22:28:47.244847+0000","last_undegraded":"2023-07-21T22:28:47.244847+0000","last_fullsized":"2023-07-21T22:28:47.244847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:41:06.382770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.251946+0000","last_change":"2023-07-21T22:28:47.251946+0000","last_active":"2023-07-21T22:28:47.251946+0000","last_peered":"2023-07-21T22:28:47.251946+0000","last_clean":"2023-07-21T22:28:47.251946+0000","last_became_active":"2023-07-21T22:28:47.247973+0000","last_became_peered":"2023-07-21T22:28:47.247973+0000","last_unstale":"2023-07-21T22:28:47.251946+0000","last_undegraded":"2023-07-21T22:28:47.251946+0000","last_fullsized":"2023-07-21T22:28:47.251946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:40.051987+0000","last_clean_scrub_stamp":"2023-07-21T22:27:40.051987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:18.015306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.245217+0000","last_change":"2023-07-21T22:27:42.097891+0000","last_active":"2023-07-21T22:28:47.245217+0000","last_peered":"2023-07-21T22:28:47.245217+0000","last_clean":"2023-07-21T22:28:47.245217+0000","last_became_active":"2023-07-21T22:27:42.097022+0000","last_became_peered":"2023-07-21T22:27:42.097022+0000","last_unstale":"2023-07-21T22:28:47.245217+0000","last_undegraded":"2023-07-21T22:28:47.245217+0000","last_fullsized":"2023-07-21T22:28:47.245217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:54:17.707405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243207+0000","last_change":"2023-07-21T22:27:42.085774+0000","last_active":"2023-07-21T22:28:47.243207+0000","last_peered":"2023-07-21T22:28:47.243207+0000","last_clean":"2023-07-21T22:28:47.243207+0000","last_became_active":"2023-07-21T22:27:42.085460+0000","last_became_peered":"2023-07-21T22:27:42.085460+0000","last_unstale":"2023-07-21T22:28:47.243207+0000","last_undegraded":"2023-07-21T22:28:47.243207+0000","last_fullsized":"2023-07-21T22:28:47.243207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:26:42.082026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.251186+0000","last_change":"2023-07-21T22:28:47.251186+0000","last_active":"2023-07-21T22:28:47.251186+0000","last_peered":"2023-07-21T22:28:47.251186+0000","last_clean":"2023-07-21T22:28:47.251186+0000","last_became_active":"2023-07-21T22:28:47.250765+0000","last_became_peered":"2023-07-21T22:28:47.250765+0000","last_unstale":"2023-07-21T22:28:47.251186+0000","last_undegraded":"2023-07-21T22:28:47.251186+0000","last_fullsized":"2023-07-21T22:28:47.251186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:12:57.007813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242433+0000","last_change":"2023-07-21T22:27:42.092918+0000","last_active":"2023-07-21T22:28:47.242433+0000","last_peered":"2023-07-21T22:28:47.242433+0000","last_clean":"2023-07-21T22:28:47.242433+0000","last_became_active":"2023-07-21T22:27:42.092484+0000","last_became_peered":"2023-07-21T22:27:42.092484+0000","last_unstale":"2023-07-21T22:28:47.242433+0000","last_undegraded":"2023-07-21T22:28:47.242433+0000","last_fullsized":"2023-07-21T22:28:47.242433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:43:03.935206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382257+0000","last_change":"2023-07-21T22:27:42.096306+0000","last_active":"2023-07-21T22:28:47.382257+0000","last_peered":"2023-07-21T22:28:47.382257+0000","last_clean":"2023-07-21T22:28:47.382257+0000","last_became_active":"2023-07-21T22:27:42.096010+0000","last_became_peered":"2023-07-21T22:27:42.096010+0000","last_unstale":"2023-07-21T22:28:47.382257+0000","last_undegraded":"2023-07-21T22:28:47.382257+0000","last_fullsized":"2023-07-21T22:28:47.382257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:20:04.344089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239283+0000","last_change":"2023-07-21T22:27:42.097964+0000","last_active":"2023-07-21T22:28:47.239283+0000","last_peered":"2023-07-21T22:28:47.239283+0000","last_clean":"2023-07-21T22:28:47.239283+0000","last_became_active":"2023-07-21T22:27:42.097666+0000","last_became_peered":"2023-07-21T22:27:42.097666+0000","last_unstale":"2023-07-21T22:28:47.239283+0000","last_undegraded":"2023-07-21T22:28:47.239283+0000","last_fullsized":"2023-07-21T22:28:47.239283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:48:04.636292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243609+0000","last_change":"2023-07-21T22:27:42.083225+0000","last_active":"2023-07-21T22:28:47.243609+0000","last_peered":"2023-07-21T22:28:47.243609+0000","last_clean":"2023-07-21T22:28:47.243609+0000","last_became_active":"2023-07-21T22:27:42.082892+0000","last_became_peered":"2023-07-21T22:27:42.082892+0000","last_unstale":"2023-07-21T22:28:47.243609+0000","last_undegraded":"2023-07-21T22:28:47.243609+0000","last_fullsized":"2023-07-21T22:28:47.243609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:57:37.483988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.243881+0000","last_change":"2023-07-21T22:27:42.090854+0000","last_active":"2023-07-21T22:28:47.243881+0000","last_peered":"2023-07-21T22:28:47.243881+0000","last_clean":"2023-07-21T22:28:47.243881+0000","last_became_active":"2023-07-21T22:27:42.090501+0000","last_became_peered":"2023-07-21T22:27:42.090501+0000","last_unstale":"2023-07-21T22:28:47.243881+0000","last_undegraded":"2023-07-21T22:28:47.243881+0000","last_fullsized":"2023-07-21T22:28:47.243881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:43:35.575019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242109+0000","last_change":"2023-07-21T22:27:42.094986+0000","last_active":"2023-07-21T22:28:47.242109+0000","last_peered":"2023-07-21T22:28:47.242109+0000","last_clean":"2023-07-21T22:28:47.242109+0000","last_became_active":"2023-07-21T22:27:42.094668+0000","last_became_peered":"2023-07-21T22:27:42.094668+0000","last_unstale":"2023-07-21T22:28:47.242109+0000","last_undegraded":"2023-07-21T22:28:47.242109+0000","last_fullsized":"2023-07-21T22:28:47.242109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:17:57.739239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.240373+0000","last_change":"2023-07-21T22:27:42.098033+0000","last_active":"2023-07-21T22:28:47.240373+0000","last_peered":"2023-07-21T22:28:47.240373+0000","last_clean":"2023-07-21T22:28:47.240373+0000","last_became_active":"2023-07-21T22:27:42.097219+0000","last_became_peered":"2023-07-21T22:27:42.097219+0000","last_unstale":"2023-07-21T22:28:47.240373+0000","last_undegraded":"2023-07-21T22:28:47.240373+0000","last_fullsized":"2023-07-21T22:28:47.240373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:58:21.227651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.241687+0000","last_change":"2023-07-21T22:27:42.092136+0000","last_active":"2023-07-21T22:28:47.241687+0000","last_peered":"2023-07-21T22:28:47.241687+0000","last_clean":"2023-07-21T22:28:47.241687+0000","last_became_active":"2023-07-21T22:27:42.091559+0000","last_became_peered":"2023-07-21T22:27:42.091559+0000","last_unstale":"2023-07-21T22:28:47.241687+0000","last_undegraded":"2023-07-21T22:28:47.241687+0000","last_fullsized":"2023-07-21T22:28:47.241687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:43:56.512118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.382811+0000","last_change":"2023-07-21T22:27:42.093227+0000","last_active":"2023-07-21T22:28:47.382811+0000","last_peered":"2023-07-21T22:28:47.382811+0000","last_clean":"2023-07-21T22:28:47.382811+0000","last_became_active":"2023-07-21T22:27:42.092918+0000","last_became_peered":"2023-07-21T22:27:42.092918+0000","last_unstale":"2023-07-21T22:28:47.382811+0000","last_undegraded":"2023-07-21T22:28:47.382811+0000","last_fullsized":"2023-07-21T22:28:47.382811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:00:34.144455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.239365+0000","last_change":"2023-07-21T22:27:42.086494+0000","last_active":"2023-07-21T22:28:47.239365+0000","last_peered":"2023-07-21T22:28:47.239365+0000","last_clean":"2023-07-21T22:28:47.239365+0000","last_became_active":"2023-07-21T22:27:42.086189+0000","last_became_peered":"2023-07-21T22:27:42.086189+0000","last_unstale":"2023-07-21T22:28:47.239365+0000","last_undegraded":"2023-07-21T22:28:47.239365+0000","last_fullsized":"2023-07-21T22:28:47.239365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:01:11.731302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-07-21T22:28:47.242968+0000","last_change":"2023-07-21T22:27:42.090874+0000","last_active":"2023-07-21T22:28:47.242968+0000","last_peered":"2023-07-21T22:28:47.242968+0000","last_clean":"2023-07-21T22:28:47.242968+0000","last_became_active":"2023-07-21T22:27:42.090597+0000","last_became_peered":"2023-07-21T22:27:42.090597+0000","last_unstale":"2023-07-21T22:28:47.242968+0000","last_undegraded":"2023-07-21T22:28:47.242968+0000","last_fullsized":"2023-07-21T22:28:47.242968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:27:41.057320+0000","last_clean_scrub_stamp":"2023-07-21T22:27:41.057320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:16:50.131217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61988,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":779,"num_read_kb":456,"num_write":230,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":986,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":185646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.629},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.52000000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.501},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.80900000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.82499999999999996}]},{"osd":9,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.78900000000000003}]},{"osd":10,"last update":"Fri Jul 21 22:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.70399999999999996}]}]},{"osd":10,"up_from":70,"seq":300647710752,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1613824,"data_stored":1085151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":8.7710000000000008,"5min":8.7710000000000008,"15min":8.7710000000000008},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":8.8610000000000007,"5min":8.8610000000000007,"15min":8.8610000000000007},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.47299999999999998,"15min":0.46999999999999997},"min":{"1min":0.29499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70399999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.505,"15min":0.501},"min":{"1min":0.40100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.38100000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.495,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.80100000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":0.73899999999999999,"15min":0.67700000000000005},"min":{"1min":0.45100000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":8.5139999999999993,"5min":8.5139999999999993,"15min":8.5139999999999993},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":0.75800000000000001,"15min":0.67800000000000005},"min":{"1min":0.47699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":8.5440000000000005,"5min":8.5440000000000005,"15min":8.5440000000000005},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.36499999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.34799999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75800000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.499}]},{"osd":5,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67900000000000005,"15min":0.67400000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.47499999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.98399999999999999,"5min":1.008,"15min":1.008},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.76400000000000001,"15min":0.72399999999999998},"min":{"1min":0.56899999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":3.972,"5min":3.972,"15min":3.972},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.087,"5min":0.76900000000000002,"15min":0.71599999999999997},"min":{"1min":0.51100000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.8570000000000002,"5min":3.8570000000000002,"15min":3.8570000000000002},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0089999999999999,"5min":1.073,"15min":1.073},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.998,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.69399999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":0.90100000000000002,"15min":0.82199999999999995},"min":{"1min":0.52900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":8.5899999999999999,"5min":8.5899999999999999,"15min":8.5899999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.82299999999999995,"15min":0.81200000000000006},"min":{"1min":0.57499999999999996,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.1469999999999998,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":0.67000000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.76000000000000001}]}]},{"osd":9,"up_from":65,"seq":279172874277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":439765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.438,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":3.0379999999999998,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60499999999999998,"15min":0.61399999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.129,"5min":3.129,"15min":3.129},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.51500000000000001,"15min":0.53000000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.436}]},{"osd":4,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.66500000000000004,"15min":0.68500000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69599999999999995,"15min":0.70199999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":2.9710000000000001,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0469999999999999,"5min":1.089,"15min":1.089},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.61899999999999999,"15min":0.63900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.66400000000000003,"15min":0.68600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.93700000000000006,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.749}]},{"osd":8,"last update":"Fri Jul 21 22:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":3.0049999999999999,"5min":3.0049999999999999,"15min":3.0049999999999999},"last":0.73499999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:28:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.72599999999999998,"15min":0.73799999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.745,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.78900000000000003}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":440121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52100000000000002,"15min":0.51500000000000001},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.62},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.35999999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.752}]},{"osd":2,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.45400000000000001,"15min":0.438},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.48099999999999998,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60099999999999998,"15min":0.59999999999999998},"min":{"1min":0.42099999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.71799999999999997}]},{"osd":4,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.434,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.61599999999999999,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.48599999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.873},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.69999999999999996,"15min":0.69699999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.997,"5min":1.097,"15min":1.097},"last":1.004}]},{"osd":7,"last update":"Fri Jul 21 22:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.55800000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.054,"5min":1.171,"15min":1.171},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.68999999999999995,"15min":0.68000000000000005},"min":{"1min":0.51000000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1830000000000001,"5min":1.232,"15min":1.232},"last":0.89300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.69799999999999995,"15min":0.68400000000000005},"min":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0680000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.94799999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.746,"15min":0.71899999999999997},"min":{"1min":0.67500000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0800000000000001,"5min":1.21,"15min":1.21},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.76900000000000002,"15min":0.745},"min":{"1min":0.68500000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.97699999999999998}]},{"osd":11,"last update":"Fri Jul 21 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.96199999999999997}]}]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28456,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716696,"statfs":{"total":95995035648,"available":95965896704,"internally_reserved":0,"allocated":1540096,"data_stored":1032522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54400000000000004,"15min":0.53800000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.436,"15min":0.437},"min":{"1min":0.32700000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46400000000000002,"15min":0.46500000000000002},"min":{"1min":0.32900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.34599999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83599999999999997,"5min":1.548,"15min":1.548},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.80700000000000005,"15min":0.88700000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82599999999999996,"5min":5.8579999999999997,"15min":5.8579999999999997},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.77600000000000002,"15min":0.82699999999999996},"min":{"1min":0.434,"5min":0.374,"15min":0.374},"max":{"1min":2.169,"5min":5.7480000000000002,"15min":5.7480000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.78000000000000003,"15min":0.85399999999999998},"min":{"1min":0.45000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.871,"5min":5.7649999999999997,"15min":5.7649999999999997},"last":0.753}]},{"osd":4,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.42999999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.41499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.80300000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.44400000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67700000000000005,"15min":0.66900000000000004},"min":{"1min":0.51600000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.91800000000000004,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90600000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.69399999999999995,"15min":0.68300000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89800000000000002,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.73599999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:29:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.91000000000000003,"15min":0.98399999999999999},"min":{"1min":0.57999999999999996,"5min":0.437,"15min":0.437},"max":{"1min":1.0009999999999999,"5min":5.7800000000000002,"15min":5.7800000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78300000000000003,"15min":0.79900000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1060000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.54200000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.79100000000000004,"15min":0.80700000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85799999999999998,"5min":1.819,"15min":1.819},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.753,"15min":0.76100000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.88300000000000001,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.56499999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.97999999999999998,"5min":1.839,"15min":1.839},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.57599999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96599999999999997,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.76900000000000002}]},{"osd":11,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.70199999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528368,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":439440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.38800000000000001,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.36499999999999999},"min":{"1min":0.254,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.624,"15min":0.56299999999999994},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":6.899,"5min":6.899,"15min":6.899},"last":0.628},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.45600000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54000000000000004,"15min":0.52000000000000002},"min":{"1min":0.44800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.64500000000000002,"15min":0.57499999999999996},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":7.0359999999999996,"5min":7.0359999999999996,"15min":7.0359999999999996},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.496,"15min":0.437},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":7.1289999999999996,"5min":7.1289999999999996,"15min":7.1289999999999996},"last":0.315},{"interface":"front","average":{"1min":1.071,"5min":0.58399999999999996,"15min":0.48599999999999999},"min":{"1min":0.32000000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":12.414999999999999,"5min":12.414999999999999,"15min":12.414999999999999},"last":0.42099999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.64900000000000002,"15min":0.58699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":6.9690000000000003,"5min":6.9690000000000003,"15min":6.9690000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.60599999999999998,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.48499999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:28:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.44700000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.60199999999999998,"15min":0.58799999999999997},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:28:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59299999999999997,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.70099999999999996,"15min":0.64000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":7.0049999999999999,"5min":7.0049999999999999,"15min":7.0049999999999999},"last":0.55600000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:28:00 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.747,"15min":0.69899999999999995},"min":{"1min":0.55000000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":7.0670000000000002,"5min":7.0670000000000002,"15min":7.0670000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70099999999999996,"15min":0.69899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.60899999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.68000000000000005,"15min":0.66700000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.516,"5min":4.516,"15min":4.516},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.78000000000000003,"15min":0.75600000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":7.1619999999999999,"5min":7.1619999999999999,"15min":7.1619999999999999},"last":0.496}]},{"osd":10,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.83599999999999997,"15min":0.80800000000000005},"min":{"1min":0.52300000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":7.0979999999999999,"5min":7.0979999999999999,"15min":7.0979999999999999},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":496289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51500000000000001,"15min":0.48499999999999999},"min":{"1min":0.41299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.48699999999999999},"min":{"1min":0.40999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.80000000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.505}]},{"osd":2,"last update":"Fri Jul 21 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.57799999999999996,"15min":0.52100000000000002},"min":{"1min":0.46800000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.79900000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56699999999999995,"15min":0.54000000000000004},"min":{"1min":0.48699999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83499999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.38300000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.872,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60199999999999998,"15min":0.58999999999999997},"min":{"1min":0.49299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.46400000000000002,"15min":0.42799999999999999},"min":{"1min":0.35999999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.46800000000000003,"15min":0.438},"min":{"1min":0.35399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.60999999999999999,"15min":0.54800000000000004},"min":{"1min":0.48999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86099999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.63400000000000001,"15min":0.57399999999999995},"min":{"1min":0.54000000000000004,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri Jul 21 22:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56299999999999994,"15min":0.53200000000000003},"min":{"1min":0.45300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.61899999999999999,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.752}]},{"osd":7,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.59699999999999998,"15min":0.55200000000000005},"min":{"1min":0.48299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.93500000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.64600000000000002,"15min":0.61199999999999999},"min":{"1min":0.437,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96799999999999997,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82999999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.70099999999999996,"15min":0.67000000000000004},"min":{"1min":0.46300000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92700000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.72499999999999998,"15min":0.69499999999999995},"min":{"1min":0.64800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77400000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:28:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74099999999999999,"15min":0.73499999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.626,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79000000000000004,"15min":0.78500000000000003},"min":{"1min":0.58299999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.80600000000000005}]},{"osd":11,"last update":"Fri Jul 21 22:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.78400000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":441163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.48899999999999999},"min":{"1min":0.372,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.746,"5min":0.873,"15min":0.873},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.501},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68700000000000006,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.78800000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55000000000000004,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70099999999999996,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.621}]},{"osd":3,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58699999999999997,"15min":0.58799999999999997},"min":{"1min":0.36099999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75700000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.441},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83199999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52400000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78500000000000003,"5min":0.879,"15min":0.879},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78500000000000003,"5min":1.002,"15min":1.002},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48299999999999998,"15min":0.49199999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58499999999999996,"5min":0.749,"15min":0.749},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79100000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.40799999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66500000000000004,"15min":0.66800000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85499999999999998,"5min":1.04,"15min":1.04},"last":0.48499999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.68400000000000005,"15min":0.69099999999999995},"min":{"1min":0.49399999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84799999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.61899999999999999,"15min":0.63500000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88600000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.68200000000000005,"15min":0.70399999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89800000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46500000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:28:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93700000000000006,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88400000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.501}]},{"osd":11,"last update":"Fri Jul 21 22:28:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69599999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":494995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.501,"15min":0.47699999999999998},"min":{"1min":0.38300000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52000000000000002,"15min":0.47099999999999997},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.68500000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.56399999999999995,"15min":0.498},"min":{"1min":0.43099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.85999999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.57299999999999995,"15min":0.52300000000000002},"min":{"1min":0.45800000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.874,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57799999999999996,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.316,"15min":0.316},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.58199999999999996,"15min":0.54900000000000004},"min":{"1min":0.44600000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.501,"15min":0.46899999999999997},"min":{"1min":0.375,"5min":0.25,"15min":0.25},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.501,"15min":0.47499999999999998},"min":{"1min":0.41199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61399999999999999,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.378,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.435,"5min":0.432,"15min":0.432},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46899999999999997}]},{"osd":8,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.58899999999999997,"15min":0.55300000000000005},"min":{"1min":0.46600000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.625,"15min":0.59699999999999998},"min":{"1min":0.51900000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.51100000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.63700000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.73099999999999998,"15min":0.71299999999999997},"min":{"1min":0.68300000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.98499999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.92100000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.85299999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":438988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.35599999999999998,"15min":0.33500000000000002},"min":{"1min":0.24399999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.629,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.26100000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.496,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.29799999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86599999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.378,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.71499999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.503}]},{"osd":2,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76000000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58399999999999996,"15min":0.56000000000000005},"min":{"1min":0.42299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.434,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.495},{"interface":"front","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.57999999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82999999999999996,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:28:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70599999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.61399999999999999,"15min":0.61899999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.749,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:28:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.50800000000000001,"15min":0.501},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78700000000000003,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.439}]},{"osd":7,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.59799999999999998},"min":{"1min":0.40200000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79000000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66100000000000003,"15min":0.65700000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85499999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.56999999999999995}]},{"osd":8,"last update":"Fri Jul 21 22:27:00 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.64300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:28:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.996,"5min":1.038,"15min":1.038},"last":0.47199999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77300000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.84899999999999998,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.64700000000000002}]},{"osd":11,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.60799999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":439613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.50900000000000001,"15min":0.48499999999999999},"min":{"1min":0.47899999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.49299999999999999},"min":{"1min":0.41799999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.41799999999999998,"15min":0.39700000000000002},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41799999999999998,"15min":0.39200000000000002},"min":{"1min":0.34200000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59099999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.44900000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.443,"5min":0.371,"15min":0.371},"max":{"1min":0.82999999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.83399999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.60899999999999999,"15min":0.54300000000000004},"min":{"1min":0.40100000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.6559999999999997,"5min":5.6559999999999997,"15min":5.6559999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.56699999999999995,"15min":0.53300000000000003},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.5430000000000001,"5min":3.5430000000000001,"15min":3.5430000000000001},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Jul 21 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.61699999999999999,"15min":0.60099999999999998},"min":{"1min":0.40300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.435,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79600000000000004,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.78100000000000003}]},{"osd":6,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.65900000000000003,"15min":0.60699999999999998},"min":{"1min":0.45300000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":5.6310000000000002,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.68899999999999995,"15min":0.629},"min":{"1min":0.53100000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":5.6790000000000003,"5min":5.6790000000000003,"15min":5.6790000000000003},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61199999999999999,"15min":0.59299999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97299999999999998,"5min":0.995,"15min":0.995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60399999999999998,"15min":0.59299999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72299999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.67400000000000004}]},{"osd":8,"last update":"Fri Jul 21 22:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.71699999999999997,"15min":0.70099999999999996},"min":{"1min":0.54400000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89000000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.86699999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80100000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":0.81299999999999994,"15min":0.755},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":7.6639999999999997,"5min":7.6639999999999997,"15min":7.6639999999999997},"last":0.754},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.81699999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78000000000000003,"15min":0.78700000000000003},"min":{"1min":0.53200000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.94599999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77500000000000002,"15min":0.77800000000000002},"min":{"1min":0.496,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94799999999999995,"5min":1.103,"15min":1.103},"last":0.86799999999999999}]},{"osd":11,"last update":"Fri Jul 21 22:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.85199999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1512,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1548288,"data_stored":1031965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52000000000000002},"min":{"1min":0.315,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.80200000000000005,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.51500000000000001,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54600000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57199999999999995,"15min":0.53100000000000003},"min":{"1min":0.45700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81899999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58799999999999997,"15min":0.54400000000000004},"min":{"1min":0.53000000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89000000000000001,"5min":1.036,"15min":1.036},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.42599999999999999,"15min":0.39100000000000001},"min":{"1min":0.32000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67700000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.43099999999999999,"15min":0.40300000000000002},"min":{"1min":0.32000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.64800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.39900000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68000000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.436},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58499999999999996,"15min":0.59499999999999997},"min":{"1min":0.39500000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76900000000000002,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61299999999999999,"15min":0.56899999999999995},"min":{"1min":0.442,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83599999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.48099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.874,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.42699999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.629},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63600000000000001,"15min":0.61299999999999999},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89400000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.61499999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:28:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.67400000000000004,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90000000000000002,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.67400000000000004,"15min":0.64400000000000002},"min":{"1min":0.58099999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92800000000000005,"5min":1.054,"15min":1.054},"last":0.72899999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:28:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.628},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89100000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.749},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61299999999999999,"15min":0.59699999999999998},"min":{"1min":0.45300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81699999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.71499999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69099999999999995,"15min":0.68700000000000006},"min":{"1min":0.53200000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.54900000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77300000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.80800000000000005,"15min":0.80600000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.99099999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77200000000000002,"15min":0.76800000000000002},"min":{"1min":0.58499999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91600000000000004,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.81299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":57646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:29:07.815 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:29:07.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.921+0000 7f8b99b04700 1 -- 172.21.15.107:0/322763843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b94100550 msgr2=0x7f8b94106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.921+0000 7f8b99b04700 1 --2- 172.21.15.107:0/322763843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b94100550 0x7f8b94106e10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00b0a0 tx=0x7f8b8c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.921+0000 7f8b99b04700 1 -- 172.21.15.107:0/322763843 shutdown_connections 2023-07-21T22:29:07.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.921+0000 7f8b99b04700 1 --2- 172.21.15.107:0/322763843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b94100550 0x7f8b94106e10 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.921+0000 7f8b99b04700 1 --2- 172.21.15.107:0/322763843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b940fe9d0 0x7f8b940fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.921+0000 7f8b99b04700 1 --2- 172.21.15.107:0/322763843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b940ffbc0 0x7f8b94100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.921+0000 7f8b99b04700 1 -- 172.21.15.107:0/322763843 >> 172.21.15.107:0/322763843 conn(0x7f8b940f9fd0 msgr2=0x7f8b940fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:07.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.921+0000 7f8b99b04700 1 -- 172.21.15.107:0/322763843 shutdown_connections 2023-07-21T22:29:07.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.921+0000 7f8b99b04700 1 -- 172.21.15.107:0/322763843 wait complete. 2023-07-21T22:29:07.970 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.921+0000 7f8b99b04700 1 Processor -- start 2023-07-21T22:29:07.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b99b04700 1 -- start start 2023-07-21T22:29:07.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b99b04700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b940fe9d0 0x7f8b94068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:07.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b99b04700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b940ffbc0 0x7f8b9419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:07.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b99b04700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b94100550 0x7f8b9419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:07.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b99b04700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9419d3f0 con 0x7f8b940ffbc0 2023-07-21T22:29:07.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b99b04700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9419d560 con 0x7f8b940fe9d0 2023-07-21T22:29:07.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b99b04700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9419d720 con 0x7f8b94100550 2023-07-21T22:29:07.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b92ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b940ffbc0 0x7f8b9419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:07.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b92ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b940ffbc0 0x7f8b9419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48088/0 (socket says 172.21.15.107:48088) 2023-07-21T22:29:07.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b92ffd700 1 -- 172.21.15.107:0/1728499950 learned_addr learned my addr 172.21.15.107:0/1728499950 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:07.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b93fff700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b94100550 0x7f8b9419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:07.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b937fe700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b940fe9d0 0x7f8b94068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:07.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b92ffd700 1 -- 172.21.15.107:0/1728499950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b94100550 msgr2=0x7f8b9419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b92ffd700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b94100550 0x7f8b9419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b92ffd700 1 -- 172.21.15.107:0/1728499950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b940fe9d0 msgr2=0x7f8b94068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:07.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b92ffd700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b940fe9d0 0x7f8b94068200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:07.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b92ffd700 1 -- 172.21.15.107:0/1728499950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c009d00 con 0x7f8b940ffbc0 2023-07-21T22:29:07.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b92ffd700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b940ffbc0 0x7f8b9419a5d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400d950 tx=0x7f8b8400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:07.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b90ff9700 1 -- 172.21.15.107:0/1728499950 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400f890 con 0x7f8b940ffbc0 2023-07-21T22:29:07.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b90ff9700 1 -- 172.21.15.107:0/1728499950 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b84010460 con 0x7f8b940ffbc0 2023-07-21T22:29:07.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.922+0000 7f8b99b04700 1 -- 172.21.15.107:0/1728499950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9419da00 con 0x7f8b940ffbc0 2023-07-21T22:29:07.973 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.923+0000 7f8b90ff9700 1 -- 172.21.15.107:0/1728499950 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400e610 con 0x7f8b940ffbc0 2023-07-21T22:29:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.923+0000 7f8b99b04700 1 -- 172.21.15.107:0/1728499950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9419df10 con 0x7f8b940ffbc0 2023-07-21T22:29:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.923+0000 7f8b99b04700 1 -- 172.21.15.107:0/1728499950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b94105ef0 con 0x7f8b940ffbc0 2023-07-21T22:29:07.974 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.925+0000 7f8b90ff9700 1 -- 172.21.15.107:0/1728499950 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8b840108d0 con 0x7f8b940ffbc0 2023-07-21T22:29:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.925+0000 7f8b90ff9700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8b800a1220 0x7f8b800a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.925+0000 7f8b90ff9700 1 -- 172.21.15.107:0/1728499950 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8b840cc810 con 0x7f8b940ffbc0 2023-07-21T22:29:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.925+0000 7f8b937fe700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8b800a1220 0x7f8b800a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.926+0000 7f8b937fe700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8b800a1220 0x7f8b800a36d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8b940ffa20 tx=0x7f8b7c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:07.975 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:07.929+0000 7f8b90ff9700 1 -- 172.21.15.107:0/1728499950 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8b84095de0 con 0x7f8b940ffbc0 2023-07-21T22:29:08.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.123+0000 7f8b99b04700 1 -- 172.21.15.107:0/1728499950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b9402ca20 con 0x7f8b940ffbc0 2023-07-21T22:29:08.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.124+0000 7f8b90ff9700 1 -- 172.21.15.107:0/1728499950 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22727 (secure 0 0 0) 0x7f8b84016090 con 0x7f8b940ffbc0 2023-07-21T22:29:08.172 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:29:08.172 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":84,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:28:47.235779+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:29:08.173 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.126+0000 7f8b99b04700 1 -- 172.21.15.107:0/1728499950 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8b800a1220 msgr2=0x7f8b800a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.126+0000 7f8b99b04700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8b800a1220 0x7f8b800a36d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8b940ffa20 tx=0x7f8b7c00b410 comp rx=0 tx=0).stop 2023-07-21T22:29:08.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.126+0000 7f8b99b04700 1 -- 172.21.15.107:0/1728499950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b940ffbc0 msgr2=0x7f8b9419a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.126+0000 7f8b99b04700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b940ffbc0 0x7f8b9419a5d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400d950 tx=0x7f8b8400dc60 comp rx=0 tx=0).stop 2023-07-21T22:29:08.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.126+0000 7f8b99b04700 1 -- 172.21.15.107:0/1728499950 shutdown_connections 2023-07-21T22:29:08.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.126+0000 7f8b99b04700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8b800a1220 0x7f8b800a36d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.126+0000 7f8b99b04700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b94100550 0x7f8b9419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.176 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.126+0000 7f8b99b04700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b940fe9d0 0x7f8b94068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.176 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.126+0000 7f8b99b04700 1 --2- 172.21.15.107:0/1728499950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b940ffbc0 0x7f8b9419a5d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.176 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.126+0000 7f8b99b04700 1 -- 172.21.15.107:0/1728499950 >> 172.21.15.107:0/1728499950 conn(0x7f8b940f9fd0 msgr2=0x7f8b94103030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:08.176 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.126+0000 7f8b99b04700 1 -- 172.21.15.107:0/1728499950 shutdown_connections 2023-07-21T22:29:08.177 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.126+0000 7f8b99b04700 1 -- 172.21.15.107:0/1728499950 wait complete. 2023-07-21T22:29:08.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:23:52.031819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:27:39.494209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:27:40.672303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:29:08.186 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:29:08.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.264+0000 7f6473dd5700 1 -- 172.21.15.107:0/909371887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646c102e80 msgr2=0x7f646c105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.264+0000 7f6473dd5700 1 --2- 172.21.15.107:0/909371887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646c102e80 0x7f646c105260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6468009990 tx=0x7f6468009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.264+0000 7f6473dd5700 1 -- 172.21.15.107:0/909371887 shutdown_connections 2023-07-21T22:29:08.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.264+0000 7f6473dd5700 1 --2- 172.21.15.107:0/909371887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f646c1057a0 0x7f646c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.264+0000 7f6473dd5700 1 --2- 172.21.15.107:0/909371887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f646c100560 0x7f646c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.264+0000 7f6473dd5700 1 --2- 172.21.15.107:0/909371887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646c102e80 0x7f646c105260 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.264+0000 7f6473dd5700 1 -- 172.21.15.107:0/909371887 >> 172.21.15.107:0/909371887 conn(0x7f646c0fa000 msgr2=0x7f646c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:08.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.264+0000 7f6473dd5700 1 -- 172.21.15.107:0/909371887 shutdown_connections 2023-07-21T22:29:08.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.264+0000 7f6473dd5700 1 -- 172.21.15.107:0/909371887 wait complete. 2023-07-21T22:29:08.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6473dd5700 1 Processor -- start 2023-07-21T22:29:08.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6473dd5700 1 -- start start 2023-07-21T22:29:08.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6473dd5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646c100560 0x7f646c195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:08.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6473dd5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f646c102e80 0x7f646c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:08.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6473dd5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f646c1057a0 0x7f646c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:08.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6473dd5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c19d3a0 con 0x7f646c100560 2023-07-21T22:29:08.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6473dd5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c19d4e0 con 0x7f646c1057a0 2023-07-21T22:29:08.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6473dd5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c19d670 con 0x7f646c102e80 2023-07-21T22:29:08.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6471b71700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646c100560 0x7f646c195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:08.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6471b71700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646c100560 0x7f646c195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48100/0 (socket says 172.21.15.107:48100) 2023-07-21T22:29:08.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6471b71700 1 -- 172.21.15.107:0/296585564 learned_addr learned my addr 172.21.15.107:0/296585564 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:08.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6472372700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f646c1057a0 0x7f646c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:08.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6471370700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f646c102e80 0x7f646c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:08.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6471b71700 1 -- 172.21.15.107:0/296585564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f646c102e80 msgr2=0x7f646c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6471b71700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f646c102e80 0x7f646c195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6471b71700 1 -- 172.21.15.107:0/296585564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f646c1057a0 msgr2=0x7f646c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6471b71700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f646c1057a0 0x7f646c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6471b71700 1 -- 172.21.15.107:0/296585564 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64680095f0 con 0x7f646c100560 2023-07-21T22:29:08.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6471370700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f646c102e80 0x7f646c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:08.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.265+0000 7f6472372700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f646c1057a0 0x7f646c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:08.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.266+0000 7f6471b71700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646c100560 0x7f646c195820 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f645c00d950 tx=0x7f645c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:08.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.266+0000 7f6462ffd700 1 -- 172.21.15.107:0/296585564 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c009b40 con 0x7f646c100560 2023-07-21T22:29:08.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.266+0000 7f6462ffd700 1 -- 172.21.15.107:0/296585564 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f645c009ca0 con 0x7f646c100560 2023-07-21T22:29:08.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.266+0000 7f6473dd5700 1 -- 172.21.15.107:0/296585564 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646c19d950 con 0x7f646c100560 2023-07-21T22:29:08.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.266+0000 7f6462ffd700 1 -- 172.21.15.107:0/296585564 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c00f620 con 0x7f646c100560 2023-07-21T22:29:08.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.266+0000 7f6473dd5700 1 -- 172.21.15.107:0/296585564 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646c19de60 con 0x7f646c100560 2023-07-21T22:29:08.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.267+0000 7f6473dd5700 1 -- 172.21.15.107:0/296585564 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646c0fbb70 con 0x7f646c100560 2023-07-21T22:29:08.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.268+0000 7f6462ffd700 1 -- 172.21.15.107:0/296585564 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f645c010460 con 0x7f646c100560 2023-07-21T22:29:08.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.268+0000 7f6462ffd700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64580a11e0 0x7f64580a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:08.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.268+0000 7f6462ffd700 1 -- 172.21.15.107:0/296585564 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f645c0cce40 con 0x7f646c100560 2023-07-21T22:29:08.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.268+0000 7f6471370700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64580a11e0 0x7f64580a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:08.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.269+0000 7f6471370700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64580a11e0 0x7f64580a3690 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f646800b5c0 tx=0x7f6468005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:08.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.272+0000 7f6462ffd700 1 -- 172.21.15.107:0/296585564 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f645c096410 con 0x7f646c100560 2023-07-21T22:29:08.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:08 smithi190 ceph-mon[98334]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:29:08.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:08 smithi132 ceph-mon[97863]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:29:08.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:08 smithi107 ceph-mon[96543]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:29:08.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.460+0000 7f6473dd5700 1 -- 172.21.15.107:0/296585564 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f646c02c9e0 con 0x7f646c100560 2023-07-21T22:29:08.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.460+0000 7f6462ffd700 1 -- 172.21.15.107:0/296585564 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f645c016020 con 0x7f646c100560 2023-07-21T22:29:08.507 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2023-07-21T22:29:08.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.462+0000 7f6473dd5700 1 -- 172.21.15.107:0/296585564 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64580a11e0 msgr2=0x7f64580a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.462+0000 7f6473dd5700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64580a11e0 0x7f64580a3690 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f646800b5c0 tx=0x7f6468005fb0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.462+0000 7f6473dd5700 1 -- 172.21.15.107:0/296585564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646c100560 msgr2=0x7f646c195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.462+0000 7f6473dd5700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646c100560 0x7f646c195820 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f645c00d950 tx=0x7f645c00dd10 comp rx=0 tx=0).stop 2023-07-21T22:29:08.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.462+0000 7f6473dd5700 1 -- 172.21.15.107:0/296585564 shutdown_connections 2023-07-21T22:29:08.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.462+0000 7f6473dd5700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f646c1057a0 0x7f646c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.462+0000 7f6473dd5700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646c100560 0x7f646c195820 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.462+0000 7f6473dd5700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f64580a11e0 0x7f64580a3690 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.462+0000 7f6473dd5700 1 --2- 172.21.15.107:0/296585564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f646c102e80 0x7f646c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.462+0000 7f6473dd5700 1 -- 172.21.15.107:0/296585564 >> 172.21.15.107:0/296585564 conn(0x7f646c0fa000 msgr2=0x7f646c1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:08.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.462+0000 7f6473dd5700 1 -- 172.21.15.107:0/296585564 shutdown_connections 2023-07-21T22:29:08.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.462+0000 7f6473dd5700 1 -- 172.21.15.107:0/296585564 wait complete. 2023-07-21T22:29:08.521 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:29:08.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.599+0000 7fdb35441700 1 -- 172.21.15.107:0/652390676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb300fefb0 msgr2=0x7fdb300ff3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.599+0000 7fdb35441700 1 --2- 172.21.15.107:0/652390676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb300fefb0 0x7fdb300ff3c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009990 tx=0x7fdb18009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.599+0000 7fdb35441700 1 -- 172.21.15.107:0/652390676 shutdown_connections 2023-07-21T22:29:08.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.599+0000 7fdb35441700 1 --2- 172.21.15.107:0/652390676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb30100670 0x7fdb30106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.599+0000 7fdb35441700 1 --2- 172.21.15.107:0/652390676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb300fefb0 0x7fdb300ff3c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.599+0000 7fdb35441700 1 --2- 172.21.15.107:0/652390676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb300ffcc0 0x7fdb30100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.599+0000 7fdb35441700 1 -- 172.21.15.107:0/652390676 >> 172.21.15.107:0/652390676 conn(0x7fdb300f9850 msgr2=0x7fdb300fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.599+0000 7fdb35441700 1 -- 172.21.15.107:0/652390676 shutdown_connections 2023-07-21T22:29:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.599+0000 7fdb35441700 1 -- 172.21.15.107:0/652390676 wait complete. 2023-07-21T22:29:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb35441700 1 Processor -- start 2023-07-21T22:29:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb35441700 1 -- start start 2023-07-21T22:29:08.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb35441700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb300fefb0 0x7fdb300fe670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:08.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb35441700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb300ffcc0 0x7fdb3019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:08.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb35441700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb30100670 0x7fdb3019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:08.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb35441700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3019d360 con 0x7fdb300fefb0 2023-07-21T22:29:08.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb35441700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3019d4d0 con 0x7fdb30100670 2023-07-21T22:29:08.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb35441700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3019d690 con 0x7fdb300ffcc0 2023-07-21T22:29:08.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb2effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb300fefb0 0x7fdb300fe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:08.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb2effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb300fefb0 0x7fdb300fe670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48112/0 (socket says 172.21.15.107:48112) 2023-07-21T22:29:08.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb2effd700 1 -- 172.21.15.107:0/1292647074 learned_addr learned my addr 172.21.15.107:0/1292647074 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:08.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb2e7fc700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb300ffcc0 0x7fdb3019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:08.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb2effd700 1 -- 172.21.15.107:0/1292647074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb300ffcc0 msgr2=0x7fdb3019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb2effd700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb300ffcc0 0x7fdb3019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb2f7fe700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb30100670 0x7fdb3019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:08.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb2effd700 1 -- 172.21.15.107:0/1292647074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb30100670 msgr2=0x7fdb3019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.600+0000 7fdb2effd700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb30100670 0x7fdb3019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.601+0000 7fdb2effd700 1 -- 172.21.15.107:0/1292647074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb180095f0 con 0x7fdb300fefb0 2023-07-21T22:29:08.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.601+0000 7fdb2f7fe700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb30100670 0x7fdb3019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:08.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.601+0000 7fdb2effd700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb300fefb0 0x7fdb300fe670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdb180095c0 tx=0x7fdb1800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:08.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.601+0000 7fdb27fff700 1 -- 172.21.15.107:0/1292647074 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1801d070 con 0x7fdb300fefb0 2023-07-21T22:29:08.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.601+0000 7fdb35441700 1 -- 172.21.15.107:0/1292647074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3019d910 con 0x7fdb300fefb0 2023-07-21T22:29:08.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.601+0000 7fdb35441700 1 -- 172.21.15.107:0/1292647074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3019dd90 con 0x7fdb300fefb0 2023-07-21T22:29:08.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.601+0000 7fdb27fff700 1 -- 172.21.15.107:0/1292647074 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb1800fb80 con 0x7fdb300fefb0 2023-07-21T22:29:08.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.601+0000 7fdb27fff700 1 -- 172.21.15.107:0/1292647074 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb18017650 con 0x7fdb300fefb0 2023-07-21T22:29:08.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.602+0000 7fdb35441700 1 -- 172.21.15.107:0/1292647074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3010a270 con 0x7fdb300fefb0 2023-07-21T22:29:08.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.603+0000 7fdb27fff700 1 -- 172.21.15.107:0/1292647074 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fdb18022470 con 0x7fdb300fefb0 2023-07-21T22:29:08.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.604+0000 7fdb27fff700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdb1c0a11e0 0x7fdb1c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:08.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.604+0000 7fdb2e7fc700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdb1c0a11e0 0x7fdb1c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:08.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.604+0000 7fdb27fff700 1 -- 172.21.15.107:0/1292647074 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdb180ce4d0 con 0x7fdb300fefb0 2023-07-21T22:29:08.659 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.604+0000 7fdb2e7fc700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdb1c0a11e0 0x7fdb1c0a3690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb20005fd0 tx=0x7fdb20005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:08.659 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.607+0000 7fdb27fff700 1 -- 172.21.15.107:0/1292647074 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdb18097b30 con 0x7fdb300fefb0 2023-07-21T22:29:08.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.795+0000 7fdb35441700 1 -- 172.21.15.107:0/1292647074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb3010a430 con 0x7fdb300fefb0 2023-07-21T22:29:08.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.796+0000 7fdb27fff700 1 -- 172.21.15.107:0/1292647074 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb3010a430 con 0x7fdb300fefb0 2023-07-21T22:29:08.842 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2023-07-21T22:29:08.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.797+0000 7fdb35441700 1 -- 172.21.15.107:0/1292647074 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdb1c0a11e0 msgr2=0x7fdb1c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.797+0000 7fdb35441700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdb1c0a11e0 0x7fdb1c0a3690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb20005fd0 tx=0x7fdb20005dd0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.797+0000 7fdb35441700 1 -- 172.21.15.107:0/1292647074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb300fefb0 msgr2=0x7fdb300fe670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.797+0000 7fdb35441700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb300fefb0 0x7fdb300fe670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdb180095c0 tx=0x7fdb1800f690 comp rx=0 tx=0).stop 2023-07-21T22:29:08.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.798+0000 7fdb35441700 1 -- 172.21.15.107:0/1292647074 shutdown_connections 2023-07-21T22:29:08.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.798+0000 7fdb35441700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb30100670 0x7fdb3019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.798+0000 7fdb35441700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb300fefb0 0x7fdb300fe670 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.798+0000 7fdb35441700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fdb1c0a11e0 0x7fdb1c0a3690 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.798+0000 7fdb35441700 1 --2- 172.21.15.107:0/1292647074 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb300ffcc0 0x7fdb3019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.798+0000 7fdb35441700 1 -- 172.21.15.107:0/1292647074 >> 172.21.15.107:0/1292647074 conn(0x7fdb300f9850 msgr2=0x7fdb300fbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:08.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.798+0000 7fdb35441700 1 -- 172.21.15.107:0/1292647074 shutdown_connections 2023-07-21T22:29:08.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.798+0000 7fdb35441700 1 -- 172.21.15.107:0/1292647074 wait complete. 2023-07-21T22:29:08.856 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:29:08.981 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.934+0000 7f42a1007700 1 -- 172.21.15.107:0/282505630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f429c0fe9f0 msgr2=0x7f429c0fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.981 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.934+0000 7f42a1007700 1 --2- 172.21.15.107:0/282505630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f429c0fe9f0 0x7f429c0fee00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f428c009990 tx=0x7f428c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.981 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.934+0000 7f42a1007700 1 -- 172.21.15.107:0/282505630 shutdown_connections 2023-07-21T22:29:08.981 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.934+0000 7f42a1007700 1 --2- 172.21.15.107:0/282505630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f429c100570 0x7f429c106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.981 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.934+0000 7f42a1007700 1 --2- 172.21.15.107:0/282505630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f429c0fe9f0 0x7f429c0fee00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.981 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.934+0000 7f42a1007700 1 --2- 172.21.15.107:0/282505630 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f429c0ffbe0 0x7f429c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.934+0000 7f42a1007700 1 -- 172.21.15.107:0/282505630 >> 172.21.15.107:0/282505630 conn(0x7f429c0fa030 msgr2=0x7f429c0fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.934+0000 7f42a1007700 1 -- 172.21.15.107:0/282505630 shutdown_connections 2023-07-21T22:29:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.934+0000 7f42a1007700 1 -- 172.21.15.107:0/282505630 wait complete. 2023-07-21T22:29:08.983 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f42a1007700 1 Processor -- start 2023-07-21T22:29:08.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f42a1007700 1 -- start start 2023-07-21T22:29:08.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f42a1007700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f429c0fe9f0 0x7f429c069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:08.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f42a1007700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f429c0ffbe0 0x7f429c19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:08.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f42a1007700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f429c100570 0x7f429c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:08.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f42a1007700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c19f4e0 con 0x7f429c100570 2023-07-21T22:29:08.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f42a1007700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c19f650 con 0x7f429c0ffbe0 2023-07-21T22:29:08.985 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f42a1007700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c19f810 con 0x7f429c0fe9f0 2023-07-21T22:29:08.985 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f429b59e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f429c100570 0x7f429c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:08.985 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f429b59e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f429c100570 0x7f429c19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48130/0 (socket says 172.21.15.107:48130) 2023-07-21T22:29:08.985 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f429b59e700 1 -- 172.21.15.107:0/1766209796 learned_addr learned my addr 172.21.15.107:0/1766209796 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:08.985 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f429ad9d700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f429c0fe9f0 0x7f429c069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:08.985 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f429a59c700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f429c0ffbe0 0x7f429c19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:08.985 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f429b59e700 1 -- 172.21.15.107:0/1766209796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f429c0fe9f0 msgr2=0x7f429c069250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.985 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f429b59e700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f429c0fe9f0 0x7f429c069250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.986 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f429b59e700 1 -- 172.21.15.107:0/1766209796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f429c0ffbe0 msgr2=0x7f429c19c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:08.986 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f429b59e700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f429c0ffbe0 0x7f429c19c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:08.986 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.935+0000 7f429b59e700 1 -- 172.21.15.107:0/1766209796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c0095f0 con 0x7f429c100570 2023-07-21T22:29:08.986 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.936+0000 7f429b59e700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f429c100570 0x7f429c19ee00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4294007d70 tx=0x7f429400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:08.986 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.936+0000 7f4293fff700 1 -- 172.21.15.107:0/1766209796 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400ad40 con 0x7f429c100570 2023-07-21T22:29:08.986 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.936+0000 7f4293fff700 1 -- 172.21.15.107:0/1766209796 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42940128e0 con 0x7f429c100570 2023-07-21T22:29:08.986 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.936+0000 7f42a1007700 1 -- 172.21.15.107:0/1766209796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429c19faf0 con 0x7f429c100570 2023-07-21T22:29:08.987 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.936+0000 7f4293fff700 1 -- 172.21.15.107:0/1766209796 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42940116b0 con 0x7f429c100570 2023-07-21T22:29:08.987 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.936+0000 7f42a1007700 1 -- 172.21.15.107:0/1766209796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429c1a0000 con 0x7f429c100570 2023-07-21T22:29:08.987 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.937+0000 7f42a1007700 1 -- 172.21.15.107:0/1766209796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429c105f30 con 0x7f429c100570 2023-07-21T22:29:08.988 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.938+0000 7f4293fff700 1 -- 172.21.15.107:0/1766209796 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f4294012420 con 0x7f429c100570 2023-07-21T22:29:08.988 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.938+0000 7f4293fff700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f42880a11e0 0x7f42880a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:08.988 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.938+0000 7f429ad9d700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f42880a11e0 0x7f42880a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:08.988 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.938+0000 7f4293fff700 1 -- 172.21.15.107:0/1766209796 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f42940ccc20 con 0x7f429c100570 2023-07-21T22:29:08.988 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.939+0000 7f429ad9d700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f42880a11e0 0x7f42880a3690 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f428c00b5c0 tx=0x7f428c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:08.989 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:08.942+0000 7f4293fff700 1 -- 172.21.15.107:0/1766209796 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4294096300 con 0x7f429c100570 2023-07-21T22:29:09.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.140+0000 7f42a1007700 1 -- 172.21.15.107:0/1766209796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f429c02ca70 con 0x7f429c100570 2023-07-21T22:29:09.187 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.140+0000 7f4293fff700 1 -- 172.21.15.107:0/1766209796 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4294099920 con 0x7f429c100570 2023-07-21T22:29:09.187 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2023-07-21T22:29:09.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.142+0000 7f42a1007700 1 -- 172.21.15.107:0/1766209796 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f42880a11e0 msgr2=0x7f42880a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:09.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.142+0000 7f42a1007700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f42880a11e0 0x7f42880a3690 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f428c00b5c0 tx=0x7f428c005960 comp rx=0 tx=0).stop 2023-07-21T22:29:09.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.142+0000 7f42a1007700 1 -- 172.21.15.107:0/1766209796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f429c100570 msgr2=0x7f429c19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:09.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.142+0000 7f42a1007700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f429c100570 0x7f429c19ee00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4294007d70 tx=0x7f429400a340 comp rx=0 tx=0).stop 2023-07-21T22:29:09.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.142+0000 7f42a1007700 1 -- 172.21.15.107:0/1766209796 shutdown_connections 2023-07-21T22:29:09.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.142+0000 7f42a1007700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f429c100570 0x7f429c19ee00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.142+0000 7f42a1007700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f42880a11e0 0x7f42880a3690 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.142+0000 7f42a1007700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f429c0fe9f0 0x7f429c069250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.142+0000 7f42a1007700 1 --2- 172.21.15.107:0/1766209796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f429c0ffbe0 0x7f429c19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.142+0000 7f42a1007700 1 -- 172.21.15.107:0/1766209796 >> 172.21.15.107:0/1766209796 conn(0x7f429c0fa030 msgr2=0x7f429c103070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:09.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.143+0000 7f42a1007700 1 -- 172.21.15.107:0/1766209796 shutdown_connections 2023-07-21T22:29:09.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.143+0000 7f42a1007700 1 -- 172.21.15.107:0/1766209796 wait complete. 2023-07-21T22:29:09.202 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:29:09.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.276+0000 7f00df70c700 1 -- 172.21.15.107:0/2701846854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d80fe9d0 msgr2=0x7f00d80fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:09.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.276+0000 7f00df70c700 1 --2- 172.21.15.107:0/2701846854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d80fe9d0 0x7f00d80fede0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009990 tx=0x7f00d4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.276+0000 7f00df70c700 1 -- 172.21.15.107:0/2701846854 shutdown_connections 2023-07-21T22:29:09.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.276+0000 7f00df70c700 1 --2- 172.21.15.107:0/2701846854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00d8100550 0x7f00d8106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.276+0000 7f00df70c700 1 --2- 172.21.15.107:0/2701846854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d80fe9d0 0x7f00d80fede0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.276+0000 7f00df70c700 1 --2- 172.21.15.107:0/2701846854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00d80ffbc0 0x7f00d8100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.276+0000 7f00df70c700 1 -- 172.21.15.107:0/2701846854 >> 172.21.15.107:0/2701846854 conn(0x7f00d80f9fd0 msgr2=0x7f00d80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:09.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.276+0000 7f00df70c700 1 -- 172.21.15.107:0/2701846854 shutdown_connections 2023-07-21T22:29:09.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.276+0000 7f00df70c700 1 -- 172.21.15.107:0/2701846854 wait complete. 2023-07-21T22:29:09.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00df70c700 1 Processor -- start 2023-07-21T22:29:09.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00df70c700 1 -- start start 2023-07-21T22:29:09.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00df70c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d80fe9d0 0x7f00d81947e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:09.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00df70c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00d80ffbc0 0x7f00d8194d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:09.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00df70c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00d8100550 0x7f00d819bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:09.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00df70c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d819c360 con 0x7f00d80fe9d0 2023-07-21T22:29:09.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00df70c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d819c4a0 con 0x7f00d80ffbc0 2023-07-21T22:29:09.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00df70c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d819c630 con 0x7f00d8100550 2023-07-21T22:29:09.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00dd4a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d80fe9d0 0x7f00d81947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:09.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00dd4a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d80fe9d0 0x7f00d81947e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48148/0 (socket says 172.21.15.107:48148) 2023-07-21T22:29:09.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00dd4a8700 1 -- 172.21.15.107:0/272937134 learned_addr learned my addr 172.21.15.107:0/272937134 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:09.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00dcca7700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00d80ffbc0 0x7f00d8194d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:09.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00ddca9700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00d8100550 0x7f00d819bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:09.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00dd4a8700 1 -- 172.21.15.107:0/272937134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00d8100550 msgr2=0x7f00d819bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:09.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00dd4a8700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00d8100550 0x7f00d819bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00dd4a8700 1 -- 172.21.15.107:0/272937134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00d80ffbc0 msgr2=0x7f00d8194d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:09.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00dd4a8700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00d80ffbc0 0x7f00d8194d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00dd4a8700 1 -- 172.21.15.107:0/272937134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d40095f0 con 0x7f00d80fe9d0 2023-07-21T22:29:09.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00ddca9700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00d8100550 0x7f00d819bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:09.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00dd4a8700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d80fe9d0 0x7f00d81947e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f00d4004760 tx=0x7f00d4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:09.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.277+0000 7f00ca7fc700 1 -- 172.21.15.107:0/272937134 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d401d070 con 0x7f00d80fe9d0 2023-07-21T22:29:09.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.278+0000 7f00ca7fc700 1 -- 172.21.15.107:0/272937134 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00d40041d0 con 0x7f00d80fe9d0 2023-07-21T22:29:09.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.278+0000 7f00df70c700 1 -- 172.21.15.107:0/272937134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d819c8b0 con 0x7f00d80fe9d0 2023-07-21T22:29:09.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.278+0000 7f00ca7fc700 1 -- 172.21.15.107:0/272937134 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d40053e0 con 0x7f00d80fe9d0 2023-07-21T22:29:09.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.278+0000 7f00df70c700 1 -- 172.21.15.107:0/272937134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d819cc70 con 0x7f00d80fe9d0 2023-07-21T22:29:09.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.279+0000 7f00df70c700 1 -- 172.21.15.107:0/272937134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d8105fb0 con 0x7f00d80fe9d0 2023-07-21T22:29:09.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.279+0000 7f00ca7fc700 1 -- 172.21.15.107:0/272937134 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f00d4005540 con 0x7f00d80fe9d0 2023-07-21T22:29:09.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.280+0000 7f00ca7fc700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f00c40a1220 0x7f00c40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:09.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.280+0000 7f00dcca7700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f00c40a1220 0x7f00c40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:09.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.280+0000 7f00ca7fc700 1 -- 172.21.15.107:0/272937134 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f00d40ce020 con 0x7f00d80fe9d0 2023-07-21T22:29:09.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.280+0000 7f00dcca7700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f00c40a1220 0x7f00c40a36d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00cc005fd0 tx=0x7f00cc00a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:09.334 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.284+0000 7f00ca7fc700 1 -- 172.21.15.107:0/272937134 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f00d40977a0 con 0x7f00d80fe9d0 2023-07-21T22:29:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:09 smithi190 ceph-mon[98334]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:29:09.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:09 smithi190 ceph-mon[98334]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:29:09.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:09 smithi190 ceph-mon[98334]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2023-07-21T22:29:09.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:09 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1728499950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:29:09.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:09 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/296585564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:29:09.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:09 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1292647074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:29:09.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:09 smithi132 ceph-mon[97863]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:29:09.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:09 smithi132 ceph-mon[97863]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:29:09.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:09 smithi132 ceph-mon[97863]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2023-07-21T22:29:09.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:09 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1728499950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:29:09.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:09 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/296585564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:29:09.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:09 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1292647074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:29:09.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:09 smithi107 ceph-mon[96543]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:29:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:09 smithi107 ceph-mon[96543]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:29:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:09 smithi107 ceph-mon[96543]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2023-07-21T22:29:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:09 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1728499950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:29:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:09 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/296585564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:29:09.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:09 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1292647074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:29:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.517+0000 7f00df70c700 1 -- 172.21.15.107:0/272937134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00d802c850 con 0x7f00d80fe9d0 2023-07-21T22:29:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.518+0000 7f00ca7fc700 1 -- 172.21.15.107:0/272937134 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10868 (secure 0 0 0) 0x7f00d409adc0 con 0x7f00d80fe9d0 2023-07-21T22:29:09.565 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:29:09.566 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:27:49.034257+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:29:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.521+0000 7f00df70c700 1 -- 172.21.15.107:0/272937134 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f00c40a1220 msgr2=0x7f00c40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.521+0000 7f00df70c700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f00c40a1220 0x7f00c40a36d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f00cc005fd0 tx=0x7f00cc00a560 comp rx=0 tx=0).stop 2023-07-21T22:29:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.521+0000 7f00df70c700 1 -- 172.21.15.107:0/272937134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d80fe9d0 msgr2=0x7f00d81947e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.521+0000 7f00df70c700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d80fe9d0 0x7f00d81947e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f00d4004760 tx=0x7f00d4004790 comp rx=0 tx=0).stop 2023-07-21T22:29:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.521+0000 7f00df70c700 1 -- 172.21.15.107:0/272937134 shutdown_connections 2023-07-21T22:29:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.521+0000 7f00df70c700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f00c40a1220 0x7f00c40a36d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.521+0000 7f00df70c700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00d8100550 0x7f00d819bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.521+0000 7f00df70c700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d80fe9d0 0x7f00d81947e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.521+0000 7f00df70c700 1 --2- 172.21.15.107:0/272937134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00d80ffbc0 0x7f00d8194d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.521+0000 7f00df70c700 1 -- 172.21.15.107:0/272937134 >> 172.21.15.107:0/272937134 conn(0x7f00d80f9fd0 msgr2=0x7f00d81030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.521+0000 7f00df70c700 1 -- 172.21.15.107:0/272937134 shutdown_connections 2023-07-21T22:29:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.521+0000 7f00df70c700 1 -- 172.21.15.107:0/272937134 wait complete. 2023-07-21T22:29:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2023-07-21T22:29:09.580 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:29:09.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.654+0000 7f490bb8b700 1 -- 172.21.15.107:0/1970712078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4904100050 msgr2=0x7f4904100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:09.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.654+0000 7f490bb8b700 1 --2- 172.21.15.107:0/1970712078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4904100050 0x7f4904100460 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4900009990 tx=0x7f4900009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.654+0000 7f490bb8b700 1 -- 172.21.15.107:0/1970712078 shutdown_connections 2023-07-21T22:29:09.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.654+0000 7f490bb8b700 1 --2- 172.21.15.107:0/1970712078 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040ff5c0 0x7f4904106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.654+0000 7f490bb8b700 1 --2- 172.21.15.107:0/1970712078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4904100050 0x7f4904100460 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.654+0000 7f490bb8b700 1 --2- 172.21.15.107:0/1970712078 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49040fec10 0x7f49040ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.654+0000 7f490bb8b700 1 -- 172.21.15.107:0/1970712078 >> 172.21.15.107:0/1970712078 conn(0x7f49040f9770 msgr2=0x7f49040fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:09.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.655+0000 7f490bb8b700 1 -- 172.21.15.107:0/1970712078 shutdown_connections 2023-07-21T22:29:09.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.655+0000 7f490bb8b700 1 -- 172.21.15.107:0/1970712078 wait complete. 2023-07-21T22:29:09.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.655+0000 7f490bb8b700 1 Processor -- start 2023-07-21T22:29:09.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.655+0000 7f490bb8b700 1 -- start start 2023-07-21T22:29:09.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.655+0000 7f490bb8b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040fec10 0x7f49040fe690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:09.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.655+0000 7f490bb8b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040ff5c0 0x7f490419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:09.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.655+0000 7f490bb8b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4904100050 0x7f490419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:09.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.655+0000 7f490bb8b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419d330 con 0x7f49040ff5c0 2023-07-21T22:29:09.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.655+0000 7f490bb8b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419d4a0 con 0x7f49040fec10 2023-07-21T22:29:09.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.655+0000 7f490bb8b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419d660 con 0x7f4904100050 2023-07-21T22:29:09.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.655+0000 7f4909126700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040ff5c0 0x7f490419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:09.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.655+0000 7f4909126700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040ff5c0 0x7f490419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48164/0 (socket says 172.21.15.107:48164) 2023-07-21T22:29:09.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.655+0000 7f4909126700 1 -- 172.21.15.107:0/203538997 learned_addr learned my addr 172.21.15.107:0/203538997 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:09.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f4909927700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040fec10 0x7f49040fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:09.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f4909126700 1 -- 172.21.15.107:0/203538997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4904100050 msgr2=0x7f490419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f490a128700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4904100050 0x7f490419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f4909126700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4904100050 0x7f490419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f4909126700 1 -- 172.21.15.107:0/203538997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040fec10 msgr2=0x7f49040fe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f4909126700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040fec10 0x7f49040fe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f4909126700 1 -- 172.21.15.107:0/203538997 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49000095f0 con 0x7f49040ff5c0 2023-07-21T22:29:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f490a128700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4904100050 0x7f490419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:09.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f4909927700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040fec10 0x7f49040fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f4909126700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040ff5c0 0x7f490419a510 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f48f400d980 tx=0x7f48f400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f48faffd700 1 -- 172.21.15.107:0/203538997 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4009b40 con 0x7f49040ff5c0 2023-07-21T22:29:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f48faffd700 1 -- 172.21.15.107:0/203538997 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f400de60 con 0x7f49040ff5c0 2023-07-21T22:29:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f490bb8b700 1 -- 172.21.15.107:0/203538997 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490419d940 con 0x7f49040ff5c0 2023-07-21T22:29:09.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f48faffd700 1 -- 172.21.15.107:0/203538997 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4005580 con 0x7f49040ff5c0 2023-07-21T22:29:09.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.656+0000 7f490bb8b700 1 -- 172.21.15.107:0/203538997 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490419de50 con 0x7f49040ff5c0 2023-07-21T22:29:09.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.657+0000 7f490bb8b700 1 -- 172.21.15.107:0/203538997 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4904105960 con 0x7f49040ff5c0 2023-07-21T22:29:09.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.658+0000 7f48faffd700 1 -- 172.21.15.107:0/203538997 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f48f4004750 con 0x7f49040ff5c0 2023-07-21T22:29:09.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.659+0000 7f48faffd700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f48f00a12a0 0x7f48f00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:09.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.659+0000 7f4909927700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f48f00a12a0 0x7f48f00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:09.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.659+0000 7f48faffd700 1 -- 172.21.15.107:0/203538997 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f48f40cce20 con 0x7f49040ff5c0 2023-07-21T22:29:09.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.659+0000 7f4909927700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f48f00a12a0 0x7f48f00a3750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4900006010 tx=0x7f490000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:09.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.662+0000 7f48faffd700 1 -- 172.21.15.107:0/203538997 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f48f4096470 con 0x7f49040ff5c0 2023-07-21T22:29:09.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.852+0000 7f490bb8b700 1 -- 172.21.15.107:0/203538997 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f490402caa0 con 0x7f49040ff5c0 2023-07-21T22:29:09.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.854+0000 7f48faffd700 1 -- 172.21.15.107:0/203538997 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22727 (secure 0 0 0) 0x7f48f4017030 con 0x7f49040ff5c0 2023-07-21T22:29:09.901 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:29:09.901 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":84,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:28:47.235779+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:29:09.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.856+0000 7f490bb8b700 1 -- 172.21.15.107:0/203538997 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f48f00a12a0 msgr2=0x7f48f00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:09.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.856+0000 7f490bb8b700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f48f00a12a0 0x7f48f00a3750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4900006010 tx=0x7f490000b540 comp rx=0 tx=0).stop 2023-07-21T22:29:09.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.856+0000 7f490bb8b700 1 -- 172.21.15.107:0/203538997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040ff5c0 msgr2=0x7f490419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:09.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.856+0000 7f490bb8b700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040ff5c0 0x7f490419a510 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f48f400d980 tx=0x7f48f400dd40 comp rx=0 tx=0).stop 2023-07-21T22:29:09.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.856+0000 7f490bb8b700 1 -- 172.21.15.107:0/203538997 shutdown_connections 2023-07-21T22:29:09.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.856+0000 7f490bb8b700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f48f00a12a0 0x7f48f00a3750 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.856+0000 7f490bb8b700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4904100050 0x7f490419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.856+0000 7f490bb8b700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49040fec10 0x7f49040fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.856+0000 7f490bb8b700 1 --2- 172.21.15.107:0/203538997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49040ff5c0 0x7f490419a510 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:09.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.856+0000 7f490bb8b700 1 -- 172.21.15.107:0/203538997 >> 172.21.15.107:0/203538997 conn(0x7f49040f9770 msgr2=0x7f49040fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:09.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.856+0000 7f490bb8b700 1 -- 172.21.15.107:0/203538997 shutdown_connections 2023-07-21T22:29:09.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:09.856+0000 7f490bb8b700 1 -- 172.21.15.107:0/203538997 wait complete. 2023-07-21T22:29:09.916 DEBUG:tasks.fwd_scrub:Yielding 2023-07-21T22:29:09.916 INFO:teuthology.run_tasks:Running task background_exec... 2023-07-21T22:29:09.930 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2023-07-21T22:29:09.931 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-07-21T22:29:09.932 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-07-21T22:29:09.932 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:29:09.932 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:29:09.934 INFO:teuthology.run_tasks:Running task workunit... 2023-07-21T22:29:09.950 INFO:tasks.workunit:Pulling workunits from ref bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:29:09.951 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-07-21T22:29:09.951 INFO:tasks.workunit:timeout=3h 2023-07-21T22:29:09.951 INFO:tasks.workunit:cleanup=True 2023-07-21T22:29:09.951 DEBUG:teuthology.orchestra.run.smithi107:> stat -- /home/ubuntu/cephtest/mnt.0 2023-07-21T22:29:09.970 INFO:teuthology.orchestra.run.smithi107.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-07-21T22:29:09.970 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-07-21T22:29:09.970 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2023-07-21T22:29:09.970 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:29:09.970 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:cephfs_t:s0 2023-07-21T22:29:09.970 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2023-07-21 22:27:48.813445859 +0000 2023-07-21T22:29:09.971 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2023-07-21 22:27:48.813445859 +0000 2023-07-21T22:29:09.971 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2023-07-21 22:29:03.247623727 +0000 2023-07-21T22:29:09.971 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2023-07-21 22:27:48.813445859 +0000 2023-07-21T22:29:09.971 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-07-21T22:29:09.971 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-07-21T22:29:10.067 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:29:10.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.021+0000 7f9d0137c700 1 -- 172.21.15.107:0/2937957225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0fe9e0 msgr2=0x7f9cfc0fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.021+0000 7f9d0137c700 1 --2- 172.21.15.107:0/2937957225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0fe9e0 0x7f9cfc0fedf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009990 tx=0x7f9cec009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.021+0000 7f9d0137c700 1 -- 172.21.15.107:0/2937957225 shutdown_connections 2023-07-21T22:29:10.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.021+0000 7f9d0137c700 1 --2- 172.21.15.107:0/2937957225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cfc100560 0x7f9cfc106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.021+0000 7f9d0137c700 1 --2- 172.21.15.107:0/2937957225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0fe9e0 0x7f9cfc0fedf0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.021+0000 7f9d0137c700 1 --2- 172.21.15.107:0/2937957225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0ffbd0 0x7f9cfc100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.021+0000 7f9d0137c700 1 -- 172.21.15.107:0/2937957225 >> 172.21.15.107:0/2937957225 conn(0x7f9cfc0fa000 msgr2=0x7f9cfc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:10.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.022+0000 7f9d0137c700 1 -- 172.21.15.107:0/2937957225 shutdown_connections 2023-07-21T22:29:10.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.022+0000 7f9d0137c700 1 -- 172.21.15.107:0/2937957225 wait complete. 2023-07-21T22:29:10.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.022+0000 7f9d0137c700 1 Processor -- start 2023-07-21T22:29:10.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.022+0000 7f9d0137c700 1 -- start start 2023-07-21T22:29:10.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.022+0000 7f9d0137c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fe9e0 0x7f9cfc069270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.022+0000 7f9d0137c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0ffbd0 0x7f9cfc19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.022+0000 7f9d0137c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cfc100560 0x7f9cfc19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.022+0000 7f9d0137c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc19d3b0 con 0x7f9cfc0ffbd0 2023-07-21T22:29:10.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.022+0000 7f9d0137c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc19d520 con 0x7f9cfc0fe9e0 2023-07-21T22:29:10.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.022+0000 7f9d0137c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc19d6e0 con 0x7f9cfc100560 2023-07-21T22:29:10.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9cfa7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0ffbd0 0x7f9cfc19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9cfa7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0ffbd0 0x7f9cfc19a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48180/0 (socket says 172.21.15.107:48180) 2023-07-21T22:29:10.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9cfa7fc700 1 -- 172.21.15.107:0/2862248648 learned_addr learned my addr 172.21.15.107:0/2862248648 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:10.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9cfb7fe700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cfc100560 0x7f9cfc19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fe9e0 0x7f9cfc069270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9cfa7fc700 1 -- 172.21.15.107:0/2862248648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cfc100560 msgr2=0x7f9cfc19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9cfa7fc700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cfc100560 0x7f9cfc19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9cfa7fc700 1 -- 172.21.15.107:0/2862248648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fe9e0 msgr2=0x7f9cfc069270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9cfa7fc700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fe9e0 0x7f9cfc069270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9cfa7fc700 1 -- 172.21.15.107:0/2862248648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec0095f0 con 0x7f9cfc0ffbd0 2023-07-21T22:29:10.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9cfa7fc700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0ffbd0 0x7f9cfc19a580 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000e9a0 tx=0x7f9cf000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:10.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9ce3fff700 1 -- 172.21.15.107:0/2862248648 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0009790 con 0x7f9cfc0ffbd0 2023-07-21T22:29:10.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9d0137c700 1 -- 172.21.15.107:0/2862248648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cfc19d9c0 con 0x7f9cfc0ffbd0 2023-07-21T22:29:10.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9ce3fff700 1 -- 172.21.15.107:0/2862248648 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cf0010430 con 0x7f9cfc0ffbd0 2023-07-21T22:29:10.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9ce3fff700 1 -- 172.21.15.107:0/2862248648 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0018680 con 0x7f9cfc0ffbd0 2023-07-21T22:29:10.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.023+0000 7f9d0137c700 1 -- 172.21.15.107:0/2862248648 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cfc19ded0 con 0x7f9cfc0ffbd0 2023-07-21T22:29:10.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.025+0000 7f9ce3fff700 1 -- 172.21.15.107:0/2862248648 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f9cf00105a0 con 0x7f9cfc0ffbd0 2023-07-21T22:29:10.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.026+0000 7f9ce3fff700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9ce40a11e0 0x7f9ce40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.026+0000 7f9ce3fff700 1 -- 172.21.15.107:0/2862248648 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9cf0014070 con 0x7f9cfc0ffbd0 2023-07-21T22:29:10.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.026+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9ce40a11e0 0x7f9ce40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.027+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9ce40a11e0 0x7f9ce40a3690 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00b2e0 tx=0x7f9cec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:10.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.027+0000 7f9d0d501b80 1 Processor -- start 2023-07-21T22:29:10.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.027+0000 7f9d0d501b80 1 -- start start 2023-07-21T22:29:10.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.027+0000 7f9d0d501b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009ab2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.027+0000 7f9d0d501b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009ab14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.027+0000 7f9d0d501b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009ab1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.027+0000 7f9d0d501b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620099b6d30 con 0x562009ab1a90 2023-07-21T22:29:10.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.027+0000 7f9d0d501b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009a09ac0 con 0x562009ab28b0 2023-07-21T22:29:10.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.027+0000 7f9d0d501b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009a02f50 con 0x562009ab3220 2023-07-21T22:29:10.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.027+0000 7f9cfb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009ab1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9cfb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009ab1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48190/0 (socket says 172.21.15.107:48190) 2023-07-21T22:29:10.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9cfb7fe700 1 -- 172.21.15.107:0/41102941 learned_addr learned my addr 172.21.15.107:0/41102941 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:10.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9cfa7fc700 1 --2- 172.21.15.107:0/41102941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009ab14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/41102941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009ab2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9cfb7fe700 1 -- 172.21.15.107:0/41102941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 msgr2=0x562009ab14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9cfb7fe700 1 --2- 172.21.15.107:0/41102941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009ab14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9cfb7fe700 1 -- 172.21.15.107:0/41102941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 msgr2=0x562009ab2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9cfb7fe700 1 --2- 172.21.15.107:0/41102941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009ab2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9cfb7fe700 1 -- 172.21.15.107:0/41102941 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562009830e60 con 0x562009ab1a90 2023-07-21T22:29:10.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9cfa7fc700 1 --2- 172.21.15.107:0/41102941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009ab14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:10.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/41102941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009ab2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:10.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9cfb7fe700 1 --2- 172.21.15.107:0/41102941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009ab1f10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400a730 tx=0x7f9cf40078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0316139330a0dfd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:10.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9ce17fa700 1 -- 172.21.15.107:0/41102941 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4018070 con 0x562009ab1a90 2023-07-21T22:29:10.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9ce17fa700 1 -- 172.21.15.107:0/41102941 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cf4010040 con 0x562009ab1a90 2023-07-21T22:29:10.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.028+0000 7f9ce17fa700 1 -- 172.21.15.107:0/41102941 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf400f050 con 0x562009ab1a90 2023-07-21T22:29:10.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.029+0000 7f9d0d501b80 1 -- 172.21.15.107:0/41102941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 msgr2=0x562009ab1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.029+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/41102941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009ab1f10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400a730 tx=0x7f9cf40078d0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.029+0000 7f9d0d501b80 1 -- 172.21.15.107:0/41102941 shutdown_connections 2023-07-21T22:29:10.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.029+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/41102941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009ab1f10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.029+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/41102941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009ab2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.029+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/41102941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009ab14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.029+0000 7f9d0d501b80 1 -- 172.21.15.107:0/41102941 >> 172.21.15.107:0/41102941 conn(0x562009aa6f70 msgr2=0x562009aac3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:10.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.029+0000 7f9d0d501b80 1 -- 172.21.15.107:0/41102941 shutdown_connections 2023-07-21T22:29:10.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.029+0000 7f9d0d501b80 1 -- 172.21.15.107:0/41102941 wait complete. 2023-07-21T22:29:10.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.029+0000 7f9d0d501b80 1 -- start start 2023-07-21T22:29:10.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.029+0000 7f9d0d501b80 1 Processor -- start 2023-07-21T22:29:10.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9d0d501b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009abb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9d0d501b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009abb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9d0d501b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009ac25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9d0d501b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620099a2b40 con 0x562009ab1a90 2023-07-21T22:29:10.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9d0d501b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620099a2840 con 0x562009ab28b0 2023-07-21T22:29:10.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9d0d501b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620098869a0 con 0x562009ab3220 2023-07-21T22:29:10.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9d0d501b80 10 client.0 _ll_register_callbacks cb 0x5620099a07e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:10.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9cfaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009abb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9cfaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009abb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48192/0 (socket says 172.21.15.107:48192) 2023-07-21T22:29:10.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9cfaffd700 1 -- 172.21.15.107:0/2575781719 learned_addr learned my addr 172.21.15.107:0/2575781719 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:10.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9cfb7fe700 1 --2- 172.21.15.107:0/2575781719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009ac25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9cfa7fc700 1 --2- 172.21.15.107:0/2575781719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009abb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9cfaffd700 1 -- 172.21.15.107:0/2575781719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 msgr2=0x562009ac25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/2575781719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009ac25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9cfaffd700 1 -- 172.21.15.107:0/2575781719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 msgr2=0x562009abb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/2575781719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009abb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9cfaffd700 1 -- 172.21.15.107:0/2575781719 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf400a410 con 0x562009ab1a90 2023-07-21T22:29:10.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/2575781719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009abb060 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0ffa30 tx=0x7f9cec01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd831a9a7e862374 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:10.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.030+0000 7f9cdeff5700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:10.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9cdeff5700 1 -- 172.21.15.107:0/2575781719 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec030420 con 0x562009ab1a90 2023-07-21T22:29:10.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9d0d501b80 1 -- 172.21.15.107:0/2575781719 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56200995c570 con 0x562009ab1a90 2023-07-21T22:29:10.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9cdeff5700 1 -- 172.21.15.107:0/2575781719 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9cec030580 con 0x562009ab1a90 2023-07-21T22:29:10.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9cdeff5700 1 -- 172.21.15.107:0/2575781719 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cec030740 con 0x562009ab1a90 2023-07-21T22:29:10.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9d0d501b80 10 client.15477 fetch_fsmap learned FSMap version 27 2023-07-21T22:29:10.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9d0d501b80 1 -- 172.21.15.107:0/2575781719 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562009819ce0 con 0x562009ab1a90 2023-07-21T22:29:10.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9cdeff5700 1 -- 172.21.15.107:0/2575781719 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cec03f080 con 0x562009ab1a90 2023-07-21T22:29:10.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9cdeff5700 1 -- 172.21.15.107:0/2575781719 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9cec039030 con 0x562009ab1a90 2023-07-21T22:29:10.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9d0d501b80 10 client.15477 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:29:10.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9d0d501b80 10 client.15477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:10.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9d0d501b80 20 client.15477 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:10.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/2575781719 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562009ac6db0 0x562009ac91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9d0d501b80 4 client.15477 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:10.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9d0d501b80 1 -- 172.21.15.107:0/2575781719 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620099fbf30 con 0x562009ac6db0 2023-07-21T22:29:10.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.031+0000 7f9cfa7fc700 1 --2- 172.21.15.107:0/2575781719 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562009ac6db0 0x562009ac91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.032+0000 7f9cdeff5700 10 client.15477 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:10.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.032+0000 7f9cfa7fc700 1 --2- 172.21.15.107:0/2575781719 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562009ac6db0 0x562009ac91c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b55fbc446b16687 server_cookie=5e7583897048c9bc in_seq=0 out_seq=0 2023-07-21T22:29:10.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.032+0000 7f9cdeff5700 1 -- 172.21.15.107:0/2575781719 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5620099fbf30 con 0x562009ac6db0 2023-07-21T22:29:10.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.032+0000 7f9cdeff5700 10 client.15477 handle_command_reply: tid=0 2023-07-21T22:29:10.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.032+0000 7f9d0d501b80 1 client.15477 shutdown 2023-07-21T22:29:10.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.033+0000 7f9d0d501b80 1 -- 172.21.15.107:0/2575781719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 msgr2=0x562009abb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.033+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/2575781719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009abb060 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0ffa30 tx=0x7f9cec01a050 comp rx=0 tx=0).stop 2023-07-21T22:29:10.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.033+0000 7f9d0d501b80 1 -- 172.21.15.107:0/2575781719 shutdown_connections 2023-07-21T22:29:10.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.033+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/2575781719 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562009ac6db0 0x562009ac91c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.033+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/2575781719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009ac25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.095 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.033+0000 7f9cdeff5700 0 client.15477 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:10.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.033+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/2575781719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009abb060 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.033+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/2575781719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009abb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.033+0000 7f9d0d501b80 1 -- 172.21.15.107:0/2575781719 >> 172.21.15.107:0/2575781719 conn(0x562009aa6f70 msgr2=0x562009aaba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:10.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.033+0000 7f9d0d501b80 1 -- 172.21.15.107:0/2575781719 shutdown_connections 2023-07-21T22:29:10.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.033+0000 7f9d0d501b80 1 -- 172.21.15.107:0/2575781719 wait complete. 2023-07-21T22:29:10.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.033+0000 7f9d0d501b80 20 client.15477 trim_cache size 0 max 16384 2023-07-21T22:29:10.097 INFO:tasks.workunit.client.0.smithi107.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-07-21T22:29:10.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.050+0000 7f9d0d501b80 1 Processor -- start 2023-07-21T22:29:10.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.050+0000 7f9d0d501b80 1 -- start start 2023-07-21T22:29:10.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9d0d501b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009ab13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9d0d501b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009ab9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9d0d501b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009ab8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9d0d501b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009abac30 con 0x562009ab1a90 2023-07-21T22:29:10.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9d0d501b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009aa9090 con 0x562009ab28b0 2023-07-21T22:29:10.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9d0d501b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009ac6b00 con 0x562009ab3220 2023-07-21T22:29:10.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9cfaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009ab13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9cfaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009ab13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48200/0 (socket says 172.21.15.107:48200) 2023-07-21T22:29:10.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9cfaffd700 1 -- 172.21.15.107:0/3054270590 learned_addr learned my addr 172.21.15.107:0/3054270590 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:10.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9cfb7fe700 1 --2- 172.21.15.107:0/3054270590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009ab8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9cfa7fc700 1 --2- 172.21.15.107:0/3054270590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009ab9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9cfaffd700 1 -- 172.21.15.107:0/3054270590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 msgr2=0x562009ab8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/3054270590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009ab8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9cfaffd700 1 -- 172.21.15.107:0/3054270590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 msgr2=0x562009ab9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/3054270590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009ab9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9cfaffd700 1 -- 172.21.15.107:0/3054270590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620098572d0 con 0x562009ab1a90 2023-07-21T22:29:10.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/3054270590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009ab13d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0098e0 tx=0x7f9cec024c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53558362d9330ea server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:10.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9ce1ffb700 1 -- 172.21.15.107:0/3054270590 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0055f0 con 0x562009ab1a90 2023-07-21T22:29:10.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9ce1ffb700 1 -- 172.21.15.107:0/3054270590 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56200995c570 con 0x562009ab1a90 2023-07-21T22:29:10.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.051+0000 7f9ce1ffb700 1 -- 172.21.15.107:0/3054270590 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0308a0 con 0x562009ab1a90 2023-07-21T22:29:10.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.052+0000 7f9d0d501b80 1 -- 172.21.15.107:0/3054270590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 msgr2=0x562009ab13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.052+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/3054270590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009ab13d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0098e0 tx=0x7f9cec024c00 comp rx=0 tx=0).stop 2023-07-21T22:29:10.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.052+0000 7f9d0d501b80 1 -- 172.21.15.107:0/3054270590 shutdown_connections 2023-07-21T22:29:10.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.052+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/3054270590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009ab8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.052+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/3054270590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009ab13d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.052+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/3054270590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009ab9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.052+0000 7f9d0d501b80 1 -- 172.21.15.107:0/3054270590 >> 172.21.15.107:0/3054270590 conn(0x562009aa6f70 msgr2=0x562009ac2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:10.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.052+0000 7f9d0d501b80 1 -- 172.21.15.107:0/3054270590 shutdown_connections 2023-07-21T22:29:10.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.052+0000 7f9d0d501b80 1 -- 172.21.15.107:0/3054270590 wait complete. 2023-07-21T22:29:10.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.052+0000 7f9d0d501b80 1 -- start start 2023-07-21T22:29:10.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9d0d501b80 1 Processor -- start 2023-07-21T22:29:10.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9d0d501b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009b17550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9d0d501b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009b17a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9d0d501b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009b1ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9d0d501b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009b1f010 con 0x562009ab1a90 2023-07-21T22:29:10.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9d0d501b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009b1f150 con 0x562009ab28b0 2023-07-21T22:29:10.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9d0d501b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562009b1f2e0 con 0x562009ab3220 2023-07-21T22:29:10.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9d0d501b80 10 client.0 _ll_register_callbacks cb 0x7f9cec00bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:10.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cfaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009b17550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cfaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009b17550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48206/0 (socket says 172.21.15.107:48206) 2023-07-21T22:29:10.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cfaffd700 1 -- 172.21.15.107:0/1490637770 learned_addr learned my addr 172.21.15.107:0/1490637770 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:10.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cfa7fc700 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009b17a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cfb7fe700 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009b1ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cfaffd700 1 -- 172.21.15.107:0/1490637770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 msgr2=0x562009b1ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009b1ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cfaffd700 1 -- 172.21.15.107:0/1490637770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 msgr2=0x562009b17a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009b17a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cfaffd700 1 -- 172.21.15.107:0/1490637770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562009819ce0 con 0x562009ab1a90 2023-07-21T22:29:10.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cfb7fe700 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009b1ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:10.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cfaffd700 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009b17550 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9cec005920 tx=0x7f9cec005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72a435e72b49412 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:10.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cdeff5700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:10.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.053+0000 7f9cdeff5700 1 -- 172.21.15.107:0/1490637770 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec024620 con 0x562009ab1a90 2023-07-21T22:29:10.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9cdeff5700 1 -- 172.21.15.107:0/1490637770 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9cec024780 con 0x562009ab1a90 2023-07-21T22:29:10.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9d0d501b80 1 -- 172.21.15.107:0/1490637770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cf4010040 con 0x562009ab1a90 2023-07-21T22:29:10.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9cdeff5700 1 -- 172.21.15.107:0/1490637770 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cec0307e0 con 0x562009ab1a90 2023-07-21T22:29:10.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9d0d501b80 10 client.15483 fetch_fsmap learned FSMap version 27 2023-07-21T22:29:10.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9d0d501b80 1 -- 172.21.15.107:0/1490637770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56200982f330 con 0x562009ab1a90 2023-07-21T22:29:10.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9cdeff5700 1 -- 172.21.15.107:0/1490637770 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cf4010040 con 0x562009ab1a90 2023-07-21T22:29:10.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9cdeff5700 1 -- 172.21.15.107:0/1490637770 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9cec04f020 con 0x562009ab1a90 2023-07-21T22:29:10.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9d0d501b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:29:10.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9d0d501b80 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:10.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9d0d501b80 20 client.15483 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:10.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562009ac6db0 0x562009b22870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9d0d501b80 4 client.15483 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:29:10.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9d0d501b80 1 -- 172.21.15.107:0/1490637770 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5620099be0d0 con 0x562009ac6db0 2023-07-21T22:29:10.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.054+0000 7f9cfa7fc700 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562009ac6db0 0x562009b22870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.055+0000 7f9cfa7fc700 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562009ac6db0 0x562009b22870 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b62687de628669 server_cookie=261669777a734d69 in_seq=0 out_seq=0 2023-07-21T22:29:10.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.055+0000 7f9cdeff5700 10 client.15483 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:10.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9cdeff5700 1 -- 172.21.15.107:0/1490637770 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5620099be0d0 con 0x562009ac6db0 2023-07-21T22:29:10.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9cdeff5700 10 client.15483 handle_command_reply: tid=0 2023-07-21T22:29:10.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9d0d501b80 1 client.15483 shutdown 2023-07-21T22:29:10.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9d0d501b80 1 -- 172.21.15.107:0/1490637770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 msgr2=0x562009b17550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009b17550 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9cec005920 tx=0x7f9cec005660 comp rx=0 tx=0).stop 2023-07-21T22:29:10.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9d0d501b80 1 -- 172.21.15.107:0/1490637770 shutdown_connections 2023-07-21T22:29:10.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562009ac6db0 0x562009b22870 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562009ab3220 0x562009b1ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562009ab1a90 0x562009b17550 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9d0d501b80 1 --2- 172.21.15.107:0/1490637770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562009ab28b0 0x562009b17a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9cdeff5700 0 client.15483 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:10.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9d0d501b80 1 -- 172.21.15.107:0/1490637770 >> 172.21.15.107:0/1490637770 conn(0x562009aa6f70 msgr2=0x562009ab9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:10.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9d0d501b80 1 -- 172.21.15.107:0/1490637770 shutdown_connections 2023-07-21T22:29:10.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9d0d501b80 1 -- 172.21.15.107:0/1490637770 wait complete. 2023-07-21T22:29:10.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.056+0000 7f9d0d501b80 20 client.15483 trim_cache size 0 max 16384 2023-07-21T22:29:10.119 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:29:10.120 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2023-07-21T22:29:10.120 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "2d8eb7f5-addb-4d3b-aa0f-c3f332cafbab", 2023-07-21T22:29:10.120 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2023-07-21T22:29:10.120 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:29:10.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.057+0000 7f9d0137c700 1 -- 172.21.15.107:0/2862248648 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9ce40a11e0 msgr2=0x7f9ce40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.057+0000 7f9d0137c700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9ce40a11e0 0x7f9ce40a3690 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00b2e0 tx=0x7f9cec005fb0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.057+0000 7f9d0137c700 1 -- 172.21.15.107:0/2862248648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0ffbd0 msgr2=0x7f9cfc19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.057+0000 7f9d0137c700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0ffbd0 0x7f9cfc19a580 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000e9a0 tx=0x7f9cf000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.057+0000 7f9d0137c700 1 -- 172.21.15.107:0/2862248648 shutdown_connections 2023-07-21T22:29:10.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.057+0000 7f9d0137c700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9ce40a11e0 0x7f9ce40a3690 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.057+0000 7f9d0137c700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9cfc100560 0x7f9cfc19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.057+0000 7f9d0137c700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9cfc0fe9e0 0x7f9cfc069270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.057+0000 7f9d0137c700 1 --2- 172.21.15.107:0/2862248648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cfc0ffbd0 0x7f9cfc19a580 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.057+0000 7f9d0137c700 1 -- 172.21.15.107:0/2862248648 >> 172.21.15.107:0/2862248648 conn(0x7f9cfc0fa000 msgr2=0x7f9cfc0fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:10.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.057+0000 7f9d0137c700 1 -- 172.21.15.107:0/2862248648 shutdown_connections 2023-07-21T22:29:10.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.057+0000 7f9d0137c700 1 -- 172.21.15.107:0/2862248648 wait complete. 2023-07-21T22:29:10.123 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:29:10.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:10 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1766209796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:29:10.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:10 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/272937134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:29:10.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:10 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/203538997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:29:10.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.243+0000 7f3ef6416700 1 -- 172.21.15.107:0/1995748308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef0100ce0 msgr2=0x7f3ef0101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.243+0000 7f3ef6416700 1 --2- 172.21.15.107:0/1995748308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef0100ce0 0x7f3ef0101130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009990 tx=0x7f3ee4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.243+0000 7f3ef6416700 1 -- 172.21.15.107:0/1995748308 shutdown_connections 2023-07-21T22:29:10.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.243+0000 7f3ef6416700 1 --2- 172.21.15.107:0/1995748308 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef0101670 0x7f3ef0107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.243+0000 7f3ef6416700 1 --2- 172.21.15.107:0/1995748308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ef00ffaf0 0x7f3ef00fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.243+0000 7f3ef6416700 1 --2- 172.21.15.107:0/1995748308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef0100ce0 0x7f3ef0101130 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.243+0000 7f3ef6416700 1 -- 172.21.15.107:0/1995748308 >> 172.21.15.107:0/1995748308 conn(0x7f3ef00fb130 msgr2=0x7f3ef00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:10.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.243+0000 7f3ef6416700 1 -- 172.21.15.107:0/1995748308 shutdown_connections 2023-07-21T22:29:10.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.243+0000 7f3ef6416700 1 -- 172.21.15.107:0/1995748308 wait complete. 2023-07-21T22:29:10.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3ef6416700 1 Processor -- start 2023-07-21T22:29:10.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3ef6416700 1 -- start start 2023-07-21T22:29:10.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3ef6416700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ef00ffaf0 0x7f3ef0195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3ef6416700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef0100ce0 0x7f3ef0195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3ef6416700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef0101670 0x7f3ef019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3ef6416700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef019d3c0 con 0x7f3ef0101670 2023-07-21T22:29:10.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3ef6416700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef019d500 con 0x7f3ef00ffaf0 2023-07-21T22:29:10.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3ef6416700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef019d690 con 0x7f3ef0100ce0 2023-07-21T22:29:10.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3ef49b3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef0101670 0x7f3ef019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3ef49b3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef0101670 0x7f3ef019cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48234/0 (socket says 172.21.15.107:48234) 2023-07-21T22:29:10.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3ef49b3700 1 -- 172.21.15.107:0/4289158692 learned_addr learned my addr 172.21.15.107:0/4289158692 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:10.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3eeffff700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ef00ffaf0 0x7f3ef0195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3eef7fe700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef0100ce0 0x7f3ef0195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.244+0000 7f3ef49b3700 1 -- 172.21.15.107:0/4289158692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef0100ce0 msgr2=0x7f3ef0195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.245+0000 7f3ef49b3700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef0100ce0 0x7f3ef0195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.245+0000 7f3ef49b3700 1 -- 172.21.15.107:0/4289158692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ef00ffaf0 msgr2=0x7f3ef0195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.245+0000 7f3ef49b3700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ef00ffaf0 0x7f3ef0195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.245+0000 7f3ef49b3700 1 -- 172.21.15.107:0/4289158692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee40095f0 con 0x7f3ef0101670 2023-07-21T22:29:10.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.245+0000 7f3ef49b3700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef0101670 0x7f3ef019cdf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800ada0 tx=0x7f3ee800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:10.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.245+0000 7f3eed7fa700 1 -- 172.21.15.107:0/4289158692 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800f070 con 0x7f3ef0101670 2023-07-21T22:29:10.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.245+0000 7f3ef6416700 1 -- 172.21.15.107:0/4289158692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef019d970 con 0x7f3ef0101670 2023-07-21T22:29:10.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.245+0000 7f3eed7fa700 1 -- 172.21.15.107:0/4289158692 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee8007b30 con 0x7f3ef0101670 2023-07-21T22:29:10.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.245+0000 7f3eed7fa700 1 -- 172.21.15.107:0/4289158692 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80053e0 con 0x7f3ef0101670 2023-07-21T22:29:10.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.245+0000 7f3ef6416700 1 -- 172.21.15.107:0/4289158692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef019ded0 con 0x7f3ef0101670 2023-07-21T22:29:10.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.247+0000 7f3eed7fa700 1 -- 172.21.15.107:0/4289158692 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3ee800c980 con 0x7f3ef0101670 2023-07-21T22:29:10.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.247+0000 7f3eed7fa700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3ed80a11e0 0x7f3ed80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.248+0000 7f3eeffff700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3ed80a11e0 0x7f3ed80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.248+0000 7f3eed7fa700 1 -- 172.21.15.107:0/4289158692 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3ee80cbfc0 con 0x7f3ef0101670 2023-07-21T22:29:10.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.248+0000 7f3eeffff700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3ed80a11e0 0x7f3ed80a3690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0100b40 tx=0x7f3ee000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:10.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.248+0000 7f3f0259bb80 1 Processor -- start 2023-07-21T22:29:10.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.248+0000 7f3f0259bb80 1 -- start start 2023-07-21T22:29:10.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.248+0000 7f3f0259bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 0x5622e36edc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.248+0000 7f3f0259bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36ec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.248+0000 7f3f0259bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36eca10 0x5622e36ece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.248+0000 7f3f0259bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e35f1cb0 con 0x5622e36ee1a0 2023-07-21T22:29:10.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.248+0000 7f3f0259bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e3644a40 con 0x5622e36ed830 2023-07-21T22:29:10.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.248+0000 7f3f0259bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e363ded0 con 0x5622e36eca10 2023-07-21T22:29:10.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3eef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3eef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36ec430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48242/0 (socket says 172.21.15.107:48242) 2023-07-21T22:29:10.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3eef7fe700 1 -- 172.21.15.107:0/3036225428 learned_addr learned my addr 172.21.15.107:0/3036225428 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:10.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3eeffff700 1 --2- 172.21.15.107:0/3036225428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 0x5622e36edc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3ef49b3700 1 --2- 172.21.15.107:0/3036225428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36eca10 0x5622e36ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3eef7fe700 1 -- 172.21.15.107:0/3036225428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36eca10 msgr2=0x5622e36ece90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3eef7fe700 1 --2- 172.21.15.107:0/3036225428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36eca10 0x5622e36ece90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3eef7fe700 1 -- 172.21.15.107:0/3036225428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 msgr2=0x5622e36edc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3eef7fe700 1 --2- 172.21.15.107:0/3036225428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 0x5622e36edc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3eef7fe700 1 -- 172.21.15.107:0/3036225428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622e346bde0 con 0x5622e36ee1a0 2023-07-21T22:29:10.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3eef7fe700 1 --2- 172.21.15.107:0/3036225428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36ec430 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4005ca0 tx=0x7f3ee4009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e257c7c458ac699 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:10.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3ed67fc700 1 -- 172.21.15.107:0/3036225428 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40053c0 con 0x5622e36ee1a0 2023-07-21T22:29:10.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3ed67fc700 1 -- 172.21.15.107:0/3036225428 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee401e040 con 0x5622e36ee1a0 2023-07-21T22:29:10.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.249+0000 7f3ed67fc700 1 -- 172.21.15.107:0/3036225428 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4004830 con 0x5622e36ee1a0 2023-07-21T22:29:10.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.250+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3036225428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 msgr2=0x5622e36ec430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.250+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3036225428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36ec430 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4005ca0 tx=0x7f3ee4009ee0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.250+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3036225428 shutdown_connections 2023-07-21T22:29:10.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.250+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3036225428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36eca10 0x5622e36ece90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.250+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3036225428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 0x5622e36edc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.250+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3036225428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36ec430 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.250+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3036225428 >> 172.21.15.107:0/3036225428 conn(0x5622e36e1ef0 msgr2=0x5622e36e7360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:10.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.250+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3036225428 shutdown_connections 2023-07-21T22:29:10.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.250+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3036225428 wait complete. 2023-07-21T22:29:10.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.250+0000 7f3f0259bb80 1 -- start start 2023-07-21T22:29:10.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3f0259bb80 1 Processor -- start 2023-07-21T22:29:10.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3f0259bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36eca10 0x5622e36f6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3f0259bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 0x5622e36f6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3f0259bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36fd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3f0259bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e35ddac0 con 0x5622e36ee1a0 2023-07-21T22:29:10.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3f0259bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e35dd7c0 con 0x5622e36ed830 2023-07-21T22:29:10.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3f0259bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e34c1920 con 0x5622e36eca10 2023-07-21T22:29:10.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3f0259bb80 10 client.0 _ll_register_callbacks cb 0x5622e35db760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:10.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3ef49b3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36fd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3ef49b3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36fd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48248/0 (socket says 172.21.15.107:48248) 2023-07-21T22:29:10.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3ef49b3700 1 -- 172.21.15.107:0/3251025448 learned_addr learned my addr 172.21.15.107:0/3251025448 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:10.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3eef7fe700 1 --2- 172.21.15.107:0/3251025448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 0x5622e36f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3eeffff700 1 --2- 172.21.15.107:0/3251025448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36eca10 0x5622e36f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3ef49b3700 1 -- 172.21.15.107:0/3251025448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36eca10 msgr2=0x5622e36f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3ef49b3700 1 --2- 172.21.15.107:0/3251025448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36eca10 0x5622e36f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3ef49b3700 1 -- 172.21.15.107:0/3251025448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 msgr2=0x5622e36f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3ef49b3700 1 --2- 172.21.15.107:0/3251025448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 0x5622e36f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.251+0000 7f3ef49b3700 1 -- 172.21.15.107:0/3251025448 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef0108860 con 0x5622e36ee1a0 2023-07-21T22:29:10.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3ef49b3700 1 --2- 172.21.15.107:0/3251025448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36fd5d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8095880 tx=0x7f3ee8004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d041fb2df69e8b68 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:10.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3ecf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:10.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3ecf7fe700 1 -- 172.21.15.107:0/3251025448 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80840c0 con 0x5622e36ee1a0 2023-07-21T22:29:10.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3ecf7fe700 1 -- 172.21.15.107:0/3251025448 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3ee8099380 con 0x5622e36ee1a0 2023-07-21T22:29:10.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3251025448 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622e35974f0 con 0x5622e36ee1a0 2023-07-21T22:29:10.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3ecf7fe700 1 -- 172.21.15.107:0/3251025448 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ee809d3b0 con 0x5622e36ee1a0 2023-07-21T22:29:10.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3f0259bb80 10 client.15495 fetch_fsmap learned FSMap version 27 2023-07-21T22:29:10.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3251025448 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5622e3454c60 con 0x5622e36ee1a0 2023-07-21T22:29:10.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3ecf7fe700 1 -- 172.21.15.107:0/3251025448 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5622e35974f0 con 0x5622e36ee1a0 2023-07-21T22:29:10.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3ecf7fe700 1 -- 172.21.15.107:0/3251025448 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3ee80995e0 con 0x5622e36ee1a0 2023-07-21T22:29:10.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3f0259bb80 10 client.15495 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:29:10.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3f0259bb80 10 client.15495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:10.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3f0259bb80 20 client.15495 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:10.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3251025448 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5622e3701dd0 0x5622e37041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3f0259bb80 4 client.15495 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:10.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.252+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3251025448 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622e3636eb0 con 0x5622e3701dd0 2023-07-21T22:29:10.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.253+0000 7f3eeffff700 1 --2- 172.21.15.107:0/3251025448 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5622e3701dd0 0x5622e37041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.253+0000 7f3ecf7fe700 10 client.15495 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:10.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.253+0000 7f3eeffff700 1 --2- 172.21.15.107:0/3251025448 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5622e3701dd0 0x5622e37041e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60f035aef0e97b9e server_cookie=c9c753b25375a6ea in_seq=0 out_seq=0 2023-07-21T22:29:10.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3ecf7fe700 1 -- 172.21.15.107:0/3251025448 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5622e3636eb0 con 0x5622e3701dd0 2023-07-21T22:29:10.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3ecf7fe700 10 client.15495 handle_command_reply: tid=0 2023-07-21T22:29:10.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3f0259bb80 1 client.15495 shutdown 2023-07-21T22:29:10.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3251025448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 msgr2=0x5622e36fd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3251025448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36fd5d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8095880 tx=0x7f3ee8004b80 comp rx=0 tx=0).stop 2023-07-21T22:29:10.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3251025448 shutdown_connections 2023-07-21T22:29:10.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3251025448 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5622e3701dd0 0x5622e37041e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3251025448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36fd5d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3251025448 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36eca10 0x5622e36f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3ecf7fe700 0 client.15495 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:10.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3251025448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 0x5622e36f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3251025448 >> 172.21.15.107:0/3251025448 conn(0x5622e36e1ef0 msgr2=0x5622e36e6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:10.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3251025448 shutdown_connections 2023-07-21T22:29:10.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3251025448 wait complete. 2023-07-21T22:29:10.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.254+0000 7f3f0259bb80 20 client.15495 trim_cache size 0 max 16384 2023-07-21T22:29:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.270+0000 7f3f0259bb80 1 Processor -- start 2023-07-21T22:29:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3f0259bb80 1 -- start start 2023-07-21T22:29:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3f0259bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36eca10 0x5622e36f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3f0259bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36ed830 0x5622e36fd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3f0259bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3f0259bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e36e8060 con 0x5622e36ee1a0 2023-07-21T22:29:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3f0259bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e36f3d30 con 0x5622e36eca10 2023-07-21T22:29:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3f0259bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e36e8540 con 0x5622e36ed830 2023-07-21T22:29:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3ef49b3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3ef49b3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36f0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48256/0 (socket says 172.21.15.107:48256) 2023-07-21T22:29:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3ef49b3700 1 -- 172.21.15.107:0/3364377403 learned_addr learned my addr 172.21.15.107:0/3364377403 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3eeffff700 1 --2- 172.21.15.107:0/3364377403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36eca10 0x5622e36f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3eef7fe700 1 --2- 172.21.15.107:0/3364377403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36ed830 0x5622e36fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3ef49b3700 1 -- 172.21.15.107:0/3364377403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36ed830 msgr2=0x5622e36fd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3ef49b3700 1 --2- 172.21.15.107:0/3364377403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36ed830 0x5622e36fd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3ef49b3700 1 -- 172.21.15.107:0/3364377403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36eca10 msgr2=0x5622e36f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3ef49b3700 1 --2- 172.21.15.107:0/3364377403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36eca10 0x5622e36f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3ef49b3700 1 -- 172.21.15.107:0/3364377403 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622e3492250 con 0x5622e36ee1a0 2023-07-21T22:29:10.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3eef7fe700 1 --2- 172.21.15.107:0/3364377403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36ed830 0x5622e36fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:10.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.271+0000 7f3ef49b3700 1 --2- 172.21.15.107:0/3364377403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36f0ad0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800cfd0 tx=0x7f3ee8093f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58db91498ad1fd7d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:10.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.272+0000 7f3ed6ffd700 1 -- 172.21.15.107:0/3364377403 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80840b0 con 0x5622e36ee1a0 2023-07-21T22:29:10.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.272+0000 7f3ed6ffd700 1 -- 172.21.15.107:0/3364377403 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee80b0ec0 con 0x5622e36ee1a0 2023-07-21T22:29:10.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.272+0000 7f3ed6ffd700 1 -- 172.21.15.107:0/3364377403 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80a1740 con 0x5622e36ee1a0 2023-07-21T22:29:10.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.272+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3364377403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 msgr2=0x5622e36f0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.272+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3364377403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36f0ad0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800cfd0 tx=0x7f3ee8093f90 comp rx=0 tx=0).stop 2023-07-21T22:29:10.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.272+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3364377403 shutdown_connections 2023-07-21T22:29:10.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.272+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3364377403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36ee1a0 0x5622e36f0ad0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.272+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3364377403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36eca10 0x5622e36f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.272+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3364377403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36ed830 0x5622e36fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.272+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3364377403 >> 172.21.15.107:0/3364377403 conn(0x5622e36e1ef0 msgr2=0x5622e36ecde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:10.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.272+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3364377403 shutdown_connections 2023-07-21T22:29:10.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.272+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3364377403 wait complete. 2023-07-21T22:29:10.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3f0259bb80 1 -- start start 2023-07-21T22:29:10.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3f0259bb80 1 Processor -- start 2023-07-21T22:29:10.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3f0259bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36eca10 0x5622e36fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.326 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3f0259bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 0x5622e36f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3f0259bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36ee1a0 0x5622e36fa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3f0259bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e36faab0 con 0x5622e36eca10 2023-07-21T22:29:10.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3f0259bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e3755c80 con 0x5622e36ed830 2023-07-21T22:29:10.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3f0259bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622e3755dc0 con 0x5622e36ee1a0 2023-07-21T22:29:10.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3f0259bb80 10 client.0 _ll_register_callbacks cb 0x7f3ee8098ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:10.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3eeffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36eca10 0x5622e36fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3eeffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36eca10 0x5622e36fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48258/0 (socket says 172.21.15.107:48258) 2023-07-21T22:29:10.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3eeffff700 1 -- 172.21.15.107:0/3247511279 learned_addr learned my addr 172.21.15.107:0/3247511279 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:10.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3eef7fe700 1 --2- 172.21.15.107:0/3247511279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 0x5622e36f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.273+0000 7f3ef49b3700 1 --2- 172.21.15.107:0/3247511279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36ee1a0 0x5622e36fa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3eeffff700 1 -- 172.21.15.107:0/3247511279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36ee1a0 msgr2=0x5622e36fa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3eeffff700 1 --2- 172.21.15.107:0/3247511279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36ee1a0 0x5622e36fa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3eeffff700 1 -- 172.21.15.107:0/3247511279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 msgr2=0x5622e36f9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:10.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3eeffff700 1 --2- 172.21.15.107:0/3247511279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 0x5622e36f9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3eeffff700 1 -- 172.21.15.107:0/3247511279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622e3454c60 con 0x5622e36eca10 2023-07-21T22:29:10.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3eeffff700 1 --2- 172.21.15.107:0/3247511279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36eca10 0x5622e36fd3f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x5622e3704af0 tx=0x7f3ee0013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7961c3241d4758b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:10.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3ecf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:10.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3ecf7fe700 1 -- 172.21.15.107:0/3247511279 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0004220 con 0x5622e36eca10 2023-07-21T22:29:10.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3ecf7fe700 1 -- 172.21.15.107:0/3247511279 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3ee0017070 con 0x5622e36eca10 2023-07-21T22:29:10.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3247511279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ee401e040 con 0x5622e36eca10 2023-07-21T22:29:10.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3ecf7fe700 1 -- 172.21.15.107:0/3247511279 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ee0027600 con 0x5622e36eca10 2023-07-21T22:29:10.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3f0259bb80 10 client.15501 fetch_fsmap learned FSMap version 27 2023-07-21T22:29:10.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3247511279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5622e346a2b0 con 0x5622e36eca10 2023-07-21T22:29:10.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3ecf7fe700 1 -- 172.21.15.107:0/3247511279 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee401e040 con 0x5622e36eca10 2023-07-21T22:29:10.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3ecf7fe700 1 -- 172.21.15.107:0/3247511279 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3ee0038050 con 0x5622e36eca10 2023-07-21T22:29:10.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3f0259bb80 10 client.15501 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:29:10.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3f0259bb80 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:10.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3f0259bb80 20 client.15501 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:10.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3247511279 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5622e3757880 0x5622e3759c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:10.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3f0259bb80 4 client.15501 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:29:10.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.274+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3247511279 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5622e35f9050 con 0x5622e3757880 2023-07-21T22:29:10.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3eef7fe700 1 --2- 172.21.15.107:0/3247511279 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5622e3757880 0x5622e3759c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:10.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3ecf7fe700 10 client.15501 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:10.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3eef7fe700 1 --2- 172.21.15.107:0/3247511279 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5622e3757880 0x5622e3759c90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd5cf0f7cc3065b6 server_cookie=867cf458d12a4093 in_seq=0 out_seq=0 2023-07-21T22:29:10.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3ecf7fe700 1 -- 172.21.15.107:0/3247511279 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5622e35f9050 con 0x5622e3757880 2023-07-21T22:29:10.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3ecf7fe700 10 client.15501 handle_command_reply: tid=0 2023-07-21T22:29:10.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3f0259bb80 1 client.15501 shutdown 2023-07-21T22:29:10.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3247511279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36eca10 msgr2=0x5622e36fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3247511279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36eca10 0x5622e36fd3f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x5622e3704af0 tx=0x7f3ee0013070 comp rx=0 tx=0).stop 2023-07-21T22:29:10.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3247511279 shutdown_connections 2023-07-21T22:29:10.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3247511279 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5622e3757880 0x5622e3759c90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3247511279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5622e36ee1a0 0x5622e36fa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3ecf7fe700 0 client.15501 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:10.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3247511279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622e36eca10 0x5622e36fd3f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.275+0000 7f3f0259bb80 1 --2- 172.21.15.107:0/3247511279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5622e36ed830 0x5622e36f9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3247511279 >> 172.21.15.107:0/3247511279 conn(0x5622e36e1ef0 msgr2=0x5622e36f3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:10.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3247511279 shutdown_connections 2023-07-21T22:29:10.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3f0259bb80 1 -- 172.21.15.107:0/3247511279 wait complete. 2023-07-21T22:29:10.334 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3f0259bb80 20 client.15501 trim_cache size 0 max 16384 2023-07-21T22:29:10.334 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:29:10.334 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:29:10.334 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:29:10.335 INFO:teuthology.orchestra.run.smithi107.stdout: "2d8eb7f5-addb-4d3b-aa0f-c3f332cafbab": { 2023-07-21T22:29:10.335 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:29:10.335 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "2d8eb7f5-addb-4d3b-aa0f-c3f332cafbab", 2023-07-21T22:29:10.335 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:29:10.335 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:29:10.335 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:29:10.335 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:29:10.337 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3ef6416700 1 -- 172.21.15.107:0/4289158692 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3ed80a11e0 msgr2=0x7f3ed80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.337 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3ef6416700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3ed80a11e0 0x7f3ed80a3690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0100b40 tx=0x7f3ee000b410 comp rx=0 tx=0).stop 2023-07-21T22:29:10.337 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3ef6416700 1 -- 172.21.15.107:0/4289158692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef0101670 msgr2=0x7f3ef019cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:10.337 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3ef6416700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef0101670 0x7f3ef019cdf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3ee800ada0 tx=0x7f3ee800c330 comp rx=0 tx=0).stop 2023-07-21T22:29:10.337 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3ef6416700 1 -- 172.21.15.107:0/4289158692 shutdown_connections 2023-07-21T22:29:10.337 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3ef6416700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef0101670 0x7f3ef019cdf0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.338 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3ef6416700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ef00ffaf0 0x7f3ef0195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.338 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3ef6416700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3ed80a11e0 0x7f3ed80a3690 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.338 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3ef6416700 1 --2- 172.21.15.107:0/4289158692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ef0100ce0 0x7f3ef0195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:10.338 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3ef6416700 1 -- 172.21.15.107:0/4289158692 >> 172.21.15.107:0/4289158692 conn(0x7f3ef00fb130 msgr2=0x7f3ef00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:10.338 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3ef6416700 1 -- 172.21.15.107:0/4289158692 shutdown_connections 2023-07-21T22:29:10.338 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:10.276+0000 7f3ef6416700 1 -- 172.21.15.107:0/4289158692 wait complete. 2023-07-21T22:29:10.339 INFO:tasks.cephfs.filesystem:scrub status for tag:2d8eb7f5-addb-4d3b-aa0f-c3f332cafbab - {'path': '/', 'tag': '2d8eb7f5-addb-4d3b-aa0f-c3f332cafbab', 'options': 'recursive,force'} 2023-07-21T22:29:10.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:10 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1766209796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:29:10.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:10 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/272937134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:29:10.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:10 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/203538997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:29:10.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:10 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1766209796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:29:10.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:10 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/272937134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:29:10.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:10 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/203538997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:29:11.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:11 smithi190 ceph-mon[98334]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 9 op/s 2023-07-21T22:29:11.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:11 smithi190 ceph-mon[98334]: scrub queued for path: / 2023-07-21T22:29:11.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:11 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:11.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:11 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:29:11.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:11 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:11.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:11 smithi190 ceph-mon[98334]: scrub summary: idle 2023-07-21T22:29:11.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:11 smithi132 ceph-mon[97863]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 9 op/s 2023-07-21T22:29:11.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:11 smithi132 ceph-mon[97863]: scrub queued for path: / 2023-07-21T22:29:11.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:11 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:11.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:11 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:29:11.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:11 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:11.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:11 smithi132 ceph-mon[97863]: scrub summary: idle 2023-07-21T22:29:11.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:11 smithi107 ceph-mon[96543]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 9 op/s 2023-07-21T22:29:11.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:11 smithi107 ceph-mon[96543]: scrub queued for path: / 2023-07-21T22:29:11.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:11 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:11.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:11 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:29:11.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:11 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:11.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:11 smithi107 ceph-mon[96543]: scrub summary: idle 2023-07-21T22:29:13.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:13 smithi190 ceph-mon[98334]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-07-21T22:29:13.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:13 smithi132 ceph-mon[97863]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-07-21T22:29:13.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:13 smithi107 ceph-mon[96543]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-07-21T22:29:15.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:15 smithi190 ceph-mon[98334]: pgmap v298: 97 pgs: 97 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-07-21T22:29:15.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:15 smithi132 ceph-mon[97863]: pgmap v298: 97 pgs: 97 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-07-21T22:29:15.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:15 smithi107 ceph-mon[96543]: pgmap v298: 97 pgs: 97 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-07-21T22:29:17.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:17 smithi190 ceph-mon[98334]: pgmap v299: 97 pgs: 97 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-07-21T22:29:17.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:17 smithi190 ceph-mon[98334]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-21T22:29:17.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:17 smithi132 ceph-mon[97863]: pgmap v299: 97 pgs: 97 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-07-21T22:29:17.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:17 smithi132 ceph-mon[97863]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-21T22:29:17.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:17 smithi107 ceph-mon[96543]: pgmap v299: 97 pgs: 97 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-07-21T22:29:17.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:17 smithi107 ceph-mon[96543]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-21T22:29:19.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:19 smithi190 ceph-mon[98334]: pgmap v300: 97 pgs: 97 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2023-07-21T22:29:19.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:19 smithi132 ceph-mon[97863]: pgmap v300: 97 pgs: 97 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2023-07-21T22:29:19.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:19 smithi107 ceph-mon[96543]: pgmap v300: 97 pgs: 97 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2023-07-21T22:29:21.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:21 smithi190 ceph-mon[98334]: pgmap v301: 97 pgs: 97 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T22:29:21.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:21 smithi132 ceph-mon[97863]: pgmap v301: 97 pgs: 97 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T22:29:21.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:21 smithi107 ceph-mon[96543]: pgmap v301: 97 pgs: 97 active+clean; 648 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T22:29:22.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:22 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:22.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:22 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:22.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:22 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:29:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.052+0000 7f9006a3b700 1 -- 172.21.15.107:0/519678173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90000ff5c0 msgr2=0x7f9000106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.052+0000 7f9006a3b700 1 --2- 172.21.15.107:0/519678173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90000ff5c0 0x7f9000106e70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800b0a0 tx=0x7f8ff800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.052+0000 7f9006a3b700 1 -- 172.21.15.107:0/519678173 shutdown_connections 2023-07-21T22:29:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.052+0000 7f9006a3b700 1 --2- 172.21.15.107:0/519678173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90000ff5c0 0x7f9000106e70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.052+0000 7f9006a3b700 1 --2- 172.21.15.107:0/519678173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90000fffc0 0x7f90001003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.052+0000 7f9006a3b700 1 --2- 172.21.15.107:0/519678173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90000fec10 0x7f90000ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.052+0000 7f9006a3b700 1 -- 172.21.15.107:0/519678173 >> 172.21.15.107:0/519678173 conn(0x7f90000f97b0 msgr2=0x7f90000fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.052+0000 7f9006a3b700 1 -- 172.21.15.107:0/519678173 shutdown_connections 2023-07-21T22:29:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.052+0000 7f9006a3b700 1 -- 172.21.15.107:0/519678173 wait complete. 2023-07-21T22:29:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.052+0000 7f9006a3b700 1 Processor -- start 2023-07-21T22:29:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f9006a3b700 1 -- start start 2023-07-21T22:29:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f9006a3b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90000fec10 0x7f9000195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f9006a3b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90000ff5c0 0x7f9000195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f9006a3b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90000fffc0 0x7f9000196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f9006a3b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90001973a0 con 0x7f90000ff5c0 2023-07-21T22:29:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f9006a3b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9000197510 con 0x7f90000fffc0 2023-07-21T22:29:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f9006a3b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90001976d0 con 0x7f90000fec10 2023-07-21T22:29:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90000ff5c0 0x7f9000195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90000ff5c0 0x7f9000195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48274/0 (socket says 172.21.15.107:48274) 2023-07-21T22:29:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f8fff7fe700 1 -- 172.21.15.107:0/1699385411 learned_addr learned my addr 172.21.15.107:0/1699385411 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f8ffffff700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90000fec10 0x7f9000195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f9004fd8700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90000fffc0 0x7f9000196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f8fff7fe700 1 -- 172.21.15.107:0/1699385411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90000fec10 msgr2=0x7f9000195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90000fec10 0x7f9000195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f8fff7fe700 1 -- 172.21.15.107:0/1699385411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90000fffc0 msgr2=0x7f9000196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90000fffc0 0x7f9000196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.053+0000 7f8fff7fe700 1 -- 172.21.15.107:0/1699385411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff8009d00 con 0x7f90000ff5c0 2023-07-21T22:29:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.054+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90000ff5c0 0x7f9000195ec0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000c9b0 tx=0x7f8ff000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.054+0000 7f8ffd7fa700 1 -- 172.21.15.107:0/1699385411 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0007b00 con 0x7f90000ff5c0 2023-07-21T22:29:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.054+0000 7f9006a3b700 1 -- 172.21.15.107:0/1699385411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90001979b0 con 0x7f90000ff5c0 2023-07-21T22:29:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.054+0000 7f8ffd7fa700 1 -- 172.21.15.107:0/1699385411 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ff0007c60 con 0x7f90000ff5c0 2023-07-21T22:29:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.054+0000 7f8ffd7fa700 1 -- 172.21.15.107:0/1699385411 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0018660 con 0x7f90000ff5c0 2023-07-21T22:29:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.054+0000 7f9006a3b700 1 -- 172.21.15.107:0/1699385411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900019e170 con 0x7f90000ff5c0 2023-07-21T22:29:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.056+0000 7f8ffd7fa700 1 -- 172.21.15.107:0/1699385411 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8ff000f450 con 0x7f90000ff5c0 2023-07-21T22:29:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.056+0000 7f8ffd7fa700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8fe80a1150 0x7f8fe80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.057+0000 7f8ffffff700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8fe80a1150 0x7f8fe80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.057+0000 7f8ffd7fa700 1 -- 172.21.15.107:0/1699385411 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8ff00cc910 con 0x7f90000ff5c0 2023-07-21T22:29:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.057+0000 7f8ffffff700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8fe80a1150 0x7f8fe80a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4005950 tx=0x7f8ff400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.057+0000 7f9012bc0b80 1 Processor -- start 2023-07-21T22:29:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.057+0000 7f9012bc0b80 1 -- start start 2023-07-21T22:29:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f9012bc0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 0x5623a443d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f9012bc0b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a443e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f9012bc0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 0x5623a443eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f9012bc0b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a433ec50 con 0x5623a443dba0 2023-07-21T22:29:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f9012bc0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a43919e0 con 0x5623a42956b0 2023-07-21T22:29:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f9012bc0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a438ae70 con 0x5623a443ea60 2023-07-21T22:29:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a443e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a443e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48282/0 (socket says 172.21.15.107:48282) 2023-07-21T22:29:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f8fff7fe700 1 -- 172.21.15.107:0/994326144 learned_addr learned my addr 172.21.15.107:0/994326144 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f8ffffff700 1 --2- 172.21.15.107:0/994326144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 0x5623a443d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f9004fd8700 1 --2- 172.21.15.107:0/994326144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 0x5623a443eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f8fff7fe700 1 -- 172.21.15.107:0/994326144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 msgr2=0x5623a443eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/994326144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 0x5623a443eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f8fff7fe700 1 -- 172.21.15.107:0/994326144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 msgr2=0x5623a443d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/994326144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 0x5623a443d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f8fff7fe700 1 -- 172.21.15.107:0/994326144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623a41b8d80 con 0x5623a443dba0 2023-07-21T22:29:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/994326144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a443e520 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0096200 tx=0x7f8ff0099730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d762961fb557c33 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f8fe67fc700 1 -- 172.21.15.107:0/994326144 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00a2150 con 0x5623a443dba0 2023-07-21T22:29:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f8fe67fc700 1 -- 172.21.15.107:0/994326144 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ff0097e40 con 0x5623a443dba0 2023-07-21T22:29:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.058+0000 7f8fe67fc700 1 -- 172.21.15.107:0/994326144 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff009a180 con 0x5623a443dba0 2023-07-21T22:29:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.059+0000 7f9012bc0b80 1 -- 172.21.15.107:0/994326144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 msgr2=0x5623a443e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.059+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/994326144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a443e520 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0096200 tx=0x7f8ff0099730 comp rx=0 tx=0).stop 2023-07-21T22:29:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.059+0000 7f9012bc0b80 1 -- 172.21.15.107:0/994326144 shutdown_connections 2023-07-21T22:29:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.059+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/994326144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 0x5623a443eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.059+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/994326144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 0x5623a443d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.059+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/994326144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a443e520 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.059+0000 7f9012bc0b80 1 -- 172.21.15.107:0/994326144 >> 172.21.15.107:0/994326144 conn(0x5623a442eae0 msgr2=0x5623a4433c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.059+0000 7f9012bc0b80 1 -- 172.21.15.107:0/994326144 shutdown_connections 2023-07-21T22:29:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.059+0000 7f9012bc0b80 1 -- 172.21.15.107:0/994326144 wait complete. 2023-07-21T22:29:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.059+0000 7f9012bc0b80 1 -- start start 2023-07-21T22:29:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9012bc0b80 1 Processor -- start 2023-07-21T22:29:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9012bc0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a42956b0 0x5623a4446bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9012bc0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a443dba0 0x5623a4447110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9012bc0b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443ea60 0x5623a444e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9012bc0b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a432aa60 con 0x5623a443ea60 2023-07-21T22:29:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9012bc0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a432a760 con 0x5623a443dba0 2023-07-21T22:29:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9012bc0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a420e8c0 con 0x5623a42956b0 2023-07-21T22:29:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9012bc0b80 10 client.0 _ll_register_callbacks cb 0x5623a4328700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9004fd8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443ea60 0x5623a444e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9004fd8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443ea60 0x5623a444e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48288/0 (socket says 172.21.15.107:48288) 2023-07-21T22:29:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9004fd8700 1 -- 172.21.15.107:0/1632174085 learned_addr learned my addr 172.21.15.107:0/1632174085 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a443dba0 0x5623a4447110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f8ffffff700 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a42956b0 0x5623a4446bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9004fd8700 1 -- 172.21.15.107:0/1632174085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a42956b0 msgr2=0x5623a4446bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9004fd8700 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a42956b0 0x5623a4446bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9004fd8700 1 -- 172.21.15.107:0/1632174085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a443dba0 msgr2=0x5623a4447110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9004fd8700 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a443dba0 0x5623a4447110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9004fd8700 1 -- 172.21.15.107:0/1632174085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900019e170 con 0x5623a443ea60 2023-07-21T22:29:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a443dba0 0x5623a4447110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f9004fd8700 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443ea60 0x5623a444e150 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800b830 tx=0x7f8ff80093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7164095d77b74a15 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.060+0000 7f8fdf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f8fdf7fe700 1 -- 172.21.15.107:0/1632174085 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff800e050 con 0x5623a443ea60 2023-07-21T22:29:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f8fdf7fe700 1 -- 172.21.15.107:0/1632174085 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8ff8005040 con 0x5623a443ea60 2023-07-21T22:29:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f9012bc0b80 1 -- 172.21.15.107:0/1632174085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623a42e4490 con 0x5623a443ea60 2023-07-21T22:29:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f8fdf7fe700 1 -- 172.21.15.107:0/1632174085 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ff8004e60 con 0x5623a443ea60 2023-07-21T22:29:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f9012bc0b80 10 client.15513 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f9012bc0b80 1 -- 172.21.15.107:0/1632174085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5623a41a1c00 con 0x5623a443ea60 2023-07-21T22:29:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f8fdf7fe700 1 -- 172.21.15.107:0/1632174085 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5623a42e4490 con 0x5623a443ea60 2023-07-21T22:29:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f8fdf7fe700 1 -- 172.21.15.107:0/1632174085 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8ff800b0d0 con 0x5623a443ea60 2023-07-21T22:29:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f9012bc0b80 10 client.15513 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f9012bc0b80 10 client.15513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f9012bc0b80 20 client.15513 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5623a4452a00 0x5623a4454e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f9012bc0b80 4 client.15513 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f9012bc0b80 1 -- 172.21.15.107:0/1632174085 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623a4383e50 con 0x5623a4452a00 2023-07-21T22:29:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.061+0000 7f8ffffff700 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5623a4452a00 0x5623a4454e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.062+0000 7f8fdf7fe700 10 client.15513 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.062+0000 7f8ffffff700 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5623a4452a00 0x5623a4454e10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d210f71769b2a5 server_cookie=a46cb5fba88f030c in_seq=0 out_seq=0 2023-07-21T22:29:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.062+0000 7f8fdf7fe700 1 -- 172.21.15.107:0/1632174085 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5623a4383e50 con 0x5623a4452a00 2023-07-21T22:29:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.062+0000 7f8fdf7fe700 10 client.15513 handle_command_reply: tid=0 2023-07-21T22:29:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.063+0000 7f9012bc0b80 1 client.15513 shutdown 2023-07-21T22:29:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.063+0000 7f9012bc0b80 1 -- 172.21.15.107:0/1632174085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443ea60 msgr2=0x5623a444e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.063+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443ea60 0x5623a444e150 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800b830 tx=0x7f8ff80093b0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.063+0000 7f9012bc0b80 1 -- 172.21.15.107:0/1632174085 shutdown_connections 2023-07-21T22:29:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.063+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5623a4452a00 0x5623a4454e10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.063+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443ea60 0x5623a444e150 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.063+0000 7f8fdf7fe700 0 client.15513 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.063+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a42956b0 0x5623a4446bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.063+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/1632174085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a443dba0 0x5623a4447110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.063+0000 7f9012bc0b80 1 -- 172.21.15.107:0/1632174085 >> 172.21.15.107:0/1632174085 conn(0x5623a442eae0 msgr2=0x5623a44333e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.063+0000 7f9012bc0b80 1 -- 172.21.15.107:0/1632174085 shutdown_connections 2023-07-21T22:29:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.063+0000 7f9012bc0b80 1 -- 172.21.15.107:0/1632174085 wait complete. 2023-07-21T22:29:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.063+0000 7f9012bc0b80 20 client.15513 trim_cache size 0 max 16384 2023-07-21T22:29:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f9012bc0b80 1 Processor -- start 2023-07-21T22:29:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f9012bc0b80 1 -- start start 2023-07-21T22:29:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f9012bc0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 0x5623a4441890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f9012bc0b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a444e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f9012bc0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 0x5623a4441dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f9012bc0b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a4434b00 con 0x5623a443dba0 2023-07-21T22:29:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f9012bc0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a4434cb0 con 0x5623a42956b0 2023-07-21T22:29:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f9012bc0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a4446080 con 0x5623a443ea60 2023-07-21T22:29:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a444e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a444e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48296/0 (socket says 172.21.15.107:48296) 2023-07-21T22:29:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f8fff7fe700 1 -- 172.21.15.107:0/377206773 learned_addr learned my addr 172.21.15.107:0/377206773 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f9004fd8700 1 --2- 172.21.15.107:0/377206773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 0x5623a4441dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f8fff7fe700 1 -- 172.21.15.107:0/377206773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 msgr2=0x5623a4441dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/377206773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 0x5623a4441dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f8fff7fe700 1 -- 172.21.15.107:0/377206773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 msgr2=0x5623a4441890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f8ffffff700 1 --2- 172.21.15.107:0/377206773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 0x5623a4441890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/377206773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 0x5623a4441890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f8fff7fe700 1 -- 172.21.15.107:0/377206773 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623a41df1f0 con 0x5623a443dba0 2023-07-21T22:29:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/377206773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a444e110 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0000c00 tx=0x7f8ff000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6850a2e36aab3382 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f8fe6ffd700 1 -- 172.21.15.107:0/377206773 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00980a0 con 0x5623a443dba0 2023-07-21T22:29:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f8fe6ffd700 1 -- 172.21.15.107:0/377206773 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ff00ca180 con 0x5623a443dba0 2023-07-21T22:29:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.078+0000 7f8fe6ffd700 1 -- 172.21.15.107:0/377206773 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0099450 con 0x5623a443dba0 2023-07-21T22:29:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.079+0000 7f9012bc0b80 1 -- 172.21.15.107:0/377206773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 msgr2=0x5623a444e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.079+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/377206773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a444e110 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0000c00 tx=0x7f8ff000fc30 comp rx=0 tx=0).stop 2023-07-21T22:29:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.079+0000 7f9012bc0b80 1 -- 172.21.15.107:0/377206773 shutdown_connections 2023-07-21T22:29:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.079+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/377206773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 0x5623a4441dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.079+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/377206773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 0x5623a4441890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.079+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/377206773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a444e110 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.079+0000 7f9012bc0b80 1 -- 172.21.15.107:0/377206773 >> 172.21.15.107:0/377206773 conn(0x5623a442eae0 msgr2=0x5623a4459770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.079+0000 7f9012bc0b80 1 -- 172.21.15.107:0/377206773 shutdown_connections 2023-07-21T22:29:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.079+0000 7f9012bc0b80 1 -- 172.21.15.107:0/377206773 wait complete. 2023-07-21T22:29:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.079+0000 7f9012bc0b80 1 -- start start 2023-07-21T22:29:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f9012bc0b80 1 Processor -- start 2023-07-21T22:29:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f9012bc0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 0x5623a444da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f9012bc0b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a444df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f9012bc0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 0x5623a444a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f9012bc0b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a444ab90 con 0x5623a443dba0 2023-07-21T22:29:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f9012bc0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a444ad00 con 0x5623a42956b0 2023-07-21T22:29:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f9012bc0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623a444aec0 con 0x5623a443ea60 2023-07-21T22:29:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f9012bc0b80 10 client.0 _ll_register_callbacks cb 0x7f8ff8009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a444df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f8fff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a444df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48302/0 (socket says 172.21.15.107:48302) 2023-07-21T22:29:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f8fff7fe700 1 -- 172.21.15.107:0/3727179312 learned_addr learned my addr 172.21.15.107:0/3727179312 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f9004fd8700 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 0x5623a444a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f8ffffff700 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 0x5623a444da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f8fff7fe700 1 -- 172.21.15.107:0/3727179312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 msgr2=0x5623a444a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 0x5623a444a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f8fff7fe700 1 -- 172.21.15.107:0/3727179312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 msgr2=0x5623a444da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 0x5623a444da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f8fff7fe700 1 -- 172.21.15.107:0/3727179312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623a41b8d80 con 0x5623a443dba0 2023-07-21T22:29:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.080+0000 7f8ffffff700 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 0x5623a444da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f8fff7fe700 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a444df90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0098050 tx=0x7f8ff0099c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f25960c3cd44f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f8fdf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f8fdf7fe700 1 -- 172.21.15.107:0/3727179312 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0099660 con 0x5623a443dba0 2023-07-21T22:29:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f8fdf7fe700 1 -- 172.21.15.107:0/3727179312 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8ff0099400 con 0x5623a443dba0 2023-07-21T22:29:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f9012bc0b80 1 -- 172.21.15.107:0/3727179312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ff0097e40 con 0x5623a443dba0 2023-07-21T22:29:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f8fdf7fe700 1 -- 172.21.15.107:0/3727179312 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ff00a1cc0 con 0x5623a443dba0 2023-07-21T22:29:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f9012bc0b80 10 client.15519 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f9012bc0b80 1 -- 172.21.15.107:0/3727179312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5623a41b7250 con 0x5623a443dba0 2023-07-21T22:29:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f8fdf7fe700 1 -- 172.21.15.107:0/3727179312 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ff0097e40 con 0x5623a443dba0 2023-07-21T22:29:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f8fdf7fe700 1 -- 172.21.15.107:0/3727179312 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8ff00b2220 con 0x5623a443dba0 2023-07-21T22:29:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f9012bc0b80 10 client.15519 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f9012bc0b80 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f9012bc0b80 20 client.15519 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5623a44a7880 0x5623a44a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f9012bc0b80 4 client.15519 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:29:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f9012bc0b80 1 -- 172.21.15.107:0/3727179312 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5623a4345ff0 con 0x5623a44a7880 2023-07-21T22:29:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.081+0000 7f8ffffff700 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5623a44a7880 0x5623a44a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.082+0000 7f8fdf7fe700 10 client.15519 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.082+0000 7f8ffffff700 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5623a44a7880 0x5623a44a9c60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802b113b166814 server_cookie=904fe9849b9aad5f in_seq=0 out_seq=0 2023-07-21T22:29:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f8fdf7fe700 1 -- 172.21.15.107:0/3727179312 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5623a4345ff0 con 0x5623a44a7880 2023-07-21T22:29:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f8fdf7fe700 10 client.15519 handle_command_reply: tid=0 2023-07-21T22:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9012bc0b80 1 client.15519 shutdown 2023-07-21T22:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9012bc0b80 1 -- 172.21.15.107:0/3727179312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 msgr2=0x5623a444df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a444df90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0098050 tx=0x7f8ff0099c00 comp rx=0 tx=0).stop 2023-07-21T22:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9012bc0b80 1 -- 172.21.15.107:0/3727179312 shutdown_connections 2023-07-21T22:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5623a44a7880 0x5623a44a9c60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5623a443ea60 0x5623a444a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623a42956b0 0x5623a444da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9012bc0b80 1 --2- 172.21.15.107:0/3727179312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623a443dba0 0x5623a444df90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f8fdf7fe700 0 client.15519 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9012bc0b80 1 -- 172.21.15.107:0/3727179312 >> 172.21.15.107:0/3727179312 conn(0x5623a442eae0 msgr2=0x5623a4441890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9012bc0b80 1 -- 172.21.15.107:0/3727179312 shutdown_connections 2023-07-21T22:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9012bc0b80 1 -- 172.21.15.107:0/3727179312 wait complete. 2023-07-21T22:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9012bc0b80 20 client.15519 trim_cache size 0 max 16384 2023-07-21T22:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9006a3b700 1 -- 172.21.15.107:0/1699385411 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8fe80a1150 msgr2=0x7f8fe80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9006a3b700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8fe80a1150 0x7f8fe80a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4005950 tx=0x7f8ff400a300 comp rx=0 tx=0).stop 2023-07-21T22:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9006a3b700 1 -- 172.21.15.107:0/1699385411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90000ff5c0 msgr2=0x7f9000195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9006a3b700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90000ff5c0 0x7f9000195ec0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000c9b0 tx=0x7f8ff000ccc0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9006a3b700 1 -- 172.21.15.107:0/1699385411 shutdown_connections 2023-07-21T22:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9006a3b700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90000fffc0 0x7f9000196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9006a3b700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8fe80a1150 0x7f8fe80a3600 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9006a3b700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f90000fec10 0x7f9000195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9006a3b700 1 --2- 172.21.15.107:0/1699385411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90000ff5c0 0x7f9000195ec0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.094+0000 7f9006a3b700 1 -- 172.21.15.107:0/1699385411 >> 172.21.15.107:0/1699385411 conn(0x7f90000f97b0 msgr2=0x7f90000fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.095+0000 7f9006a3b700 1 -- 172.21.15.107:0/1699385411 shutdown_connections 2023-07-21T22:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:23.095+0000 7f9006a3b700 1 -- 172.21.15.107:0/1699385411 wait complete. 2023-07-21T22:29:23.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:23 smithi132 ceph-mon[97863]: pgmap v302: 97 pgs: 97 active+clean; 652 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2023-07-21T22:29:23.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:23 smithi107 ceph-mon[96543]: pgmap v302: 97 pgs: 97 active+clean; 652 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2023-07-21T22:29:23.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:23 smithi190 ceph-mon[98334]: pgmap v302: 97 pgs: 97 active+clean; 652 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2023-07-21T22:29:25.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:25 smithi132 ceph-mon[97863]: pgmap v303: 97 pgs: 97 active+clean; 658 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2023-07-21T22:29:25.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:25 smithi107 ceph-mon[96543]: pgmap v303: 97 pgs: 97 active+clean; 658 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2023-07-21T22:29:25.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:25 smithi190 ceph-mon[98334]: pgmap v303: 97 pgs: 97 active+clean; 658 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2023-07-21T22:29:27.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:27 smithi132 ceph-mon[97863]: pgmap v304: 97 pgs: 97 active+clean; 658 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2023-07-21T22:29:27.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:27 smithi107 ceph-mon[96543]: pgmap v304: 97 pgs: 97 active+clean; 658 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2023-07-21T22:29:27.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:27 smithi190 ceph-mon[98334]: pgmap v304: 97 pgs: 97 active+clean; 658 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2023-07-21T22:29:29.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:29 smithi132 ceph-mon[97863]: pgmap v305: 97 pgs: 97 active+clean; 665 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 16 op/s 2023-07-21T22:29:29.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:29 smithi107 ceph-mon[96543]: pgmap v305: 97 pgs: 97 active+clean; 665 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 16 op/s 2023-07-21T22:29:29.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:29 smithi190 ceph-mon[98334]: pgmap v305: 97 pgs: 97 active+clean; 665 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.6 KiB/s wr, 16 op/s 2023-07-21T22:29:31.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:31 smithi132 ceph-mon[97863]: pgmap v306: 97 pgs: 97 active+clean; 665 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2023-07-21T22:29:31.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:31 smithi107 ceph-mon[96543]: pgmap v306: 97 pgs: 97 active+clean; 665 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2023-07-21T22:29:31.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:31 smithi190 ceph-mon[98334]: pgmap v306: 97 pgs: 97 active+clean; 665 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2023-07-21T22:29:33.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:33 smithi132 ceph-mon[97863]: pgmap v307: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.6 KiB/s wr, 16 op/s 2023-07-21T22:29:33.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:33 smithi107 ceph-mon[96543]: pgmap v307: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.6 KiB/s wr, 16 op/s 2023-07-21T22:29:33.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:33 smithi190 ceph-mon[98334]: pgmap v307: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.6 KiB/s wr, 16 op/s 2023-07-21T22:29:35.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:35 smithi132 ceph-mon[97863]: pgmap v308: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2023-07-21T22:29:35.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:35 smithi107 ceph-mon[96543]: pgmap v308: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2023-07-21T22:29:35.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:35 smithi190 ceph-mon[98334]: pgmap v308: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2023-07-21T22:29:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.207+0000 7f0efb785700 1 -- 172.21.15.107:0/3539937674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef41056e0 msgr2=0x7f0ef4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.207+0000 7f0efb785700 1 --2- 172.21.15.107:0/3539937674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef41056e0 0x7f0ef4107b90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009ec0 tx=0x7f0eec00b330 comp rx=0 tx=0).stop 2023-07-21T22:29:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.207+0000 7f0efb785700 1 -- 172.21.15.107:0/3539937674 shutdown_connections 2023-07-21T22:29:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.207+0000 7f0efb785700 1 --2- 172.21.15.107:0/3539937674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef41056e0 0x7f0ef4107b90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.207+0000 7f0efb785700 1 --2- 172.21.15.107:0/3539937674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ef4068600 0x7f0ef41026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.207+0000 7f0efb785700 1 --2- 172.21.15.107:0/3539937674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef4102c10 0x7f0ef4105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.207+0000 7f0efb785700 1 -- 172.21.15.107:0/3539937674 >> 172.21.15.107:0/3539937674 conn(0x7f0ef40fa000 msgr2=0x7f0ef40fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.207+0000 7f0efb785700 1 -- 172.21.15.107:0/3539937674 shutdown_connections 2023-07-21T22:29:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.207+0000 7f0efb785700 1 -- 172.21.15.107:0/3539937674 wait complete. 2023-07-21T22:29:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.208+0000 7f0efb785700 1 Processor -- start 2023-07-21T22:29:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.208+0000 7f0efb785700 1 -- start start 2023-07-21T22:29:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.208+0000 7f0efb785700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef4068600 0x7f0ef4195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.208+0000 7f0efb785700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ef4102c10 0x7f0ef4195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.208+0000 7f0efb785700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef41056e0 0x7f0ef4196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.208+0000 7f0efb785700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef41974b0 con 0x7f0ef41056e0 2023-07-21T22:29:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.208+0000 7f0efb785700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4197620 con 0x7f0ef4102c10 2023-07-21T22:29:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.208+0000 7f0efb785700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef41977e0 con 0x7f0ef4068600 2023-07-21T22:29:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.208+0000 7f0ef9d22700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef41056e0 0x7f0ef4196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.208+0000 7f0ef9d22700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef41056e0 0x7f0ef4196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48318/0 (socket says 172.21.15.107:48318) 2023-07-21T22:29:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.208+0000 7f0ef9d22700 1 -- 172.21.15.107:0/3630718852 learned_addr learned my addr 172.21.15.107:0/3630718852 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.208+0000 7f0ef9521700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef4068600 0x7f0ef4195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.208+0000 7f0ef8d20700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ef4102c10 0x7f0ef4195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.209+0000 7f0ef9d22700 1 -- 172.21.15.107:0/3630718852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef4068600 msgr2=0x7f0ef4195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.209+0000 7f0ef9d22700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef4068600 0x7f0ef4195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.209+0000 7f0ef9d22700 1 -- 172.21.15.107:0/3630718852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ef4102c10 msgr2=0x7f0ef4195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.209+0000 7f0ef9d22700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ef4102c10 0x7f0ef4195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.209+0000 7f0ef9d22700 1 -- 172.21.15.107:0/3630718852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec005e70 con 0x7f0ef41056e0 2023-07-21T22:29:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.209+0000 7f0ef9d22700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef41056e0 0x7f0ef4196e50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0eec006070 tx=0x7f0eec008fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.209+0000 7f0eea7fc700 1 -- 172.21.15.107:0/3630718852 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec00e040 con 0x7f0ef41056e0 2023-07-21T22:29:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.209+0000 7f0efb785700 1 -- 172.21.15.107:0/3630718852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef4197a60 con 0x7f0ef41056e0 2023-07-21T22:29:36.259 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.209+0000 7f0eea7fc700 1 -- 172.21.15.107:0/3630718852 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0eec00be80 con 0x7f0ef41056e0 2023-07-21T22:29:36.259 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.209+0000 7f0eea7fc700 1 -- 172.21.15.107:0/3630718852 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec01baf0 con 0x7f0ef41056e0 2023-07-21T22:29:36.259 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.209+0000 7f0efb785700 1 -- 172.21.15.107:0/3630718852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef419e280 con 0x7f0ef41056e0 2023-07-21T22:29:36.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.211+0000 7f0eea7fc700 1 -- 172.21.15.107:0/3630718852 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0eec01d670 con 0x7f0ef41056e0 2023-07-21T22:29:36.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.212+0000 7f0eea7fc700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0ee00a1150 0x7f0ee00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.212+0000 7f0ef9521700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0ee00a1150 0x7f0ee00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.212+0000 7f0eea7fc700 1 -- 172.21.15.107:0/3630718852 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0eec0ce2d0 con 0x7f0ef41056e0 2023-07-21T22:29:36.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.212+0000 7f0ef9521700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0ee00a1150 0x7f0ee00a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0005950 tx=0x7f0ef000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.213+0000 7f0f0790ab80 1 Processor -- start 2023-07-21T22:29:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.213+0000 7f0f0790ab80 1 -- start start 2023-07-21T22:29:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0f0790ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 0x55acf2cb82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0f0790ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2cb8830 0x55acf2cb9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0f0790ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb9580 0x55acf2cb99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0f0790ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf2bbdc50 con 0x55acf2b146b0 2023-07-21T22:29:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0f0790ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf2c109e0 con 0x55acf2cb9580 2023-07-21T22:29:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0f0790ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf2c09e70 con 0x55acf2cb8830 2023-07-21T22:29:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0ef9521700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 0x55acf2cb82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0ef9521700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 0x55acf2cb82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48322/0 (socket says 172.21.15.107:48322) 2023-07-21T22:29:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0ef9521700 1 -- 172.21.15.107:0/1915997490 learned_addr learned my addr 172.21.15.107:0/1915997490 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0ef8d20700 1 --2- 172.21.15.107:0/1915997490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2cb8830 0x55acf2cb9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0ef9d22700 1 --2- 172.21.15.107:0/1915997490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb9580 0x55acf2cb99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0ef9521700 1 -- 172.21.15.107:0/1915997490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2cb8830 msgr2=0x55acf2cb9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0ef9521700 1 --2- 172.21.15.107:0/1915997490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2cb8830 0x55acf2cb9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0ef9521700 1 -- 172.21.15.107:0/1915997490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb9580 msgr2=0x55acf2cb99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0ef9521700 1 --2- 172.21.15.107:0/1915997490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb9580 0x55acf2cb99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0ef9521700 1 -- 172.21.15.107:0/1915997490 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf2a37d80 con 0x55acf2b146b0 2023-07-21T22:29:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0ef9d22700 1 --2- 172.21.15.107:0/1915997490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb9580 0x55acf2cb99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0ef9521700 1 --2- 172.21.15.107:0/1915997490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 0x55acf2cb82f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0010f70 tx=0x7f0ef0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846bb5b2651eeed3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.214+0000 7f0edf7fe700 1 -- 172.21.15.107:0/1915997490 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0011db0 con 0x55acf2b146b0 2023-07-21T22:29:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.215+0000 7f0edf7fe700 1 -- 172.21.15.107:0/1915997490 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ef0015070 con 0x55acf2b146b0 2023-07-21T22:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.215+0000 7f0edf7fe700 1 -- 172.21.15.107:0/1915997490 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef001c610 con 0x55acf2b146b0 2023-07-21T22:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.215+0000 7f0f0790ab80 1 -- 172.21.15.107:0/1915997490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 msgr2=0x55acf2cb82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.215+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/1915997490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 0x55acf2cb82f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0010f70 tx=0x7f0ef0013000 comp rx=0 tx=0).stop 2023-07-21T22:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.215+0000 7f0f0790ab80 1 -- 172.21.15.107:0/1915997490 shutdown_connections 2023-07-21T22:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.215+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/1915997490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb9580 0x55acf2cb99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.215+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/1915997490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 0x55acf2cb82f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.215+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/1915997490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2cb8830 0x55acf2cb9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.215+0000 7f0f0790ab80 1 -- 172.21.15.107:0/1915997490 >> 172.21.15.107:0/1915997490 conn(0x55acf2cadae0 msgr2=0x55acf2cb2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.215+0000 7f0f0790ab80 1 -- 172.21.15.107:0/1915997490 shutdown_connections 2023-07-21T22:29:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.215+0000 7f0f0790ab80 1 -- 172.21.15.107:0/1915997490 wait complete. 2023-07-21T22:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0f0790ab80 1 -- start start 2023-07-21T22:29:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0f0790ab80 1 Processor -- start 2023-07-21T22:29:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0f0790ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 0x55acf2cc17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0f0790ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 0x55acf2cc1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0f0790ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2cb9580 0x55acf2cc8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0f0790ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf2ba9a60 con 0x55acf2b146b0 2023-07-21T22:29:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0f0790ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf2ba9760 con 0x55acf2cb8830 2023-07-21T22:29:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0f0790ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf2a8d8c0 con 0x55acf2cb9580 2023-07-21T22:29:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0f0790ab80 10 client.0 _ll_register_callbacks cb 0x55acf2ba7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0ef9521700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 0x55acf2cc17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0ef9521700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 0x55acf2cc17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48328/0 (socket says 172.21.15.107:48328) 2023-07-21T22:29:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0ef9521700 1 -- 172.21.15.107:0/478229631 learned_addr learned my addr 172.21.15.107:0/478229631 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0ef9d22700 1 --2- 172.21.15.107:0/478229631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2cb9580 0x55acf2cc8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.216+0000 7f0ef8d20700 1 --2- 172.21.15.107:0/478229631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 0x55acf2cc1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0ef9521700 1 -- 172.21.15.107:0/478229631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2cb9580 msgr2=0x55acf2cc8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0ef9521700 1 --2- 172.21.15.107:0/478229631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2cb9580 0x55acf2cc8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0ef9521700 1 -- 172.21.15.107:0/478229631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 msgr2=0x55acf2cc1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0ef9521700 1 --2- 172.21.15.107:0/478229631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 0x55acf2cc1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0ef9521700 1 -- 172.21.15.107:0/478229631 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef0010c20 con 0x55acf2b146b0 2023-07-21T22:29:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0ef9521700 1 --2- 172.21.15.107:0/478229631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 0x55acf2cc17e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0011a00 tx=0x7f0ef0011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26222bed825be591 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0edcff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0edcff9700 1 -- 172.21.15.107:0/478229631 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef002c070 con 0x55acf2b146b0 2023-07-21T22:29:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0edcff9700 1 -- 172.21.15.107:0/478229631 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0ef0017070 con 0x55acf2b146b0 2023-07-21T22:29:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0f0790ab80 1 -- 172.21.15.107:0/478229631 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acf2b63490 con 0x55acf2b146b0 2023-07-21T22:29:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0edcff9700 1 -- 172.21.15.107:0/478229631 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ef001e630 con 0x55acf2b146b0 2023-07-21T22:29:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0f0790ab80 10 client.15531 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0f0790ab80 1 -- 172.21.15.107:0/478229631 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55acf2a20c00 con 0x55acf2b146b0 2023-07-21T22:29:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.217+0000 7f0edcff9700 1 -- 172.21.15.107:0/478229631 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55acf2b63490 con 0x55acf2b146b0 2023-07-21T22:29:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.218+0000 7f0edcff9700 1 -- 172.21.15.107:0/478229631 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0ef00263f0 con 0x55acf2b146b0 2023-07-21T22:29:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.218+0000 7f0f0790ab80 10 client.15531 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.218+0000 7f0f0790ab80 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.218+0000 7f0f0790ab80 20 client.15531 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.218+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/478229631 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55acf2ccd440 0x55acf2ccf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.218+0000 7f0f0790ab80 4 client.15531 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.218+0000 7f0f0790ab80 1 -- 172.21.15.107:0/478229631 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acf2c02e50 con 0x55acf2ccd440 2023-07-21T22:29:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.218+0000 7f0ef8d20700 1 --2- 172.21.15.107:0/478229631 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55acf2ccd440 0x55acf2ccf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.218+0000 7f0edcff9700 10 client.15531 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:36.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.218+0000 7f0ef8d20700 1 --2- 172.21.15.107:0/478229631 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55acf2ccd440 0x55acf2ccf850 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3651f3c7a3ab860 server_cookie=21e7588daf30a77a in_seq=0 out_seq=0 2023-07-21T22:29:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0edcff9700 1 -- 172.21.15.107:0/478229631 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55acf2c02e50 con 0x55acf2ccd440 2023-07-21T22:29:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0edcff9700 10 client.15531 handle_command_reply: tid=0 2023-07-21T22:29:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0f0790ab80 1 client.15531 shutdown 2023-07-21T22:29:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0f0790ab80 1 -- 172.21.15.107:0/478229631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 msgr2=0x55acf2cc17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/478229631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 0x55acf2cc17e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0011a00 tx=0x7f0ef0011de0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0f0790ab80 1 -- 172.21.15.107:0/478229631 shutdown_connections 2023-07-21T22:29:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/478229631 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55acf2ccd440 0x55acf2ccf850 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/478229631 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2cb9580 0x55acf2cc8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/478229631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2b146b0 0x55acf2cc17e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0edcff9700 0 client.15531 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/478229631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 0x55acf2cc1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0f0790ab80 1 -- 172.21.15.107:0/478229631 >> 172.21.15.107:0/478229631 conn(0x55acf2cadae0 msgr2=0x55acf2cb23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0f0790ab80 1 -- 172.21.15.107:0/478229631 shutdown_connections 2023-07-21T22:29:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.219+0000 7f0f0790ab80 1 -- 172.21.15.107:0/478229631 wait complete. 2023-07-21T22:29:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.220+0000 7f0f0790ab80 20 client.15531 trim_cache size 0 max 16384 2023-07-21T22:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.234+0000 7f0f0790ab80 1 Processor -- start 2023-07-21T22:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.234+0000 7f0f0790ab80 1 -- start start 2023-07-21T22:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.234+0000 7f0f0790ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2b146b0 0x55acf2cb9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.234+0000 7f0f0790ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 0x55acf2cc8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.234+0000 7f0f0790ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 0x55acf2cc0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.234+0000 7f0f0790ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf2cafc00 con 0x55acf2cb9580 2023-07-21T22:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.234+0000 7f0f0790ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf2cb3880 con 0x55acf2cb8830 2023-07-21T22:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.234+0000 7f0f0790ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf2cbf690 con 0x55acf2b146b0 2023-07-21T22:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.234+0000 7f0ef9d22700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 0x55acf2cc0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.234+0000 7f0ef9d22700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 0x55acf2cc0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48340/0 (socket says 172.21.15.107:48340) 2023-07-21T22:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.234+0000 7f0ef9d22700 1 -- 172.21.15.107:0/221852900 learned_addr learned my addr 172.21.15.107:0/221852900 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.234+0000 7f0ef8d20700 1 --2- 172.21.15.107:0/221852900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 0x55acf2cc8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.235+0000 7f0ef9521700 1 --2- 172.21.15.107:0/221852900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2b146b0 0x55acf2cb9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.235+0000 7f0ef9d22700 1 -- 172.21.15.107:0/221852900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2b146b0 msgr2=0x55acf2cb9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.235+0000 7f0ef9d22700 1 --2- 172.21.15.107:0/221852900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2b146b0 0x55acf2cb9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.235+0000 7f0ef9d22700 1 -- 172.21.15.107:0/221852900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 msgr2=0x55acf2cc8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.235+0000 7f0ef9d22700 1 --2- 172.21.15.107:0/221852900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 0x55acf2cc8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.235+0000 7f0ef9d22700 1 -- 172.21.15.107:0/221852900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf2a5e1f0 con 0x55acf2cb9580 2023-07-21T22:29:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.235+0000 7f0ef8d20700 1 --2- 172.21.15.107:0/221852900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 0x55acf2cc8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.235+0000 7f0ef9d22700 1 --2- 172.21.15.107:0/221852900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 0x55acf2cc0500 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00ba80 tx=0x7f0eec0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6500897a79d82e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.235+0000 7f0edffff700 1 -- 172.21.15.107:0/221852900 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec09b750 con 0x55acf2cb9580 2023-07-21T22:29:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.235+0000 7f0edffff700 1 -- 172.21.15.107:0/221852900 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0eec09f780 con 0x55acf2cb9580 2023-07-21T22:29:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.235+0000 7f0edffff700 1 -- 172.21.15.107:0/221852900 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec09bc10 con 0x55acf2cb9580 2023-07-21T22:29:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.236+0000 7f0f0790ab80 1 -- 172.21.15.107:0/221852900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 msgr2=0x55acf2cc0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.236+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/221852900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 0x55acf2cc0500 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00ba80 tx=0x7f0eec0bb040 comp rx=0 tx=0).stop 2023-07-21T22:29:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.236+0000 7f0f0790ab80 1 -- 172.21.15.107:0/221852900 shutdown_connections 2023-07-21T22:29:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.236+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/221852900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 0x55acf2cc0500 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.236+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/221852900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2b146b0 0x55acf2cb9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.236+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/221852900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 0x55acf2cc8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.236+0000 7f0f0790ab80 1 -- 172.21.15.107:0/221852900 >> 172.21.15.107:0/221852900 conn(0x55acf2cadae0 msgr2=0x55acf2cd41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.236+0000 7f0f0790ab80 1 -- 172.21.15.107:0/221852900 shutdown_connections 2023-07-21T22:29:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.236+0000 7f0f0790ab80 1 -- 172.21.15.107:0/221852900 wait complete. 2023-07-21T22:29:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.236+0000 7f0f0790ab80 1 -- start start 2023-07-21T22:29:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.236+0000 7f0f0790ab80 1 Processor -- start 2023-07-21T22:29:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0f0790ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2b146b0 0x55acf2cc2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0f0790ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 0x55acf2cc33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0f0790ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 0x55acf2cc7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0f0790ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf2cc82f0 con 0x55acf2cb9580 2023-07-21T22:29:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0f0790ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf2cc8460 con 0x55acf2cb8830 2023-07-21T22:29:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0f0790ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf2cc8620 con 0x55acf2b146b0 2023-07-21T22:29:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0f0790ab80 10 client.0 _ll_register_callbacks cb 0x7f0ee400b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0ef9d22700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 0x55acf2cc7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0ef9d22700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 0x55acf2cc7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48346/0 (socket says 172.21.15.107:48346) 2023-07-21T22:29:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0ef9d22700 1 -- 172.21.15.107:0/3163655407 learned_addr learned my addr 172.21.15.107:0/3163655407 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0ef9521700 1 --2- 172.21.15.107:0/3163655407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2b146b0 0x55acf2cc2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0ef8d20700 1 --2- 172.21.15.107:0/3163655407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 0x55acf2cc33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0ef9d22700 1 -- 172.21.15.107:0/3163655407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2b146b0 msgr2=0x55acf2cc2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0ef9d22700 1 --2- 172.21.15.107:0/3163655407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2b146b0 0x55acf2cc2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0ef9d22700 1 -- 172.21.15.107:0/3163655407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 msgr2=0x55acf2cc33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0ef9d22700 1 --2- 172.21.15.107:0/3163655407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 0x55acf2cc33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0ef9d22700 1 -- 172.21.15.107:0/3163655407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee4004410 con 0x55acf2cb9580 2023-07-21T22:29:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0ef9d22700 1 --2- 172.21.15.107:0/3163655407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 0x55acf2cc7db0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0eec01de60 tx=0x7f0eec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2659b0e0fd4ad5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0edcff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0edcff9700 1 -- 172.21.15.107:0/3163655407 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec09ae40 con 0x55acf2cb9580 2023-07-21T22:29:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0edcff9700 1 -- 172.21.15.107:0/3163655407 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0eec0b7030 con 0x55acf2cb9580 2023-07-21T22:29:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.237+0000 7f0f0790ab80 1 -- 172.21.15.107:0/3163655407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ef0015070 con 0x55acf2cb9580 2023-07-21T22:29:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.238+0000 7f0edcff9700 1 -- 172.21.15.107:0/3163655407 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0eec0a3b10 con 0x55acf2cb9580 2023-07-21T22:29:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.238+0000 7f0f0790ab80 10 client.15537 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.238+0000 7f0f0790ab80 1 -- 172.21.15.107:0/3163655407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f0ef419e280 con 0x55acf2cb9580 2023-07-21T22:29:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.238+0000 7f0edcff9700 1 -- 172.21.15.107:0/3163655407 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ef0015070 con 0x55acf2cb9580 2023-07-21T22:29:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.238+0000 7f0edcff9700 1 -- 172.21.15.107:0/3163655407 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0eec0c7d90 con 0x55acf2cb9580 2023-07-21T22:29:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.238+0000 7f0f0790ab80 10 client.15537 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.238+0000 7f0f0790ab80 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.238+0000 7f0f0790ab80 20 client.15537 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.238+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/3163655407 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55acf2d26050 0x55acf2d28460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.238+0000 7f0f0790ab80 4 client.15537 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.238+0000 7f0f0790ab80 1 -- 172.21.15.107:0/3163655407 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55acf2bc4ff0 con 0x55acf2d26050 2023-07-21T22:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.238+0000 7f0ef9521700 1 --2- 172.21.15.107:0/3163655407 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55acf2d26050 0x55acf2d28460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.239+0000 7f0edcff9700 10 client.15537 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.239+0000 7f0ef9521700 1 --2- 172.21.15.107:0/3163655407 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55acf2d26050 0x55acf2d28460 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0befb3831e7d88 server_cookie=41577be2a7b1cc3 in_seq=0 out_seq=0 2023-07-21T22:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0edcff9700 1 -- 172.21.15.107:0/3163655407 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55acf2bc4ff0 con 0x55acf2d26050 2023-07-21T22:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0edcff9700 10 client.15537 handle_command_reply: tid=0 2023-07-21T22:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0f0790ab80 1 client.15537 shutdown 2023-07-21T22:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0f0790ab80 1 -- 172.21.15.107:0/3163655407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 msgr2=0x55acf2cc7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/3163655407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 0x55acf2cc7db0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0eec01de60 tx=0x7f0eec00b330 comp rx=0 tx=0).stop 2023-07-21T22:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0f0790ab80 1 -- 172.21.15.107:0/3163655407 shutdown_connections 2023-07-21T22:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/3163655407 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55acf2d26050 0x55acf2d28460 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/3163655407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acf2cb9580 0x55acf2cc7db0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0edcff9700 0 client.15537 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/3163655407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55acf2b146b0 0x55acf2cc2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0f0790ab80 1 --2- 172.21.15.107:0/3163655407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55acf2cb8830 0x55acf2cc33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0f0790ab80 1 -- 172.21.15.107:0/3163655407 >> 172.21.15.107:0/3163655407 conn(0x55acf2cadae0 msgr2=0x55acf2cd41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0f0790ab80 1 -- 172.21.15.107:0/3163655407 shutdown_connections 2023-07-21T22:29:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.246+0000 7f0f0790ab80 1 -- 172.21.15.107:0/3163655407 wait complete. 2023-07-21T22:29:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.247+0000 7f0f0790ab80 20 client.15537 trim_cache size 0 max 16384 2023-07-21T22:29:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:29:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:29:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:29:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.247+0000 7f0efb785700 1 -- 172.21.15.107:0/3630718852 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0ee00a1150 msgr2=0x7f0ee00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.247+0000 7f0efb785700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0ee00a1150 0x7f0ee00a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0005950 tx=0x7f0ef000b410 comp rx=0 tx=0).stop 2023-07-21T22:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.247+0000 7f0efb785700 1 -- 172.21.15.107:0/3630718852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef41056e0 msgr2=0x7f0ef4196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.247+0000 7f0efb785700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef41056e0 0x7f0ef4196e50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0eec006070 tx=0x7f0eec008fe0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.247+0000 7f0efb785700 1 -- 172.21.15.107:0/3630718852 shutdown_connections 2023-07-21T22:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.247+0000 7f0efb785700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ef41056e0 0x7f0ef4196e50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.247+0000 7f0efb785700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0ee00a1150 0x7f0ee00a3600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.247+0000 7f0efb785700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ef4068600 0x7f0ef4195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.247+0000 7f0efb785700 1 --2- 172.21.15.107:0/3630718852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ef4102c10 0x7f0ef4195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.247+0000 7f0efb785700 1 -- 172.21.15.107:0/3630718852 >> 172.21.15.107:0/3630718852 conn(0x7f0ef40fa000 msgr2=0x7f0ef4100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.247+0000 7f0efb785700 1 -- 172.21.15.107:0/3630718852 shutdown_connections 2023-07-21T22:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:36.247+0000 7f0efb785700 1 -- 172.21.15.107:0/3630718852 wait complete. 2023-07-21T22:29:37.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:37 smithi132 ceph-mon[97863]: pgmap v309: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2023-07-21T22:29:37.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:37 smithi107 ceph-mon[96543]: pgmap v309: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2023-07-21T22:29:37.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:37 smithi190 ceph-mon[98334]: pgmap v309: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2023-07-21T22:29:39.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:39 smithi132 ceph-mon[97863]: pgmap v310: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2023-07-21T22:29:39.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:39 smithi107 ceph-mon[96543]: pgmap v310: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2023-07-21T22:29:39.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:39 smithi190 ceph-mon[98334]: pgmap v310: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2023-07-21T22:29:40.339 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:29:40.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.448+0000 7f14b0ea2700 1 -- 172.21.15.107:0/4164362209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14ac100560 msgr2=0x7f14ac106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.448+0000 7f14b0ea2700 1 --2- 172.21.15.107:0/4164362209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14ac100560 0x7f14ac106e20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f14a400b0a0 tx=0x7f14a400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.448+0000 7f14b0ea2700 1 -- 172.21.15.107:0/4164362209 shutdown_connections 2023-07-21T22:29:40.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.448+0000 7f14b0ea2700 1 --2- 172.21.15.107:0/4164362209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14ac100560 0x7f14ac106e20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.448+0000 7f14b0ea2700 1 --2- 172.21.15.107:0/4164362209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14ac0fe9e0 0x7f14ac0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.448+0000 7f14b0ea2700 1 --2- 172.21.15.107:0/4164362209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14ac0ffbd0 0x7f14ac100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.448+0000 7f14b0ea2700 1 -- 172.21.15.107:0/4164362209 >> 172.21.15.107:0/4164362209 conn(0x7f14ac0fa000 msgr2=0x7f14ac0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:40.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.448+0000 7f14b0ea2700 1 -- 172.21.15.107:0/4164362209 shutdown_connections 2023-07-21T22:29:40.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.448+0000 7f14b0ea2700 1 -- 172.21.15.107:0/4164362209 wait complete. 2023-07-21T22:29:40.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.449+0000 7f14b0ea2700 1 Processor -- start 2023-07-21T22:29:40.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.449+0000 7f14b0ea2700 1 -- start start 2023-07-21T22:29:40.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.449+0000 7f14b0ea2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14ac0fe9e0 0x7f14ac194800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.449+0000 7f14b0ea2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14ac0ffbd0 0x7f14ac194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.449+0000 7f14b0ea2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14ac100560 0x7f14ac19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.449+0000 7f14b0ea2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac19c380 con 0x7f14ac100560 2023-07-21T22:29:40.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.449+0000 7f14b0ea2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac19c4c0 con 0x7f14ac0fe9e0 2023-07-21T22:29:40.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.449+0000 7f14b0ea2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac19c650 con 0x7f14ac0ffbd0 2023-07-21T22:29:40.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.449+0000 7f14aa59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14ac0fe9e0 0x7f14ac194800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.449+0000 7f14aad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14ac100560 0x7f14ac19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.449+0000 7f14aa59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14ac0fe9e0 0x7f14ac194800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:39182/0 (socket says 172.21.15.107:39182) 2023-07-21T22:29:40.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.449+0000 7f14aa59c700 1 -- 172.21.15.107:0/3486273761 learned_addr learned my addr 172.21.15.107:0/3486273761 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:40.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.450+0000 7f14a9d9b700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14ac0ffbd0 0x7f14ac194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.450+0000 7f14aad9d700 1 -- 172.21.15.107:0/3486273761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14ac0ffbd0 msgr2=0x7f14ac194d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.450+0000 7f14aad9d700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14ac0ffbd0 0x7f14ac194d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.450+0000 7f14aad9d700 1 -- 172.21.15.107:0/3486273761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14ac0fe9e0 msgr2=0x7f14ac194800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.450+0000 7f14aad9d700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14ac0fe9e0 0x7f14ac194800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.450+0000 7f14aad9d700 1 -- 172.21.15.107:0/3486273761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a4009d00 con 0x7f14ac100560 2023-07-21T22:29:40.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.450+0000 7f14aa59c700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14ac0fe9e0 0x7f14ac194800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:40.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.450+0000 7f14aad9d700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14ac100560 0x7f14ac19bdb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f14a400b0a0 tx=0x7f14a4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.450+0000 7f149b7fe700 1 -- 172.21.15.107:0/3486273761 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a400e040 con 0x7f14ac100560 2023-07-21T22:29:40.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.450+0000 7f149b7fe700 1 -- 172.21.15.107:0/3486273761 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14a401db50 con 0x7f14ac100560 2023-07-21T22:29:40.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.450+0000 7f149b7fe700 1 -- 172.21.15.107:0/3486273761 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a401bba0 con 0x7f14ac100560 2023-07-21T22:29:40.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.450+0000 7f14b0ea2700 1 -- 172.21.15.107:0/3486273761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ac067670 con 0x7f14ac100560 2023-07-21T22:29:40.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.450+0000 7f14b0ea2700 1 -- 172.21.15.107:0/3486273761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac067b80 con 0x7f14ac100560 2023-07-21T22:29:40.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.453+0000 7f149b7fe700 1 -- 172.21.15.107:0/3486273761 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f14a401dcc0 con 0x7f14ac100560 2023-07-21T22:29:40.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.453+0000 7f149b7fe700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f14940a11d0 0x7f14940a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.454+0000 7f14aa59c700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f14940a11d0 0x7f14940a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.454+0000 7f149b7fe700 1 -- 172.21.15.107:0/3486273761 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f14a40ce580 con 0x7f14ac100560 2023-07-21T22:29:40.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.454+0000 7f14aa59c700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f14940a11d0 0x7f14940a3680 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f149c0105c0 tx=0x7f149c00b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.455+0000 7f14bd027b80 1 Processor -- start 2023-07-21T22:29:40.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.455+0000 7f14bd027b80 1 -- start start 2023-07-21T22:29:40.503 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.455+0000 7f14bd027b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 0x558f32b66c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.503 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.455+0000 7f14bd027b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b671a0 0x558f32b65430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.503 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.455+0000 7f14bd027b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b65a10 0x558f32b65e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.503 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.455+0000 7f14bd027b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32a6acb0 con 0x558f32b671a0 2023-07-21T22:29:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.455+0000 7f14bd027b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32abda40 con 0x558f32b66830 2023-07-21T22:29:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.455+0000 7f14bd027b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32ab6ed0 con 0x558f32b65a10 2023-07-21T22:29:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.455+0000 7f14a9d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b671a0 0x558f32b65430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.455+0000 7f14a9d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b671a0 0x558f32b65430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48366/0 (socket says 172.21.15.107:48366) 2023-07-21T22:29:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.455+0000 7f14a9d9b700 1 -- 172.21.15.107:0/2234791110 learned_addr learned my addr 172.21.15.107:0/2234791110 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.455+0000 7f14aa59c700 1 --2- 172.21.15.107:0/2234791110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 0x558f32b66c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.455+0000 7f14aad9d700 1 --2- 172.21.15.107:0/2234791110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b65a10 0x558f32b65e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.456+0000 7f14a9d9b700 1 -- 172.21.15.107:0/2234791110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b65a10 msgr2=0x558f32b65e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.456+0000 7f14a9d9b700 1 --2- 172.21.15.107:0/2234791110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b65a10 0x558f32b65e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.456+0000 7f14a9d9b700 1 -- 172.21.15.107:0/2234791110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 msgr2=0x558f32b66c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.456+0000 7f14a9d9b700 1 --2- 172.21.15.107:0/2234791110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 0x558f32b66c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.456+0000 7f14a9d9b700 1 -- 172.21.15.107:0/2234791110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f328e4de0 con 0x558f32b671a0 2023-07-21T22:29:40.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.456+0000 7f14aa59c700 1 --2- 172.21.15.107:0/2234791110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 0x558f32b66c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:40.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.456+0000 7f14a9d9b700 1 --2- 172.21.15.107:0/2234791110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b671a0 0x558f32b65430 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f14a000df00 tx=0x7f14a0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cefa852606a34dd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.456+0000 7f1498ff9700 1 -- 172.21.15.107:0/2234791110 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0012050 con 0x558f32b671a0 2023-07-21T22:29:40.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.456+0000 7f1498ff9700 1 -- 172.21.15.107:0/2234791110 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14a0013040 con 0x558f32b671a0 2023-07-21T22:29:40.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.456+0000 7f1498ff9700 1 -- 172.21.15.107:0/2234791110 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0017620 con 0x558f32b671a0 2023-07-21T22:29:40.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.457+0000 7f14bd027b80 1 -- 172.21.15.107:0/2234791110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b671a0 msgr2=0x558f32b65430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.457+0000 7f14bd027b80 1 --2- 172.21.15.107:0/2234791110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b671a0 0x558f32b65430 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f14a000df00 tx=0x7f14a0009580 comp rx=0 tx=0).stop 2023-07-21T22:29:40.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.457+0000 7f14bd027b80 1 -- 172.21.15.107:0/2234791110 shutdown_connections 2023-07-21T22:29:40.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.457+0000 7f14bd027b80 1 --2- 172.21.15.107:0/2234791110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b65a10 0x558f32b65e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.457+0000 7f14bd027b80 1 --2- 172.21.15.107:0/2234791110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 0x558f32b66c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.457+0000 7f14bd027b80 1 --2- 172.21.15.107:0/2234791110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b671a0 0x558f32b65430 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.457+0000 7f14bd027b80 1 -- 172.21.15.107:0/2234791110 >> 172.21.15.107:0/2234791110 conn(0x558f32b5aef0 msgr2=0x558f32b60360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:40.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.457+0000 7f14bd027b80 1 -- 172.21.15.107:0/2234791110 shutdown_connections 2023-07-21T22:29:40.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.457+0000 7f14bd027b80 1 -- 172.21.15.107:0/2234791110 wait complete. 2023-07-21T22:29:40.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.457+0000 7f14bd027b80 1 -- start start 2023-07-21T22:29:40.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.457+0000 7f14bd027b80 1 Processor -- start 2023-07-21T22:29:40.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14bd027b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b65a10 0x558f32b6f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14bd027b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 0x558f32b6f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14bd027b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b671a0 0x558f32b765d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14bd027b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32a56ac0 con 0x558f32b65a10 2023-07-21T22:29:40.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14bd027b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32a567c0 con 0x558f32b66830 2023-07-21T22:29:40.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14bd027b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f3293a920 con 0x558f32b671a0 2023-07-21T22:29:40.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14bd027b80 10 client.0 _ll_register_callbacks cb 0x558f32a54760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:40.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14aa59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b65a10 0x558f32b6f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14aa59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b65a10 0x558f32b6f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48368/0 (socket says 172.21.15.107:48368) 2023-07-21T22:29:40.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14aa59c700 1 -- 172.21.15.107:0/579119371 learned_addr learned my addr 172.21.15.107:0/579119371 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:40.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14aad9d700 1 --2- 172.21.15.107:0/579119371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b671a0 0x558f32b765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14aa59c700 1 -- 172.21.15.107:0/579119371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b671a0 msgr2=0x558f32b765d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14aa59c700 1 --2- 172.21.15.107:0/579119371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b671a0 0x558f32b765d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14aa59c700 1 -- 172.21.15.107:0/579119371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 msgr2=0x558f32b6f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.512 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14aa59c700 1 --2- 172.21.15.107:0/579119371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 0x558f32b6f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.512 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14aa59c700 1 -- 172.21.15.107:0/579119371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c011440 con 0x558f32b65a10 2023-07-21T22:29:40.512 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14aad9d700 1 --2- 172.21.15.107:0/579119371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b671a0 0x558f32b765d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:40.512 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f14aa59c700 1 --2- 172.21.15.107:0/579119371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b65a10 0x558f32b6f050 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f149c011d10 tx=0x7f149c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d616bd7b858c22b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.512 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.458+0000 7f1491ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:40.512 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f1491ffb700 1 -- 172.21.15.107:0/579119371 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c011ea0 con 0x558f32b65a10 2023-07-21T22:29:40.512 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f1491ffb700 1 -- 172.21.15.107:0/579119371 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f149c017070 con 0x558f32b65a10 2023-07-21T22:29:40.512 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f14bd027b80 1 -- 172.21.15.107:0/579119371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f32a104f0 con 0x558f32b65a10 2023-07-21T22:29:40.513 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f1491ffb700 1 -- 172.21.15.107:0/579119371 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f149c004220 con 0x558f32b65a10 2023-07-21T22:29:40.513 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f14bd027b80 10 client.15549 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:40.513 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f14bd027b80 1 -- 172.21.15.107:0/579119371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f14a000dbb0 con 0x558f32b65a10 2023-07-21T22:29:40.513 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f1491ffb700 1 -- 172.21.15.107:0/579119371 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558f32a104f0 con 0x558f32b65a10 2023-07-21T22:29:40.513 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f1491ffb700 1 -- 172.21.15.107:0/579119371 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f149c003980 con 0x558f32b65a10 2023-07-21T22:29:40.513 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f14bd027b80 10 client.15549 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:40.513 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f14bd027b80 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:40.514 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f14bd027b80 20 client.15549 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:40.514 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f14bd027b80 1 --2- 172.21.15.107:0/579119371 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f32b7af30 0x558f32b7d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.514 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f14bd027b80 4 client.15549 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:40.514 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.459+0000 7f14bd027b80 1 -- 172.21.15.107:0/579119371 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f32aafeb0 con 0x558f32b7af30 2023-07-21T22:29:40.514 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.460+0000 7f14a9d9b700 1 --2- 172.21.15.107:0/579119371 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f32b7af30 0x558f32b7d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.514 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.460+0000 7f1491ffb700 10 client.15549 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:40.514 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.460+0000 7f14a9d9b700 1 --2- 172.21.15.107:0/579119371 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f32b7af30 0x558f32b7d340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac81a65e35a9314 server_cookie=332379eb6fa57854 in_seq=0 out_seq=0 2023-07-21T22:29:40.514 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f1491ffb700 1 -- 172.21.15.107:0/579119371 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558f32aafeb0 con 0x558f32b7af30 2023-07-21T22:29:40.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f1491ffb700 10 client.15549 handle_command_reply: tid=0 2023-07-21T22:29:40.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f14bd027b80 1 client.15549 shutdown 2023-07-21T22:29:40.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f14bd027b80 1 -- 172.21.15.107:0/579119371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b65a10 msgr2=0x558f32b6f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f14bd027b80 1 --2- 172.21.15.107:0/579119371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b65a10 0x558f32b6f050 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f149c011d10 tx=0x7f149c0047c0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f14bd027b80 1 -- 172.21.15.107:0/579119371 shutdown_connections 2023-07-21T22:29:40.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f14bd027b80 1 --2- 172.21.15.107:0/579119371 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f32b7af30 0x558f32b7d340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f14bd027b80 1 --2- 172.21.15.107:0/579119371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b671a0 0x558f32b765d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f1491ffb700 0 client.15549 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:40.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f14bd027b80 1 --2- 172.21.15.107:0/579119371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b65a10 0x558f32b6f050 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f14bd027b80 1 --2- 172.21.15.107:0/579119371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 0x558f32b6f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f14bd027b80 1 -- 172.21.15.107:0/579119371 >> 172.21.15.107:0/579119371 conn(0x558f32b5aef0 msgr2=0x558f32b5fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:40.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f14bd027b80 1 -- 172.21.15.107:0/579119371 shutdown_connections 2023-07-21T22:29:40.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f14bd027b80 1 -- 172.21.15.107:0/579119371 wait complete. 2023-07-21T22:29:40.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.461+0000 7f14bd027b80 20 client.15549 trim_cache size 0 max 16384 2023-07-21T22:29:40.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.487+0000 7f14bd027b80 1 Processor -- start 2023-07-21T22:29:40.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.487+0000 7f14bd027b80 1 -- start start 2023-07-21T22:29:40.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14bd027b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b65a10 0x558f32b6cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14bd027b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 0x558f32b76590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14bd027b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b671a0 0x558f32b6d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14bd027b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32b70220 con 0x558f32b65a10 2023-07-21T22:29:40.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14bd027b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32b57e50 con 0x558f32b66830 2023-07-21T22:29:40.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14bd027b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32b612b0 con 0x558f32b671a0 2023-07-21T22:29:40.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14aa59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b65a10 0x558f32b6cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14aa59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b65a10 0x558f32b6cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48376/0 (socket says 172.21.15.107:48376) 2023-07-21T22:29:40.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14aa59c700 1 -- 172.21.15.107:0/243507648 learned_addr learned my addr 172.21.15.107:0/243507648 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:40.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14a9d9b700 1 --2- 172.21.15.107:0/243507648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 0x558f32b76590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14aad9d700 1 --2- 172.21.15.107:0/243507648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b671a0 0x558f32b6d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.538 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14a9d9b700 1 -- 172.21.15.107:0/243507648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b671a0 msgr2=0x558f32b6d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.538 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14a9d9b700 1 --2- 172.21.15.107:0/243507648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b671a0 0x558f32b6d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.538 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14a9d9b700 1 -- 172.21.15.107:0/243507648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b65a10 msgr2=0x558f32b6cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.538 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14a9d9b700 1 --2- 172.21.15.107:0/243507648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b65a10 0x558f32b6cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.538 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14a9d9b700 1 -- 172.21.15.107:0/243507648 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f328cdc60 con 0x558f32b66830 2023-07-21T22:29:40.538 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.488+0000 7f14aad9d700 1 --2- 172.21.15.107:0/243507648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b671a0 0x558f32b6d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:40.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.489+0000 7f14a9d9b700 1 --2- 172.21.15.107:0/243507648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 0x558f32b76590 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14a000ded0 tx=0x7f14a000d620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b827183f3cc1c7f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.489+0000 7f14997fa700 1 -- 172.21.15.107:0/243507648 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a002c070 con 0x558f32b66830 2023-07-21T22:29:40.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.489+0000 7f14997fa700 1 -- 172.21.15.107:0/243507648 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14a0009de0 con 0x558f32b66830 2023-07-21T22:29:40.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.489+0000 7f14997fa700 1 -- 172.21.15.107:0/243507648 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0027070 con 0x558f32b66830 2023-07-21T22:29:40.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.489+0000 7f14bd027b80 1 -- 172.21.15.107:0/243507648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 msgr2=0x558f32b76590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.489+0000 7f14bd027b80 1 --2- 172.21.15.107:0/243507648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 0x558f32b76590 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14a000ded0 tx=0x7f14a000d620 comp rx=0 tx=0).stop 2023-07-21T22:29:40.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 -- 172.21.15.107:0/243507648 shutdown_connections 2023-07-21T22:29:40.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 --2- 172.21.15.107:0/243507648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b671a0 0x558f32b6d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 --2- 172.21.15.107:0/243507648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b65a10 0x558f32b6cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 --2- 172.21.15.107:0/243507648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b66830 0x558f32b76590 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 -- 172.21.15.107:0/243507648 >> 172.21.15.107:0/243507648 conn(0x558f32b5aef0 msgr2=0x558f32b6de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:40.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 -- 172.21.15.107:0/243507648 shutdown_connections 2023-07-21T22:29:40.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 -- 172.21.15.107:0/243507648 wait complete. 2023-07-21T22:29:40.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 -- start start 2023-07-21T22:29:40.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 Processor -- start 2023-07-21T22:29:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b65a10 0x558f32b761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b66830 0x558f32b72c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b671a0 0x558f32b731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32b738a0 con 0x558f32b66830 2023-07-21T22:29:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32b76860 con 0x558f32b671a0 2023-07-21T22:29:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f32b769f0 con 0x558f32b65a10 2023-07-21T22:29:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.490+0000 7f14bd027b80 10 client.0 _ll_register_callbacks cb 0x7f149c0118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14aa59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b65a10 0x558f32b761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14a9d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b66830 0x558f32b72c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14aa59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b65a10 0x558f32b761e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:36940/0 (socket says 172.21.15.107:36940) 2023-07-21T22:29:40.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14aad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b671a0 0x558f32b731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14aa59c700 1 -- 172.21.15.107:0/2771033030 learned_addr learned my addr 172.21.15.107:0/2771033030 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:40.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14a9d9b700 1 -- 172.21.15.107:0/2771033030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b65a10 msgr2=0x558f32b761e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14a9d9b700 1 --2- 172.21.15.107:0/2771033030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b65a10 0x558f32b761e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14a9d9b700 1 -- 172.21.15.107:0/2771033030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b671a0 msgr2=0x558f32b731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:40.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14a9d9b700 1 --2- 172.21.15.107:0/2771033030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b671a0 0x558f32b731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14a9d9b700 1 -- 172.21.15.107:0/2771033030 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac067b80 con 0x558f32b66830 2023-07-21T22:29:40.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14aa59c700 1 --2- 172.21.15.107:0/2771033030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b65a10 0x558f32b761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:40.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14aad9d700 1 --2- 172.21.15.107:0/2771033030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b671a0 0x558f32b731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:40.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14a9d9b700 1 --2- 172.21.15.107:0/2771033030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b66830 0x558f32b72c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x558f32b5cb80 tx=0x7f14a000d690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b9aa8cfff3d7db server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f1491ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:40.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f1491ffb700 1 -- 172.21.15.107:0/2771033030 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a002c070 con 0x558f32b66830 2023-07-21T22:29:40.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f14bd027b80 1 -- 172.21.15.107:0/2771033030 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14a0013040 con 0x558f32b66830 2023-07-21T22:29:40.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.491+0000 7f1491ffb700 1 -- 172.21.15.107:0/2771033030 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f14a0005040 con 0x558f32b66830 2023-07-21T22:29:40.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.492+0000 7f1491ffb700 1 -- 172.21.15.107:0/2771033030 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14a0027070 con 0x558f32b66830 2023-07-21T22:29:40.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.492+0000 7f14bd027b80 10 client.15555 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:40.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.492+0000 7f14bd027b80 1 -- 172.21.15.107:0/2771033030 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558f328e4de0 con 0x558f32b66830 2023-07-21T22:29:40.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.492+0000 7f1491ffb700 1 -- 172.21.15.107:0/2771033030 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14a0036020 con 0x558f32b66830 2023-07-21T22:29:40.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.492+0000 7f1491ffb700 1 -- 172.21.15.107:0/2771033030 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f14a002f020 con 0x558f32b66830 2023-07-21T22:29:40.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.492+0000 7f14bd027b80 10 client.15555 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:40.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.492+0000 7f14bd027b80 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:40.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.492+0000 7f14bd027b80 20 client.15555 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:40.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.492+0000 7f14bd027b80 1 --2- 172.21.15.107:0/2771033030 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f32bcf390 0x558f32bd1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.492+0000 7f14bd027b80 4 client.15555 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:29:40.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.492+0000 7f14bd027b80 1 -- 172.21.15.107:0/2771033030 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558f32a72050 con 0x558f32bcf390 2023-07-21T22:29:40.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.492+0000 7f14aa59c700 1 --2- 172.21.15.107:0/2771033030 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f32bcf390 0x558f32bd1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.546 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f1491ffb700 10 client.15555 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:40.546 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f14aa59c700 1 --2- 172.21.15.107:0/2771033030 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f32bcf390 0x558f32bd1770 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8c68f3f04be280 server_cookie=edb0bca9a5a50b71 in_seq=0 out_seq=0 2023-07-21T22:29:40.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f1491ffb700 1 -- 172.21.15.107:0/2771033030 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558f32a72050 con 0x558f32bcf390 2023-07-21T22:29:40.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f1491ffb700 10 client.15555 handle_command_reply: tid=0 2023-07-21T22:29:40.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f14bd027b80 1 client.15555 shutdown 2023-07-21T22:29:40.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f14bd027b80 1 -- 172.21.15.107:0/2771033030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b66830 msgr2=0x558f32b72c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f14bd027b80 1 --2- 172.21.15.107:0/2771033030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b66830 0x558f32b72c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x558f32b5cb80 tx=0x7f14a000d690 comp rx=0 tx=0).stop 2023-07-21T22:29:40.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f14bd027b80 1 -- 172.21.15.107:0/2771033030 shutdown_connections 2023-07-21T22:29:40.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f14bd027b80 1 --2- 172.21.15.107:0/2771033030 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f32bcf390 0x558f32bd1770 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f14bd027b80 1 --2- 172.21.15.107:0/2771033030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f32b671a0 0x558f32b731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f14bd027b80 1 --2- 172.21.15.107:0/2771033030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f32b65a10 0x558f32b761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f14bd027b80 1 --2- 172.21.15.107:0/2771033030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f32b66830 0x558f32b72c80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f1491ffb700 0 client.15555 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:40.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f14bd027b80 1 -- 172.21.15.107:0/2771033030 >> 172.21.15.107:0/2771033030 conn(0x558f32b5aef0 msgr2=0x558f32b6cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:40.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f14bd027b80 1 -- 172.21.15.107:0/2771033030 shutdown_connections 2023-07-21T22:29:40.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f14bd027b80 1 -- 172.21.15.107:0/2771033030 wait complete. 2023-07-21T22:29:40.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.493+0000 7f14bd027b80 20 client.15555 trim_cache size 0 max 16384 2023-07-21T22:29:40.549 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:29:40.549 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2023-07-21T22:29:40.549 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2023-07-21T22:29:40.549 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:29:40.550 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.494+0000 7f14b0ea2700 1 -- 172.21.15.107:0/3486273761 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f14940a11d0 msgr2=0x7f14940a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.550 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.494+0000 7f14b0ea2700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f14940a11d0 0x7f14940a3680 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f149c0105c0 tx=0x7f149c00b4a0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.550 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.494+0000 7f14b0ea2700 1 -- 172.21.15.107:0/3486273761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14ac100560 msgr2=0x7f14ac19bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.550 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.494+0000 7f14b0ea2700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14ac100560 0x7f14ac19bdb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f14a400b0a0 tx=0x7f14a4004d00 comp rx=0 tx=0).stop 2023-07-21T22:29:40.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.494+0000 7f14b0ea2700 1 -- 172.21.15.107:0/3486273761 shutdown_connections 2023-07-21T22:29:40.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.494+0000 7f14b0ea2700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14ac100560 0x7f14ac19bdb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.494+0000 7f14b0ea2700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f14ac0fe9e0 0x7f14ac194800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.494+0000 7f14b0ea2700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f14940a11d0 0x7f14940a3680 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.494+0000 7f14b0ea2700 1 --2- 172.21.15.107:0/3486273761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14ac0ffbd0 0x7f14ac194d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.494+0000 7f14b0ea2700 1 -- 172.21.15.107:0/3486273761 >> 172.21.15.107:0/3486273761 conn(0x7f14ac0fa000 msgr2=0x7f14ac0fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:40.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.494+0000 7f14b0ea2700 1 -- 172.21.15.107:0/3486273761 shutdown_connections 2023-07-21T22:29:40.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.494+0000 7f14b0ea2700 1 -- 172.21.15.107:0/3486273761 wait complete. 2023-07-21T22:29:40.555 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:29:40.555 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:29:40.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.637+0000 7f627c2b6700 1 -- 172.21.15.107:0/3416076315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62740fe9d0 msgr2=0x7f62740fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.638+0000 7f627c2b6700 1 --2- 172.21.15.107:0/3416076315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62740fe9d0 0x7f62740fede0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6270009990 tx=0x7f6270009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.638+0000 7f627c2b6700 1 -- 172.21.15.107:0/3416076315 shutdown_connections 2023-07-21T22:29:40.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.638+0000 7f627c2b6700 1 --2- 172.21.15.107:0/3416076315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6274100550 0x7f6274106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.638+0000 7f627c2b6700 1 --2- 172.21.15.107:0/3416076315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62740fe9d0 0x7f62740fede0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.638+0000 7f627c2b6700 1 --2- 172.21.15.107:0/3416076315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62740ffbc0 0x7f6274100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.638+0000 7f627c2b6700 1 -- 172.21.15.107:0/3416076315 >> 172.21.15.107:0/3416076315 conn(0x7f62740f9fd0 msgr2=0x7f62740fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:40.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.638+0000 7f627c2b6700 1 -- 172.21.15.107:0/3416076315 shutdown_connections 2023-07-21T22:29:40.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.638+0000 7f627c2b6700 1 -- 172.21.15.107:0/3416076315 wait complete. 2023-07-21T22:29:40.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.638+0000 7f627c2b6700 1 Processor -- start 2023-07-21T22:29:40.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.638+0000 7f627c2b6700 1 -- start start 2023-07-21T22:29:40.687 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627c2b6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62740fe9d0 0x7f6274195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.687 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627c2b6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62740ffbc0 0x7f6274195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.687 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627c2b6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6274100550 0x7f627419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.687 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627c2b6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627419d3a0 con 0x7f6274100550 2023-07-21T22:29:40.687 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627c2b6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627419d4e0 con 0x7f62740fe9d0 2023-07-21T22:29:40.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627c2b6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627419d670 con 0x7f62740ffbc0 2023-07-21T22:29:40.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627a853700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6274100550 0x7f627419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627a052700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62740fe9d0 0x7f6274195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627a853700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6274100550 0x7f627419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48400/0 (socket says 172.21.15.107:48400) 2023-07-21T22:29:40.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627a052700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62740fe9d0 0x7f6274195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:39224/0 (socket says 172.21.15.107:39224) 2023-07-21T22:29:40.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627a853700 1 -- 172.21.15.107:0/2800999411 learned_addr learned my addr 172.21.15.107:0/2800999411 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:40.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f6279851700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62740ffbc0 0x7f6274195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627a853700 1 -- 172.21.15.107:0/2800999411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62740ffbc0 msgr2=0x7f6274195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627a853700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62740ffbc0 0x7f6274195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627a853700 1 -- 172.21.15.107:0/2800999411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62740fe9d0 msgr2=0x7f6274195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:40.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627a853700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62740fe9d0 0x7f6274195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:40.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627a853700 1 -- 172.21.15.107:0/2800999411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62700095f0 con 0x7f6274100550 2023-07-21T22:29:40.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627a052700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62740fe9d0 0x7f6274195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:40.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f6279851700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62740ffbc0 0x7f6274195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:40.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.639+0000 7f627a853700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6274100550 0x7f627419cdd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f626c00ef30 tx=0x7f626c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.640+0000 7f626b7fe700 1 -- 172.21.15.107:0/2800999411 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c018070 con 0x7f6274100550 2023-07-21T22:29:40.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.640+0000 7f626b7fe700 1 -- 172.21.15.107:0/2800999411 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f626c00f040 con 0x7f6274100550 2023-07-21T22:29:40.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.640+0000 7f627c2b6700 1 -- 172.21.15.107:0/2800999411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6274107880 con 0x7f6274100550 2023-07-21T22:29:40.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.640+0000 7f626b7fe700 1 -- 172.21.15.107:0/2800999411 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c00cd00 con 0x7f6274100550 2023-07-21T22:29:40.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.640+0000 7f627c2b6700 1 -- 172.21.15.107:0/2800999411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6274107d90 con 0x7f6274100550 2023-07-21T22:29:40.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.641+0000 7f627c2b6700 1 -- 172.21.15.107:0/2800999411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627410b370 con 0x7f6274100550 2023-07-21T22:29:40.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.642+0000 7f626b7fe700 1 -- 172.21.15.107:0/2800999411 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f626c003a00 con 0x7f6274100550 2023-07-21T22:29:40.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.643+0000 7f626b7fe700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62600a11a0 0x7f62600a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:40.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.643+0000 7f627a052700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62600a11a0 0x7f62600a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:40.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.643+0000 7f626b7fe700 1 -- 172.21.15.107:0/2800999411 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f626c0cc450 con 0x7f6274100550 2023-07-21T22:29:40.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.644+0000 7f627a052700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62600a11a0 0x7f62600a3650 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f627000b2e0 tx=0x7f6270005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:40.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.649+0000 7f626b7fe700 1 -- 172.21.15.107:0/2800999411 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f626c095c10 con 0x7f6274100550 2023-07-21T22:29:41.001 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.955+0000 7f627c2b6700 1 -- 172.21.15.107:0/2800999411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f627410b530 con 0x7f6274100550 2023-07-21T22:29:41.002 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.956+0000 7f626b7fe700 1 -- 172.21.15.107:0/2800999411 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10869 (secure 0 0 0) 0x7f626c099230 con 0x7f6274100550 2023-07-21T22:29:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:29:41.002 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:29:21.093850+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:29:41.004 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.958+0000 7f627c2b6700 1 -- 172.21.15.107:0/2800999411 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62600a11a0 msgr2=0x7f62600a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.004 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.958+0000 7f627c2b6700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62600a11a0 0x7f62600a3650 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f627000b2e0 tx=0x7f6270005fb0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.004 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.958+0000 7f627c2b6700 1 -- 172.21.15.107:0/2800999411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6274100550 msgr2=0x7f627419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.004 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.958+0000 7f627c2b6700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6274100550 0x7f627419cdd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f626c00ef30 tx=0x7f626c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.005 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.958+0000 7f627c2b6700 1 -- 172.21.15.107:0/2800999411 shutdown_connections 2023-07-21T22:29:41.005 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.958+0000 7f627c2b6700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6274100550 0x7f627419cdd0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.005 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.958+0000 7f627c2b6700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62740fe9d0 0x7f6274195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.005 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.958+0000 7f627c2b6700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62600a11a0 0x7f62600a3650 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.005 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.958+0000 7f627c2b6700 1 --2- 172.21.15.107:0/2800999411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62740ffbc0 0x7f6274195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.005 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.958+0000 7f627c2b6700 1 -- 172.21.15.107:0/2800999411 >> 172.21.15.107:0/2800999411 conn(0x7f62740f9fd0 msgr2=0x7f62740fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.005 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.959+0000 7f627c2b6700 1 -- 172.21.15.107:0/2800999411 shutdown_connections 2023-07-21T22:29:41.005 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:40.959+0000 7f627c2b6700 1 -- 172.21.15.107:0/2800999411 wait complete. 2023-07-21T22:29:41.006 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2023-07-21T22:29:41.016 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-21T22:29:41.179 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.132+0000 7f4d268f6700 1 -- 172.21.15.107:0/8364648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d20107c20 msgr2=0x7f4d2010e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.179 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.132+0000 7f4d268f6700 1 --2- 172.21.15.107:0/8364648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d20107c20 0x7f4d2010e4e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800b0a0 tx=0x7f4d1800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.132+0000 7f4d268f6700 1 -- 172.21.15.107:0/8364648 shutdown_connections 2023-07-21T22:29:41.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.132+0000 7f4d268f6700 1 --2- 172.21.15.107:0/8364648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d20107c20 0x7f4d2010e4e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.132+0000 7f4d268f6700 1 --2- 172.21.15.107:0/8364648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d201060a0 0x7f4d201064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.132+0000 7f4d268f6700 1 --2- 172.21.15.107:0/8364648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d20107290 0x7f4d201076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.132+0000 7f4d268f6700 1 -- 172.21.15.107:0/8364648 >> 172.21.15.107:0/8364648 conn(0x7f4d200750d0 msgr2=0x7f4d20077520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.180 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.133+0000 7f4d268f6700 1 -- 172.21.15.107:0/8364648 shutdown_connections 2023-07-21T22:29:41.181 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.133+0000 7f4d268f6700 1 -- 172.21.15.107:0/8364648 wait complete. 2023-07-21T22:29:41.182 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.133+0000 7f4d268f6700 1 Processor -- start 2023-07-21T22:29:41.183 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.133+0000 7f4d268f6700 1 -- start start 2023-07-21T22:29:41.183 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.133+0000 7f4d268f6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d201060a0 0x7f4d2019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.183 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.133+0000 7f4d268f6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d20107290 0x7f4d2019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.183 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d268f6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d20107c20 0x7f4d201a3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.183 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d268f6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d201a3a40 con 0x7f4d20107c20 2023-07-21T22:29:41.183 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d268f6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d201a3b80 con 0x7f4d20107290 2023-07-21T22:29:41.184 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d268f6700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d201a3d10 con 0x7f4d201060a0 2023-07-21T22:29:41.184 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d24e93700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d20107c20 0x7f4d201a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.184 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d24e93700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d20107c20 0x7f4d201a3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48414/0 (socket says 172.21.15.107:48414) 2023-07-21T22:29:41.184 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d24e93700 1 -- 172.21.15.107:0/4275714724 learned_addr learned my addr 172.21.15.107:0/4275714724 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.184 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d1ffff700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d201060a0 0x7f4d2019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.184 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d17fff700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d20107290 0x7f4d2019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d24e93700 1 -- 172.21.15.107:0/4275714724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d201060a0 msgr2=0x7f4d2019bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d24e93700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d201060a0 0x7f4d2019bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d24e93700 1 -- 172.21.15.107:0/4275714724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d20107290 msgr2=0x7f4d2019c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d24e93700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d20107290 0x7f4d2019c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d24e93700 1 -- 172.21.15.107:0/4275714724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d18009d00 con 0x7f4d20107c20 2023-07-21T22:29:41.185 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d17fff700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d20107290 0x7f4d2019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:41.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d24e93700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d20107c20 0x7f4d201a3470 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800b780 tx=0x7f4d1800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d1dffb700 1 -- 172.21.15.107:0/4275714724 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800e040 con 0x7f4d20107c20 2023-07-21T22:29:41.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d1dffb700 1 -- 172.21.15.107:0/4275714724 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d18004e60 con 0x7f4d20107c20 2023-07-21T22:29:41.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d268f6700 1 -- 172.21.15.107:0/4275714724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d201a3f90 con 0x7f4d20107c20 2023-07-21T22:29:41.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d1dffb700 1 -- 172.21.15.107:0/4275714724 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1801b740 con 0x7f4d20107c20 2023-07-21T22:29:41.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.134+0000 7f4d268f6700 1 -- 172.21.15.107:0/4275714724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d201a43c0 con 0x7f4d20107c20 2023-07-21T22:29:41.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.137+0000 7f4d1dffb700 1 -- 172.21.15.107:0/4275714724 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4d1801da40 con 0x7f4d20107c20 2023-07-21T22:29:41.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.138+0000 7f4d1dffb700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4d0c0a11d0 0x7f4d0c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.138+0000 7f4d1ffff700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4d0c0a11d0 0x7f4d0c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.138+0000 7f4d1dffb700 1 -- 172.21.15.107:0/4275714724 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4d180ce030 con 0x7f4d20107c20 2023-07-21T22:29:41.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.139+0000 7f4d1ffff700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4d0c0a11d0 0x7f4d0c0a3680 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4d201070f0 tx=0x7f4d08007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.139+0000 7f4d32a7bb80 1 Processor -- start 2023-07-21T22:29:41.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.139+0000 7f4d32a7bb80 1 -- start start 2023-07-21T22:29:41.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.139+0000 7f4d32a7bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7b830 0x561e2fb7bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.139+0000 7f4d32a7bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 0x561e2fb7a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.139+0000 7f4d32a7bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7aa10 0x561e2fb7ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.139+0000 7f4d32a7bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2fa7fcb0 con 0x561e2fb7c1a0 2023-07-21T22:29:41.192 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.139+0000 7f4d32a7bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2fad2a40 con 0x561e2fb7b830 2023-07-21T22:29:41.192 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.139+0000 7f4d32a7bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2facbed0 con 0x561e2fb7aa10 2023-07-21T22:29:41.192 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d17fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 0x561e2fb7a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.192 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d17fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 0x561e2fb7a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48422/0 (socket says 172.21.15.107:48422) 2023-07-21T22:29:41.192 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d24e93700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7aa10 0x561e2fb7ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.192 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d17fff700 1 -- 172.21.15.107:0/4207230603 learned_addr learned my addr 172.21.15.107:0/4207230603 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.193 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d1ffff700 1 --2- 172.21.15.107:0/4207230603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7b830 0x561e2fb7bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.193 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d24e93700 1 -- 172.21.15.107:0/4207230603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7b830 msgr2=0x561e2fb7bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.193 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d24e93700 1 --2- 172.21.15.107:0/4207230603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7b830 0x561e2fb7bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.193 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d24e93700 1 -- 172.21.15.107:0/4207230603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 msgr2=0x561e2fb7a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.193 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d24e93700 1 --2- 172.21.15.107:0/4207230603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 0x561e2fb7a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.193 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d24e93700 1 -- 172.21.15.107:0/4207230603 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e2f8f9de0 con 0x561e2fb7aa10 2023-07-21T22:29:41.193 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d17fff700 1 --2- 172.21.15.107:0/4207230603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 0x561e2fb7a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:41.194 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d24e93700 1 --2- 172.21.15.107:0/4207230603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7aa10 0x561e2fb7ae90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4d180cbe80 tx=0x7f4d1801dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13608a8f00cf8fdd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.194 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d167fc700 1 -- 172.21.15.107:0/4207230603 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1809b4b0 con 0x561e2fb7aa10 2023-07-21T22:29:41.194 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d167fc700 1 -- 172.21.15.107:0/4207230603 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d1809f4e0 con 0x561e2fb7aa10 2023-07-21T22:29:41.194 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.140+0000 7f4d167fc700 1 -- 172.21.15.107:0/4207230603 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1809b970 con 0x561e2fb7aa10 2023-07-21T22:29:41.194 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.141+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/4207230603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7aa10 msgr2=0x561e2fb7ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.194 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.141+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/4207230603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7aa10 0x561e2fb7ae90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4d180cbe80 tx=0x7f4d1801dde0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.194 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.141+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/4207230603 shutdown_connections 2023-07-21T22:29:41.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.141+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/4207230603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7aa10 0x561e2fb7ae90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.141+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/4207230603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7b830 0x561e2fb7bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.141+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/4207230603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 0x561e2fb7a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.141+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/4207230603 >> 172.21.15.107:0/4207230603 conn(0x561e2fb6fef0 msgr2=0x561e2fb75360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.141+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/4207230603 shutdown_connections 2023-07-21T22:29:41.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.141+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/4207230603 wait complete. 2023-07-21T22:29:41.195 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.142+0000 7f4d32a7bb80 1 -- start start 2023-07-21T22:29:41.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.142+0000 7f4d32a7bb80 1 Processor -- start 2023-07-21T22:29:41.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.142+0000 7f4d32a7bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7aa10 0x561e2fb83ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.142+0000 7f4d32a7bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7b830 0x561e2fb843e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.142+0000 7f4d32a7bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7c1a0 0x561e2fb8b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.142+0000 7f4d32a7bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2fa6bac0 con 0x561e2fb7b830 2023-07-21T22:29:41.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.142+0000 7f4d32a7bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2fa6b7c0 con 0x561e2fb7aa10 2023-07-21T22:29:41.196 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.142+0000 7f4d32a7bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2f94f920 con 0x561e2fb7c1a0 2023-07-21T22:29:41.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.142+0000 7f4d32a7bb80 10 client.0 _ll_register_callbacks cb 0x561e2fa69760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:41.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.142+0000 7f4d1ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7aa10 0x561e2fb83ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.142+0000 7f4d1ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7aa10 0x561e2fb83ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:39252/0 (socket says 172.21.15.107:39252) 2023-07-21T22:29:41.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.142+0000 7f4d1ffff700 1 -- 172.21.15.107:0/657554425 learned_addr learned my addr 172.21.15.107:0/657554425 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.143+0000 7f4d1ffff700 1 -- 172.21.15.107:0/657554425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7c1a0 msgr2=0x561e2fb8b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.143+0000 7f4d1ffff700 1 --2- 172.21.15.107:0/657554425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7c1a0 0x561e2fb8b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.197 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.143+0000 7f4d1ffff700 1 -- 172.21.15.107:0/657554425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7b830 msgr2=0x561e2fb843e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.143+0000 7f4d1ffff700 1 --2- 172.21.15.107:0/657554425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7b830 0x561e2fb843e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.143+0000 7f4d1ffff700 1 -- 172.21.15.107:0/657554425 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d10009900 con 0x561e2fb7aa10 2023-07-21T22:29:41.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.143+0000 7f4d1ffff700 1 --2- 172.21.15.107:0/657554425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7aa10 0x561e2fb83ea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4d080116c0 tx=0x7f4d080047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c31dff4ac0680fd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.143+0000 7f4cff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:29:41.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.143+0000 7f4cff7fe700 1 -- 172.21.15.107:0/657554425 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08004250 con 0x561e2fb7aa10 2023-07-21T22:29:41.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.143+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/657554425 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e2fa254f0 con 0x561e2fb7aa10 2023-07-21T22:29:41.198 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.143+0000 7f4cff7fe700 1 -- 172.21.15.107:0/657554425 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4d08017070 con 0x561e2fb7aa10 2023-07-21T22:29:41.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.143+0000 7f4cff7fe700 1 -- 172.21.15.107:0/657554425 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d080043b0 con 0x561e2fb7aa10 2023-07-21T22:29:41.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.143+0000 7f4d32a7bb80 10 client.25133 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:41.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.143+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/657554425 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f4d201a43c0 con 0x561e2fb7aa10 2023-07-21T22:29:41.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.144+0000 7f4cff7fe700 1 -- 172.21.15.107:0/657554425 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561e2fa254f0 con 0x561e2fb7aa10 2023-07-21T22:29:41.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.144+0000 7f4cff7fe700 1 -- 172.21.15.107:0/657554425 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4d0801f430 con 0x561e2fb7aa10 2023-07-21T22:29:41.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.144+0000 7f4d32a7bb80 10 client.25133 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:41.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.144+0000 7f4d32a7bb80 10 client.25133 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-21T22:29:41.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.144+0000 7f4d32a7bb80 20 client.25133 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:41.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.144+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/657554425 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x561e2fb8fdd0 0x561e2fb921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.144+0000 7f4d32a7bb80 4 client.25133 mds_command: new command op to 24437 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:41.201 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.144+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/657554425 --> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e2fac4eb0 con 0x561e2fb8fdd0 2023-07-21T22:29:41.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.144+0000 7f4d17fff700 1 --2- 172.21.15.107:0/657554425 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x561e2fb8fdd0 0x561e2fb921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.145+0000 7f4cff7fe700 10 client.25133 ms_handle_connect on v2:172.21.15.132:6832/3293513462 2023-07-21T22:29:41.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.145+0000 7f4d17fff700 1 --2- 172.21.15.107:0/657554425 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x561e2fb8fdd0 0x561e2fb921e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=442c512ee25b8f77 server_cookie=9cc001d190716909 in_seq=0 out_seq=0 2023-07-21T22:29:41.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.145+0000 7f4cff7fe700 1 -- 172.21.15.107:0/657554425 <== mds.4 v2:172.21.15.132:6832/3293513462 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561e2fac4eb0 con 0x561e2fb8fdd0 2023-07-21T22:29:41.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.145+0000 7f4cff7fe700 10 client.25133 handle_command_reply: tid=0 2023-07-21T22:29:41.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.145+0000 7f4d32a7bb80 1 client.25133 shutdown 2023-07-21T22:29:41.202 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.146+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/657554425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7aa10 msgr2=0x561e2fb83ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.146+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/657554425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7aa10 0x561e2fb83ea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4d080116c0 tx=0x7f4d080047c0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.146+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/657554425 shutdown_connections 2023-07-21T22:29:41.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.146+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/657554425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7c1a0 0x561e2fb8b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.146+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/657554425 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x561e2fb8fdd0 0x561e2fb921e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.146+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/657554425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7aa10 0x561e2fb83ea0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.146+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/657554425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7b830 0x561e2fb843e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.203 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.146+0000 7f4cff7fe700 0 client.25133 ms_handle_reset on v2:172.21.15.132:6832/3293513462 2023-07-21T22:29:41.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.146+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/657554425 >> 172.21.15.107:0/657554425 conn(0x561e2fb6fef0 msgr2=0x561e2fb74a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.146+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/657554425 shutdown_connections 2023-07-21T22:29:41.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.146+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/657554425 wait complete. 2023-07-21T22:29:41.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.146+0000 7f4d32a7bb80 20 client.25133 trim_cache size 0 max 16384 2023-07-21T22:29:41.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d32a7bb80 1 Processor -- start 2023-07-21T22:29:41.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d32a7bb80 1 -- start start 2023-07-21T22:29:41.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d32a7bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7aa10 0x561e2fb81f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d32a7bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7b830 0x561e2fb824d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d32a7bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 0x561e2fb8b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d32a7bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2fb839f0 con 0x561e2fb7c1a0 2023-07-21T22:29:41.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d32a7bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2fb8d110 con 0x561e2fb7aa10 2023-07-21T22:29:41.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d32a7bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2fb8fad0 con 0x561e2fb7b830 2023-07-21T22:29:41.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d24e93700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 0x561e2fb8b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d24e93700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 0x561e2fb8b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48434/0 (socket says 172.21.15.107:48434) 2023-07-21T22:29:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d24e93700 1 -- 172.21.15.107:0/3546964742 learned_addr learned my addr 172.21.15.107:0/3546964742 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d17fff700 1 --2- 172.21.15.107:0/3546964742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7b830 0x561e2fb824d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d1ffff700 1 --2- 172.21.15.107:0/3546964742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7aa10 0x561e2fb81f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d24e93700 1 -- 172.21.15.107:0/3546964742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7b830 msgr2=0x561e2fb824d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d24e93700 1 --2- 172.21.15.107:0/3546964742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7b830 0x561e2fb824d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d24e93700 1 -- 172.21.15.107:0/3546964742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7aa10 msgr2=0x561e2fb81f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d24e93700 1 --2- 172.21.15.107:0/3546964742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7aa10 0x561e2fb81f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d24e93700 1 -- 172.21.15.107:0/3546964742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e2f8e2c60 con 0x561e2fb7c1a0 2023-07-21T22:29:41.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.167+0000 7f4d1ffff700 1 --2- 172.21.15.107:0/3546964742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7aa10 0x561e2fb81f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:41.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.168+0000 7f4d24e93700 1 --2- 172.21.15.107:0/3546964742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 0x561e2fb8b3e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009fd0 tx=0x7f4d1801dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f059b1035f853af server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.168+0000 7f4d16ffd700 1 -- 172.21.15.107:0/3546964742 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1809b4b0 con 0x561e2fb7c1a0 2023-07-21T22:29:41.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.168+0000 7f4d16ffd700 1 -- 172.21.15.107:0/3546964742 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d180a3870 con 0x561e2fb7c1a0 2023-07-21T22:29:41.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.168+0000 7f4d16ffd700 1 -- 172.21.15.107:0/3546964742 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1809a9a0 con 0x561e2fb7c1a0 2023-07-21T22:29:41.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.168+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/3546964742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 msgr2=0x561e2fb8b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.168+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/3546964742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 0x561e2fb8b3e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009fd0 tx=0x7f4d1801dd50 comp rx=0 tx=0).stop 2023-07-21T22:29:41.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.169+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/3546964742 shutdown_connections 2023-07-21T22:29:41.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.169+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/3546964742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7c1a0 0x561e2fb8b3e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.169+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/3546964742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7aa10 0x561e2fb81f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.169+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/3546964742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7b830 0x561e2fb824d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.169+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/3546964742 >> 172.21.15.107:0/3546964742 conn(0x561e2fb6fef0 msgr2=0x561e2fb7ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.169+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/3546964742 shutdown_connections 2023-07-21T22:29:41.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.169+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/3546964742 wait complete. 2023-07-21T22:29:41.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.169+0000 7f4d32a7bb80 1 -- start start 2023-07-21T22:29:41.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.169+0000 7f4d32a7bb80 1 Processor -- start 2023-07-21T22:29:41.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d32a7bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7aa10 0x561e2fb874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d32a7bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7b830 0x561e2fb87a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d32a7bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7c1a0 0x561e2fb8aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d32a7bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2fb8af70 con 0x561e2fb7aa10 2023-07-21T22:29:41.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d32a7bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2fb8b0e0 con 0x561e2fb7c1a0 2023-07-21T22:29:41.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d32a7bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2fb8b2a0 con 0x561e2fb7b830 2023-07-21T22:29:41.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d32a7bb80 10 client.0 _ll_register_callbacks cb 0x7f4d08010cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:41.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d1ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7aa10 0x561e2fb874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d1ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7aa10 0x561e2fb874d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48438/0 (socket says 172.21.15.107:48438) 2023-07-21T22:29:41.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d1ffff700 1 -- 172.21.15.107:0/915938163 learned_addr learned my addr 172.21.15.107:0/915938163 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d24e93700 1 --2- 172.21.15.107:0/915938163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7c1a0 0x561e2fb8aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d1ffff700 1 -- 172.21.15.107:0/915938163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7b830 msgr2=0x561e2fb87a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d1ffff700 1 --2- 172.21.15.107:0/915938163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7b830 0x561e2fb87a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d1ffff700 1 -- 172.21.15.107:0/915938163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7c1a0 msgr2=0x561e2fb8aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d1ffff700 1 --2- 172.21.15.107:0/915938163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7c1a0 0x561e2fb8aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d1ffff700 1 -- 172.21.15.107:0/915938163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e2f8f9de0 con 0x561e2fb7aa10 2023-07-21T22:29:41.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d1ffff700 1 --2- 172.21.15.107:0/915938163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7aa10 0x561e2fb874d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4d08007a30 tx=0x7f4d08012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a927dce64e03fe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4cff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:41.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4cff7fe700 1 -- 172.21.15.107:0/915938163 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0801fc00 con 0x561e2fb7aa10 2023-07-21T22:29:41.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4cff7fe700 1 -- 172.21.15.107:0/915938163 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4d08017040 con 0x561e2fb7aa10 2023-07-21T22:29:41.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.170+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/915938163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d1809f4e0 con 0x561e2fb7aa10 2023-07-21T22:29:41.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.171+0000 7f4cff7fe700 1 -- 172.21.15.107:0/915938163 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d0801fd60 con 0x561e2fb7aa10 2023-07-21T22:29:41.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.171+0000 7f4d32a7bb80 10 client.15576 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:41.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.171+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/915938163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561e2f920250 con 0x561e2fb7aa10 2023-07-21T22:29:41.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.171+0000 7f4cff7fe700 1 -- 172.21.15.107:0/915938163 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d08049080 con 0x561e2fb7aa10 2023-07-21T22:29:41.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.171+0000 7f4cff7fe700 1 -- 172.21.15.107:0/915938163 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4d08043030 con 0x561e2fb7aa10 2023-07-21T22:29:41.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.171+0000 7f4d32a7bb80 10 client.15576 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:41.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.171+0000 7f4d32a7bb80 10 client.15576 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-21T22:29:41.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.171+0000 7f4d32a7bb80 20 client.15576 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:41.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.171+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/915938163 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x561e2fbe4e80 0x561e2fbe7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.171+0000 7f4d32a7bb80 4 client.15576 mds_command: new command op to 24437 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:29:41.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.171+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/915938163 --> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561e2fa87050 con 0x561e2fbe4e80 2023-07-21T22:29:41.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.171+0000 7f4d17fff700 1 --2- 172.21.15.107:0/915938163 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x561e2fbe4e80 0x561e2fbe7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4cff7fe700 10 client.15576 ms_handle_connect on v2:172.21.15.132:6832/3293513462 2023-07-21T22:29:41.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4d17fff700 1 --2- 172.21.15.107:0/915938163 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x561e2fbe4e80 0x561e2fbe7290 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=83d3133c1ddd961b server_cookie=360c6f877cab4e68 in_seq=0 out_seq=0 2023-07-21T22:29:41.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4cff7fe700 1 -- 172.21.15.107:0/915938163 <== mds.4 v2:172.21.15.132:6832/3293513462 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561e2fa87050 con 0x561e2fbe4e80 2023-07-21T22:29:41.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4cff7fe700 10 client.15576 handle_command_reply: tid=0 2023-07-21T22:29:41.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4d32a7bb80 1 client.15576 shutdown 2023-07-21T22:29:41.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/915938163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7aa10 msgr2=0x561e2fb874d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/915938163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7aa10 0x561e2fb874d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4d08007a30 tx=0x7f4d08012f90 comp rx=0 tx=0).stop 2023-07-21T22:29:41.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/915938163 shutdown_connections 2023-07-21T22:29:41.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/915938163 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x561e2fbe4e80 0x561e2fbe7290 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/915938163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e2fb7c1a0 0x561e2fb8aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4cff7fe700 0 client.15576 ms_handle_reset on v2:172.21.15.132:6832/3293513462 2023-07-21T22:29:41.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/915938163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e2fb7aa10 0x561e2fb874d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4d32a7bb80 1 --2- 172.21.15.107:0/915938163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e2fb7b830 0x561e2fb87a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.172+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/915938163 >> 172.21.15.107:0/915938163 conn(0x561e2fb6fef0 msgr2=0x561e2fb81f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/915938163 shutdown_connections 2023-07-21T22:29:41.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d32a7bb80 1 -- 172.21.15.107:0/915938163 wait complete. 2023-07-21T22:29:41.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d32a7bb80 20 client.15576 trim_cache size 0 max 16384 2023-07-21T22:29:41.236 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:29:41.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d268f6700 1 -- 172.21.15.107:0/4275714724 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4d0c0a11d0 msgr2=0x7f4d0c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d268f6700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4d0c0a11d0 0x7f4d0c0a3680 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4d201070f0 tx=0x7f4d08007400 comp rx=0 tx=0).stop 2023-07-21T22:29:41.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d268f6700 1 -- 172.21.15.107:0/4275714724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d20107c20 msgr2=0x7f4d201a3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d268f6700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d20107c20 0x7f4d201a3470 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800b780 tx=0x7f4d1800bd00 comp rx=0 tx=0).stop 2023-07-21T22:29:41.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d268f6700 1 -- 172.21.15.107:0/4275714724 shutdown_connections 2023-07-21T22:29:41.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d268f6700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d20107c20 0x7f4d201a3470 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d268f6700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4d0c0a11d0 0x7f4d0c0a3680 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d268f6700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d201060a0 0x7f4d2019bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d268f6700 1 --2- 172.21.15.107:0/4275714724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d20107290 0x7f4d2019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d268f6700 1 -- 172.21.15.107:0/4275714724 >> 172.21.15.107:0/4275714724 conn(0x7f4d200750d0 msgr2=0x7f4d2010a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d268f6700 1 -- 172.21.15.107:0/4275714724 shutdown_connections 2023-07-21T22:29:41.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.173+0000 7f4d268f6700 1 -- 172.21.15.107:0/4275714724 wait complete. 2023-07-21T22:29:41.242 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-21T22:29:41.265 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:41 smithi107 ceph-mon[96543]: pgmap v311: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-07-21T22:29:41.265 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:41 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2800999411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:29:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.338+0000 7f4b7115b700 1 -- 172.21.15.107:0/1533043403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b6c100560 msgr2=0x7f4b6c106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.338+0000 7f4b7115b700 1 --2- 172.21.15.107:0/1533043403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b6c100560 0x7f4b6c106e20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400b0a0 tx=0x7f4b6400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.386 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.339+0000 7f4b7115b700 1 -- 172.21.15.107:0/1533043403 shutdown_connections 2023-07-21T22:29:41.386 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.339+0000 7f4b7115b700 1 --2- 172.21.15.107:0/1533043403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b6c100560 0x7f4b6c106e20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.386 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.339+0000 7f4b7115b700 1 --2- 172.21.15.107:0/1533043403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c0fe9e0 0x7f4b6c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.386 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.339+0000 7f4b7115b700 1 --2- 172.21.15.107:0/1533043403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b6c0ffbd0 0x7f4b6c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.386 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.339+0000 7f4b7115b700 1 -- 172.21.15.107:0/1533043403 >> 172.21.15.107:0/1533043403 conn(0x7f4b6c0fa000 msgr2=0x7f4b6c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.386 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.339+0000 7f4b7115b700 1 -- 172.21.15.107:0/1533043403 shutdown_connections 2023-07-21T22:29:41.386 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.339+0000 7f4b7115b700 1 -- 172.21.15.107:0/1533043403 wait complete. 2023-07-21T22:29:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.339+0000 7f4b7115b700 1 Processor -- start 2023-07-21T22:29:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b7115b700 1 -- start start 2023-07-21T22:29:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b7115b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c0fe9e0 0x7f4b6c195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b7115b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b6c0ffbd0 0x7f4b6c195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b7115b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b6c100560 0x7f4b6c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b7115b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c19d490 con 0x7f4b6c100560 2023-07-21T22:29:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b7115b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c19d5d0 con 0x7f4b6c0fe9e0 2023-07-21T22:29:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b7115b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c19d760 con 0x7f4b6c0ffbd0 2023-07-21T22:29:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b6b59e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b6c100560 0x7f4b6c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b6ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c0fe9e0 0x7f4b6c195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b6a59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b6c0ffbd0 0x7f4b6c195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b6ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c0fe9e0 0x7f4b6c195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:39280/0 (socket says 172.21.15.107:39280) 2023-07-21T22:29:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b6ad9d700 1 -- 172.21.15.107:0/1518657157 learned_addr learned my addr 172.21.15.107:0/1518657157 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b6b59e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b6c100560 0x7f4b6c19cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48456/0 (socket says 172.21.15.107:48456) 2023-07-21T22:29:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b6ad9d700 1 -- 172.21.15.107:0/1518657157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b6c0ffbd0 msgr2=0x7f4b6c195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b6ad9d700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b6c0ffbd0 0x7f4b6c195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b6ad9d700 1 -- 172.21.15.107:0/1518657157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b6c100560 msgr2=0x7f4b6c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b6ad9d700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b6c100560 0x7f4b6c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b6ad9d700 1 -- 172.21.15.107:0/1518657157 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b64009d00 con 0x7f4b6c0fe9e0 2023-07-21T22:29:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b6a59c700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b6c0ffbd0 0x7f4b6c195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.340+0000 7f4b6b59e700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b6c100560 0x7f4b6c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.341+0000 7f4b6ad9d700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c0fe9e0 0x7f4b6c195910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00e970 tx=0x7f4b5c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.341+0000 7f4b53fff700 1 -- 172.21.15.107:0/1518657157 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c009790 con 0x7f4b6c0fe9e0 2023-07-21T22:29:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.341+0000 7f4b7115b700 1 -- 172.21.15.107:0/1518657157 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6c19da40 con 0x7f4b6c0fe9e0 2023-07-21T22:29:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.341+0000 7f4b53fff700 1 -- 172.21.15.107:0/1518657157 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5c00c7e0 con 0x7f4b6c0fe9e0 2023-07-21T22:29:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.341+0000 7f4b53fff700 1 -- 172.21.15.107:0/1518657157 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c003c70 con 0x7f4b6c0fe9e0 2023-07-21T22:29:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.341+0000 7f4b7115b700 1 -- 172.21.15.107:0/1518657157 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c19dfa0 con 0x7f4b6c0fe9e0 2023-07-21T22:29:41.394 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.343+0000 7f4b53fff700 1 -- 172.21.15.107:0/1518657157 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4b5c00cc50 con 0x7f4b6c0fe9e0 2023-07-21T22:29:41.394 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.344+0000 7f4b53fff700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4b540a11d0 0x7f4b540a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.394 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.344+0000 7f4b6a59c700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4b540a11d0 0x7f4b540a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.394 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.344+0000 7f4b53fff700 1 -- 172.21.15.107:0/1518657157 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4b5c014070 con 0x7f4b6c0fe9e0 2023-07-21T22:29:41.394 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.345+0000 7f4b6a59c700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4b540a11d0 0x7f4b540a3680 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4b60011490 tx=0x7f4b60009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.395 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.345+0000 7f4b7d2e0b80 1 Processor -- start 2023-07-21T22:29:41.395 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.345+0000 7f4b7d2e0b80 1 -- start start 2023-07-21T22:29:41.395 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b7d2e0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6804a80 0x5602c6804e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.395 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b7d2e0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 0x5602c6805a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.395 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b7d2e0b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 0x5602c6806460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.395 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b7d2e0b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602c6705cb0 con 0x5602c6805ff0 2023-07-21T22:29:41.395 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b7d2e0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602c6758a40 con 0x5602c6804a80 2023-07-21T22:29:41.395 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b7d2e0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602c6751ed0 con 0x5602c68053d0 2023-07-21T22:29:41.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b6b59e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 0x5602c6806460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b6b59e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 0x5602c6806460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48464/0 (socket says 172.21.15.107:48464) 2023-07-21T22:29:41.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b6b59e700 1 -- 172.21.15.107:0/10326210 learned_addr learned my addr 172.21.15.107:0/10326210 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b6a59c700 1 --2- 172.21.15.107:0/10326210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 0x5602c6805a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b6ad9d700 1 --2- 172.21.15.107:0/10326210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6804a80 0x5602c6804e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b6b59e700 1 -- 172.21.15.107:0/10326210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 msgr2=0x5602c6805a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b6b59e700 1 --2- 172.21.15.107:0/10326210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 0x5602c6805a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b6b59e700 1 -- 172.21.15.107:0/10326210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6804a80 msgr2=0x5602c6804e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b6b59e700 1 --2- 172.21.15.107:0/10326210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6804a80 0x5602c6804e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b6b59e700 1 -- 172.21.15.107:0/10326210 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602c657fde0 con 0x5602c6805ff0 2023-07-21T22:29:41.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b6a59c700 1 --2- 172.21.15.107:0/10326210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 0x5602c6805a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:41.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b6b59e700 1 --2- 172.21.15.107:0/10326210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 0x5602c6806460 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009c00 tx=0x7f4b64009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62461f64ca572d99 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.346+0000 7f4b517fa700 1 -- 172.21.15.107:0/10326210 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400e040 con 0x5602c6805ff0 2023-07-21T22:29:41.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.347+0000 7f4b517fa700 1 -- 172.21.15.107:0/10326210 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b64006020 con 0x5602c6805ff0 2023-07-21T22:29:41.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.347+0000 7f4b517fa700 1 -- 172.21.15.107:0/10326210 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6401c440 con 0x5602c6805ff0 2023-07-21T22:29:41.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.347+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/10326210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 msgr2=0x5602c6806460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.347+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/10326210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 0x5602c6806460 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009c00 tx=0x7f4b64009c30 comp rx=0 tx=0).stop 2023-07-21T22:29:41.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.347+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/10326210 shutdown_connections 2023-07-21T22:29:41.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.347+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/10326210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 0x5602c6806460 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.347+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/10326210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6804a80 0x5602c6804e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.347+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/10326210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 0x5602c6805a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.347+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/10326210 >> 172.21.15.107:0/10326210 conn(0x5602c67f5ef0 msgr2=0x5602c67fb360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.399 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.347+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/10326210 shutdown_connections 2023-07-21T22:29:41.399 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.348+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/10326210 wait complete. 2023-07-21T22:29:41.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.348+0000 7f4b7d2e0b80 1 -- start start 2023-07-21T22:29:41.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.348+0000 7f4b7d2e0b80 1 Processor -- start 2023-07-21T22:29:41.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.348+0000 7f4b7d2e0b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 0x5602c680e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.348+0000 7f4b7d2e0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 0x5602c680e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.348+0000 7f4b7d2e0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6805ff0 0x5602c6815980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.348+0000 7f4b7d2e0b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602c66f1ac0 con 0x5602c6804a80 2023-07-21T22:29:41.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.348+0000 7f4b7d2e0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602c66f17c0 con 0x5602c6805ff0 2023-07-21T22:29:41.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.348+0000 7f4b7d2e0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602c65d5920 con 0x5602c68053d0 2023-07-21T22:29:41.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.348+0000 7f4b7d2e0b80 10 client.0 _ll_register_callbacks cb 0x5602c66ef760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:41.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.348+0000 7f4b6ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 0x5602c680e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.348+0000 7f4b6ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 0x5602c680e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48466/0 (socket says 172.21.15.107:48466) 2023-07-21T22:29:41.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.348+0000 7f4b6ad9d700 1 -- 172.21.15.107:0/3392450313 learned_addr learned my addr 172.21.15.107:0/3392450313 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b6a59c700 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 0x5602c680e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b6b59e700 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6805ff0 0x5602c6815980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b6ad9d700 1 -- 172.21.15.107:0/3392450313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 msgr2=0x5602c680e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b6ad9d700 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 0x5602c680e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b6ad9d700 1 -- 172.21.15.107:0/3392450313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6805ff0 msgr2=0x5602c6815980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b6ad9d700 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6805ff0 0x5602c6815980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b6ad9d700 1 -- 172.21.15.107:0/3392450313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c067670 con 0x5602c6804a80 2023-07-21T22:29:41.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b6b59e700 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6805ff0 0x5602c6815980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:41.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b6ad9d700 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 0x5602c680e400 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0ca550 tx=0x7f4b5c099f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7956b0826f651be server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:41.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b4eff5700 1 -- 172.21.15.107:0/3392450313 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c09a2c0 con 0x5602c6804a80 2023-07-21T22:29:41.404 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b4eff5700 1 -- 172.21.15.107:0/3392450313 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4b5c0b8210 con 0x5602c6804a80 2023-07-21T22:29:41.405 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/3392450313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602c66ab4f0 con 0x5602c6804a80 2023-07-21T22:29:41.405 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b4eff5700 1 -- 172.21.15.107:0/3392450313 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b5c099350 con 0x5602c6804a80 2023-07-21T22:29:41.405 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b7d2e0b80 10 client.15588 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:41.405 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/3392450313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5602c6568c60 con 0x5602c6804a80 2023-07-21T22:29:41.405 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.349+0000 7f4b4eff5700 1 -- 172.21.15.107:0/3392450313 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5602c66ab4f0 con 0x5602c6804a80 2023-07-21T22:29:41.405 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.350+0000 7f4b4eff5700 1 -- 172.21.15.107:0/3392450313 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4b5c0adb60 con 0x5602c6804a80 2023-07-21T22:29:41.405 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.350+0000 7f4b7d2e0b80 10 client.15588 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:41.405 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.350+0000 7f4b7d2e0b80 10 client.15588 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T22:29:41.406 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.350+0000 7f4b7d2e0b80 20 client.15588 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:41.406 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.350+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5602c681a1b0 0x5602c681c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.406 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.350+0000 7f4b7d2e0b80 4 client.15588 mds_command: new command op to 24455 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:41.406 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.350+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/3392450313 --> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602c674aeb0 con 0x5602c681a1b0 2023-07-21T22:29:41.406 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.350+0000 7f4b6a59c700 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5602c681a1b0 0x5602c681c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.406 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.351+0000 7f4b4eff5700 10 client.15588 ms_handle_connect on v2:172.21.15.132:6834/1022724305 2023-07-21T22:29:41.406 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.351+0000 7f4b6a59c700 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5602c681a1b0 0x5602c681c5c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e2b0b914f5cef9bc server_cookie=5559854afb1746d3 in_seq=0 out_seq=0 2023-07-21T22:29:41.407 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.351+0000 7f4b4eff5700 1 -- 172.21.15.107:0/3392450313 <== mds.3 v2:172.21.15.132:6834/1022724305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5602c674aeb0 con 0x5602c681a1b0 2023-07-21T22:29:41.407 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.351+0000 7f4b4eff5700 10 client.15588 handle_command_reply: tid=0 2023-07-21T22:29:41.407 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.351+0000 7f4b7d2e0b80 1 client.15588 shutdown 2023-07-21T22:29:41.407 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.351+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/3392450313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 msgr2=0x5602c680e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.407 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.351+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 0x5602c680e400 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0ca550 tx=0x7f4b5c099f10 comp rx=0 tx=0).stop 2023-07-21T22:29:41.407 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.351+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/3392450313 shutdown_connections 2023-07-21T22:29:41.407 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.351+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5602c681a1b0 0x5602c681c5c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.407 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.352+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6805ff0 0x5602c6815980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.352+0000 7f4b4eff5700 0 client.15588 ms_handle_reset on v2:172.21.15.132:6834/1022724305 2023-07-21T22:29:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.352+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 0x5602c680e400 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.352+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/3392450313 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 0x5602c680e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.352+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/3392450313 >> 172.21.15.107:0/3392450313 conn(0x5602c67f5ef0 msgr2=0x5602c67faa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.352+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/3392450313 shutdown_connections 2023-07-21T22:29:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.352+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/3392450313 wait complete. 2023-07-21T22:29:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.352+0000 7f4b7d2e0b80 20 client.15588 trim_cache size 0 max 16384 2023-07-21T22:29:41.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b7d2e0b80 1 Processor -- start 2023-07-21T22:29:41.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b7d2e0b80 1 -- start start 2023-07-21T22:29:41.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b7d2e0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c6804a80 0x5602c680dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b7d2e0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c68053d0 0x5602c680d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b7d2e0b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 0x5602c6808e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b7d2e0b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602c67fc5e0 con 0x5602c6805ff0 2023-07-21T22:29:41.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b7d2e0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602c6806780 con 0x5602c68053d0 2023-07-21T22:29:41.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b7d2e0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602c681c5c0 con 0x5602c6804a80 2023-07-21T22:29:41.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b6b59e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 0x5602c6808e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b6b59e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 0x5602c6808e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48478/0 (socket says 172.21.15.107:48478) 2023-07-21T22:29:41.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b6b59e700 1 -- 172.21.15.107:0/3260288962 learned_addr learned my addr 172.21.15.107:0/3260288962 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b6ad9d700 1 --2- 172.21.15.107:0/3260288962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c6804a80 0x5602c680dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b6a59c700 1 --2- 172.21.15.107:0/3260288962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c68053d0 0x5602c680d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b6b59e700 1 -- 172.21.15.107:0/3260288962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c6804a80 msgr2=0x5602c680dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b6b59e700 1 --2- 172.21.15.107:0/3260288962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c6804a80 0x5602c680dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b6b59e700 1 -- 172.21.15.107:0/3260288962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c68053d0 msgr2=0x5602c680d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b6b59e700 1 --2- 172.21.15.107:0/3260288962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c68053d0 0x5602c680d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.374+0000 7f4b6b59e700 1 -- 172.21.15.107:0/3260288962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602c65a6250 con 0x5602c6805ff0 2023-07-21T22:29:41.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.375+0000 7f4b6b59e700 1 --2- 172.21.15.107:0/3260288962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 0x5602c6808e60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400b3f0 tx=0x7f4b6400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9374d8946a9cfb39 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.375+0000 7f4b51ffb700 1 -- 172.21.15.107:0/3260288962 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400e040 con 0x5602c6805ff0 2023-07-21T22:29:41.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.375+0000 7f4b51ffb700 1 -- 172.21.15.107:0/3260288962 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b64003ce0 con 0x5602c6805ff0 2023-07-21T22:29:41.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.375+0000 7f4b51ffb700 1 -- 172.21.15.107:0/3260288962 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64005490 con 0x5602c6805ff0 2023-07-21T22:29:41.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/3260288962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 msgr2=0x5602c6808e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/3260288962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 0x5602c6808e60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400b3f0 tx=0x7f4b6400b7b0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/3260288962 shutdown_connections 2023-07-21T22:29:41.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/3260288962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6805ff0 0x5602c6808e60 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/3260288962 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c6804a80 0x5602c680dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/3260288962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c68053d0 0x5602c680d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/3260288962 >> 172.21.15.107:0/3260288962 conn(0x5602c67f5ef0 msgr2=0x5602c6820f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/3260288962 shutdown_connections 2023-07-21T22:29:41.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/3260288962 wait complete. 2023-07-21T22:29:41.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 -- start start 2023-07-21T22:29:41.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 Processor -- start 2023-07-21T22:29:41.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 0x5602c68040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 0x5602c68026c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6805ff0 0x5602c6815940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602c6802c00 con 0x5602c6804a80 2023-07-21T22:29:41.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.376+0000 7f4b7d2e0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602c6802d70 con 0x5602c6805ff0 2023-07-21T22:29:41.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b7d2e0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602c6802f30 con 0x5602c68053d0 2023-07-21T22:29:41.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b7d2e0b80 10 client.0 _ll_register_callbacks cb 0x5602c66ef760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:41.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b6ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 0x5602c68040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b6ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 0x5602c68040b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48480/0 (socket says 172.21.15.107:48480) 2023-07-21T22:29:41.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b6ad9d700 1 -- 172.21.15.107:0/2551190948 learned_addr learned my addr 172.21.15.107:0/2551190948 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b6b59e700 1 --2- 172.21.15.107:0/2551190948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6805ff0 0x5602c6815940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b6ad9d700 1 -- 172.21.15.107:0/2551190948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 msgr2=0x5602c68026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b6ad9d700 1 --2- 172.21.15.107:0/2551190948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 0x5602c68026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b6ad9d700 1 -- 172.21.15.107:0/2551190948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6805ff0 msgr2=0x5602c6815940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b6ad9d700 1 --2- 172.21.15.107:0/2551190948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6805ff0 0x5602c6815940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b6ad9d700 1 -- 172.21.15.107:0/2551190948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602c657fde0 con 0x5602c6804a80 2023-07-21T22:29:41.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b6ad9d700 1 --2- 172.21.15.107:0/2551190948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 0x5602c68040b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c096880 tx=0x7f4b5c0a2430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbcf2fe34f9eaf4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:41.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b4eff5700 1 -- 172.21.15.107:0/2551190948 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c099550 con 0x5602c6804a80 2023-07-21T22:29:41.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b4eff5700 1 -- 172.21.15.107:0/2551190948 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4b5c095d20 con 0x5602c6804a80 2023-07-21T22:29:41.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.377+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/2551190948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b64006020 con 0x5602c6804a80 2023-07-21T22:29:41.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.378+0000 7f4b4eff5700 1 -- 172.21.15.107:0/2551190948 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b5c0961a0 con 0x5602c6804a80 2023-07-21T22:29:41.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.378+0000 7f4b7d2e0b80 10 client.15594 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:41.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.378+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/2551190948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5602c657e2b0 con 0x5602c6804a80 2023-07-21T22:29:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.378+0000 7f4b4eff5700 1 -- 172.21.15.107:0/2551190948 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b64006020 con 0x5602c6804a80 2023-07-21T22:29:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.378+0000 7f4b4eff5700 1 -- 172.21.15.107:0/2551190948 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4b5c07ad90 con 0x5602c6804a80 2023-07-21T22:29:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.378+0000 7f4b7d2e0b80 10 client.15594 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.378+0000 7f4b7d2e0b80 10 client.15594 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T22:29:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.378+0000 7f4b7d2e0b80 20 client.15594 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.378+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/2551190948 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5602c68734c0 0x5602c68758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.378+0000 7f4b7d2e0b80 4 client.15594 mds_command: new command op to 24455 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:29:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.378+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/2551190948 --> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5602c670d050 con 0x5602c68734c0 2023-07-21T22:29:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b6a59c700 1 --2- 172.21.15.107:0/2551190948 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5602c68734c0 0x5602c68758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b6a59c700 1 --2- 172.21.15.107:0/2551190948 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5602c68734c0 0x5602c68758a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=22e85ff080447420 server_cookie=e8fa577573222994 in_seq=0 out_seq=0 2023-07-21T22:29:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b4eff5700 10 client.15594 ms_handle_connect on v2:172.21.15.132:6834/1022724305 2023-07-21T22:29:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b4eff5700 1 -- 172.21.15.107:0/2551190948 <== mds.3 v2:172.21.15.132:6834/1022724305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5602c670d050 con 0x5602c68734c0 2023-07-21T22:29:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b4eff5700 10 client.15594 handle_command_reply: tid=0 2023-07-21T22:29:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b7d2e0b80 1 client.15594 shutdown 2023-07-21T22:29:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/2551190948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 msgr2=0x5602c68040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/2551190948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 0x5602c68040b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c096880 tx=0x7f4b5c0a2430 comp rx=0 tx=0).stop 2023-07-21T22:29:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/2551190948 shutdown_connections 2023-07-21T22:29:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/2551190948 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5602c68734c0 0x5602c68758a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/2551190948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5602c6805ff0 0x5602c6815940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b4eff5700 0 client.15594 ms_handle_reset on v2:172.21.15.132:6834/1022724305 2023-07-21T22:29:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/2551190948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5602c6804a80 0x5602c68040b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b7d2e0b80 1 --2- 172.21.15.107:0/2551190948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602c68053d0 0x5602c68026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/2551190948 >> 172.21.15.107:0/2551190948 conn(0x5602c67f5ef0 msgr2=0x5602c6808e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/2551190948 shutdown_connections 2023-07-21T22:29:41.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.379+0000 7f4b7d2e0b80 1 -- 172.21.15.107:0/2551190948 wait complete. 2023-07-21T22:29:41.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.380+0000 7f4b7d2e0b80 20 client.15594 trim_cache size 0 max 16384 2023-07-21T22:29:41.435 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:29:41.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.380+0000 7f4b7115b700 1 -- 172.21.15.107:0/1518657157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4b540a11d0 msgr2=0x7f4b540a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.380+0000 7f4b7115b700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4b540a11d0 0x7f4b540a3680 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4b60011490 tx=0x7f4b60009260 comp rx=0 tx=0).stop 2023-07-21T22:29:41.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.380+0000 7f4b7115b700 1 -- 172.21.15.107:0/1518657157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c0fe9e0 msgr2=0x7f4b6c195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.380+0000 7f4b7115b700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c0fe9e0 0x7f4b6c195910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00e970 tx=0x7f4b5c00ec80 comp rx=0 tx=0).stop 2023-07-21T22:29:41.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.380+0000 7f4b7115b700 1 -- 172.21.15.107:0/1518657157 shutdown_connections 2023-07-21T22:29:41.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.380+0000 7f4b7115b700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b6c100560 0x7f4b6c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.380+0000 7f4b7115b700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b6c0fe9e0 0x7f4b6c195910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.380+0000 7f4b7115b700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4b540a11d0 0x7f4b540a3680 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.380+0000 7f4b7115b700 1 --2- 172.21.15.107:0/1518657157 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b6c0ffbd0 0x7f4b6c195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.380+0000 7f4b7115b700 1 -- 172.21.15.107:0/1518657157 >> 172.21.15.107:0/1518657157 conn(0x7f4b6c0fa000 msgr2=0x7f4b6c102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.380+0000 7f4b7115b700 1 -- 172.21.15.107:0/1518657157 shutdown_connections 2023-07-21T22:29:41.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.380+0000 7f4b7115b700 1 -- 172.21.15.107:0/1518657157 wait complete. 2023-07-21T22:29:41.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:41 smithi132 ceph-mon[97863]: pgmap v311: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-07-21T22:29:41.438 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:41 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2800999411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:29:41.441 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:29:41.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.587+0000 7fe6cccc4700 1 -- 172.21.15.107:0/1120062498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c81005c0 msgr2=0x7fe6c81029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.587+0000 7fe6cccc4700 1 --2- 172.21.15.107:0/1120062498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c81005c0 0x7fe6c81029a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009990 tx=0x7fe6b8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.587+0000 7fe6cccc4700 1 -- 172.21.15.107:0/1120062498 shutdown_connections 2023-07-21T22:29:41.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.587+0000 7fe6cccc4700 1 --2- 172.21.15.107:0/1120062498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c8105800 0x7fe6c8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.587+0000 7fe6cccc4700 1 --2- 172.21.15.107:0/1120062498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c81005c0 0x7fe6c81029a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.587+0000 7fe6cccc4700 1 --2- 172.21.15.107:0/1120062498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6c8102ee0 0x7fe6c81052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.587+0000 7fe6cccc4700 1 -- 172.21.15.107:0/1120062498 >> 172.21.15.107:0/1120062498 conn(0x7fe6c80fa000 msgr2=0x7fe6c80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.587+0000 7fe6cccc4700 1 -- 172.21.15.107:0/1120062498 shutdown_connections 2023-07-21T22:29:41.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.587+0000 7fe6cccc4700 1 -- 172.21.15.107:0/1120062498 wait complete. 2023-07-21T22:29:41.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.588+0000 7fe6cccc4700 1 Processor -- start 2023-07-21T22:29:41.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.588+0000 7fe6cccc4700 1 -- start start 2023-07-21T22:29:41.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.588+0000 7fe6cccc4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c81005c0 0x7fe6c80fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.588+0000 7fe6cccc4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c8102ee0 0x7fe6c80fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.588+0000 7fe6cccc4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6c8105800 0x7fe6c80feab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.588+0000 7fe6cccc4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c80ff250 con 0x7fe6c8102ee0 2023-07-21T22:29:41.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.588+0000 7fe6cccc4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c80ff3c0 con 0x7fe6c8105800 2023-07-21T22:29:41.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.588+0000 7fe6cccc4700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c819f7b0 con 0x7fe6c81005c0 2023-07-21T22:29:41.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.588+0000 7fe6c659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c81005c0 0x7fe6c80fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.588+0000 7fe6c659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c81005c0 0x7fe6c80fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37052/0 (socket says 172.21.15.107:37052) 2023-07-21T22:29:41.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.588+0000 7fe6c659c700 1 -- 172.21.15.107:0/2461514117 learned_addr learned my addr 172.21.15.107:0/2461514117 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.588+0000 7fe6c6d9d700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6c8105800 0x7fe6c80feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.588+0000 7fe6c5d9b700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c8102ee0 0x7fe6c80fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.589+0000 7fe6c659c700 1 -- 172.21.15.107:0/2461514117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6c8105800 msgr2=0x7fe6c80feab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.589+0000 7fe6c659c700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6c8105800 0x7fe6c80feab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.589+0000 7fe6c659c700 1 -- 172.21.15.107:0/2461514117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c8102ee0 msgr2=0x7fe6c80fe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.589+0000 7fe6c659c700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c8102ee0 0x7fe6c80fe570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.589+0000 7fe6c659c700 1 -- 172.21.15.107:0/2461514117 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b80095f0 con 0x7fe6c81005c0 2023-07-21T22:29:41.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.589+0000 7fe6c659c700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c81005c0 0x7fe6c80fff30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80059d0 tx=0x7fe6b80047a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.589+0000 7fe6b77fe700 1 -- 172.21.15.107:0/2461514117 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b801d070 con 0x7fe6c81005c0 2023-07-21T22:29:41.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.589+0000 7fe6b77fe700 1 -- 172.21.15.107:0/2461514117 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6b800fb90 con 0x7fe6c81005c0 2023-07-21T22:29:41.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.589+0000 7fe6b77fe700 1 -- 172.21.15.107:0/2461514117 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8017650 con 0x7fe6c81005c0 2023-07-21T22:29:41.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.589+0000 7fe6cccc4700 1 -- 172.21.15.107:0/2461514117 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c819fa30 con 0x7fe6c81005c0 2023-07-21T22:29:41.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.589+0000 7fe6cccc4700 1 -- 172.21.15.107:0/2461514117 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c819fe00 con 0x7fe6c81005c0 2023-07-21T22:29:41.641 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.592+0000 7fe6b77fe700 1 -- 172.21.15.107:0/2461514117 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe6b800fd00 con 0x7fe6c81005c0 2023-07-21T22:29:41.641 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.593+0000 7fe6b77fe700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe6b00a1150 0x7fe6b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.641 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.593+0000 7fe6c5d9b700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe6b00a1150 0x7fe6b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.593+0000 7fe6b77fe700 1 -- 172.21.15.107:0/2461514117 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe6b80ce040 con 0x7fe6c81005c0 2023-07-21T22:29:41.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.593+0000 7fe6d8e49b80 1 Processor -- start 2023-07-21T22:29:41.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.593+0000 7fe6d8e49b80 1 -- start start 2023-07-21T22:29:41.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6c5d9b700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe6b00a1150 0x7fe6b00a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00ac50 tx=0x7fe6bc00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6d8e49b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f9060 0x5586457f9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6d8e49b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f99b0 0x5586457f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6d8e49b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f8220 0x5586457f8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6d8e49b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586456f4cb0 con 0x5586457f99b0 2023-07-21T22:29:41.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6d8e49b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558645747a40 con 0x5586457f8220 2023-07-21T22:29:41.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6d8e49b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558645740ed0 con 0x5586457f9060 2023-07-21T22:29:41.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6c5d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f99b0 0x5586457f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6c5d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f99b0 0x5586457f7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48504/0 (socket says 172.21.15.107:48504) 2023-07-21T22:29:41.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6c5d9b700 1 -- 172.21.15.107:0/3777012814 learned_addr learned my addr 172.21.15.107:0/3777012814 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6c659c700 1 --2- 172.21.15.107:0/3777012814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f9060 0x5586457f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6c6d9d700 1 --2- 172.21.15.107:0/3777012814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f8220 0x5586457f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6c5d9b700 1 -- 172.21.15.107:0/3777012814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f9060 msgr2=0x5586457f9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6c5d9b700 1 --2- 172.21.15.107:0/3777012814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f9060 0x5586457f9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6c5d9b700 1 -- 172.21.15.107:0/3777012814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f8220 msgr2=0x5586457f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6c5d9b700 1 --2- 172.21.15.107:0/3777012814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f8220 0x5586457f8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6c5d9b700 1 -- 172.21.15.107:0/3777012814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55864556ede0 con 0x5586457f99b0 2023-07-21T22:29:41.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.594+0000 7fe6c5d9b700 1 --2- 172.21.15.107:0/3777012814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f99b0 0x5586457f7c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00ff90 tx=0x7fe6bc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb72ac48bd52aa1a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.595+0000 7fe6b4ff9700 1 -- 172.21.15.107:0/3777012814 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc003a90 con 0x5586457f99b0 2023-07-21T22:29:41.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.595+0000 7fe6b4ff9700 1 -- 172.21.15.107:0/3777012814 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6bc003bf0 con 0x5586457f99b0 2023-07-21T22:29:41.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.595+0000 7fe6b4ff9700 1 -- 172.21.15.107:0/3777012814 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc01d660 con 0x5586457f99b0 2023-07-21T22:29:41.645 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:41 smithi190 ceph-mon[98334]: pgmap v311: 97 pgs: 97 active+clean; 669 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2023-07-21T22:29:41.646 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:41 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2800999411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:29:41.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.595+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/3777012814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f99b0 msgr2=0x5586457f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.595+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/3777012814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f99b0 0x5586457f7c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00ff90 tx=0x7fe6bc012000 comp rx=0 tx=0).stop 2023-07-21T22:29:41.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.596+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/3777012814 shutdown_connections 2023-07-21T22:29:41.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.596+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/3777012814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f8220 0x5586457f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.596+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/3777012814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f9060 0x5586457f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.596+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/3777012814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f99b0 0x5586457f7c40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.596+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/3777012814 >> 172.21.15.107:0/3777012814 conn(0x5586457e4ef0 msgr2=0x5586457ea200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.596+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/3777012814 shutdown_connections 2023-07-21T22:29:41.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.596+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/3777012814 wait complete. 2023-07-21T22:29:41.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.596+0000 7fe6d8e49b80 1 -- start start 2023-07-21T22:29:41.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.596+0000 7fe6d8e49b80 1 Processor -- start 2023-07-21T22:29:41.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6d8e49b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 0x558645801900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6d8e49b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f9060 0x558645801e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6d8e49b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f99b0 0x558645808e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6d8e49b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586456e0ac0 con 0x5586457f8220 2023-07-21T22:29:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6d8e49b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586456e07c0 con 0x5586457f9060 2023-07-21T22:29:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6d8e49b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586455c4920 con 0x5586457f99b0 2023-07-21T22:29:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6d8e49b80 10 client.0 _ll_register_callbacks cb 0x5586456de760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6c659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 0x558645801900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6c659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 0x558645801900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48508/0 (socket says 172.21.15.107:48508) 2023-07-21T22:29:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6c659c700 1 -- 172.21.15.107:0/407968734 learned_addr learned my addr 172.21.15.107:0/407968734 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6c5d9b700 1 --2- 172.21.15.107:0/407968734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f9060 0x558645801e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6c6d9d700 1 --2- 172.21.15.107:0/407968734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f99b0 0x558645808e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6c659c700 1 -- 172.21.15.107:0/407968734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f99b0 msgr2=0x558645808e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6c659c700 1 --2- 172.21.15.107:0/407968734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f99b0 0x558645808e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6c659c700 1 -- 172.21.15.107:0/407968734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f9060 msgr2=0x558645801e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6c659c700 1 --2- 172.21.15.107:0/407968734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f9060 0x558645801e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6c659c700 1 -- 172.21.15.107:0/407968734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6bc00fc40 con 0x5586457f8220 2023-07-21T22:29:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6c659c700 1 --2- 172.21.15.107:0/407968734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 0x558645801900 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80cbe90 tx=0x7fe6b809ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4725809ec4a9a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.597+0000 7fe6adffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6adffb700 1 -- 172.21.15.107:0/407968734 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b80b0020 con 0x5586457f8220 2023-07-21T22:29:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6adffb700 1 -- 172.21.15.107:0/407968734 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe6b809f4f0 con 0x5586457f8220 2023-07-21T22:29:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/407968734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55864569a4f0 con 0x5586457f8220 2023-07-21T22:29:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6adffb700 1 -- 172.21.15.107:0/407968734 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6b80a3880 con 0x5586457f8220 2023-07-21T22:29:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6d8e49b80 10 client.15606 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/407968734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558645557c60 con 0x5586457f8220 2023-07-21T22:29:41.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6adffb700 1 -- 172.21.15.107:0/407968734 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55864569a4f0 con 0x5586457f8220 2023-07-21T22:29:41.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6adffb700 1 -- 172.21.15.107:0/407968734 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe6b80972b0 con 0x5586457f8220 2023-07-21T22:29:41.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6d8e49b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:41.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6d8e49b80 10 client.15606 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-07-21T22:29:41.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6d8e49b80 20 client.15606 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:41.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/407968734 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x55864580d2c0 0x55864580f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6d8e49b80 4 client.15606 mds_command: new command op to 24466 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:41.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.598+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/407968734 --> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558645739eb0 con 0x55864580d2c0 2023-07-21T22:29:41.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.599+0000 7fe6c5d9b700 1 --2- 172.21.15.107:0/407968734 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x55864580d2c0 0x55864580f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.599+0000 7fe6adffb700 10 client.15606 ms_handle_connect on v2:172.21.15.190:6834/2286851849 2023-07-21T22:29:41.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.599+0000 7fe6c5d9b700 1 --2- 172.21.15.107:0/407968734 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x55864580d2c0 0x55864580f6a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19720cb237775eda server_cookie=5f26151b7dd094a5 in_seq=0 out_seq=0 2023-07-21T22:29:41.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6adffb700 1 -- 172.21.15.107:0/407968734 <== mds.2 v2:172.21.15.190:6834/2286851849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558645739eb0 con 0x55864580d2c0 2023-07-21T22:29:41.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6adffb700 10 client.15606 handle_command_reply: tid=0 2023-07-21T22:29:41.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6d8e49b80 1 client.15606 shutdown 2023-07-21T22:29:41.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/407968734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 msgr2=0x558645801900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/407968734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 0x558645801900 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80cbe90 tx=0x7fe6b809ba00 comp rx=0 tx=0).stop 2023-07-21T22:29:41.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/407968734 shutdown_connections 2023-07-21T22:29:41.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/407968734 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x55864580d2c0 0x55864580f6a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/407968734 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f99b0 0x558645808e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6adffb700 0 client.15606 ms_handle_reset on v2:172.21.15.190:6834/2286851849 2023-07-21T22:29:41.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/407968734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 0x558645801900 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/407968734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f9060 0x558645801e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/407968734 >> 172.21.15.107:0/407968734 conn(0x5586457e4ef0 msgr2=0x5586457f6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/407968734 shutdown_connections 2023-07-21T22:29:41.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/407968734 wait complete. 2023-07-21T22:29:41.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.600+0000 7fe6d8e49b80 20 client.15606 trim_cache size 0 max 16384 2023-07-21T22:29:41.661 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6d8e49b80 1 Processor -- start 2023-07-21T22:29:41.661 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6d8e49b80 1 -- start start 2023-07-21T22:29:41.661 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6d8e49b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 0x5586457f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.661 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6d8e49b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f9060 0x558645808e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.661 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6d8e49b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f99b0 0x558645814000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6d8e49b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558645800db0 con 0x5586457f8220 2023-07-21T22:29:41.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6d8e49b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586457fcc40 con 0x5586457f9060 2023-07-21T22:29:41.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6d8e49b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586457e20a0 con 0x5586457f99b0 2023-07-21T22:29:41.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6c659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 0x5586457f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6c659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 0x5586457f7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48518/0 (socket says 172.21.15.107:48518) 2023-07-21T22:29:41.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6c659c700 1 -- 172.21.15.107:0/1882700256 learned_addr learned my addr 172.21.15.107:0/1882700256 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.662 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6c6d9d700 1 --2- 172.21.15.107:0/1882700256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f99b0 0x558645814000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6c659c700 1 -- 172.21.15.107:0/1882700256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f99b0 msgr2=0x558645814000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6c659c700 1 --2- 172.21.15.107:0/1882700256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f99b0 0x558645814000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6c659c700 1 -- 172.21.15.107:0/1882700256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f9060 msgr2=0x558645808e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:29:41.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6c659c700 1 --2- 172.21.15.107:0/1882700256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f9060 0x558645808e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6c659c700 1 -- 172.21.15.107:0/1882700256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558645595250 con 0x5586457f8220 2023-07-21T22:29:41.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.614+0000 7fe6c6d9d700 1 --2- 172.21.15.107:0/1882700256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f99b0 0x558645814000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:41.663 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.615+0000 7fe6c659c700 1 --2- 172.21.15.107:0/1882700256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 0x5586457f7bd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8099810 tx=0x7fe6b8099a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1844f36a4ee422 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.666 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.615+0000 7fe6b57fa700 1 -- 172.21.15.107:0/1882700256 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b80b0070 con 0x5586457f8220 2023-07-21T22:29:41.666 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.615+0000 7fe6b57fa700 1 -- 172.21.15.107:0/1882700256 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6b809f280 con 0x5586457f8220 2023-07-21T22:29:41.666 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.615+0000 7fe6b57fa700 1 -- 172.21.15.107:0/1882700256 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b80a1370 con 0x5586457f8220 2023-07-21T22:29:41.666 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.615+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/1882700256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 msgr2=0x5586457f7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.667 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.615+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/1882700256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 0x5586457f7bd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8099810 tx=0x7fe6b8099a70 comp rx=0 tx=0).stop 2023-07-21T22:29:41.667 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.615+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/1882700256 shutdown_connections 2023-07-21T22:29:41.667 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.615+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/1882700256 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f99b0 0x558645814000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.667 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.615+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/1882700256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f8220 0x5586457f7bd0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.667 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.615+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/1882700256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f9060 0x558645808e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.667 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.615+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/1882700256 >> 172.21.15.107:0/1882700256 conn(0x5586457e4ef0 msgr2=0x558645800640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.667 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.616+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/1882700256 shutdown_connections 2023-07-21T22:29:41.667 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.616+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/1882700256 wait complete. 2023-07-21T22:29:41.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.616+0000 7fe6d8e49b80 1 -- start start 2023-07-21T22:29:41.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.616+0000 7fe6d8e49b80 1 Processor -- start 2023-07-21T22:29:41.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.616+0000 7fe6d8e49b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f8220 0x558645807330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.616+0000 7fe6d8e49b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f9060 0x558645807870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.616+0000 7fe6d8e49b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f99b0 0x558645808130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.616+0000 7fe6d8e49b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558645808670 con 0x5586457f99b0 2023-07-21T22:29:41.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.616+0000 7fe6d8e49b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586458087e0 con 0x5586457f8220 2023-07-21T22:29:41.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.616+0000 7fe6d8e49b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586458089a0 con 0x5586457f9060 2023-07-21T22:29:41.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.616+0000 7fe6d8e49b80 10 client.0 _ll_register_callbacks cb 0x7fe6b809ae40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:41.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6c5d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f9060 0x558645807870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6c659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f8220 0x558645807330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6c5d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f9060 0x558645807870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37082/0 (socket says 172.21.15.107:37082) 2023-07-21T22:29:41.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6c5d9b700 1 -- 172.21.15.107:0/424067268 learned_addr learned my addr 172.21.15.107:0/424067268 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6c6d9d700 1 --2- 172.21.15.107:0/424067268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f99b0 0x558645808130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6c6d9d700 1 -- 172.21.15.107:0/424067268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f9060 msgr2=0x558645807870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6c6d9d700 1 --2- 172.21.15.107:0/424067268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f9060 0x558645807870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6c6d9d700 1 -- 172.21.15.107:0/424067268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f8220 msgr2=0x558645807330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6c6d9d700 1 --2- 172.21.15.107:0/424067268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f8220 0x558645807330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6c6d9d700 1 -- 172.21.15.107:0/424067268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558645557c60 con 0x5586457f99b0 2023-07-21T22:29:41.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6c6d9d700 1 --2- 172.21.15.107:0/424067268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f99b0 0x558645808130 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55864580a010 tx=0x7fe6c0007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5235bf29449064d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6adffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:41.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6adffb700 1 -- 172.21.15.107:0/424067268 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000e040 con 0x5586457f99b0 2023-07-21T22:29:41.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6adffb700 1 -- 172.21.15.107:0/424067268 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe6c0004bb0 con 0x5586457f99b0 2023-07-21T22:29:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/424067268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6bc003bf0 con 0x5586457f99b0 2023-07-21T22:29:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.617+0000 7fe6adffb700 1 -- 172.21.15.107:0/424067268 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6c0004d70 con 0x5586457f99b0 2023-07-21T22:29:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.618+0000 7fe6d8e49b80 10 client.15612 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.618+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/424067268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55864556d2b0 con 0x5586457f99b0 2023-07-21T22:29:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.618+0000 7fe6adffb700 1 -- 172.21.15.107:0/424067268 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6bc003bf0 con 0x5586457f99b0 2023-07-21T22:29:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.618+0000 7fe6adffb700 1 -- 172.21.15.107:0/424067268 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe6c0028050 con 0x5586457f99b0 2023-07-21T22:29:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.618+0000 7fe6d8e49b80 10 client.15612 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:41.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.618+0000 7fe6d8e49b80 10 client.15612 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-07-21T22:29:41.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.618+0000 7fe6d8e49b80 20 client.15612 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:41.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.618+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/424067268 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x558645862160 0x558645864570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.618+0000 7fe6d8e49b80 4 client.15612 mds_command: new command op to 24466 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:29:41.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.618+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/424067268 --> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5586456fc050 con 0x558645862160 2023-07-21T22:29:41.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.618+0000 7fe6c659c700 1 --2- 172.21.15.107:0/424067268 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x558645862160 0x558645864570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6adffb700 10 client.15612 ms_handle_connect on v2:172.21.15.190:6834/2286851849 2023-07-21T22:29:41.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6c659c700 1 --2- 172.21.15.107:0/424067268 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x558645862160 0x558645864570 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e2706a2afaebb4b server_cookie=ddc57fa92b6e1778 in_seq=0 out_seq=0 2023-07-21T22:29:41.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6adffb700 1 -- 172.21.15.107:0/424067268 <== mds.2 v2:172.21.15.190:6834/2286851849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5586456fc050 con 0x558645862160 2023-07-21T22:29:41.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6adffb700 10 client.15612 handle_command_reply: tid=0 2023-07-21T22:29:41.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6d8e49b80 1 client.15612 shutdown 2023-07-21T22:29:41.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/424067268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f99b0 msgr2=0x558645808130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/424067268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f99b0 0x558645808130 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55864580a010 tx=0x7fe6c0007b30 comp rx=0 tx=0).stop 2023-07-21T22:29:41.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/424067268 shutdown_connections 2023-07-21T22:29:41.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/424067268 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x558645862160 0x558645864570 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/424067268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586457f99b0 0x558645808130 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/424067268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5586457f8220 0x558645807330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6d8e49b80 1 --2- 172.21.15.107:0/424067268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5586457f9060 0x558645807870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/424067268 >> 172.21.15.107:0/424067268 conn(0x5586457e4ef0 msgr2=0x558645800640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6adffb700 0 client.15612 ms_handle_reset on v2:172.21.15.190:6834/2286851849 2023-07-21T22:29:41.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/424067268 shutdown_connections 2023-07-21T22:29:41.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6d8e49b80 1 -- 172.21.15.107:0/424067268 wait complete. 2023-07-21T22:29:41.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.619+0000 7fe6d8e49b80 20 client.15612 trim_cache size 0 max 16384 2023-07-21T22:29:41.675 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:29:41.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.620+0000 7fe6cccc4700 1 -- 172.21.15.107:0/2461514117 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe6b00a1150 msgr2=0x7fe6b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.620+0000 7fe6cccc4700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe6b00a1150 0x7fe6b00a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00ac50 tx=0x7fe6bc00a380 comp rx=0 tx=0).stop 2023-07-21T22:29:41.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.620+0000 7fe6cccc4700 1 -- 172.21.15.107:0/2461514117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c81005c0 msgr2=0x7fe6c80fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.620+0000 7fe6cccc4700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c81005c0 0x7fe6c80fff30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80059d0 tx=0x7fe6b80047a0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.620+0000 7fe6cccc4700 1 -- 172.21.15.107:0/2461514117 shutdown_connections 2023-07-21T22:29:41.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.620+0000 7fe6cccc4700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6c8105800 0x7fe6c80feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.620+0000 7fe6cccc4700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe6b00a1150 0x7fe6b00a3600 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.620+0000 7fe6cccc4700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6c81005c0 0x7fe6c80fff30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.620+0000 7fe6cccc4700 1 --2- 172.21.15.107:0/2461514117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6c8102ee0 0x7fe6c80fe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.620+0000 7fe6cccc4700 1 -- 172.21.15.107:0/2461514117 >> 172.21.15.107:0/2461514117 conn(0x7fe6c80fa000 msgr2=0x7fe6c8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.620+0000 7fe6cccc4700 1 -- 172.21.15.107:0/2461514117 shutdown_connections 2023-07-21T22:29:41.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.620+0000 7fe6cccc4700 1 -- 172.21.15.107:0/2461514117 wait complete. 2023-07-21T22:29:41.680 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:29:41.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.781+0000 7f0fbd050700 1 -- 172.21.15.107:0/3953445823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fb81017f0 msgr2=0x7f0fb81080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.781+0000 7f0fbd050700 1 --2- 172.21.15.107:0/3953445823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fb81017f0 0x7f0fb81080b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000b0a0 tx=0x7f0fb000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.781+0000 7f0fbd050700 1 -- 172.21.15.107:0/3953445823 shutdown_connections 2023-07-21T22:29:41.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.781+0000 7f0fbd050700 1 --2- 172.21.15.107:0/3953445823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fb81017f0 0x7f0fb81080b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.781+0000 7f0fbd050700 1 --2- 172.21.15.107:0/3953445823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fb80fddf0 0x7f0fb80fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.781+0000 7f0fbd050700 1 --2- 172.21.15.107:0/3953445823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fb8100e60 0x7f0fb81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.781+0000 7f0fbd050700 1 -- 172.21.15.107:0/3953445823 >> 172.21.15.107:0/3953445823 conn(0x7f0fb80f97b0 msgr2=0x7f0fb80fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.781+0000 7f0fbd050700 1 -- 172.21.15.107:0/3953445823 shutdown_connections 2023-07-21T22:29:41.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.781+0000 7f0fbd050700 1 -- 172.21.15.107:0/3953445823 wait complete. 2023-07-21T22:29:41.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.782+0000 7f0fbd050700 1 Processor -- start 2023-07-21T22:29:41.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.782+0000 7f0fbd050700 1 -- start start 2023-07-21T22:29:41.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.782+0000 7f0fbd050700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fb80fddf0 0x7f0fb81958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.782+0000 7f0fbd050700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fb8100e60 0x7f0fb8195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.782+0000 7f0fbd050700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fb81017f0 0x7f0fb819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.782+0000 7f0fbd050700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb819d470 con 0x7f0fb8100e60 2023-07-21T22:29:41.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.782+0000 7f0fbd050700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb819d5b0 con 0x7f0fb81017f0 2023-07-21T22:29:41.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.782+0000 7f0fbd050700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb819d740 con 0x7f0fb80fddf0 2023-07-21T22:29:41.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.782+0000 7f0fb659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fb8100e60 0x7f0fb8195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.832 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.782+0000 7f0fb659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fb8100e60 0x7f0fb8195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48536/0 (socket says 172.21.15.107:48536) 2023-07-21T22:29:41.832 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.782+0000 7f0fb6d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fb80fddf0 0x7f0fb81958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.832 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.782+0000 7f0fb659c700 1 -- 172.21.15.107:0/3820025849 learned_addr learned my addr 172.21.15.107:0/3820025849 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.832 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.782+0000 7f0fb759e700 1 --2- 172.21.15.107:0/3820025849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fb81017f0 0x7f0fb819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.832 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.783+0000 7f0fb659c700 1 -- 172.21.15.107:0/3820025849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fb80fddf0 msgr2=0x7f0fb81958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.832 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.783+0000 7f0fb659c700 1 --2- 172.21.15.107:0/3820025849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fb80fddf0 0x7f0fb81958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.832 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.783+0000 7f0fb659c700 1 -- 172.21.15.107:0/3820025849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fb81017f0 msgr2=0x7f0fb819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.833 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.783+0000 7f0fb659c700 1 --2- 172.21.15.107:0/3820025849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fb81017f0 0x7f0fb819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.833 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.783+0000 7f0fb659c700 1 -- 172.21.15.107:0/3820025849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb0009d00 con 0x7f0fb8100e60 2023-07-21T22:29:41.833 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.783+0000 7f0fb659c700 1 --2- 172.21.15.107:0/3820025849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fb8100e60 0x7f0fb8195e30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00e9a0 tx=0x7f0fac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.833 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.783+0000 7f0f9ffff700 1 -- 172.21.15.107:0/3820025849 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac009790 con 0x7f0fb8100e60 2023-07-21T22:29:41.833 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.783+0000 7f0fbd050700 1 -- 172.21.15.107:0/3820025849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb80fed70 con 0x7f0fb8100e60 2023-07-21T22:29:41.833 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.783+0000 7f0f9ffff700 1 -- 172.21.15.107:0/3820025849 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fac010430 con 0x7f0fb8100e60 2023-07-21T22:29:41.833 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.783+0000 7f0f9ffff700 1 -- 172.21.15.107:0/3820025849 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac018680 con 0x7f0fb8100e60 2023-07-21T22:29:41.833 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.783+0000 7f0fbd050700 1 -- 172.21.15.107:0/3820025849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb80ff2d0 con 0x7f0fb8100e60 2023-07-21T22:29:41.835 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.786+0000 7f0f9ffff700 1 -- 172.21.15.107:0/3820025849 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0fac0105a0 con 0x7f0fb8100e60 2023-07-21T22:29:41.835 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.787+0000 7f0f9ffff700 1 --2- 172.21.15.107:0/3820025849 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0fa00a1150 0x7f0fa00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.835 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.787+0000 7f0f9ffff700 1 -- 172.21.15.107:0/3820025849 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0fac014070 con 0x7f0fb8100e60 2023-07-21T22:29:41.835 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.787+0000 7f0fc91d5b80 1 Processor -- start 2023-07-21T22:29:41.836 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.787+0000 7f0fc91d5b80 1 -- start start 2023-07-21T22:29:41.836 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.787+0000 7f0fb6d9d700 1 --2- 172.21.15.107:0/3820025849 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0fa00a1150 0x7f0fa00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.836 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.787+0000 7f0fc91d5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f166251e80 0x55f166252290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.836 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.787+0000 7f0fc91d5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1662527d0 0x55f166252e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.836 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.787+0000 7f0fc91d5b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f166253860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.836 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.787+0000 7f0fc91d5b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16614ecb0 con 0x55f1662533f0 2023-07-21T22:29:41.836 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.787+0000 7f0fc91d5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1661a1a40 con 0x55f1662527d0 2023-07-21T22:29:41.836 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.787+0000 7f0fc91d5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16619aed0 con 0x55f166251e80 2023-07-21T22:29:41.837 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0fb759e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f166253860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.837 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0fb759e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f166253860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48546/0 (socket says 172.21.15.107:48546) 2023-07-21T22:29:41.837 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0fb759e700 1 -- 172.21.15.107:0/1615762911 learned_addr learned my addr 172.21.15.107:0/1615762911 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.837 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0fb659c700 1 --2- 172.21.15.107:0/1615762911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1662527d0 0x55f166252e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.837 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0fb6d9d700 1 --2- 172.21.15.107:0/1615762911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f166251e80 0x55f166252290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.837 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0fb6d9d700 1 --2- 172.21.15.107:0/3820025849 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0fa00a1150 0x7f0fa00a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0fb80685a0 tx=0x7f0fa800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.837 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0fb759e700 1 -- 172.21.15.107:0/1615762911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f166251e80 msgr2=0x55f166252290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.838 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0fb759e700 1 --2- 172.21.15.107:0/1615762911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f166251e80 0x55f166252290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.838 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0fb759e700 1 -- 172.21.15.107:0/1615762911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1662527d0 msgr2=0x55f166252e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.838 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0fb759e700 1 --2- 172.21.15.107:0/1615762911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1662527d0 0x55f166252e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.838 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0fb759e700 1 -- 172.21.15.107:0/1615762911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f165fc8de0 con 0x55f1662533f0 2023-07-21T22:29:41.838 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0fb659c700 1 --2- 172.21.15.107:0/1615762911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1662527d0 0x55f166252e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:41.838 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0fb759e700 1 --2- 172.21.15.107:0/1615762911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f166253860 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00092e0 tx=0x7f0fb0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76f103dbb6ce336 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.838 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0f9d7fa700 1 -- 172.21.15.107:0/1615762911 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb000e040 con 0x55f1662533f0 2023-07-21T22:29:41.838 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0f9d7fa700 1 -- 172.21.15.107:0/1615762911 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb000bab0 con 0x55f1662533f0 2023-07-21T22:29:41.839 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.788+0000 7f0f9d7fa700 1 -- 172.21.15.107:0/1615762911 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb001c680 con 0x55f1662533f0 2023-07-21T22:29:41.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.789+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/1615762911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 msgr2=0x55f166253860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.789+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/1615762911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f166253860 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00092e0 tx=0x7f0fb0009310 comp rx=0 tx=0).stop 2023-07-21T22:29:41.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.789+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/1615762911 shutdown_connections 2023-07-21T22:29:41.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.789+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/1615762911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f166253860 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.789+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/1615762911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f166251e80 0x55f166252290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.789+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/1615762911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1662527d0 0x55f166252e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.789+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/1615762911 >> 172.21.15.107:0/1615762911 conn(0x55f16623eef0 msgr2=0x55f166244360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.789+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/1615762911 shutdown_connections 2023-07-21T22:29:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.789+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/1615762911 wait complete. 2023-07-21T22:29:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fc91d5b80 1 -- start start 2023-07-21T22:29:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fc91d5b80 1 Processor -- start 2023-07-21T22:29:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fc91d5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f166251e80 0x55f16624aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fc91d5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1662527d0 0x55f16624b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fc91d5b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f166250040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fc91d5b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16613aac0 con 0x55f1662533f0 2023-07-21T22:29:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fc91d5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16613a7c0 con 0x55f1662527d0 2023-07-21T22:29:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fc91d5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16601e920 con 0x55f166251e80 2023-07-21T22:29:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fc91d5b80 10 client.0 _ll_register_callbacks cb 0x55f166138760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fb659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1662527d0 0x55f16624b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fb6d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f166251e80 0x55f16624aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fb759e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f166250040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fb6d9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f166251e80 0x55f16624aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37106/0 (socket says 172.21.15.107:37106) 2023-07-21T22:29:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fb6d9d700 1 -- 172.21.15.107:0/909046721 learned_addr learned my addr 172.21.15.107:0/909046721 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fb659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1662527d0 0x55f16624b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:39378/0 (socket says 172.21.15.107:39378) 2023-07-21T22:29:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fb6d9d700 1 -- 172.21.15.107:0/909046721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1662527d0 msgr2=0x55f16624b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fb6d9d700 1 --2- 172.21.15.107:0/909046721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1662527d0 0x55f16624b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fb6d9d700 1 -- 172.21.15.107:0/909046721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 msgr2=0x55f166250040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fb6d9d700 1 --2- 172.21.15.107:0/909046721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f166250040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fb6d9d700 1 -- 172.21.15.107:0/909046721 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb80ff2d0 con 0x55f166251e80 2023-07-21T22:29:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fb659c700 1 --2- 172.21.15.107:0/909046721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1662527d0 0x55f16624b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.790+0000 7f0fb759e700 1 --2- 172.21.15.107:0/909046721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f166250040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.791+0000 7f0fb6d9d700 1 --2- 172.21.15.107:0/909046721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f166251e80 0x55f16624aad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80116f0 tx=0x7f0fa80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2603a5b4db0751c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.791+0000 7f0f9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:29:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.791+0000 7f0f9aff5700 1 -- 172.21.15.107:0/909046721 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8004220 con 0x55f166251e80 2023-07-21T22:29:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.791+0000 7f0f9aff5700 1 -- 172.21.15.107:0/909046721 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0fa8017070 con 0x55f166251e80 2023-07-21T22:29:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.791+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/909046721 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1660f44f0 con 0x55f166251e80 2023-07-21T22:29:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.791+0000 7f0f9aff5700 1 -- 172.21.15.107:0/909046721 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fa8011e90 con 0x55f166251e80 2023-07-21T22:29:41.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.791+0000 7f0fc91d5b80 10 client.25228 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:41.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.791+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/909046721 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f0fb8108870 con 0x55f166251e80 2023-07-21T22:29:41.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.791+0000 7f0f9aff5700 1 -- 172.21.15.107:0/909046721 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f1660f44f0 con 0x55f166251e80 2023-07-21T22:29:41.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.792+0000 7f0f9aff5700 1 -- 172.21.15.107:0/909046721 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0fa8030050 con 0x55f166251e80 2023-07-21T22:29:41.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.792+0000 7f0fc91d5b80 10 client.25228 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:41.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.792+0000 7f0fc91d5b80 10 client.25228 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-21T22:29:41.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.792+0000 7f0fc91d5b80 20 client.25228 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:41.847 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.792+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/909046721 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55f16626fe60 0x55f166272270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.792+0000 7f0fc91d5b80 4 client.25228 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:41.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.792+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/909046721 --> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f166193eb0 con 0x55f16626fe60 2023-07-21T22:29:41.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.792+0000 7f0fb659c700 1 --2- 172.21.15.107:0/909046721 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55f16626fe60 0x55f166272270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.792+0000 7f0f9aff5700 10 client.25228 ms_handle_connect on v2:172.21.15.190:6838/2960500982 2023-07-21T22:29:41.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.792+0000 7f0fb659c700 1 --2- 172.21.15.107:0/909046721 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55f16626fe60 0x55f166272270 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e74aa19d0c49cdc4 server_cookie=c7c8b7def61fa0e1 in_seq=0 out_seq=0 2023-07-21T22:29:41.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0f9aff5700 1 -- 172.21.15.107:0/909046721 <== mds.1 v2:172.21.15.190:6838/2960500982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f166193eb0 con 0x55f16626fe60 2023-07-21T22:29:41.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0f9aff5700 10 client.25228 handle_command_reply: tid=0 2023-07-21T22:29:41.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0fc91d5b80 1 client.25228 shutdown 2023-07-21T22:29:41.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/909046721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f166251e80 msgr2=0x55f16624aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/909046721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f166251e80 0x55f16624aad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80116f0 tx=0x7f0fa80047c0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/909046721 shutdown_connections 2023-07-21T22:29:41.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/909046721 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55f16626fe60 0x55f166272270 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/909046721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f166250040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0f9aff5700 0 client.25228 ms_handle_reset on v2:172.21.15.190:6838/2960500982 2023-07-21T22:29:41.849 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/909046721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f166251e80 0x55f16624aad0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/909046721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f1662527d0 0x55f16624b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/909046721 >> 172.21.15.107:0/909046721 conn(0x55f16623eef0 msgr2=0x55f166243a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/909046721 shutdown_connections 2023-07-21T22:29:41.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/909046721 wait complete. 2023-07-21T22:29:41.850 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.793+0000 7f0fc91d5b80 20 client.25228 trim_cache size 0 max 16384 2023-07-21T22:29:41.853 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.807+0000 7f0fc91d5b80 1 Processor -- start 2023-07-21T22:29:41.853 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.807+0000 7f0fc91d5b80 1 -- start start 2023-07-21T22:29:41.854 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.807+0000 7f0fc91d5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f166251e80 0x55f1662537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.854 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.807+0000 7f0fc91d5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1662527d0 0x55f166276bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.807+0000 7f0fc91d5b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f16627faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.807+0000 7f0fc91d5b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f166245060 con 0x55f1662533f0 2023-07-21T22:29:41.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.807+0000 7f0fc91d5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f166277110 con 0x55f166251e80 2023-07-21T22:29:41.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.807+0000 7f0fc91d5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16626fb60 con 0x55f1662527d0 2023-07-21T22:29:41.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0fb659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1662527d0 0x55f166276bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0fb759e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f16627faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.855 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0fb659c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1662527d0 0x55f166276bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37116/0 (socket says 172.21.15.107:37116) 2023-07-21T22:29:41.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0fb659c700 1 -- 172.21.15.107:0/4247532713 learned_addr learned my addr 172.21.15.107:0/4247532713 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0fb6d9d700 1 --2- 172.21.15.107:0/4247532713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f166251e80 0x55f1662537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0fb759e700 1 -- 172.21.15.107:0/4247532713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1662527d0 msgr2=0x55f166276bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0fb759e700 1 --2- 172.21.15.107:0/4247532713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1662527d0 0x55f166276bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0fb759e700 1 -- 172.21.15.107:0/4247532713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f166251e80 msgr2=0x55f1662537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0fb759e700 1 --2- 172.21.15.107:0/4247532713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f166251e80 0x55f1662537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.856 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0fb759e700 1 -- 172.21.15.107:0/4247532713 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f165fb1c60 con 0x55f1662533f0 2023-07-21T22:29:41.857 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0fb759e700 1 --2- 172.21.15.107:0/4247532713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f16627faf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009610 tx=0x7f0fb00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f6813bc763688d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.859 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0f9dffb700 1 -- 172.21.15.107:0/4247532713 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb000e040 con 0x55f1662533f0 2023-07-21T22:29:41.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0f9dffb700 1 -- 172.21.15.107:0/4247532713 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb00076a0 con 0x55f1662533f0 2023-07-21T22:29:41.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.808+0000 7f0f9dffb700 1 -- 172.21.15.107:0/4247532713 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00057e0 con 0x55f1662533f0 2023-07-21T22:29:41.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.809+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/4247532713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 msgr2=0x55f16627faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.809+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/4247532713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f16627faf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009610 tx=0x7f0fb00099e0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.809+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/4247532713 shutdown_connections 2023-07-21T22:29:41.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.809+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/4247532713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f16627faf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.809+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/4247532713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f166251e80 0x55f1662537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.809+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/4247532713 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1662527d0 0x55f166276bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.809+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/4247532713 >> 172.21.15.107:0/4247532713 conn(0x55f16623eef0 msgr2=0x55f166243a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.809+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/4247532713 shutdown_connections 2023-07-21T22:29:41.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.809+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/4247532713 wait complete. 2023-07-21T22:29:41.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.809+0000 7f0fc91d5b80 1 -- start start 2023-07-21T22:29:41.861 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.809+0000 7f0fc91d5b80 1 Processor -- start 2023-07-21T22:29:41.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fc91d5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f166251e80 0x55f16627d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fc91d5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1662527d0 0x55f16627d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fc91d5b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f16627e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fc91d5b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16627ed10 con 0x55f1662533f0 2023-07-21T22:29:41.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fc91d5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16627ee80 con 0x55f166251e80 2023-07-21T22:29:41.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fc91d5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f16627f040 con 0x55f1662527d0 2023-07-21T22:29:41.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fc91d5b80 10 client.0 _ll_register_callbacks cb 0x55f166138760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:41.862 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fb759e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f16627e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.863 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fb759e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f16627e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48564/0 (socket says 172.21.15.107:48564) 2023-07-21T22:29:41.863 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fb759e700 1 -- 172.21.15.107:0/3862357649 learned_addr learned my addr 172.21.15.107:0/3862357649 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:41.863 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fb6d9d700 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f166251e80 0x55f16627d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.863 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fb659c700 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1662527d0 0x55f16627d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.863 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fb759e700 1 -- 172.21.15.107:0/3862357649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1662527d0 msgr2=0x55f16627d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.863 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fb759e700 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1662527d0 0x55f16627d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.863 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fb759e700 1 -- 172.21.15.107:0/3862357649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f166251e80 msgr2=0x55f16627d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:41.863 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fb759e700 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f166251e80 0x55f16627d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fb759e700 1 -- 172.21.15.107:0/3862357649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f165fc8de0 con 0x55f1662533f0 2023-07-21T22:29:41.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.810+0000 7f0fb6d9d700 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f166251e80 0x55f16627d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:41.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0fb759e700 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f16627e7d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009380 tx=0x7f0fb0005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d4902663371a9c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:41.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0f9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:41.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0f9aff5700 1 -- 172.21.15.107:0/3862357649 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb000e040 con 0x55f1662533f0 2023-07-21T22:29:41.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0f9aff5700 1 -- 172.21.15.107:0/3862357649 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0fb0005770 con 0x55f1662533f0 2023-07-21T22:29:41.864 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/3862357649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fb000bab0 con 0x55f1662533f0 2023-07-21T22:29:41.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0f9aff5700 1 -- 172.21.15.107:0/3862357649 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fb0007890 con 0x55f1662533f0 2023-07-21T22:29:41.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0fc91d5b80 10 client.15630 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:41.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/3862357649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f165fef250 con 0x55f1662533f0 2023-07-21T22:29:41.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0f9aff5700 1 -- 172.21.15.107:0/3862357649 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fb000bab0 con 0x55f1662533f0 2023-07-21T22:29:41.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0f9aff5700 1 -- 172.21.15.107:0/3862357649 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0fb0019030 con 0x55f1662533f0 2023-07-21T22:29:41.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0fc91d5b80 10 client.15630 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:41.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0fc91d5b80 10 client.15630 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-21T22:29:41.865 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0fc91d5b80 20 client.15630 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:41.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55f1662c4740 0x55f1662c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:41.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0fc91d5b80 4 client.15630 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:29:41.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.811+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/3862357649 --> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f166156050 con 0x55f1662c4740 2023-07-21T22:29:41.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0fb6d9d700 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55f1662c4740 0x55f1662c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:41.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0f9aff5700 10 client.15630 ms_handle_connect on v2:172.21.15.190:6838/2960500982 2023-07-21T22:29:41.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0fb6d9d700 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55f1662c4740 0x55f1662c6b50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90b8db3e6da2bb8f server_cookie=bfa248e9f03ba366 in_seq=0 out_seq=0 2023-07-21T22:29:41.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0f9aff5700 1 -- 172.21.15.107:0/3862357649 <== mds.1 v2:172.21.15.190:6838/2960500982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f166156050 con 0x55f1662c4740 2023-07-21T22:29:41.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0f9aff5700 10 client.15630 handle_command_reply: tid=0 2023-07-21T22:29:41.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0fc91d5b80 1 client.15630 shutdown 2023-07-21T22:29:41.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/3862357649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 msgr2=0x55f16627e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f16627e7d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009380 tx=0x7f0fb0005280 comp rx=0 tx=0).stop 2023-07-21T22:29:41.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/3862357649 shutdown_connections 2023-07-21T22:29:41.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55f1662c4740 0x55f1662c6b50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1662533f0 0x55f16627e7d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f166251e80 0x55f16627d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0fc91d5b80 1 --2- 172.21.15.107:0/3862357649 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f1662527d0 0x55f16627d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/3862357649 >> 172.21.15.107:0/3862357649 conn(0x55f16623eef0 msgr2=0x55f1662597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.812+0000 7f0f9aff5700 0 client.15630 ms_handle_reset on v2:172.21.15.190:6838/2960500982 2023-07-21T22:29:41.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/3862357649 shutdown_connections 2023-07-21T22:29:41.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fc91d5b80 1 -- 172.21.15.107:0/3862357649 wait complete. 2023-07-21T22:29:41.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fc91d5b80 20 client.15630 trim_cache size 0 max 16384 2023-07-21T22:29:41.869 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:29:41.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fbd050700 1 -- 172.21.15.107:0/3820025849 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0fa00a1150 msgr2=0x7f0fa00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fbd050700 1 --2- 172.21.15.107:0/3820025849 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0fa00a1150 0x7f0fa00a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0fb80685a0 tx=0x7f0fa800b410 comp rx=0 tx=0).stop 2023-07-21T22:29:41.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fbd050700 1 -- 172.21.15.107:0/3820025849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fb8100e60 msgr2=0x7f0fb8195e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:41.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fbd050700 1 --2- 172.21.15.107:0/3820025849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fb8100e60 0x7f0fb8195e30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00e9a0 tx=0x7f0fac00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fbd050700 1 -- 172.21.15.107:0/3820025849 shutdown_connections 2023-07-21T22:29:41.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fbd050700 1 --2- 172.21.15.107:0/3820025849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fb81017f0 0x7f0fb819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fbd050700 1 --2- 172.21.15.107:0/3820025849 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0fa00a1150 0x7f0fa00a3600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fbd050700 1 --2- 172.21.15.107:0/3820025849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0fb80fddf0 0x7f0fb81958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fbd050700 1 --2- 172.21.15.107:0/3820025849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fb8100e60 0x7f0fb8195e30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:41.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fbd050700 1 -- 172.21.15.107:0/3820025849 >> 172.21.15.107:0/3820025849 conn(0x7f0fb80f97b0 msgr2=0x7f0fb80fb950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:41.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fbd050700 1 -- 172.21.15.107:0/3820025849 shutdown_connections 2023-07-21T22:29:41.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.813+0000 7f0fbd050700 1 -- 172.21.15.107:0/3820025849 wait complete. 2023-07-21T22:29:41.873 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:29:42.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.972+0000 7f057dca3700 1 -- 172.21.15.107:0/1004483769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05780ff620 msgr2=0x7f0578106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:42.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.972+0000 7f057dca3700 1 --2- 172.21.15.107:0/1004483769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05780ff620 0x7f0578106ef0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f057000b0a0 tx=0x7f057000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.972+0000 7f057dca3700 1 -- 172.21.15.107:0/1004483769 shutdown_connections 2023-07-21T22:29:42.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.972+0000 7f057dca3700 1 --2- 172.21.15.107:0/1004483769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05780ff620 0x7f0578106ef0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.972+0000 7f057dca3700 1 --2- 172.21.15.107:0/1004483769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05780fff60 0x7f0578100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.972+0000 7f057dca3700 1 --2- 172.21.15.107:0/1004483769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05780fec90 0x7f05780ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.972+0000 7f057dca3700 1 -- 172.21.15.107:0/1004483769 >> 172.21.15.107:0/1004483769 conn(0x7f05780f9770 msgr2=0x7f05780fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:42.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.972+0000 7f057dca3700 1 -- 172.21.15.107:0/1004483769 shutdown_connections 2023-07-21T22:29:42.020 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.972+0000 7f057dca3700 1 -- 172.21.15.107:0/1004483769 wait complete. 2023-07-21T22:29:42.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.973+0000 7f057dca3700 1 Processor -- start 2023-07-21T22:29:42.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.973+0000 7f057dca3700 1 -- start start 2023-07-21T22:29:42.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.973+0000 7f057dca3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05780fec90 0x7f0578193660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.973+0000 7f057dca3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05780ff620 0x7f0578193ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.973+0000 7f057dca3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05780fff60 0x7f057819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.973+0000 7f057dca3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057819b1e0 con 0x7f05780fec90 2023-07-21T22:29:42.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.973+0000 7f057dca3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057819b320 con 0x7f05780ff620 2023-07-21T22:29:42.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.973+0000 7f057dca3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057819b4b0 con 0x7f05780fff60 2023-07-21T22:29:42.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.973+0000 7f05777fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05780fec90 0x7f0578193660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.973+0000 7f05777fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05780fec90 0x7f0578193660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48576/0 (socket says 172.21.15.107:48576) 2023-07-21T22:29:42.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.973+0000 7f05777fe700 1 -- 172.21.15.107:0/2489622911 learned_addr learned my addr 172.21.15.107:0/2489622911 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:42.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f0577fff700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05780fff60 0x7f057819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f0576ffd700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05780ff620 0x7f0578193ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f05777fe700 1 -- 172.21.15.107:0/2489622911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05780fff60 msgr2=0x7f057819ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:42.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f05777fe700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05780fff60 0x7f057819ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f05777fe700 1 -- 172.21.15.107:0/2489622911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05780ff620 msgr2=0x7f0578193ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:42.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f05777fe700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05780ff620 0x7f0578193ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f05777fe700 1 -- 172.21.15.107:0/2489622911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0570009d00 con 0x7f05780fec90 2023-07-21T22:29:42.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f0576ffd700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05780ff620 0x7f0578193ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:42.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f05777fe700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05780fec90 0x7f0578193660 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f05680099f0 tx=0x7f0568009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:42.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f0574ff9700 1 -- 172.21.15.107:0/2489622911 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568012070 con 0x7f05780fec90 2023-07-21T22:29:42.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f057dca3700 1 -- 172.21.15.107:0/2489622911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057819b790 con 0x7f05780fec90 2023-07-21T22:29:42.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f057dca3700 1 -- 172.21.15.107:0/2489622911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057819bcf0 con 0x7f05780fec90 2023-07-21T22:29:42.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f0574ff9700 1 -- 172.21.15.107:0/2489622911 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f056800ce30 con 0x7f05780fec90 2023-07-21T22:29:42.025 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.974+0000 7f0574ff9700 1 -- 172.21.15.107:0/2489622911 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05680038e0 con 0x7f05780fec90 2023-07-21T22:29:42.027 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.977+0000 7f0574ff9700 1 -- 172.21.15.107:0/2489622911 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f056800c970 con 0x7f05780fec90 2023-07-21T22:29:42.027 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.977+0000 7f0574ff9700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f05600a1150 0x7f05600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.027 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.978+0000 7f0576ffd700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f05600a1150 0x7f05600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.027 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.978+0000 7f0574ff9700 1 -- 172.21.15.107:0/2489622911 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f05680cbec0 con 0x7f05780fec90 2023-07-21T22:29:42.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.978+0000 7f0589e28b80 1 Processor -- start 2023-07-21T22:29:42.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.978+0000 7f0576ffd700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f05600a1150 0x7f05600a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f056c005fd0 tx=0x7f056c005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:42.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.978+0000 7f0589e28b80 1 -- start start 2023-07-21T22:29:42.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.978+0000 7f0589e28b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9c2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.978+0000 7f0589e28b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.978+0000 7f0589e28b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c1220 0x55f7ae9c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.978+0000 7f0589e28b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ae8bdcb0 con 0x55f7ae9c2040 2023-07-21T22:29:42.028 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.978+0000 7f0589e28b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ae910a40 con 0x55f7ae9c29b0 2023-07-21T22:29:42.029 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.978+0000 7f0589e28b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ae909ed0 con 0x55f7ae9c1220 2023-07-21T22:29:42.029 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f05777fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.029 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f05777fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9c2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48584/0 (socket says 172.21.15.107:48584) 2023-07-21T22:29:42.029 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f05777fe700 1 -- 172.21.15.107:0/990728348 learned_addr learned my addr 172.21.15.107:0/990728348 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:42.029 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f0577fff700 1 --2- 172.21.15.107:0/990728348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c1220 0x55f7ae9c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.029 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f0576ffd700 1 --2- 172.21.15.107:0/990728348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.029 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f05777fe700 1 -- 172.21.15.107:0/990728348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c1220 msgr2=0x55f7ae9c16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:42.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f05777fe700 1 --2- 172.21.15.107:0/990728348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c1220 0x55f7ae9c16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f05777fe700 1 -- 172.21.15.107:0/990728348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c29b0 msgr2=0x55f7ae9c0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:42.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f05777fe700 1 --2- 172.21.15.107:0/990728348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9c0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f05777fe700 1 -- 172.21.15.107:0/990728348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7ae737de0 con 0x55f7ae9c2040 2023-07-21T22:29:42.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f0576ffd700 1 --2- 172.21.15.107:0/990728348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:42.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f05777fe700 1 --2- 172.21.15.107:0/990728348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9c2450 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f05680a11d0 tx=0x7f0568099150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d782c04334318b90 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:42.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f055dffb700 1 -- 172.21.15.107:0/990728348 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568098c10 con 0x55f7ae9c2040 2023-07-21T22:29:42.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f055dffb700 1 -- 172.21.15.107:0/990728348 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05680992b0 con 0x55f7ae9c2040 2023-07-21T22:29:42.031 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.979+0000 7f055dffb700 1 -- 172.21.15.107:0/990728348 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05680a1270 con 0x55f7ae9c2040 2023-07-21T22:29:42.031 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.980+0000 7f0589e28b80 1 -- 172.21.15.107:0/990728348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 msgr2=0x55f7ae9c2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:42.031 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.980+0000 7f0589e28b80 1 --2- 172.21.15.107:0/990728348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9c2450 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f05680a11d0 tx=0x7f0568099150 comp rx=0 tx=0).stop 2023-07-21T22:29:42.031 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.980+0000 7f0589e28b80 1 -- 172.21.15.107:0/990728348 shutdown_connections 2023-07-21T22:29:42.031 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.980+0000 7f0589e28b80 1 --2- 172.21.15.107:0/990728348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c1220 0x55f7ae9c16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.031 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.980+0000 7f0589e28b80 1 --2- 172.21.15.107:0/990728348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9c2450 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.031 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.980+0000 7f0589e28b80 1 --2- 172.21.15.107:0/990728348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.031 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.980+0000 7f0589e28b80 1 -- 172.21.15.107:0/990728348 >> 172.21.15.107:0/990728348 conn(0x55f7ae9adef0 msgr2=0x55f7ae9b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:42.032 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.980+0000 7f0589e28b80 1 -- 172.21.15.107:0/990728348 shutdown_connections 2023-07-21T22:29:42.032 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.980+0000 7f0589e28b80 1 -- 172.21.15.107:0/990728348 wait complete. 2023-07-21T22:29:42.032 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.981+0000 7f0589e28b80 1 -- start start 2023-07-21T22:29:42.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.981+0000 7f0589e28b80 1 Processor -- start 2023-07-21T22:29:42.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.981+0000 7f0589e28b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c1220 0x55f7ae9ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.981+0000 7f0589e28b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c2040 0x55f7ae9cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.981+0000 7f0589e28b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9d1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.981+0000 7f0589e28b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ae8a9ac0 con 0x55f7ae9c1220 2023-07-21T22:29:42.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.981+0000 7f0589e28b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ae8a97c0 con 0x55f7ae9c29b0 2023-07-21T22:29:42.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.981+0000 7f0589e28b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ae78d920 con 0x55f7ae9c2040 2023-07-21T22:29:42.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.981+0000 7f0589e28b80 10 client.0 _ll_register_callbacks cb 0x55f7ae8a7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:42.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.981+0000 7f05777fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c1220 0x55f7ae9ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.981+0000 7f05777fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c1220 0x55f7ae9ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48590/0 (socket says 172.21.15.107:48590) 2023-07-21T22:29:42.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.981+0000 7f05777fe700 1 -- 172.21.15.107:0/2811175817 learned_addr learned my addr 172.21.15.107:0/2811175817 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:42.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.981+0000 7f0577fff700 1 --2- 172.21.15.107:0/2811175817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9d1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f05777fe700 1 -- 172.21.15.107:0/2811175817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c2040 msgr2=0x55f7ae9cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:42.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f05777fe700 1 --2- 172.21.15.107:0/2811175817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c2040 0x55f7ae9cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f05777fe700 1 -- 172.21.15.107:0/2811175817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c29b0 msgr2=0x55f7ae9d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:42.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f05777fe700 1 --2- 172.21.15.107:0/2811175817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9d1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f05777fe700 1 -- 172.21.15.107:0/2811175817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057819bcf0 con 0x55f7ae9c1220 2023-07-21T22:29:42.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f05777fe700 1 --2- 172.21.15.107:0/2811175817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c1220 0x55f7ae9ca8c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0568095d20 tx=0x7f0568005ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c71a29e3f478e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:42.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f0556ffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:42.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f0556ffd700 1 -- 172.21.15.107:0/2811175817 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05680a1470 con 0x55f7ae9c1220 2023-07-21T22:29:42.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f0556ffd700 1 -- 172.21.15.107:0/2811175817 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f05680a1700 con 0x55f7ae9c1220 2023-07-21T22:29:42.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f0589e28b80 1 -- 172.21.15.107:0/2811175817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7ae8634f0 con 0x55f7ae9c1220 2023-07-21T22:29:42.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f0556ffd700 1 -- 172.21.15.107:0/2811175817 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0568082b00 con 0x55f7ae9c1220 2023-07-21T22:29:42.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f0589e28b80 10 client.15642 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:42.038 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f0589e28b80 1 -- 172.21.15.107:0/2811175817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f7ae720c60 con 0x55f7ae9c1220 2023-07-21T22:29:42.038 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.982+0000 7f0556ffd700 1 -- 172.21.15.107:0/2811175817 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05680c7080 con 0x55f7ae9c1220 2023-07-21T22:29:42.038 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.983+0000 7f0556ffd700 1 -- 172.21.15.107:0/2811175817 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0568098210 con 0x55f7ae9c1220 2023-07-21T22:29:42.038 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.983+0000 7f0589e28b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:42.038 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.983+0000 7f0589e28b80 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:42.038 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.983+0000 7f0589e28b80 20 client.15642 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:42.038 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.983+0000 7f0589e28b80 1 --2- 172.21.15.107:0/2811175817 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f7ae9d6350 0x55f7ae9d8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.038 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.983+0000 7f0589e28b80 4 client.15642 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:42.039 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.983+0000 7f0589e28b80 1 -- 172.21.15.107:0/2811175817 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7ae902eb0 con 0x55f7ae9d6350 2023-07-21T22:29:42.039 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.983+0000 7f0576ffd700 1 --2- 172.21.15.107:0/2811175817 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f7ae9d6350 0x55f7ae9d8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.039 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.983+0000 7f0556ffd700 10 client.15642 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:42.039 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.983+0000 7f0576ffd700 1 --2- 172.21.15.107:0/2811175817 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f7ae9d6350 0x55f7ae9d8760 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4bfc1ab5f8b84ab server_cookie=6e48257e4c0c4b6f in_seq=0 out_seq=0 2023-07-21T22:29:42.039 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0556ffd700 1 -- 172.21.15.107:0/2811175817 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f7ae902eb0 con 0x55f7ae9d6350 2023-07-21T22:29:42.039 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0556ffd700 10 client.15642 handle_command_reply: tid=0 2023-07-21T22:29:42.039 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0589e28b80 1 client.15642 shutdown 2023-07-21T22:29:42.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0589e28b80 1 -- 172.21.15.107:0/2811175817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c1220 msgr2=0x55f7ae9ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:42.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0589e28b80 1 --2- 172.21.15.107:0/2811175817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c1220 0x55f7ae9ca8c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0568095d20 tx=0x7f0568005ba0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0589e28b80 1 -- 172.21.15.107:0/2811175817 shutdown_connections 2023-07-21T22:29:42.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0589e28b80 1 --2- 172.21.15.107:0/2811175817 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f7ae9d6350 0x55f7ae9d8760 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0589e28b80 1 --2- 172.21.15.107:0/2811175817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9d1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0589e28b80 1 --2- 172.21.15.107:0/2811175817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c1220 0x55f7ae9ca8c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0589e28b80 1 --2- 172.21.15.107:0/2811175817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c2040 0x55f7ae9cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0589e28b80 1 -- 172.21.15.107:0/2811175817 >> 172.21.15.107:0/2811175817 conn(0x55f7ae9adef0 msgr2=0x55f7ae9b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:42.041 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0556ffd700 0 client.15642 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:42.041 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0589e28b80 1 -- 172.21.15.107:0/2811175817 shutdown_connections 2023-07-21T22:29:42.041 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.984+0000 7f0589e28b80 1 -- 172.21.15.107:0/2811175817 wait complete. 2023-07-21T22:29:42.041 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.985+0000 7f0589e28b80 20 client.15642 trim_cache size 0 max 16384 2023-07-21T22:29:42.044 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.997+0000 7f0589e28b80 1 Processor -- start 2023-07-21T22:29:42.044 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.997+0000 7f0589e28b80 1 -- start start 2023-07-21T22:29:42.044 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.997+0000 7f0589e28b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c1220 0x55f7ae9c15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.045 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.997+0000 7f0589e28b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.997+0000 7f0589e28b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.997+0000 7f0589e28b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ae9ca490 con 0x55f7ae9c2040 2023-07-21T22:29:42.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.997+0000 7f0589e28b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ae9b43c0 con 0x55f7ae9c1220 2023-07-21T22:29:42.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.997+0000 7f0589e28b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ae9c1b30 con 0x55f7ae9c29b0 2023-07-21T22:29:42.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.998+0000 7f0576ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.998+0000 7f0576ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48600/0 (socket says 172.21.15.107:48600) 2023-07-21T22:29:42.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.998+0000 7f0576ffd700 1 -- 172.21.15.107:0/3044141465 learned_addr learned my addr 172.21.15.107:0/3044141465 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:42.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.998+0000 7f05777fe700 1 --2- 172.21.15.107:0/3044141465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c1220 0x55f7ae9c15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.998+0000 7f0576ffd700 1 -- 172.21.15.107:0/3044141465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c29b0 msgr2=0x55f7ae9d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:42.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.998+0000 7f0576ffd700 1 --2- 172.21.15.107:0/3044141465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.998+0000 7f0576ffd700 1 -- 172.21.15.107:0/3044141465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c1220 msgr2=0x55f7ae9c15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:42.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.998+0000 7f0576ffd700 1 --2- 172.21.15.107:0/3044141465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c1220 0x55f7ae9c15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.998+0000 7f0576ffd700 1 -- 172.21.15.107:0/3044141465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7ae75e250 con 0x55f7ae9c2040 2023-07-21T22:29:42.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.998+0000 7f0576ffd700 1 --2- 172.21.15.107:0/3044141465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9d1dd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55f7ae9d9040 tx=0x7f056c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46f64618894a523 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:42.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.998+0000 7f055e7fc700 1 -- 172.21.15.107:0/3044141465 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c024070 con 0x55f7ae9c2040 2023-07-21T22:29:42.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.998+0000 7f055e7fc700 1 -- 172.21.15.107:0/3044141465 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f056c00edc0 con 0x55f7ae9c2040 2023-07-21T22:29:42.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.998+0000 7f055e7fc700 1 -- 172.21.15.107:0/3044141465 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c004a90 con 0x55f7ae9c2040 2023-07-21T22:29:42.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.999+0000 7f0589e28b80 1 -- 172.21.15.107:0/3044141465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 msgr2=0x55f7ae9d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:42.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.999+0000 7f0589e28b80 1 --2- 172.21.15.107:0/3044141465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9d1dd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55f7ae9d9040 tx=0x7f056c009f90 comp rx=0 tx=0).stop 2023-07-21T22:29:42.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.999+0000 7f0589e28b80 1 -- 172.21.15.107:0/3044141465 shutdown_connections 2023-07-21T22:29:42.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.999+0000 7f0589e28b80 1 --2- 172.21.15.107:0/3044141465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.999+0000 7f0589e28b80 1 --2- 172.21.15.107:0/3044141465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c1220 0x55f7ae9c15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.999+0000 7f0589e28b80 1 --2- 172.21.15.107:0/3044141465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9d1dd0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.999+0000 7f0589e28b80 1 -- 172.21.15.107:0/3044141465 >> 172.21.15.107:0/3044141465 conn(0x55f7ae9adef0 msgr2=0x55f7ae9dd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:42.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.999+0000 7f0589e28b80 1 -- 172.21.15.107:0/3044141465 shutdown_connections 2023-07-21T22:29:42.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.999+0000 7f0589e28b80 1 -- 172.21.15.107:0/3044141465 wait complete. 2023-07-21T22:29:42.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:41.999+0000 7f0589e28b80 1 -- start start 2023-07-21T22:29:42.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0589e28b80 1 Processor -- start 2023-07-21T22:29:42.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0589e28b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c1220 0x55f7ae9d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0589e28b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0589e28b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0589e28b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ae9ce570 con 0x55f7ae9c2040 2023-07-21T22:29:42.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0589e28b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ae9ce6e0 con 0x55f7ae9c1220 2023-07-21T22:29:42.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0589e28b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7ae9ce8a0 con 0x55f7ae9c29b0 2023-07-21T22:29:42.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0589e28b80 10 client.0 _ll_register_callbacks cb 0x7f057000d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:42.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0576ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0576ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9d1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48606/0 (socket says 172.21.15.107:48606) 2023-07-21T22:29:42.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0576ffd700 1 -- 172.21.15.107:0/433296701 learned_addr learned my addr 172.21.15.107:0/433296701 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:42.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f05777fe700 1 --2- 172.21.15.107:0/433296701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c1220 0x55f7ae9d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0576ffd700 1 -- 172.21.15.107:0/433296701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c29b0 msgr2=0x55f7ae9ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:42.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0576ffd700 1 --2- 172.21.15.107:0/433296701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.000+0000 7f0576ffd700 1 -- 172.21.15.107:0/433296701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c1220 msgr2=0x55f7ae9d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:42.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.001+0000 7f0576ffd700 1 --2- 172.21.15.107:0/433296701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c1220 0x55f7ae9d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.001+0000 7f0576ffd700 1 -- 172.21.15.107:0/433296701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c00eaf0 con 0x55f7ae9c2040 2023-07-21T22:29:42.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.001+0000 7f05777fe700 1 --2- 172.21.15.107:0/433296701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c1220 0x55f7ae9d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:42.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.001+0000 7f0576ffd700 1 --2- 172.21.15.107:0/433296701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9d1ad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f056c000c00 tx=0x7f056c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f4c013590cf96d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:42.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.001+0000 7f0556ffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:42.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.001+0000 7f0556ffd700 1 -- 172.21.15.107:0/433296701 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c024050 con 0x55f7ae9c2040 2023-07-21T22:29:42.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.001+0000 7f0556ffd700 1 -- 172.21.15.107:0/433296701 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f056c025400 con 0x55f7ae9c2040 2023-07-21T22:29:42.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.001+0000 7f0589e28b80 1 -- 172.21.15.107:0/433296701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05680992b0 con 0x55f7ae9c2040 2023-07-21T22:29:42.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.001+0000 7f0556ffd700 1 -- 172.21.15.107:0/433296701 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f056c0255c0 con 0x55f7ae9c2040 2023-07-21T22:29:42.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.001+0000 7f0589e28b80 10 client.15648 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:42.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.001+0000 7f0589e28b80 1 -- 172.21.15.107:0/433296701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f7ae7362b0 con 0x55f7ae9c2040 2023-07-21T22:29:42.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.001+0000 7f0556ffd700 1 -- 172.21.15.107:0/433296701 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05680992b0 con 0x55f7ae9c2040 2023-07-21T22:29:42.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.002+0000 7f0556ffd700 1 -- 172.21.15.107:0/433296701 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f056c02e5e0 con 0x55f7ae9c2040 2023-07-21T22:29:42.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.002+0000 7f0589e28b80 10 client.15648 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:42.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.002+0000 7f0589e28b80 10 client.15648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:42.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.002+0000 7f0589e28b80 20 client.15648 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:42.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.002+0000 7f0589e28b80 1 --2- 172.21.15.107:0/433296701 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f7aea2b670 0x55f7aea2da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:42.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.002+0000 7f0589e28b80 4 client.15648 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:29:42.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.002+0000 7f0589e28b80 1 -- 172.21.15.107:0/433296701 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f7ae8c5050 con 0x55f7aea2b670 2023-07-21T22:29:42.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.002+0000 7f05777fe700 1 --2- 172.21.15.107:0/433296701 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f7aea2b670 0x55f7aea2da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:42.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.002+0000 7f0556ffd700 10 client.15648 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:42.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.002+0000 7f05777fe700 1 --2- 172.21.15.107:0/433296701 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f7aea2b670 0x55f7aea2da80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9abc6d72267ba4dd server_cookie=b6e5d286792c772d in_seq=0 out_seq=0 2023-07-21T22:29:42.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0556ffd700 1 -- 172.21.15.107:0/433296701 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f7ae8c5050 con 0x55f7aea2b670 2023-07-21T22:29:42.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0556ffd700 10 client.15648 handle_command_reply: tid=0 2023-07-21T22:29:42.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0589e28b80 1 client.15648 shutdown 2023-07-21T22:29:42.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0589e28b80 1 -- 172.21.15.107:0/433296701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 msgr2=0x55f7ae9d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:42.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0589e28b80 1 --2- 172.21.15.107:0/433296701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9d1ad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f056c000c00 tx=0x7f056c005810 comp rx=0 tx=0).stop 2023-07-21T22:29:42.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0589e28b80 1 -- 172.21.15.107:0/433296701 shutdown_connections 2023-07-21T22:29:42.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0589e28b80 1 --2- 172.21.15.107:0/433296701 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f7aea2b670 0x55f7aea2da80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0589e28b80 1 --2- 172.21.15.107:0/433296701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7ae9c29b0 0x55f7ae9ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0589e28b80 1 --2- 172.21.15.107:0/433296701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7ae9c1220 0x55f7ae9d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0589e28b80 1 --2- 172.21.15.107:0/433296701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7ae9c2040 0x55f7ae9d1ad0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0589e28b80 1 -- 172.21.15.107:0/433296701 >> 172.21.15.107:0/433296701 conn(0x55f7ae9adef0 msgr2=0x55f7ae9d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:42.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0556ffd700 0 client.15648 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:42.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0589e28b80 1 -- 172.21.15.107:0/433296701 shutdown_connections 2023-07-21T22:29:42.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0589e28b80 1 -- 172.21.15.107:0/433296701 wait complete. 2023-07-21T22:29:42.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.003+0000 7f0589e28b80 20 client.15648 trim_cache size 0 max 16384 2023-07-21T22:29:42.059 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:29:42.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.004+0000 7f057dca3700 1 -- 172.21.15.107:0/2489622911 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f05600a1150 msgr2=0x7f05600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:42.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.004+0000 7f057dca3700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f05600a1150 0x7f05600a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f056c005fd0 tx=0x7f056c005ee0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.004+0000 7f057dca3700 1 -- 172.21.15.107:0/2489622911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05780fec90 msgr2=0x7f0578193660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:42.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.004+0000 7f057dca3700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05780fec90 0x7f0578193660 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f05680099f0 tx=0x7f0568009d00 comp rx=0 tx=0).stop 2023-07-21T22:29:42.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.004+0000 7f057dca3700 1 -- 172.21.15.107:0/2489622911 shutdown_connections 2023-07-21T22:29:42.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.004+0000 7f057dca3700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f05600a1150 0x7f05600a3600 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.004+0000 7f057dca3700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05780fff60 0x7f057819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.004+0000 7f057dca3700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05780fec90 0x7f0578193660 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.004+0000 7f057dca3700 1 --2- 172.21.15.107:0/2489622911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05780ff620 0x7f0578193ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:42.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.004+0000 7f057dca3700 1 -- 172.21.15.107:0/2489622911 >> 172.21.15.107:0/2489622911 conn(0x7f05780f9770 msgr2=0x7f05780fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:42.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.004+0000 7f057dca3700 1 -- 172.21.15.107:0/2489622911 shutdown_connections 2023-07-21T22:29:42.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:42.004+0000 7f057dca3700 1 -- 172.21.15.107:0/2489622911 wait complete. 2023-07-21T22:29:43.063 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:29:43.064 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:29:43.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.161+0000 7fef02de0700 1 -- 172.21.15.107:0/2086351042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc0fe9e0 msgr2=0x7feefc0fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.161+0000 7fef02de0700 1 --2- 172.21.15.107:0/2086351042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc0fe9e0 0x7feefc0fedf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7feef0009990 tx=0x7feef0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.161+0000 7fef02de0700 1 -- 172.21.15.107:0/2086351042 shutdown_connections 2023-07-21T22:29:43.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.161+0000 7fef02de0700 1 --2- 172.21.15.107:0/2086351042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc100560 0x7feefc106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.161+0000 7fef02de0700 1 --2- 172.21.15.107:0/2086351042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc0fe9e0 0x7feefc0fedf0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.161+0000 7fef02de0700 1 --2- 172.21.15.107:0/2086351042 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feefc0ffbd0 0x7feefc100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.209 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.161+0000 7fef02de0700 1 -- 172.21.15.107:0/2086351042 >> 172.21.15.107:0/2086351042 conn(0x7feefc0fa000 msgr2=0x7feefc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:43.209 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.161+0000 7fef02de0700 1 -- 172.21.15.107:0/2086351042 shutdown_connections 2023-07-21T22:29:43.209 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.161+0000 7fef02de0700 1 -- 172.21.15.107:0/2086351042 wait complete. 2023-07-21T22:29:43.210 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef02de0700 1 Processor -- start 2023-07-21T22:29:43.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef02de0700 1 -- start start 2023-07-21T22:29:43.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef02de0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feefc0fe9e0 0x7feefc068240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef02de0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc0ffbd0 0x7feefc19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef02de0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc100560 0x7feefc19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef02de0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc19d360 con 0x7feefc100560 2023-07-21T22:29:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef02de0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc19d4d0 con 0x7feefc0fe9e0 2023-07-21T22:29:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef02de0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc19d690 con 0x7feefc0ffbd0 2023-07-21T22:29:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef0137d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc100560 0x7feefc19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef00b7c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feefc0fe9e0 0x7feefc068240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef0137d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc100560 0x7feefc19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48620/0 (socket says 172.21.15.107:48620) 2023-07-21T22:29:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef0137d700 1 -- 172.21.15.107:0/1650816580 learned_addr learned my addr 172.21.15.107:0/1650816580 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef0137d700 1 -- 172.21.15.107:0/1650816580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc0ffbd0 msgr2=0x7feefc19a530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T22:29:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.162+0000 7fef0137d700 1 --2- 172.21.15.107:0/1650816580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc0ffbd0 0x7feefc19a530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.163+0000 7fef0137d700 1 -- 172.21.15.107:0/1650816580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feefc0fe9e0 msgr2=0x7feefc068240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.163+0000 7fef0137d700 1 --2- 172.21.15.107:0/1650816580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feefc0fe9e0 0x7feefc068240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.163+0000 7fef0137d700 1 -- 172.21.15.107:0/1650816580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef00095f0 con 0x7feefc100560 2023-07-21T22:29:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.163+0000 7fef0137d700 1 --2- 172.21.15.107:0/1650816580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc100560 0x7feefc19cbc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7feef4007ee0 tx=0x7feef400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.163+0000 7feef9ffb700 1 -- 172.21.15.107:0/1650816580 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4017070 con 0x7feefc100560 2023-07-21T22:29:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.163+0000 7feef9ffb700 1 -- 172.21.15.107:0/1650816580 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feef400f040 con 0x7feefc100560 2023-07-21T22:29:43.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.163+0000 7fef02de0700 1 -- 172.21.15.107:0/1650816580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc19d970 con 0x7feefc100560 2023-07-21T22:29:43.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.163+0000 7feef9ffb700 1 -- 172.21.15.107:0/1650816580 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4005740 con 0x7feefc100560 2023-07-21T22:29:43.214 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.163+0000 7fef02de0700 1 -- 172.21.15.107:0/1650816580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc19de30 con 0x7feefc100560 2023-07-21T22:29:43.215 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.166+0000 7feef9ffb700 1 -- 172.21.15.107:0/1650816580 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7feef400dc60 con 0x7feefc100560 2023-07-21T22:29:43.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.166+0000 7feef9ffb700 1 --2- 172.21.15.107:0/1650816580 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7feeec0a1150 0x7feeec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.167+0000 7fef00b7c700 1 --2- 172.21.15.107:0/1650816580 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7feeec0a1150 0x7feeec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.167+0000 7feef9ffb700 1 -- 172.21.15.107:0/1650816580 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7feef40cc170 con 0x7feefc100560 2023-07-21T22:29:43.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.167+0000 7fef00b7c700 1 --2- 172.21.15.107:0/1650816580 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7feeec0a1150 0x7feeec0a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feef0006010 tx=0x7feef000b560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:43.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7fef0ef65b80 1 Processor -- start 2023-07-21T22:29:43.216 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7fef0ef65b80 1 -- start start 2023-07-21T22:29:43.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7fef0ef65b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 0x56513f192cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7fef0ef65b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f193220 0x56513f1914b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7fef0ef65b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f191a90 0x56513f191f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7fef0ef65b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513f096d30 con 0x56513f193220 2023-07-21T22:29:43.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7fef0ef65b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513f0e9ac0 con 0x56513f191a90 2023-07-21T22:29:43.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7fef0ef65b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513f0e2f50 con 0x56513f1928b0 2023-07-21T22:29:43.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7feefbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f193220 0x56513f1914b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7feefbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f193220 0x56513f1914b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48628/0 (socket says 172.21.15.107:48628) 2023-07-21T22:29:43.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7feefbfff700 1 -- 172.21.15.107:0/677553560 learned_addr learned my addr 172.21.15.107:0/677553560 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:43.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7fef00b7c700 1 --2- 172.21.15.107:0/677553560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 0x56513f192cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7fef0137d700 1 --2- 172.21.15.107:0/677553560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f191a90 0x56513f191f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7feefbfff700 1 -- 172.21.15.107:0/677553560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 msgr2=0x56513f192cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7feefbfff700 1 --2- 172.21.15.107:0/677553560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 0x56513f192cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7feefbfff700 1 -- 172.21.15.107:0/677553560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f191a90 msgr2=0x56513f191f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7feefbfff700 1 --2- 172.21.15.107:0/677553560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f191a90 0x56513f191f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7feefbfff700 1 -- 172.21.15.107:0/677553560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56513ef10e60 con 0x56513f193220 2023-07-21T22:29:43.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.168+0000 7fef0137d700 1 --2- 172.21.15.107:0/677553560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f191a90 0x56513f191f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:43.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.169+0000 7feefbfff700 1 --2- 172.21.15.107:0/677553560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f193220 0x56513f1914b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7feee800aa10 tx=0x7feee800ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59582a5bf12ab4d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:43.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.169+0000 7feee6ffd700 1 -- 172.21.15.107:0/677553560 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee800cc10 con 0x56513f193220 2023-07-21T22:29:43.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.169+0000 7feee6ffd700 1 -- 172.21.15.107:0/677553560 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feee800cd70 con 0x56513f193220 2023-07-21T22:29:43.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.169+0000 7feee6ffd700 1 -- 172.21.15.107:0/677553560 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8018660 con 0x56513f193220 2023-07-21T22:29:43.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.170+0000 7fef0ef65b80 1 -- 172.21.15.107:0/677553560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f193220 msgr2=0x56513f1914b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.170+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/677553560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f193220 0x56513f1914b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7feee800aa10 tx=0x7feee800ad20 comp rx=0 tx=0).stop 2023-07-21T22:29:43.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.170+0000 7fef0ef65b80 1 -- 172.21.15.107:0/677553560 shutdown_connections 2023-07-21T22:29:43.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.170+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/677553560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f191a90 0x56513f191f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.170+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/677553560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 0x56513f192cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.170+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/677553560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f193220 0x56513f1914b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.170+0000 7fef0ef65b80 1 -- 172.21.15.107:0/677553560 >> 172.21.15.107:0/677553560 conn(0x56513f186f70 msgr2=0x56513f18c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:43.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.170+0000 7fef0ef65b80 1 -- 172.21.15.107:0/677553560 shutdown_connections 2023-07-21T22:29:43.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.170+0000 7fef0ef65b80 1 -- 172.21.15.107:0/677553560 wait complete. 2023-07-21T22:29:43.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.170+0000 7fef0ef65b80 1 -- start start 2023-07-21T22:29:43.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.170+0000 7fef0ef65b80 1 Processor -- start 2023-07-21T22:29:43.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef0ef65b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 0x56513f19b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef0ef65b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 0x56513f19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef0ef65b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f193220 0x56513f1a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef0ef65b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513f082b40 con 0x56513f191a90 2023-07-21T22:29:43.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef0ef65b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513f082840 con 0x56513f193220 2023-07-21T22:29:43.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef0ef65b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513ef669a0 con 0x56513f1928b0 2023-07-21T22:29:43.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef0ef65b80 10 client.0 _ll_register_callbacks cb 0x56513f0807e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:43.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef00b7c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 0x56513f19b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef00b7c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 0x56513f19b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48632/0 (socket says 172.21.15.107:48632) 2023-07-21T22:29:43.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef00b7c700 1 -- 172.21.15.107:0/3400280056 learned_addr learned my addr 172.21.15.107:0/3400280056 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:43.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7feefbfff700 1 --2- 172.21.15.107:0/3400280056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 0x56513f19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef0137d700 1 --2- 172.21.15.107:0/3400280056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f193220 0x56513f1a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef00b7c700 1 -- 172.21.15.107:0/3400280056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 msgr2=0x56513f19b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef00b7c700 1 --2- 172.21.15.107:0/3400280056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 0x56513f19b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef00b7c700 1 -- 172.21.15.107:0/3400280056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f193220 msgr2=0x56513f1a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef00b7c700 1 --2- 172.21.15.107:0/3400280056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f193220 0x56513f1a26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef00b7c700 1 -- 172.21.15.107:0/3400280056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee800a640 con 0x56513f191a90 2023-07-21T22:29:43.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7fef00b7c700 1 --2- 172.21.15.107:0/3400280056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 0x56513f19b170 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7feefc0ffa30 tx=0x7feef0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae02a533a74fdb11 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:43.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7feedbfff700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:43.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.171+0000 7feedbfff700 1 -- 172.21.15.107:0/3400280056 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef00040b0 con 0x56513f191a90 2023-07-21T22:29:43.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7feedbfff700 1 -- 172.21.15.107:0/3400280056 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7feef0004500 con 0x56513f191a90 2023-07-21T22:29:43.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3400280056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56513f03c570 con 0x56513f191a90 2023-07-21T22:29:43.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7feedbfff700 1 -- 172.21.15.107:0/3400280056 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7feef0004210 con 0x56513f191a90 2023-07-21T22:29:43.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7fef0ef65b80 10 client.15660 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:43.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3400280056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56513eef9ce0 con 0x56513f191a90 2023-07-21T22:29:43.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7feedbfff700 1 -- 172.21.15.107:0/3400280056 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feef003f020 con 0x56513f191a90 2023-07-21T22:29:43.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7feedbfff700 1 -- 172.21.15.107:0/3400280056 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7feef002f070 con 0x56513f191a90 2023-07-21T22:29:43.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7fef0ef65b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:43.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7fef0ef65b80 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:43.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7fef0ef65b80 20 client.15660 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:43.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/3400280056 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56513f1a6f60 0x56513f1a9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7fef0ef65b80 4 client.15660 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:43.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3400280056 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56513f0dbf30 con 0x56513f1a6f60 2023-07-21T22:29:43.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.172+0000 7feefbfff700 1 --2- 172.21.15.107:0/3400280056 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56513f1a6f60 0x56513f1a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.173+0000 7feedbfff700 10 client.15660 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:43.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.173+0000 7feefbfff700 1 --2- 172.21.15.107:0/3400280056 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56513f1a6f60 0x56513f1a9370 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e766215429c300 server_cookie=62298ddd239cc5d1 in_seq=0 out_seq=0 2023-07-21T22:29:43.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.173+0000 7feedbfff700 1 -- 172.21.15.107:0/3400280056 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56513f0dbf30 con 0x56513f1a6f60 2023-07-21T22:29:43.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.173+0000 7feedbfff700 10 client.15660 handle_command_reply: tid=0 2023-07-21T22:29:43.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.174+0000 7fef0ef65b80 1 client.15660 shutdown 2023-07-21T22:29:43.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.174+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3400280056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 msgr2=0x56513f19b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.174+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/3400280056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 0x56513f19b170 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7feefc0ffa30 tx=0x7feef0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.174+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3400280056 shutdown_connections 2023-07-21T22:29:43.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.174+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/3400280056 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56513f1a6f60 0x56513f1a9370 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.174+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/3400280056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f193220 0x56513f1a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.174+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/3400280056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 0x56513f19b170 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.174+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/3400280056 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 0x56513f19b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.174+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3400280056 >> 172.21.15.107:0/3400280056 conn(0x56513f186f70 msgr2=0x56513f18ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:43.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.174+0000 7feedbfff700 0 client.15660 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:43.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.174+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3400280056 shutdown_connections 2023-07-21T22:29:43.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.174+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3400280056 wait complete. 2023-07-21T22:29:43.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.174+0000 7fef0ef65b80 20 client.15660 trim_cache size 0 max 16384 2023-07-21T22:29:43.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef0ef65b80 1 Processor -- start 2023-07-21T22:29:43.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef0ef65b80 1 -- start start 2023-07-21T22:29:43.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef0ef65b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f186f70 0x56513f1a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef0ef65b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f191a90 0x56513f1a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef0ef65b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f1928b0 0x56513f199e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef0ef65b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513f18c190 con 0x56513f186f70 2023-07-21T22:29:43.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef0ef65b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513f198e00 con 0x56513f1928b0 2023-07-21T22:29:43.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef0ef65b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513f18b660 con 0x56513f191a90 2023-07-21T22:29:43.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef00b7c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f186f70 0x56513f1a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef00b7c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f186f70 0x56513f1a8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48640/0 (socket says 172.21.15.107:48640) 2023-07-21T22:29:43.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef00b7c700 1 -- 172.21.15.107:0/879743 learned_addr learned my addr 172.21.15.107:0/879743 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7feefbfff700 1 --2- 172.21.15.107:0/879743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f191a90 0x56513f1a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef0137d700 1 --2- 172.21.15.107:0/879743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f1928b0 0x56513f199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef00b7c700 1 -- 172.21.15.107:0/879743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f191a90 msgr2=0x56513f1a26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef00b7c700 1 --2- 172.21.15.107:0/879743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f191a90 0x56513f1a26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef00b7c700 1 -- 172.21.15.107:0/879743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f1928b0 msgr2=0x56513f199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef00b7c700 1 --2- 172.21.15.107:0/879743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f1928b0 0x56513f199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7fef00b7c700 1 -- 172.21.15.107:0/879743 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56513f18d3b0 con 0x56513f186f70 2023-07-21T22:29:43.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.201+0000 7feefbfff700 1 --2- 172.21.15.107:0/879743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f191a90 0x56513f1a26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:43.251 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.202+0000 7fef00b7c700 1 --2- 172.21.15.107:0/879743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f186f70 0x56513f1a8f70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7feef000f740 tx=0x7feef000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643585e15afc8512 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:43.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.202+0000 7feee77fe700 1 -- 172.21.15.107:0/879743 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0003f30 con 0x56513f186f70 2023-07-21T22:29:43.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.202+0000 7feee77fe700 1 -- 172.21.15.107:0/879743 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feef003f070 con 0x56513f186f70 2023-07-21T22:29:43.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.202+0000 7feee77fe700 1 -- 172.21.15.107:0/879743 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0044430 con 0x56513f186f70 2023-07-21T22:29:43.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.203+0000 7fef0ef65b80 1 -- 172.21.15.107:0/879743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f186f70 msgr2=0x56513f1a8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.203+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/879743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f186f70 0x56513f1a8f70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7feef000f740 tx=0x7feef000bfd0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.203+0000 7fef0ef65b80 1 -- 172.21.15.107:0/879743 shutdown_connections 2023-07-21T22:29:43.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.203+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/879743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f1928b0 0x56513f199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.203+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/879743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f186f70 0x56513f1a8f70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.203+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/879743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f191a90 0x56513f1a26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.203+0000 7fef0ef65b80 1 -- 172.21.15.107:0/879743 >> 172.21.15.107:0/879743 conn(0x56513f1916c0 msgr2=0x56513f18ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:43.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.203+0000 7fef0ef65b80 1 -- 172.21.15.107:0/879743 shutdown_connections 2023-07-21T22:29:43.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.203+0000 7fef0ef65b80 1 -- 172.21.15.107:0/879743 wait complete. 2023-07-21T22:29:43.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.203+0000 7fef0ef65b80 1 -- start start 2023-07-21T22:29:43.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.203+0000 7fef0ef65b80 1 Processor -- start 2023-07-21T22:29:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7fef0ef65b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f186f70 0x56513f1a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7fef0ef65b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 0x56513f19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7fef0ef65b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 0x56513f19f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7fef0ef65b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513f19f8d0 con 0x56513f191a90 2023-07-21T22:29:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7fef0ef65b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513f19fa40 con 0x56513f186f70 2023-07-21T22:29:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7fef0ef65b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56513f1fae30 con 0x56513f1928b0 2023-07-21T22:29:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7fef0ef65b80 10 client.0 _ll_register_callbacks cb 0x56513f0807e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:43.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7feefbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 0x56513f19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7feefbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 0x56513f19ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48648/0 (socket says 172.21.15.107:48648) 2023-07-21T22:29:43.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7feefbfff700 1 -- 172.21.15.107:0/3634650280 learned_addr learned my addr 172.21.15.107:0/3634650280 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:43.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7fef00b7c700 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f186f70 0x56513f1a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7fef0137d700 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 0x56513f19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7feefbfff700 1 -- 172.21.15.107:0/3634650280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 msgr2=0x56513f19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7feefbfff700 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 0x56513f19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7feefbfff700 1 -- 172.21.15.107:0/3634650280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f186f70 msgr2=0x56513f1a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7feefbfff700 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f186f70 0x56513f1a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7feefbfff700 1 -- 172.21.15.107:0/3634650280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56513eef9ce0 con 0x56513f191a90 2023-07-21T22:29:43.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7fef00b7c700 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f186f70 0x56513f1a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:29:43.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7feefbfff700 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 0x56513f19ee50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7feee8017000 tx=0x7feee800a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d87daae0e6966f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:43.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.204+0000 7feedbfff700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:43.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7feedbfff700 1 -- 172.21.15.107:0/3634650280 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee801f040 con 0x56513f191a90 2023-07-21T22:29:43.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3634650280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feee800cd70 con 0x56513f191a90 2023-07-21T22:29:43.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7feedbfff700 1 -- 172.21.15.107:0/3634650280 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7feee8027aa0 con 0x56513f191a90 2023-07-21T22:29:43.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7feedbfff700 1 -- 172.21.15.107:0/3634650280 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7feee800c6a0 con 0x56513f191a90 2023-07-21T22:29:43.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7fef0ef65b80 10 client.15666 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:43.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3634650280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56513ef372d0 con 0x56513f191a90 2023-07-21T22:29:43.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7feedbfff700 1 -- 172.21.15.107:0/3634650280 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feee800cd70 con 0x56513f191a90 2023-07-21T22:29:43.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7feedbfff700 1 -- 172.21.15.107:0/3634650280 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7feee8016020 con 0x56513f191a90 2023-07-21T22:29:43.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7fef0ef65b80 10 client.15666 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:43.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7fef0ef65b80 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:43.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7fef0ef65b80 20 client.15666 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:43.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56513f193220 0x56513f1fe8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7fef0ef65b80 4 client.15666 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:29:43.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.205+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3634650280 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56513f09e0d0 con 0x56513f193220 2023-07-21T22:29:43.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.206+0000 7fef00b7c700 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56513f193220 0x56513f1fe8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.206+0000 7feedbfff700 10 client.15666 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:43.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.206+0000 7fef00b7c700 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56513f193220 0x56513f1fe8e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8abc812490d1e0f server_cookie=e57bf2cbc3160eda in_seq=0 out_seq=0 2023-07-21T22:29:43.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.207+0000 7feedbfff700 1 -- 172.21.15.107:0/3634650280 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56513f09e0d0 con 0x56513f193220 2023-07-21T22:29:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.207+0000 7feedbfff700 10 client.15666 handle_command_reply: tid=0 2023-07-21T22:29:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.207+0000 7fef0ef65b80 1 client.15666 shutdown 2023-07-21T22:29:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.207+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3634650280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 msgr2=0x56513f19ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.207+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 0x56513f19ee50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7feee8017000 tx=0x7feee800a320 comp rx=0 tx=0).stop 2023-07-21T22:29:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.207+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3634650280 shutdown_connections 2023-07-21T22:29:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.207+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56513f193220 0x56513f1fe8e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.208+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56513f1928b0 0x56513f19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.208+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56513f186f70 0x56513f1a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.208+0000 7fef0ef65b80 1 --2- 172.21.15.107:0/3634650280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56513f191a90 0x56513f19ee50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.208+0000 7feedbfff700 0 client.15666 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.208+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3634650280 >> 172.21.15.107:0/3634650280 conn(0x56513f1916c0 msgr2=0x56513f199050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.208+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3634650280 shutdown_connections 2023-07-21T22:29:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.208+0000 7fef0ef65b80 1 -- 172.21.15.107:0/3634650280 wait complete. 2023-07-21T22:29:43.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.208+0000 7fef0ef65b80 20 client.15666 trim_cache size 0 max 16384 2023-07-21T22:29:43.263 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:29:43.263 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2023-07-21T22:29:43.264 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "4b7bfc2d-247e-4de8-a2a6-fd6bfc207cfa", 2023-07-21T22:29:43.264 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2023-07-21T22:29:43.264 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:29:43.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.208+0000 7fef02de0700 1 -- 172.21.15.107:0/1650816580 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7feeec0a1150 msgr2=0x7feeec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.208+0000 7fef02de0700 1 --2- 172.21.15.107:0/1650816580 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7feeec0a1150 0x7feeec0a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feef0006010 tx=0x7feef000b560 comp rx=0 tx=0).stop 2023-07-21T22:29:43.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.209+0000 7fef02de0700 1 -- 172.21.15.107:0/1650816580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc100560 msgr2=0x7feefc19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.209+0000 7fef02de0700 1 --2- 172.21.15.107:0/1650816580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc100560 0x7feefc19cbc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7feef4007ee0 tx=0x7feef400d3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.209+0000 7fef02de0700 1 -- 172.21.15.107:0/1650816580 shutdown_connections 2023-07-21T22:29:43.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.209+0000 7fef02de0700 1 --2- 172.21.15.107:0/1650816580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc100560 0x7feefc19cbc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.209+0000 7fef02de0700 1 --2- 172.21.15.107:0/1650816580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feefc0fe9e0 0x7feefc068240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.209+0000 7fef02de0700 1 --2- 172.21.15.107:0/1650816580 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7feeec0a1150 0x7feeec0a3600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.209+0000 7fef02de0700 1 --2- 172.21.15.107:0/1650816580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feefc0ffbd0 0x7feefc19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.209+0000 7fef02de0700 1 -- 172.21.15.107:0/1650816580 >> 172.21.15.107:0/1650816580 conn(0x7feefc0fa000 msgr2=0x7feefc0fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:43.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.209+0000 7fef02de0700 1 -- 172.21.15.107:0/1650816580 shutdown_connections 2023-07-21T22:29:43.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.209+0000 7fef02de0700 1 -- 172.21.15.107:0/1650816580 wait complete. 2023-07-21T22:29:43.269 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:29:43.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.379+0000 7f4c45766700 1 -- 172.21.15.107:0/1819025147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c400ffc00 msgr2=0x7f4c40100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.379+0000 7f4c45766700 1 --2- 172.21.15.107:0/1819025147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c400ffc00 0x7f4c40100070 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4c34009990 tx=0x7f4c34009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.379+0000 7f4c45766700 1 -- 172.21.15.107:0/1819025147 shutdown_connections 2023-07-21T22:29:43.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.379+0000 7f4c45766700 1 --2- 172.21.15.107:0/1819025147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c401005b0 0x7f4c40106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.379+0000 7f4c45766700 1 --2- 172.21.15.107:0/1819025147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c400feef0 0x7f4c400ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.426 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.379+0000 7f4c45766700 1 --2- 172.21.15.107:0/1819025147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c400ffc00 0x7f4c40100070 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.379+0000 7f4c45766700 1 -- 172.21.15.107:0/1819025147 >> 172.21.15.107:0/1819025147 conn(0x7f4c400f9770 msgr2=0x7f4c400fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:43.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.379+0000 7f4c45766700 1 -- 172.21.15.107:0/1819025147 shutdown_connections 2023-07-21T22:29:43.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.379+0000 7f4c45766700 1 -- 172.21.15.107:0/1819025147 wait complete. 2023-07-21T22:29:43.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c45766700 1 Processor -- start 2023-07-21T22:29:43.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c45766700 1 -- start start 2023-07-21T22:29:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c45766700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c400feef0 0x7f4c40194710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c45766700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c400ffc00 0x7f4c40194c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c45766700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c401005b0 0x7f4c4019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c45766700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4019c290 con 0x7f4c400feef0 2023-07-21T22:29:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c45766700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c400fda40 con 0x7f4c400ffc00 2023-07-21T22:29:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c45766700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c400fdbd0 con 0x7f4c401005b0 2023-07-21T22:29:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c3effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c400feef0 0x7f4c40194710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c3effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c400feef0 0x7f4c40194710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48660/0 (socket says 172.21.15.107:48660) 2023-07-21T22:29:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c3effd700 1 -- 172.21.15.107:0/2918315303 learned_addr learned my addr 172.21.15.107:0/2918315303 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c3e7fc700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c400ffc00 0x7f4c40194c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c3f7fe700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c401005b0 0x7f4c4019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c3effd700 1 -- 172.21.15.107:0/2918315303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c401005b0 msgr2=0x7f4c4019bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c3effd700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c401005b0 0x7f4c4019bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c3effd700 1 -- 172.21.15.107:0/2918315303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c400ffc00 msgr2=0x7f4c40194c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c3effd700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c400ffc00 0x7f4c40194c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.380+0000 7f4c3effd700 1 -- 172.21.15.107:0/2918315303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c340095f0 con 0x7f4c400feef0 2023-07-21T22:29:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.381+0000 7f4c3effd700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c400feef0 0x7f4c40194710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000d980 tx=0x7f4c3000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.381+0000 7f4c27fff700 1 -- 172.21.15.107:0/2918315303 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3000de60 con 0x7f4c400feef0 2023-07-21T22:29:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.381+0000 7f4c45766700 1 -- 172.21.15.107:0/2918315303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c400fdeb0 con 0x7f4c400feef0 2023-07-21T22:29:43.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.381+0000 7f4c27fff700 1 -- 172.21.15.107:0/2918315303 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c300044e0 con 0x7f4c400feef0 2023-07-21T22:29:43.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.381+0000 7f4c27fff700 1 -- 172.21.15.107:0/2918315303 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30003be0 con 0x7f4c400feef0 2023-07-21T22:29:43.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.381+0000 7f4c45766700 1 -- 172.21.15.107:0/2918315303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c400fe410 con 0x7f4c400feef0 2023-07-21T22:29:43.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:43 smithi132 ceph-mon[97863]: pgmap v312: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2023-07-21T22:29:43.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.383+0000 7f4c27fff700 1 -- 172.21.15.107:0/2918315303 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4c30009bb0 con 0x7f4c400feef0 2023-07-21T22:29:43.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.384+0000 7f4c27fff700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4c280a1150 0x7f4c280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.384+0000 7f4c27fff700 1 -- 172.21.15.107:0/2918315303 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4c300cbed0 con 0x7f4c400feef0 2023-07-21T22:29:43.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.384+0000 7f4c3e7fc700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4c280a1150 0x7f4c280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.384+0000 7f4c3e7fc700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4c280a1150 0x7f4c280a3600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4c34006010 tx=0x7f4c3400b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:43.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.385+0000 7f4c518ebb80 1 Processor -- start 2023-07-21T22:29:43.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.385+0000 7f4c518ebb80 1 -- start start 2023-07-21T22:29:43.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.385+0000 7f4c518ebb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8be0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c518ebb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 0x558fd8bdf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c518ebb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8bdfa10 0x558fd8bdfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c518ebb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd8ae4cb0 con 0x558fd8be11a0 2023-07-21T22:29:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c518ebb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd8b37a40 con 0x558fd8be0830 2023-07-21T22:29:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c518ebb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd8b30ed0 con 0x558fd8bdfa10 2023-07-21T22:29:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c3e7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 0x558fd8bdf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c3e7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 0x558fd8bdf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48670/0 (socket says 172.21.15.107:48670) 2023-07-21T22:29:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c3e7fc700 1 -- 172.21.15.107:0/1403573390 learned_addr learned my addr 172.21.15.107:0/1403573390 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:43.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c3f7fe700 1 --2- 172.21.15.107:0/1403573390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8bdfa10 0x558fd8bdfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c3effd700 1 --2- 172.21.15.107:0/1403573390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8be0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c3e7fc700 1 -- 172.21.15.107:0/1403573390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8bdfa10 msgr2=0x558fd8bdfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c3e7fc700 1 --2- 172.21.15.107:0/1403573390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8bdfa10 0x558fd8bdfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c3e7fc700 1 -- 172.21.15.107:0/1403573390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 msgr2=0x558fd8be0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c3e7fc700 1 --2- 172.21.15.107:0/1403573390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8be0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c3e7fc700 1 -- 172.21.15.107:0/1403573390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fd895ede0 con 0x558fd8be11a0 2023-07-21T22:29:43.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c3effd700 1 --2- 172.21.15.107:0/1403573390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8be0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:43.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c3e7fc700 1 --2- 172.21.15.107:0/1403573390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 0x558fd8bdf430 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4c400fec10 tx=0x7f4c3401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5897324afa724c26 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:43.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.386+0000 7f4c257fa700 1 -- 172.21.15.107:0/1403573390 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3402b070 con 0x558fd8be11a0 2023-07-21T22:29:43.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.387+0000 7f4c257fa700 1 -- 172.21.15.107:0/1403573390 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c3401e040 con 0x558fd8be11a0 2023-07-21T22:29:43.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.387+0000 7f4c257fa700 1 -- 172.21.15.107:0/1403573390 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34004e50 con 0x558fd8be11a0 2023-07-21T22:29:43.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.387+0000 7f4c518ebb80 1 -- 172.21.15.107:0/1403573390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 msgr2=0x558fd8bdf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.387+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/1403573390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 0x558fd8bdf430 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4c400fec10 tx=0x7f4c3401f000 comp rx=0 tx=0).stop 2023-07-21T22:29:43.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.387+0000 7f4c518ebb80 1 -- 172.21.15.107:0/1403573390 shutdown_connections 2023-07-21T22:29:43.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/1403573390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8bdfa10 0x558fd8bdfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/1403573390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8be0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/1403573390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 0x558fd8bdf430 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 -- 172.21.15.107:0/1403573390 >> 172.21.15.107:0/1403573390 conn(0x558fd8bd4ef0 msgr2=0x558fd8bda360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:43.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 -- 172.21.15.107:0/1403573390 shutdown_connections 2023-07-21T22:29:43.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 -- 172.21.15.107:0/1403573390 wait complete. 2023-07-21T22:29:43.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 -- start start 2023-07-21T22:29:43.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 Processor -- start 2023-07-21T22:29:43.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 0x558fd8be8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8be94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8be11a0 0x558fd8bf0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd8ad0ac0 con 0x558fd8bdfa10 2023-07-21T22:29:43.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd8ad07c0 con 0x558fd8be0830 2023-07-21T22:29:43.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd89b4920 con 0x558fd8be11a0 2023-07-21T22:29:43.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c518ebb80 10 client.0 _ll_register_callbacks cb 0x558fd8ace760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:43.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c3effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 0x558fd8be8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c3effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 0x558fd8be8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48674/0 (socket says 172.21.15.107:48674) 2023-07-21T22:29:43.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.388+0000 7f4c3effd700 1 -- 172.21.15.107:0/662518170 learned_addr learned my addr 172.21.15.107:0/662518170 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:43.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c3f7fe700 1 --2- 172.21.15.107:0/662518170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8be11a0 0x558fd8bf0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c3effd700 1 -- 172.21.15.107:0/662518170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8be11a0 msgr2=0x558fd8bf0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c3effd700 1 --2- 172.21.15.107:0/662518170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8be11a0 0x558fd8bf0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c3effd700 1 -- 172.21.15.107:0/662518170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 msgr2=0x558fd8be94f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:29:43.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c3effd700 1 --2- 172.21.15.107:0/662518170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8be94f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c3effd700 1 -- 172.21.15.107:0/662518170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c40107750 con 0x558fd8bdfa10 2023-07-21T22:29:43.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c3effd700 1 --2- 172.21.15.107:0/662518170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 0x558fd8be8fb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009520 tx=0x7f4c300993f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3933904bf0556 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:43.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:43.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c22ff5700 1 -- 172.21.15.107:0/662518170 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c300989b0 con 0x558fd8bdfa10 2023-07-21T22:29:43.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c22ff5700 1 -- 172.21.15.107:0/662518170 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4c300a1710 con 0x558fd8bdfa10 2023-07-21T22:29:43.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c518ebb80 1 -- 172.21.15.107:0/662518170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fd8a8a4f0 con 0x558fd8bdfa10 2023-07-21T22:29:43.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c22ff5700 1 -- 172.21.15.107:0/662518170 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c300a1280 con 0x558fd8bdfa10 2023-07-21T22:29:43.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c518ebb80 10 client.15678 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:43.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c518ebb80 1 -- 172.21.15.107:0/662518170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558fd8947c60 con 0x558fd8bdfa10 2023-07-21T22:29:43.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.389+0000 7f4c22ff5700 1 -- 172.21.15.107:0/662518170 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c300c7020 con 0x558fd8bdfa10 2023-07-21T22:29:43.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.390+0000 7f4c22ff5700 1 -- 172.21.15.107:0/662518170 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4c300950b0 con 0x558fd8bdfa10 2023-07-21T22:29:43.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.390+0000 7f4c518ebb80 10 client.15678 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:43.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.390+0000 7f4c518ebb80 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:43.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.390+0000 7f4c518ebb80 20 client.15678 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:43.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.390+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/662518170 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558fd8bf4d10 0x558fd8bf7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.390+0000 7f4c518ebb80 4 client.15678 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:43.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.390+0000 7f4c518ebb80 1 -- 172.21.15.107:0/662518170 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fd8b29eb0 con 0x558fd8bf4d10 2023-07-21T22:29:43.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.390+0000 7f4c3e7fc700 1 --2- 172.21.15.107:0/662518170 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558fd8bf4d10 0x558fd8bf7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.391+0000 7f4c22ff5700 10 client.15678 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:43.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.391+0000 7f4c3e7fc700 1 --2- 172.21.15.107:0/662518170 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558fd8bf4d10 0x558fd8bf7120 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f0e4c7fcb5c827 server_cookie=2e02825e9570bd4f in_seq=0 out_seq=0 2023-07-21T22:29:43.446 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:43 smithi107 ceph-mon[96543]: pgmap v312: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2023-07-21T22:29:43.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.391+0000 7f4c22ff5700 1 -- 172.21.15.107:0/662518170 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558fd8b29eb0 con 0x558fd8bf4d10 2023-07-21T22:29:43.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.391+0000 7f4c22ff5700 10 client.15678 handle_command_reply: tid=0 2023-07-21T22:29:43.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.391+0000 7f4c518ebb80 1 client.15678 shutdown 2023-07-21T22:29:43.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.391+0000 7f4c518ebb80 1 -- 172.21.15.107:0/662518170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 msgr2=0x558fd8be8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.391+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/662518170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 0x558fd8be8fb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009520 tx=0x7f4c300993f0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.391+0000 7f4c518ebb80 1 -- 172.21.15.107:0/662518170 shutdown_connections 2023-07-21T22:29:43.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.391+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/662518170 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558fd8bf4d10 0x558fd8bf7120 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.391+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/662518170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8be11a0 0x558fd8bf0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.391+0000 7f4c22ff5700 0 client.15678 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:43.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.391+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/662518170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 0x558fd8be8fb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.391+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/662518170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8be94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.392+0000 7f4c518ebb80 1 -- 172.21.15.107:0/662518170 >> 172.21.15.107:0/662518170 conn(0x558fd8bd4ef0 msgr2=0x558fd8bd9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:43.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.392+0000 7f4c518ebb80 1 -- 172.21.15.107:0/662518170 shutdown_connections 2023-07-21T22:29:43.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.392+0000 7f4c518ebb80 1 -- 172.21.15.107:0/662518170 wait complete. 2023-07-21T22:29:43.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.392+0000 7f4c518ebb80 20 client.15678 trim_cache size 0 max 16384 2023-07-21T22:29:43.460 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.413+0000 7f4c518ebb80 1 Processor -- start 2023-07-21T22:29:43.460 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.413+0000 7f4c518ebb80 1 -- start start 2023-07-21T22:29:43.460 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c518ebb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8bdfa10 0x558fd8bdf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.460 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c518ebb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8bf04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.461 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c518ebb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 0x558fd8bd9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.461 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c518ebb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd8bf4aa0 con 0x558fd8be11a0 2023-07-21T22:29:43.461 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c518ebb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd8bd1e50 con 0x558fd8be0830 2023-07-21T22:29:43.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c518ebb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd8bdbc90 con 0x558fd8bdfa10 2023-07-21T22:29:43.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c3f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 0x558fd8bd9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c3f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 0x558fd8bd9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48684/0 (socket says 172.21.15.107:48684) 2023-07-21T22:29:43.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c3f7fe700 1 -- 172.21.15.107:0/2082334293 learned_addr learned my addr 172.21.15.107:0/2082334293 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:43.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c3effd700 1 --2- 172.21.15.107:0/2082334293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8bdfa10 0x558fd8bdf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c3e7fc700 1 --2- 172.21.15.107:0/2082334293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8bf04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c3f7fe700 1 -- 172.21.15.107:0/2082334293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8bdfa10 msgr2=0x558fd8bdf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c3f7fe700 1 --2- 172.21.15.107:0/2082334293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8bdfa10 0x558fd8bdf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c3f7fe700 1 -- 172.21.15.107:0/2082334293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 msgr2=0x558fd8bf04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c3f7fe700 1 --2- 172.21.15.107:0/2082334293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8bf04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c3f7fe700 1 -- 172.21.15.107:0/2082334293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fd8985250 con 0x558fd8be11a0 2023-07-21T22:29:43.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.414+0000 7f4c3f7fe700 1 --2- 172.21.15.107:0/2082334293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 0x558fd8bd9a00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009fd0 tx=0x7f4c3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5aa1ac6eee3503d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:43.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.415+0000 7f4c25ffb700 1 -- 172.21.15.107:0/2082334293 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800e030 con 0x558fd8be11a0 2023-07-21T22:29:43.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.415+0000 7f4c25ffb700 1 -- 172.21.15.107:0/2082334293 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c3800f040 con 0x558fd8be11a0 2023-07-21T22:29:43.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.415+0000 7f4c25ffb700 1 -- 172.21.15.107:0/2082334293 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38016660 con 0x558fd8be11a0 2023-07-21T22:29:43.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.415+0000 7f4c518ebb80 1 -- 172.21.15.107:0/2082334293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 msgr2=0x558fd8bd9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.415+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/2082334293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 0x558fd8bd9a00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009fd0 tx=0x7f4c3800b040 comp rx=0 tx=0).stop 2023-07-21T22:29:43.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 -- 172.21.15.107:0/2082334293 shutdown_connections 2023-07-21T22:29:43.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/2082334293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8be11a0 0x558fd8bd9a00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/2082334293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8bdfa10 0x558fd8bdf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/2082334293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8bf04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 -- 172.21.15.107:0/2082334293 >> 172.21.15.107:0/2082334293 conn(0x558fd8bd4ef0 msgr2=0x558fd8bdfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:43.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 -- 172.21.15.107:0/2082334293 shutdown_connections 2023-07-21T22:29:43.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 -- 172.21.15.107:0/2082334293 wait complete. 2023-07-21T22:29:43.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 -- start start 2023-07-21T22:29:43.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 Processor -- start 2023-07-21T22:29:43.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 0x558fd8bf01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8becc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8be11a0 0x558fd8bed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd8bed710 con 0x558fd8bdfa10 2023-07-21T22:29:43.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd8bed880 con 0x558fd8be0830 2023-07-21T22:29:43.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fd8c48be0 con 0x558fd8be11a0 2023-07-21T22:29:43.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.416+0000 7f4c518ebb80 10 client.0 _ll_register_callbacks cb 0x7f4c30098c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:43.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c3effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 0x558fd8bf01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c3effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 0x558fd8bf01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48686/0 (socket says 172.21.15.107:48686) 2023-07-21T22:29:43.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c3e7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8becc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c3effd700 1 -- 172.21.15.107:0/3916964008 learned_addr learned my addr 172.21.15.107:0/3916964008 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:43.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c3f7fe700 1 --2- 172.21.15.107:0/3916964008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8be11a0 0x558fd8bed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c3effd700 1 -- 172.21.15.107:0/3916964008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8be11a0 msgr2=0x558fd8bed1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c3effd700 1 --2- 172.21.15.107:0/3916964008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8be11a0 0x558fd8bed1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c3effd700 1 -- 172.21.15.107:0/3916964008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 msgr2=0x558fd8becc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:43.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c3effd700 1 --2- 172.21.15.107:0/3916964008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8becc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c3effd700 1 -- 172.21.15.107:0/3916964008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c380091f0 con 0x558fd8bdfa10 2023-07-21T22:29:43.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c3effd700 1 --2- 172.21.15.107:0/3916964008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 0x558fd8bf01f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4c300c9a00 tx=0x7f4c30098f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b700c6c24c75d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:43.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:29:43.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c22ff5700 1 -- 172.21.15.107:0/3916964008 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30099090 con 0x558fd8bdfa10 2023-07-21T22:29:43.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c22ff5700 1 -- 172.21.15.107:0/3916964008 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4c300c8080 con 0x558fd8bdfa10 2023-07-21T22:29:43.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.417+0000 7f4c518ebb80 1 -- 172.21.15.107:0/3916964008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c3401e040 con 0x558fd8bdfa10 2023-07-21T22:29:43.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.418+0000 7f4c22ff5700 1 -- 172.21.15.107:0/3916964008 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c300a1710 con 0x558fd8bdfa10 2023-07-21T22:29:43.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.418+0000 7f4c518ebb80 10 client.15684 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:43.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.418+0000 7f4c518ebb80 1 -- 172.21.15.107:0/3916964008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558fd895d2b0 con 0x558fd8bdfa10 2023-07-21T22:29:43.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.418+0000 7f4c22ff5700 1 -- 172.21.15.107:0/3916964008 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c3401e040 con 0x558fd8bdfa10 2023-07-21T22:29:43.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.418+0000 7f4c22ff5700 1 -- 172.21.15.107:0/3916964008 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4c3007b080 con 0x558fd8bdfa10 2023-07-21T22:29:43.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.418+0000 7f4c518ebb80 10 client.15684 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:43.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.418+0000 7f4c518ebb80 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:43.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.418+0000 7f4c518ebb80 20 client.15684 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:43.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.418+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/3916964008 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558fd8c4a080 0x558fd8c4c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:43.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.418+0000 7f4c518ebb80 4 client.15684 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:29:43.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.418+0000 7f4c518ebb80 1 -- 172.21.15.107:0/3916964008 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558fd8aec050 con 0x558fd8c4a080 2023-07-21T22:29:43.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.418+0000 7f4c3e7fc700 1 --2- 172.21.15.107:0/3916964008 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558fd8c4a080 0x558fd8c4c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:43.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c22ff5700 10 client.15684 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:43.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c3e7fc700 1 --2- 172.21.15.107:0/3916964008 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558fd8c4a080 0x558fd8c4c490 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b1332b0067e0a7b server_cookie=94c1146e4d46d792 in_seq=0 out_seq=0 2023-07-21T22:29:43.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c22ff5700 1 -- 172.21.15.107:0/3916964008 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558fd8aec050 con 0x558fd8c4a080 2023-07-21T22:29:43.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c22ff5700 10 client.15684 handle_command_reply: tid=0 2023-07-21T22:29:43.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c518ebb80 1 client.15684 shutdown 2023-07-21T22:29:43.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c518ebb80 1 -- 172.21.15.107:0/3916964008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 msgr2=0x558fd8bf01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/3916964008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 0x558fd8bf01f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4c300c9a00 tx=0x7f4c30098f30 comp rx=0 tx=0).stop 2023-07-21T22:29:43.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c518ebb80 1 -- 172.21.15.107:0/3916964008 shutdown_connections 2023-07-21T22:29:43.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/3916964008 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558fd8c4a080 0x558fd8c4c490 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/3916964008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fd8be11a0 0x558fd8bed1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c22ff5700 0 client.15684 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:43.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/3916964008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558fd8bdfa10 0x558fd8bf01f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c518ebb80 1 --2- 172.21.15.107:0/3916964008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558fd8be0830 0x558fd8becc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c518ebb80 1 -- 172.21.15.107:0/3916964008 >> 172.21.15.107:0/3916964008 conn(0x558fd8bd4ef0 msgr2=0x558fd8bd9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:43.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c518ebb80 1 -- 172.21.15.107:0/3916964008 shutdown_connections 2023-07-21T22:29:43.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c518ebb80 1 -- 172.21.15.107:0/3916964008 wait complete. 2023-07-21T22:29:43.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.419+0000 7f4c518ebb80 20 client.15684 trim_cache size 0 max 16384 2023-07-21T22:29:43.475 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:29:43.475 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:29:43.475 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:29:43.475 INFO:teuthology.orchestra.run.smithi107.stdout: "4b7bfc2d-247e-4de8-a2a6-fd6bfc207cfa": { 2023-07-21T22:29:43.475 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:29:43.476 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "4b7bfc2d-247e-4de8-a2a6-fd6bfc207cfa", 2023-07-21T22:29:43.476 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:29:43.476 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:29:43.476 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:29:43.476 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:29:43.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.420+0000 7f4c45766700 1 -- 172.21.15.107:0/2918315303 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4c280a1150 msgr2=0x7f4c280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.420+0000 7f4c45766700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4c280a1150 0x7f4c280a3600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4c34006010 tx=0x7f4c3400b260 comp rx=0 tx=0).stop 2023-07-21T22:29:43.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.420+0000 7f4c45766700 1 -- 172.21.15.107:0/2918315303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c400feef0 msgr2=0x7f4c40194710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:43.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.420+0000 7f4c45766700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c400feef0 0x7f4c40194710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000d980 tx=0x7f4c3000dd40 comp rx=0 tx=0).stop 2023-07-21T22:29:43.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.420+0000 7f4c45766700 1 -- 172.21.15.107:0/2918315303 shutdown_connections 2023-07-21T22:29:43.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.420+0000 7f4c45766700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4c280a1150 0x7f4c280a3600 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.420+0000 7f4c45766700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c401005b0 0x7f4c4019bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.420+0000 7f4c45766700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c400feef0 0x7f4c40194710 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.420+0000 7f4c45766700 1 --2- 172.21.15.107:0/2918315303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c400ffc00 0x7f4c40194c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:43.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.420+0000 7f4c45766700 1 -- 172.21.15.107:0/2918315303 >> 172.21.15.107:0/2918315303 conn(0x7f4c400f9770 msgr2=0x7f4c400fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:43.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.420+0000 7f4c45766700 1 -- 172.21.15.107:0/2918315303 shutdown_connections 2023-07-21T22:29:43.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:29:43.420+0000 7f4c45766700 1 -- 172.21.15.107:0/2918315303 wait complete. 2023-07-21T22:29:43.480 INFO:tasks.cephfs.filesystem:scrub status for tag:4b7bfc2d-247e-4de8-a2a6-fd6bfc207cfa - {'path': '/', 'tag': '4b7bfc2d-247e-4de8-a2a6-fd6bfc207cfa', 'options': 'recursive,force'} 2023-07-21T22:29:43.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:43 smithi190 ceph-mon[98334]: pgmap v312: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2023-07-21T22:29:44.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:44 smithi107 ceph-mon[96543]: scrub queued for path: / 2023-07-21T22:29:44.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:44 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:44.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:44 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:29:44.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:44 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:29:44.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:44 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:44.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:44 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:44 smithi190 ceph-mon[98334]: scrub queued for path: / 2023-07-21T22:29:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:44 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:44 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:29:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:44 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:29:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:44 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:44.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:44 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:44.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:44 smithi132 ceph-mon[97863]: scrub queued for path: / 2023-07-21T22:29:44.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:44 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:44.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:44 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:29:44.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:44 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:29:44.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:44 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:44.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:44 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:29:45.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:45 smithi107 ceph-mon[96543]: pgmap v313: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T22:29:45.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:45 smithi190 ceph-mon[98334]: pgmap v313: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T22:29:45.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:45 smithi132 ceph-mon[97863]: pgmap v313: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-21T22:29:46.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:46 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:29:46.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:46 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:29:46.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:46 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:29:47.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:47 smithi190 ceph-mon[98334]: pgmap v314: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-21T22:29:47.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:47 smithi132 ceph-mon[97863]: pgmap v314: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-21T22:29:47.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:47 smithi107 ceph-mon[96543]: pgmap v314: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-21T22:29:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.412+0000 7fcf598e8700 1 -- 172.21.15.107:0/2437323933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf540ffc10 msgr2=0x7fcf54100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.412+0000 7fcf598e8700 1 --2- 172.21.15.107:0/2437323933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf540ffc10 0x7fcf54100060 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009990 tx=0x7fcf44009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.412+0000 7fcf598e8700 1 -- 172.21.15.107:0/2437323933 shutdown_connections 2023-07-21T22:29:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.412+0000 7fcf598e8700 1 --2- 172.21.15.107:0/2437323933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf541005a0 0x7fcf54106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.412+0000 7fcf598e8700 1 --2- 172.21.15.107:0/2437323933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf540fea20 0x7fcf540fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.412+0000 7fcf598e8700 1 --2- 172.21.15.107:0/2437323933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf540ffc10 0x7fcf54100060 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.412+0000 7fcf598e8700 1 -- 172.21.15.107:0/2437323933 >> 172.21.15.107:0/2437323933 conn(0x7fcf540fa020 msgr2=0x7fcf540fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.412+0000 7fcf598e8700 1 -- 172.21.15.107:0/2437323933 shutdown_connections 2023-07-21T22:29:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.412+0000 7fcf598e8700 1 -- 172.21.15.107:0/2437323933 wait complete. 2023-07-21T22:29:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.413+0000 7fcf598e8700 1 Processor -- start 2023-07-21T22:29:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.413+0000 7fcf598e8700 1 -- start start 2023-07-21T22:29:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.413+0000 7fcf598e8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf540fea20 0x7fcf54193700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.413+0000 7fcf598e8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf540ffc10 0x7fcf54193c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.413+0000 7fcf598e8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf541005a0 0x7fcf54194b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.413+0000 7fcf598e8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf54195170 con 0x7fcf541005a0 2023-07-21T22:29:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.413+0000 7fcf598e8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf541952e0 con 0x7fcf540fea20 2023-07-21T22:29:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.413+0000 7fcf598e8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf541954a0 con 0x7fcf540ffc10 2023-07-21T22:29:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf52ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf540fea20 0x7fcf54193700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf541005a0 0x7fcf54194b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf541005a0 0x7fcf54194b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48702/0 (socket says 172.21.15.107:48702) 2023-07-21T22:29:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf52ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf540fea20 0x7fcf54193700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:39526/0 (socket says 172.21.15.107:39526) 2023-07-21T22:29:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf537fe700 1 -- 172.21.15.107:0/808467543 learned_addr learned my addr 172.21.15.107:0/808467543 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf527fc700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf540ffc10 0x7fcf54193c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf52ffd700 1 -- 172.21.15.107:0/808467543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf540ffc10 msgr2=0x7fcf54193c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf52ffd700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf540ffc10 0x7fcf54193c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf52ffd700 1 -- 172.21.15.107:0/808467543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf541005a0 msgr2=0x7fcf54194b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf52ffd700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf541005a0 0x7fcf54194b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf52ffd700 1 -- 172.21.15.107:0/808467543 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf440095f0 con 0x7fcf540fea20 2023-07-21T22:29:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf527fc700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf540ffc10 0x7fcf54193c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf52ffd700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf540fea20 0x7fcf54193700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcf480096d0 tx=0x7fcf480099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.414+0000 7fcf588e6700 1 -- 172.21.15.107:0/808467543 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf48018070 con 0x7fcf540fea20 2023-07-21T22:29:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.415+0000 7fcf598e8700 1 -- 172.21.15.107:0/808467543 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf54195780 con 0x7fcf540fea20 2023-07-21T22:29:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.415+0000 7fcf588e6700 1 -- 172.21.15.107:0/808467543 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf480044e0 con 0x7fcf540fea20 2023-07-21T22:29:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.415+0000 7fcf598e8700 1 -- 172.21.15.107:0/808467543 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf54195f30 con 0x7fcf540fea20 2023-07-21T22:29:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.415+0000 7fcf588e6700 1 -- 172.21.15.107:0/808467543 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf48010040 con 0x7fcf540fea20 2023-07-21T22:29:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.417+0000 7fcf588e6700 1 -- 172.21.15.107:0/808467543 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fcf48004020 con 0x7fcf540fea20 2023-07-21T22:29:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.418+0000 7fcf588e6700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcf3c0a11d0 0x7fcf3c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.418+0000 7fcf588e6700 1 -- 172.21.15.107:0/808467543 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcf480cbec0 con 0x7fcf540fea20 2023-07-21T22:29:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.418+0000 7fcf527fc700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcf3c0a11d0 0x7fcf3c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.419+0000 7fcf527fc700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcf3c0a11d0 0x7fcf3c0a3680 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400b2e0 tx=0x7fcf44005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.419+0000 7fcf65a6db80 1 Processor -- start 2023-07-21T22:29:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.419+0000 7fcf65a6db80 1 -- start start 2023-07-21T22:29:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf65a6db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582109506b0 0x558210af42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf65a6db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af4830 0x558210af5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf65a6db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af5580 0x558210af59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf65a6db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582109f9c50 con 0x5582109506b0 2023-07-21T22:29:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf65a6db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210a4c9e0 con 0x558210af4830 2023-07-21T22:29:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf65a6db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210a45e70 con 0x558210af5580 2023-07-21T22:29:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf52ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582109506b0 0x558210af42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf52ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582109506b0 0x558210af42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48708/0 (socket says 172.21.15.107:48708) 2023-07-21T22:29:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf52ffd700 1 -- 172.21.15.107:0/1313676627 learned_addr learned my addr 172.21.15.107:0/1313676627 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf527fc700 1 --2- 172.21.15.107:0/1313676627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af4830 0x558210af5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf52ffd700 1 -- 172.21.15.107:0/1313676627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af5580 msgr2=0x558210af59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf52ffd700 1 --2- 172.21.15.107:0/1313676627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af5580 0x558210af59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf52ffd700 1 -- 172.21.15.107:0/1313676627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af4830 msgr2=0x558210af5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf52ffd700 1 --2- 172.21.15.107:0/1313676627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af4830 0x558210af5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.420+0000 7fcf52ffd700 1 -- 172.21.15.107:0/1313676627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558210873d80 con 0x5582109506b0 2023-07-21T22:29:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.421+0000 7fcf52ffd700 1 --2- 172.21.15.107:0/1313676627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582109506b0 0x558210af42f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcf480046c0 tx=0x7fcf48098ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211bfb0e638f614e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.421+0000 7fcf39ffb700 1 -- 172.21.15.107:0/1313676627 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4809d2e0 con 0x5582109506b0 2023-07-21T22:29:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.421+0000 7fcf39ffb700 1 -- 172.21.15.107:0/1313676627 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf480a1700 con 0x5582109506b0 2023-07-21T22:29:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.421+0000 7fcf39ffb700 1 -- 172.21.15.107:0/1313676627 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf480a14d0 con 0x5582109506b0 2023-07-21T22:29:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.422+0000 7fcf65a6db80 1 -- 172.21.15.107:0/1313676627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582109506b0 msgr2=0x558210af42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.422+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/1313676627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582109506b0 0x558210af42f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcf480046c0 tx=0x7fcf48098ef0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.422+0000 7fcf65a6db80 1 -- 172.21.15.107:0/1313676627 shutdown_connections 2023-07-21T22:29:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.422+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/1313676627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af5580 0x558210af59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.422+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/1313676627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582109506b0 0x558210af42f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.422+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/1313676627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af4830 0x558210af5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.422+0000 7fcf65a6db80 1 -- 172.21.15.107:0/1313676627 >> 172.21.15.107:0/1313676627 conn(0x558210ae9ae0 msgr2=0x558210aeec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.422+0000 7fcf65a6db80 1 -- 172.21.15.107:0/1313676627 shutdown_connections 2023-07-21T22:29:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.422+0000 7fcf65a6db80 1 -- 172.21.15.107:0/1313676627 wait complete. 2023-07-21T22:29:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.422+0000 7fcf65a6db80 1 -- start start 2023-07-21T22:29:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.423+0000 7fcf65a6db80 1 Processor -- start 2023-07-21T22:29:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.423+0000 7fcf65a6db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582109506b0 0x558210afd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.423+0000 7fcf65a6db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af4830 0x558210afde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.423+0000 7fcf65a6db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558210af5580 0x558210b04ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.423+0000 7fcf65a6db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582109e5a60 con 0x558210af5580 2023-07-21T22:29:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.423+0000 7fcf65a6db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582109e5760 con 0x558210af4830 2023-07-21T22:29:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.423+0000 7fcf65a6db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582108c98c0 con 0x5582109506b0 2023-07-21T22:29:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.423+0000 7fcf65a6db80 10 client.0 _ll_register_callbacks cb 0x5582109e3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558210af5580 0x558210b04ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558210af5580 0x558210b04ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48714/0 (socket says 172.21.15.107:48714) 2023-07-21T22:29:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf537fe700 1 -- 172.21.15.107:0/486175011 learned_addr learned my addr 172.21.15.107:0/486175011 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf527fc700 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af4830 0x558210afde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf52ffd700 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582109506b0 0x558210afd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf527fc700 1 -- 172.21.15.107:0/486175011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582109506b0 msgr2=0x558210afd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf527fc700 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582109506b0 0x558210afd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf527fc700 1 -- 172.21.15.107:0/486175011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558210af5580 msgr2=0x558210b04ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf527fc700 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558210af5580 0x558210b04ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf527fc700 1 -- 172.21.15.107:0/486175011 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf54195f30 con 0x558210af4830 2023-07-21T22:29:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf52ffd700 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582109506b0 0x558210afd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf527fc700 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af4830 0x558210afde60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcf440059d0 tx=0x7fcf44009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae96a7c58f694fdd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:29:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.424+0000 7fcf32ffd700 1 -- 172.21.15.107:0/486175011 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf44024d40 con 0x558210af4830 2023-07-21T22:29:49.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf32ffd700 1 -- 172.21.15.107:0/486175011 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcf440303f0 con 0x558210af4830 2023-07-21T22:29:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf65a6db80 1 -- 172.21.15.107:0/486175011 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55821099f490 con 0x558210af4830 2023-07-21T22:29:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf32ffd700 1 -- 172.21.15.107:0/486175011 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf44024ea0 con 0x558210af4830 2023-07-21T22:29:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf65a6db80 10 client.25223 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf65a6db80 1 -- 172.21.15.107:0/486175011 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55821085cc00 con 0x558210af4830 2023-07-21T22:29:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf32ffd700 1 -- 172.21.15.107:0/486175011 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf4403f080 con 0x558210af4830 2023-07-21T22:29:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf32ffd700 1 -- 172.21.15.107:0/486175011 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fcf44039030 con 0x558210af4830 2023-07-21T22:29:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf65a6db80 10 client.25223 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:49.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf65a6db80 10 client.25223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf65a6db80 20 client.25223 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558210b1a6d0 0x558210b1cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf65a6db80 4 client.25223 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf65a6db80 1 -- 172.21.15.107:0/486175011 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558210a3ee50 con 0x558210b1a6d0 2023-07-21T22:29:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.425+0000 7fcf52ffd700 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558210b1a6d0 0x558210b1cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.426+0000 7fcf32ffd700 10 client.25223 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.426+0000 7fcf52ffd700 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558210b1a6d0 0x558210b1cae0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff40ce82f26c92d4 server_cookie=5f4b30e005c06abe in_seq=0 out_seq=0 2023-07-21T22:29:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.426+0000 7fcf32ffd700 1 -- 172.21.15.107:0/486175011 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558210a3ee50 con 0x558210b1a6d0 2023-07-21T22:29:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.426+0000 7fcf32ffd700 10 client.25223 handle_command_reply: tid=0 2023-07-21T22:29:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.427+0000 7fcf65a6db80 1 client.25223 shutdown 2023-07-21T22:29:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.427+0000 7fcf65a6db80 1 -- 172.21.15.107:0/486175011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af4830 msgr2=0x558210afde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.427+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af4830 0x558210afde60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcf440059d0 tx=0x7fcf44009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.427+0000 7fcf65a6db80 1 -- 172.21.15.107:0/486175011 shutdown_connections 2023-07-21T22:29:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.427+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558210b1a6d0 0x558210b1cae0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.427+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558210af5580 0x558210b04ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.427+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5582109506b0 0x558210afd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.427+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/486175011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af4830 0x558210afde60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.427+0000 7fcf65a6db80 1 -- 172.21.15.107:0/486175011 >> 172.21.15.107:0/486175011 conn(0x558210ae9ae0 msgr2=0x558210aee3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.427+0000 7fcf32ffd700 0 client.25223 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.427+0000 7fcf65a6db80 1 -- 172.21.15.107:0/486175011 shutdown_connections 2023-07-21T22:29:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.427+0000 7fcf65a6db80 1 -- 172.21.15.107:0/486175011 wait complete. 2023-07-21T22:29:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.427+0000 7fcf65a6db80 20 client.25223 trim_cache size 0 max 16384 2023-07-21T22:29:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.455+0000 7fcf65a6db80 1 Processor -- start 2023-07-21T22:29:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.455+0000 7fcf65a6db80 1 -- start start 2023-07-21T22:29:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.455+0000 7fcf65a6db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582109506b0 0x558210afc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.455+0000 7fcf65a6db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af4830 0x558210b04e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.455+0000 7fcf65a6db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af5580 0x558210af84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.455+0000 7fcf65a6db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210af06e0 con 0x5582109506b0 2023-07-21T22:29:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.455+0000 7fcf65a6db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210aefaf0 con 0x558210af5580 2023-07-21T22:29:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.455+0000 7fcf65a6db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210afd4c0 con 0x558210af4830 2023-07-21T22:29:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.456+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af5580 0x558210af84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.456+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af5580 0x558210af84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:39552/0 (socket says 172.21.15.107:39552) 2023-07-21T22:29:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.456+0000 7fcf537fe700 1 -- 172.21.15.107:0/539779277 learned_addr learned my addr 172.21.15.107:0/539779277 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.456+0000 7fcf527fc700 1 --2- 172.21.15.107:0/539779277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af4830 0x558210b04e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.456+0000 7fcf52ffd700 1 --2- 172.21.15.107:0/539779277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582109506b0 0x558210afc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.456+0000 7fcf537fe700 1 -- 172.21.15.107:0/539779277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af4830 msgr2=0x558210b04e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.456+0000 7fcf537fe700 1 --2- 172.21.15.107:0/539779277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af4830 0x558210b04e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.456+0000 7fcf537fe700 1 -- 172.21.15.107:0/539779277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582109506b0 msgr2=0x558210afc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.456+0000 7fcf537fe700 1 --2- 172.21.15.107:0/539779277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582109506b0 0x558210afc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.456+0000 7fcf537fe700 1 -- 172.21.15.107:0/539779277 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55821089a1f0 con 0x558210af5580 2023-07-21T22:29:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.456+0000 7fcf537fe700 1 --2- 172.21.15.107:0/539779277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af5580 0x558210af84d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00a7a0 tx=0x7fcf4c00aab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d12880562f78828 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.456+0000 7fcf3a7fc700 1 -- 172.21.15.107:0/539779277 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00f040 con 0x558210af5580 2023-07-21T22:29:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.457+0000 7fcf3a7fc700 1 -- 172.21.15.107:0/539779277 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf4c010040 con 0x558210af5580 2023-07-21T22:29:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.457+0000 7fcf3a7fc700 1 -- 172.21.15.107:0/539779277 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c038460 con 0x558210af5580 2023-07-21T22:29:49.507 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.458+0000 7fcf65a6db80 1 -- 172.21.15.107:0/539779277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af5580 msgr2=0x558210af84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.458+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/539779277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af5580 0x558210af84d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00a7a0 tx=0x7fcf4c00aab0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.458+0000 7fcf65a6db80 1 -- 172.21.15.107:0/539779277 shutdown_connections 2023-07-21T22:29:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.458+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/539779277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558210af5580 0x558210af84d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.458+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/539779277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5582109506b0 0x558210afc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.458+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/539779277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af4830 0x558210b04e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.458+0000 7fcf65a6db80 1 -- 172.21.15.107:0/539779277 >> 172.21.15.107:0/539779277 conn(0x558210ae9ae0 msgr2=0x558210afba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.458+0000 7fcf65a6db80 1 -- 172.21.15.107:0/539779277 shutdown_connections 2023-07-21T22:29:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.458+0000 7fcf65a6db80 1 -- 172.21.15.107:0/539779277 wait complete. 2023-07-21T22:29:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.458+0000 7fcf65a6db80 1 -- start start 2023-07-21T22:29:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf65a6db80 1 Processor -- start 2023-07-21T22:29:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf65a6db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582109506b0 0x558210b008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf65a6db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558210af4830 0x558210b00e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf65a6db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af5580 0x558210b03e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf65a6db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210b04380 con 0x558210af4830 2023-07-21T22:29:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf65a6db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210b044f0 con 0x5582109506b0 2023-07-21T22:29:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf65a6db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558210b046b0 con 0x558210af5580 2023-07-21T22:29:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf65a6db80 10 client.0 _ll_register_callbacks cb 0x7fcf4400f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af5580 0x558210b03e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf527fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558210af4830 0x558210b00e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af5580 0x558210b03e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37286/0 (socket says 172.21.15.107:37286) 2023-07-21T22:29:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf537fe700 1 -- 172.21.15.107:0/3771371635 learned_addr learned my addr 172.21.15.107:0/3771371635 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:29:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf537fe700 1 -- 172.21.15.107:0/3771371635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582109506b0 msgr2=0x558210b008e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:29:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf537fe700 1 --2- 172.21.15.107:0/3771371635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582109506b0 0x558210b008e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf537fe700 1 -- 172.21.15.107:0/3771371635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558210af4830 msgr2=0x558210b00e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.459+0000 7fcf537fe700 1 --2- 172.21.15.107:0/3771371635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558210af4830 0x558210b00e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.460+0000 7fcf537fe700 1 -- 172.21.15.107:0/3771371635 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4c00a450 con 0x558210af5580 2023-07-21T22:29:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.460+0000 7fcf537fe700 1 --2- 172.21.15.107:0/3771371635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af5580 0x558210b03e40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c008010 tx=0x7fcf4c005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f20fb8ba84169fd7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.461+0000 7fcf32ffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:29:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.461+0000 7fcf32ffd700 1 -- 172.21.15.107:0/3771371635 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00f040 con 0x558210af5580 2023-07-21T22:29:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.461+0000 7fcf32ffd700 1 -- 172.21.15.107:0/3771371635 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcf4c003c10 con 0x558210af5580 2023-07-21T22:29:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.461+0000 7fcf65a6db80 1 -- 172.21.15.107:0/3771371635 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf480a1700 con 0x558210af5580 2023-07-21T22:29:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.461+0000 7fcf32ffd700 1 -- 172.21.15.107:0/3771371635 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf4c003dd0 con 0x558210af5580 2023-07-21T22:29:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.461+0000 7fcf65a6db80 10 client.25291 fetch_fsmap learned FSMap version 28 2023-07-21T22:29:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.461+0000 7fcf65a6db80 1 -- 172.21.15.107:0/3771371635 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558210872250 con 0x558210af5580 2023-07-21T22:29:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.461+0000 7fcf32ffd700 1 -- 172.21.15.107:0/3771371635 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf480a1700 con 0x558210af5580 2023-07-21T22:29:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.461+0000 7fcf32ffd700 1 -- 172.21.15.107:0/3771371635 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fcf4c048020 con 0x558210af5580 2023-07-21T22:29:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.462+0000 7fcf65a6db80 10 client.25291 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:29:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.462+0000 7fcf65a6db80 10 client.25291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.462+0000 7fcf65a6db80 20 client.25291 populate_metadata read hostname 'smithi107' 2023-07-21T22:29:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.462+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/3771371635 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558210b09f70 0x558210b0c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.462+0000 7fcf65a6db80 4 client.25291 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:29:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.462+0000 7fcf65a6db80 1 -- 172.21.15.107:0/3771371635 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558210a00ff0 con 0x558210b09f70 2023-07-21T22:29:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.462+0000 7fcf52ffd700 1 --2- 172.21.15.107:0/3771371635 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558210b09f70 0x558210b0c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.462+0000 7fcf32ffd700 10 client.25291 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.462+0000 7fcf52ffd700 1 --2- 172.21.15.107:0/3771371635 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558210b09f70 0x558210b0c380 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17987024537371e8 server_cookie=757f40d414f6fa0c in_seq=0 out_seq=0 2023-07-21T22:29:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf32ffd700 1 -- 172.21.15.107:0/3771371635 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558210a00ff0 con 0x558210b09f70 2023-07-21T22:29:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf32ffd700 10 client.25291 handle_command_reply: tid=0 2023-07-21T22:29:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf65a6db80 1 client.25291 shutdown 2023-07-21T22:29:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf65a6db80 1 -- 172.21.15.107:0/3771371635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af5580 msgr2=0x558210b03e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/3771371635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af5580 0x558210b03e40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c008010 tx=0x7fcf4c005b90 comp rx=0 tx=0).stop 2023-07-21T22:29:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf65a6db80 1 -- 172.21.15.107:0/3771371635 shutdown_connections 2023-07-21T22:29:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/3771371635 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558210b09f70 0x558210b0c380 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/3771371635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558210af5580 0x558210b03e40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/3771371635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5582109506b0 0x558210b008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf65a6db80 1 --2- 172.21.15.107:0/3771371635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558210af4830 0x558210b00e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf65a6db80 1 -- 172.21.15.107:0/3771371635 >> 172.21.15.107:0/3771371635 conn(0x558210ae9ae0 msgr2=0x558210afc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf32ffd700 0 client.25291 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:29:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf65a6db80 1 -- 172.21.15.107:0/3771371635 shutdown_connections 2023-07-21T22:29:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.466+0000 7fcf65a6db80 1 -- 172.21.15.107:0/3771371635 wait complete. 2023-07-21T22:29:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.467+0000 7fcf65a6db80 20 client.25291 trim_cache size 0 max 16384 2023-07-21T22:29:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:29:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:29:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:29:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:29:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.469+0000 7fcf598e8700 1 -- 172.21.15.107:0/808467543 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcf3c0a11d0 msgr2=0x7fcf3c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.469+0000 7fcf598e8700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcf3c0a11d0 0x7fcf3c0a3680 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400b2e0 tx=0x7fcf44005960 comp rx=0 tx=0).stop 2023-07-21T22:29:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.469+0000 7fcf598e8700 1 -- 172.21.15.107:0/808467543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf540fea20 msgr2=0x7fcf54193700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.469+0000 7fcf598e8700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf540fea20 0x7fcf54193700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcf480096d0 tx=0x7fcf480099e0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.472+0000 7fcf598e8700 1 -- 172.21.15.107:0/808467543 shutdown_connections 2023-07-21T22:29:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.472+0000 7fcf598e8700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf541005a0 0x7fcf54194b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.472+0000 7fcf598e8700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf540fea20 0x7fcf54193700 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.472+0000 7fcf598e8700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fcf3c0a11d0 0x7fcf3c0a3680 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.472+0000 7fcf598e8700 1 --2- 172.21.15.107:0/808467543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf540ffc10 0x7fcf54193c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.472+0000 7fcf598e8700 1 -- 172.21.15.107:0/808467543 >> 172.21.15.107:0/808467543 conn(0x7fcf540fa020 msgr2=0x7fcf540fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.477+0000 7fcf598e8700 1 -- 172.21.15.107:0/808467543 shutdown_connections 2023-07-21T22:29:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:29:49.477+0000 7fcf598e8700 1 -- 172.21.15.107:0/808467543 wait complete. 2023-07-21T22:29:49.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:49 smithi190 ceph-mon[98334]: pgmap v315: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-07-21T22:29:49.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:49 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:49.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:49 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:49.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:49 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:49.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:49 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:49.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:49 smithi132 ceph-mon[97863]: pgmap v315: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-07-21T22:29:49.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:49 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:49.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:49 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:49.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:49 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:49.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:49 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:49.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:49 smithi107 ceph-mon[96543]: pgmap v315: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-07-21T22:29:49.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:49 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:49.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:49 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:49.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:49 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:49.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:49 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:50.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:50.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:50.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:29:50.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:29:50.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:50.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:50.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:50.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:29:50.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:29:50.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:50.691 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:50.691 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:50.691 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:29:50.691 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:29:50.692 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:29:51.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:51 smithi190 ceph-mon[98334]: pgmap v316: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 10 op/s 2023-07-21T22:29:51.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:51 smithi190 ceph-mon[98334]: scrub summary: idle 2023-07-21T22:29:51.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:51 smithi132 ceph-mon[97863]: pgmap v316: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 10 op/s 2023-07-21T22:29:51.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:51 smithi132 ceph-mon[97863]: scrub summary: idle 2023-07-21T22:29:51.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:51 smithi107 ceph-mon[96543]: pgmap v316: 97 pgs: 97 active+clean; 674 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 10 op/s 2023-07-21T22:29:51.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:51 smithi107 ceph-mon[96543]: scrub summary: idle 2023-07-21T22:29:52.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:52 smithi190 ceph-mon[98334]: scrub summary: idle 2023-07-21T22:29:52.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:52 smithi132 ceph-mon[97863]: scrub summary: idle 2023-07-21T22:29:52.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:52 smithi107 ceph-mon[96543]: scrub summary: idle 2023-07-21T22:29:53.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:53 smithi190 ceph-mon[98334]: pgmap v317: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-07-21T22:29:53.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:53 smithi132 ceph-mon[97863]: pgmap v317: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-07-21T22:29:53.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:53 smithi107 ceph-mon[96543]: pgmap v317: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-07-21T22:29:55.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:55 smithi190 ceph-mon[98334]: pgmap v318: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 10 op/s 2023-07-21T22:29:55.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:55 smithi132 ceph-mon[97863]: pgmap v318: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 10 op/s 2023-07-21T22:29:55.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:55 smithi107 ceph-mon[96543]: pgmap v318: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 10 op/s 2023-07-21T22:29:57.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:57 smithi190 ceph-mon[98334]: pgmap v319: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 10 op/s 2023-07-21T22:29:57.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:57 smithi132 ceph-mon[97863]: pgmap v319: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 10 op/s 2023-07-21T22:29:57.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:57 smithi107 ceph-mon[96543]: pgmap v319: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 10 op/s 2023-07-21T22:29:59.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:29:59 smithi190 ceph-mon[98334]: pgmap v320: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 597 B/s wr, 12 op/s 2023-07-21T22:29:59.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:29:59 smithi132 ceph-mon[97863]: pgmap v320: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 597 B/s wr, 12 op/s 2023-07-21T22:29:59.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:29:59 smithi107 ceph-mon[96543]: pgmap v320: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 597 B/s wr, 12 op/s 2023-07-21T22:30:00.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:00 smithi190 ceph-mon[98334]: overall HEALTH_OK 2023-07-21T22:30:00.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:00 smithi132 ceph-mon[97863]: overall HEALTH_OK 2023-07-21T22:30:00.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:00 smithi107 ceph-mon[96543]: overall HEALTH_OK 2023-07-21T22:30:01.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:01 smithi190 ceph-mon[98334]: pgmap v321: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-07-21T22:30:01.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:01 smithi132 ceph-mon[97863]: pgmap v321: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-07-21T22:30:01.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:01 smithi107 ceph-mon[96543]: pgmap v321: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-07-21T22:30:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.603+0000 7fa9c3ab2700 1 -- 172.21.15.107:0/3875015125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9bc0feaa0 msgr2=0x7fa9bc0feeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.603+0000 7fa9c3ab2700 1 --2- 172.21.15.107:0/3875015125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9bc0feaa0 0x7fa9bc0feeb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009990 tx=0x7fa9ac009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.603+0000 7fa9c3ab2700 1 -- 172.21.15.107:0/3875015125 shutdown_connections 2023-07-21T22:30:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.603+0000 7fa9c3ab2700 1 --2- 172.21.15.107:0/3875015125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9bc100620 0x7fa9bc106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.603+0000 7fa9c3ab2700 1 --2- 172.21.15.107:0/3875015125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9bc0feaa0 0x7fa9bc0feeb0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.603+0000 7fa9c3ab2700 1 --2- 172.21.15.107:0/3875015125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9bc0ffc90 0x7fa9bc1000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.603+0000 7fa9c3ab2700 1 -- 172.21.15.107:0/3875015125 >> 172.21.15.107:0/3875015125 conn(0x7fa9bc0fa0c0 msgr2=0x7fa9bc0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.603+0000 7fa9c3ab2700 1 -- 172.21.15.107:0/3875015125 shutdown_connections 2023-07-21T22:30:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.603+0000 7fa9c3ab2700 1 -- 172.21.15.107:0/3875015125 wait complete. 2023-07-21T22:30:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c3ab2700 1 Processor -- start 2023-07-21T22:30:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c3ab2700 1 -- start start 2023-07-21T22:30:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c3ab2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9bc0feaa0 0x7fa9bc194840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c3ab2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9bc0ffc90 0x7fa9bc194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c3ab2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9bc100620 0x7fa9bc195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c3ab2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc196260 con 0x7fa9bc100620 2023-07-21T22:30:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c3ab2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc1963d0 con 0x7fa9bc0feaa0 2023-07-21T22:30:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c3ab2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc196590 con 0x7fa9bc0ffc90 2023-07-21T22:30:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c204f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9bc100620 0x7fa9bc195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c204f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9bc100620 0x7fa9bc195c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48742/0 (socket says 172.21.15.107:48742) 2023-07-21T22:30:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c204f700 1 -- 172.21.15.107:0/3655386801 learned_addr learned my addr 172.21.15.107:0/3655386801 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c104d700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9bc0ffc90 0x7fa9bc194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c184e700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9bc0feaa0 0x7fa9bc194840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c204f700 1 -- 172.21.15.107:0/3655386801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9bc0ffc90 msgr2=0x7fa9bc194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c204f700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9bc0ffc90 0x7fa9bc194d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c204f700 1 -- 172.21.15.107:0/3655386801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9bc0feaa0 msgr2=0x7fa9bc194840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c204f700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9bc0feaa0 0x7fa9bc194840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c204f700 1 -- 172.21.15.107:0/3655386801 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac0095f0 con 0x7fa9bc100620 2023-07-21T22:30:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.604+0000 7fa9c204f700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9bc100620 0x7fa9bc195c00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4007f10 tx=0x7fa9b400f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.605+0000 7fa9b2ffd700 1 -- 172.21.15.107:0/3655386801 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b400fd30 con 0x7fa9bc100620 2023-07-21T22:30:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.605+0000 7fa9b2ffd700 1 -- 172.21.15.107:0/3655386801 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9b400fe90 con 0x7fa9bc100620 2023-07-21T22:30:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.605+0000 7fa9c3ab2700 1 -- 172.21.15.107:0/3655386801 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9bc196870 con 0x7fa9bc100620 2023-07-21T22:30:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.605+0000 7fa9b2ffd700 1 -- 172.21.15.107:0/3655386801 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4005020 con 0x7fa9bc100620 2023-07-21T22:30:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.605+0000 7fa9c3ab2700 1 -- 172.21.15.107:0/3655386801 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9bc068a30 con 0x7fa9bc100620 2023-07-21T22:30:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.607+0000 7fa9b2ffd700 1 -- 172.21.15.107:0/3655386801 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa9b4012460 con 0x7fa9bc100620 2023-07-21T22:30:02.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.608+0000 7fa9b2ffd700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa9a80a1150 0x7fa9a80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.608+0000 7fa9c184e700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa9a80a1150 0x7fa9a80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.608+0000 7fa9b2ffd700 1 -- 172.21.15.107:0/3655386801 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa9b40ceb80 con 0x7fa9bc100620 2023-07-21T22:30:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.608+0000 7fa9c184e700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa9a80a1150 0x7fa9a80a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00b5c0 tx=0x7fa9ac005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9cfc37b80 1 Processor -- start 2023-07-21T22:30:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9cfc37b80 1 -- start start 2023-07-21T22:30:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9cfc37b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 0x5633387b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9cfc37b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 0x5633387b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9cfc37b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 0x5633387b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9cfc37b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633386b5c50 con 0x56333860c6b0 2023-07-21T22:30:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9cfc37b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633387089e0 con 0x5633387b4ba0 2023-07-21T22:30:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9cfc37b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563338701e70 con 0x5633387b5a60 2023-07-21T22:30:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9c184e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 0x5633387b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9c184e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 0x5633387b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48746/0 (socket says 172.21.15.107:48746) 2023-07-21T22:30:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9c184e700 1 -- 172.21.15.107:0/2542486856 learned_addr learned my addr 172.21.15.107:0/2542486856 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9c204f700 1 --2- 172.21.15.107:0/2542486856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 0x5633387b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9c104d700 1 --2- 172.21.15.107:0/2542486856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 0x5633387b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9c184e700 1 -- 172.21.15.107:0/2542486856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 msgr2=0x5633387b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9c184e700 1 --2- 172.21.15.107:0/2542486856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 0x5633387b5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9c184e700 1 -- 172.21.15.107:0/2542486856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 msgr2=0x5633387b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.609+0000 7fa9c184e700 1 --2- 172.21.15.107:0/2542486856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 0x5633387b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.610+0000 7fa9c184e700 1 -- 172.21.15.107:0/2542486856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56333852fd80 con 0x56333860c6b0 2023-07-21T22:30:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.610+0000 7fa9c184e700 1 --2- 172.21.15.107:0/2542486856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 0x5633387b4660 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac015000 tx=0x7fa9ac020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b4f1129faacdbf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.610+0000 7fa9a7fff700 1 -- 172.21.15.107:0/2542486856 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00be00 con 0x56333860c6b0 2023-07-21T22:30:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.610+0000 7fa9a7fff700 1 -- 172.21.15.107:0/2542486856 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9ac01e040 con 0x56333860c6b0 2023-07-21T22:30:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.610+0000 7fa9a7fff700 1 -- 172.21.15.107:0/2542486856 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac004780 con 0x56333860c6b0 2023-07-21T22:30:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.610+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/2542486856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 msgr2=0x5633387b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.611+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/2542486856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 0x5633387b4660 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac015000 tx=0x7fa9ac020000 comp rx=0 tx=0).stop 2023-07-21T22:30:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.611+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/2542486856 shutdown_connections 2023-07-21T22:30:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.611+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/2542486856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 0x5633387b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.611+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/2542486856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 0x5633387b4660 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.611+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/2542486856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 0x5633387b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.611+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/2542486856 >> 172.21.15.107:0/2542486856 conn(0x5633387a5ae0 msgr2=0x5633387aac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.611+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/2542486856 shutdown_connections 2023-07-21T22:30:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.611+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/2542486856 wait complete. 2023-07-21T22:30:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.611+0000 7fa9cfc37b80 1 -- start start 2023-07-21T22:30:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.611+0000 7fa9cfc37b80 1 Processor -- start 2023-07-21T22:30:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9cfc37b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56333860c6b0 0x5633387bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9cfc37b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b4ba0 0x5633387be0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9cfc37b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 0x5633387c5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9cfc37b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633386a1a60 con 0x5633387b4ba0 2023-07-21T22:30:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9cfc37b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633386a1760 con 0x56333860c6b0 2023-07-21T22:30:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9cfc37b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633385858c0 con 0x5633387b5a60 2023-07-21T22:30:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9cfc37b80 10 client.0 _ll_register_callbacks cb 0x56333869f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9c104d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b4ba0 0x5633387be0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9c104d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b4ba0 0x5633387be0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48754/0 (socket says 172.21.15.107:48754) 2023-07-21T22:30:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9c104d700 1 -- 172.21.15.107:0/3110552383 learned_addr learned my addr 172.21.15.107:0/3110552383 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9c204f700 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 0x5633387c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9c184e700 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56333860c6b0 0x5633387bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9c104d700 1 -- 172.21.15.107:0/3110552383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 msgr2=0x5633387c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9c104d700 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 0x5633387c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9c104d700 1 -- 172.21.15.107:0/3110552383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56333860c6b0 msgr2=0x5633387bdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9c104d700 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56333860c6b0 0x5633387bdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9c104d700 1 -- 172.21.15.107:0/3110552383 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9bc107810 con 0x5633387b4ba0 2023-07-21T22:30:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9c184e700 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56333860c6b0 0x5633387bdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:30:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9c104d700 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b4ba0 0x5633387be0c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8012860 tx=0x7fa9b8012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f18340bbfa2458d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9a57fa700 1 -- 172.21.15.107:0/3110552383 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b800a7d0 con 0x5633387b4ba0 2023-07-21T22:30:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9a57fa700 1 -- 172.21.15.107:0/3110552383 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa9b8010880 con 0x5633387b4ba0 2023-07-21T22:30:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.612+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3110552383 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56333865b490 con 0x5633387b4ba0 2023-07-21T22:30:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.613+0000 7fa9a57fa700 1 -- 172.21.15.107:0/3110552383 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9b8010a40 con 0x5633387b4ba0 2023-07-21T22:30:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.613+0000 7fa9cfc37b80 10 client.15714 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.613+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3110552383 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563338518c00 con 0x5633387b4ba0 2023-07-21T22:30:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.613+0000 7fa9a57fa700 1 -- 172.21.15.107:0/3110552383 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9b8029080 con 0x5633387b4ba0 2023-07-21T22:30:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.613+0000 7fa9a57fa700 1 -- 172.21.15.107:0/3110552383 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa9b8023030 con 0x5633387b4ba0 2023-07-21T22:30:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.613+0000 7fa9cfc37b80 10 client.15714 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.613+0000 7fa9cfc37b80 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.613+0000 7fa9cfc37b80 20 client.15714 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.613+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5633387c9a30 0x5633387cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.613+0000 7fa9cfc37b80 4 client.15714 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:02.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.613+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3110552383 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633386fae50 con 0x5633387c9a30 2023-07-21T22:30:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.613+0000 7fa9c184e700 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5633387c9a30 0x5633387cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.614+0000 7fa9a57fa700 10 client.15714 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.614+0000 7fa9c184e700 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5633387c9a30 0x5633387cbe40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc8375dd43543fae server_cookie=ba8b9f075ebf9b35 in_seq=0 out_seq=0 2023-07-21T22:30:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.614+0000 7fa9a57fa700 1 -- 172.21.15.107:0/3110552383 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5633386fae50 con 0x5633387c9a30 2023-07-21T22:30:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.614+0000 7fa9a57fa700 10 client.15714 handle_command_reply: tid=0 2023-07-21T22:30:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.614+0000 7fa9cfc37b80 1 client.15714 shutdown 2023-07-21T22:30:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.614+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3110552383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b4ba0 msgr2=0x5633387be0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.614+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b4ba0 0x5633387be0c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8012860 tx=0x7fa9b8012b70 comp rx=0 tx=0).stop 2023-07-21T22:30:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.615+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3110552383 shutdown_connections 2023-07-21T22:30:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.615+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5633387c9a30 0x5633387cbe40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.615+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 0x5633387c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.615+0000 7fa9a57fa700 0 client.15714 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.615+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56333860c6b0 0x5633387bdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.615+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3110552383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b4ba0 0x5633387be0c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.615+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3110552383 >> 172.21.15.107:0/3110552383 conn(0x5633387a5ae0 msgr2=0x5633387aa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.615+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3110552383 shutdown_connections 2023-07-21T22:30:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.615+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3110552383 wait complete. 2023-07-21T22:30:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.615+0000 7fa9cfc37b80 20 client.15714 trim_cache size 0 max 16384 2023-07-21T22:30:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.629+0000 7fa9cfc37b80 1 Processor -- start 2023-07-21T22:30:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.629+0000 7fa9cfc37b80 1 -- start start 2023-07-21T22:30:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.629+0000 7fa9cfc37b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 0x5633387d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.629+0000 7fa9cfc37b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 0x5633387c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.629+0000 7fa9cfc37b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 0x5633387aab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.629+0000 7fa9cfc37b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633387b5000 con 0x56333860c6b0 2023-07-21T22:30:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.629+0000 7fa9cfc37b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633387be830 con 0x5633387b4ba0 2023-07-21T22:30:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.629+0000 7fa9cfc37b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633387c6dc0 con 0x5633387b5a60 2023-07-21T22:30:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.629+0000 7fa9c184e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 0x5633387d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.629+0000 7fa9c184e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 0x5633387d0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48760/0 (socket says 172.21.15.107:48760) 2023-07-21T22:30:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.629+0000 7fa9c184e700 1 -- 172.21.15.107:0/3975454486 learned_addr learned my addr 172.21.15.107:0/3975454486 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.630+0000 7fa9c104d700 1 --2- 172.21.15.107:0/3975454486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 0x5633387c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.630+0000 7fa9c184e700 1 -- 172.21.15.107:0/3975454486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 msgr2=0x5633387aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.630+0000 7fa9c204f700 1 --2- 172.21.15.107:0/3975454486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 0x5633387aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.630+0000 7fa9c184e700 1 --2- 172.21.15.107:0/3975454486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 0x5633387aab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.630+0000 7fa9c184e700 1 -- 172.21.15.107:0/3975454486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 msgr2=0x5633387c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.630+0000 7fa9c184e700 1 --2- 172.21.15.107:0/3975454486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 0x5633387c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.630+0000 7fa9c184e700 1 -- 172.21.15.107:0/3975454486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633385561f0 con 0x56333860c6b0 2023-07-21T22:30:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.630+0000 7fa9c104d700 1 --2- 172.21.15.107:0/3975454486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 0x5633387c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:30:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.630+0000 7fa9c184e700 1 --2- 172.21.15.107:0/3975454486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 0x5633387d0780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009ce0 tx=0x7fa9ac0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b3aee3ac704868 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.630+0000 7fa9b0dac700 1 -- 172.21.15.107:0/3975454486 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac0059d0 con 0x56333860c6b0 2023-07-21T22:30:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.630+0000 7fa9b0dac700 1 -- 172.21.15.107:0/3975454486 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9ac00bda0 con 0x56333860c6b0 2023-07-21T22:30:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.630+0000 7fa9b0dac700 1 -- 172.21.15.107:0/3975454486 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac003b80 con 0x56333860c6b0 2023-07-21T22:30:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.631+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3975454486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 msgr2=0x5633387d0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.631+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3975454486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 0x5633387d0780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009ce0 tx=0x7fa9ac0092d0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.631+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3975454486 shutdown_connections 2023-07-21T22:30:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.631+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3975454486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633387b5a60 0x5633387aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.631+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3975454486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333860c6b0 0x5633387d0780 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.631+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3975454486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 0x5633387c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.631+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3975454486 >> 172.21.15.107:0/3975454486 conn(0x5633387a5ae0 msgr2=0x5633387bc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.631+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3975454486 shutdown_connections 2023-07-21T22:30:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.631+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3975454486 wait complete. 2023-07-21T22:30:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.631+0000 7fa9cfc37b80 1 -- start start 2023-07-21T22:30:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.631+0000 7fa9cfc37b80 1 Processor -- start 2023-07-21T22:30:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9cfc37b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56333860c6b0 0x5633387c3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9cfc37b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 0x5633387c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9cfc37b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b5a60 0x5633387c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9cfc37b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633387c4510 con 0x5633387b5a60 2023-07-21T22:30:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9cfc37b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633387c4680 con 0x5633387b4ba0 2023-07-21T22:30:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9cfc37b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633387c4840 con 0x56333860c6b0 2023-07-21T22:30:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9cfc37b80 10 client.0 _ll_register_callbacks cb 0x7fa9b800acd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9c204f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b5a60 0x5633387c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9c204f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b5a60 0x5633387c3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48770/0 (socket says 172.21.15.107:48770) 2023-07-21T22:30:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9c204f700 1 -- 172.21.15.107:0/3074424312 learned_addr learned my addr 172.21.15.107:0/3074424312 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9c184e700 1 --2- 172.21.15.107:0/3074424312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56333860c6b0 0x5633387c3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9c204f700 1 -- 172.21.15.107:0/3074424312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56333860c6b0 msgr2=0x5633387c3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9c204f700 1 --2- 172.21.15.107:0/3074424312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56333860c6b0 0x5633387c3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9c104d700 1 --2- 172.21.15.107:0/3074424312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 0x5633387c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9c204f700 1 -- 172.21.15.107:0/3074424312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 msgr2=0x5633387c37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9c204f700 1 --2- 172.21.15.107:0/3074424312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 0x5633387c37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9c204f700 1 -- 172.21.15.107:0/3074424312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56333852fd80 con 0x5633387b5a60 2023-07-21T22:30:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9c204f700 1 --2- 172.21.15.107:0/3074424312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b5a60 0x5633387c3dd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa9b409bf00 tx=0x7fa9b40a4440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8287042ed258675 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9a57fa700 1 -- 172.21.15.107:0/3074424312 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b409c3d0 con 0x5633387b5a60 2023-07-21T22:30:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9a57fa700 1 -- 172.21.15.107:0/3074424312 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa9b409a3d0 con 0x5633387b5a60 2023-07-21T22:30:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3074424312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9ac01e040 con 0x5633387b5a60 2023-07-21T22:30:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.632+0000 7fa9a57fa700 1 -- 172.21.15.107:0/3074424312 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9b409b460 con 0x5633387b5a60 2023-07-21T22:30:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.633+0000 7fa9cfc37b80 10 client.15720 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.633+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3074424312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56333852e250 con 0x5633387b5a60 2023-07-21T22:30:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.633+0000 7fa9a57fa700 1 -- 172.21.15.107:0/3074424312 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9ac01e040 con 0x5633387b5a60 2023-07-21T22:30:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.633+0000 7fa9a57fa700 1 -- 172.21.15.107:0/3074424312 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa9b40ba1e0 con 0x5633387b5a60 2023-07-21T22:30:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.633+0000 7fa9cfc37b80 10 client.15720 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.633+0000 7fa9cfc37b80 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.633+0000 7fa9cfc37b80 20 client.15720 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.633+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3074424312 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56333881e4b0 0x5633388208c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.633+0000 7fa9cfc37b80 4 client.15720 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:30:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.633+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3074424312 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5633386bcff0 con 0x56333881e4b0 2023-07-21T22:30:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.633+0000 7fa9c184e700 1 --2- 172.21.15.107:0/3074424312 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56333881e4b0 0x5633388208c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.634+0000 7fa9a57fa700 10 client.15720 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.634+0000 7fa9c184e700 1 --2- 172.21.15.107:0/3074424312 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56333881e4b0 0x5633388208c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca6437824962f1ab server_cookie=3d62718c9e90bc06 in_seq=0 out_seq=0 2023-07-21T22:30:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9a57fa700 1 -- 172.21.15.107:0/3074424312 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5633386bcff0 con 0x56333881e4b0 2023-07-21T22:30:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9a57fa700 10 client.15720 handle_command_reply: tid=0 2023-07-21T22:30:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9cfc37b80 1 client.15720 shutdown 2023-07-21T22:30:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3074424312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b5a60 msgr2=0x5633387c3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3074424312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b5a60 0x5633387c3dd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa9b409bf00 tx=0x7fa9b40a4440 comp rx=0 tx=0).stop 2023-07-21T22:30:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3074424312 shutdown_connections 2023-07-21T22:30:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3074424312 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56333881e4b0 0x5633388208c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3074424312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5633387b5a60 0x5633387c3dd0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9a57fa700 0 client.15720 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3074424312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56333860c6b0 0x5633387c3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9cfc37b80 1 --2- 172.21.15.107:0/3074424312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5633387b4ba0 0x5633387c37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3074424312 >> 172.21.15.107:0/3074424312 conn(0x5633387a5ae0 msgr2=0x5633387d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3074424312 shutdown_connections 2023-07-21T22:30:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9cfc37b80 1 -- 172.21.15.107:0/3074424312 wait complete. 2023-07-21T22:30:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9cfc37b80 20 client.15720 trim_cache size 0 max 16384 2023-07-21T22:30:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:30:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:30:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:30:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:30:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9c3ab2700 1 -- 172.21.15.107:0/3655386801 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa9a80a1150 msgr2=0x7fa9a80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9c3ab2700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa9a80a1150 0x7fa9a80a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00b5c0 tx=0x7fa9ac005960 comp rx=0 tx=0).stop 2023-07-21T22:30:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9c3ab2700 1 -- 172.21.15.107:0/3655386801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9bc100620 msgr2=0x7fa9bc195c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.638+0000 7fa9c3ab2700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9bc100620 0x7fa9bc195c00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4007f10 tx=0x7fa9b400f5c0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.639+0000 7fa9c3ab2700 1 -- 172.21.15.107:0/3655386801 shutdown_connections 2023-07-21T22:30:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.639+0000 7fa9c3ab2700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9bc100620 0x7fa9bc195c00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.639+0000 7fa9c3ab2700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9bc0feaa0 0x7fa9bc194840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.639+0000 7fa9c3ab2700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa9a80a1150 0x7fa9a80a3600 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.639+0000 7fa9c3ab2700 1 --2- 172.21.15.107:0/3655386801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9bc0ffc90 0x7fa9bc194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.639+0000 7fa9c3ab2700 1 -- 172.21.15.107:0/3655386801 >> 172.21.15.107:0/3655386801 conn(0x7fa9bc0fa0c0 msgr2=0x7fa9bc1031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.639+0000 7fa9c3ab2700 1 -- 172.21.15.107:0/3655386801 shutdown_connections 2023-07-21T22:30:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:02.639+0000 7fa9c3ab2700 1 -- 172.21.15.107:0/3655386801 wait complete. 2023-07-21T22:30:03.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:03 smithi190 ceph-mon[98334]: pgmap v322: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2023-07-21T22:30:03.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:03 smithi132 ceph-mon[97863]: pgmap v322: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2023-07-21T22:30:03.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:03 smithi107 ceph-mon[96543]: pgmap v322: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2023-07-21T22:30:05.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:05 smithi190 ceph-mon[98334]: pgmap v323: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:30:05.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:05 smithi132 ceph-mon[97863]: pgmap v323: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:30:05.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:05 smithi107 ceph-mon[96543]: pgmap v323: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-21T22:30:06.786 INFO:tasks.workunit.client.0.smithi107.stderr:Note: switching to 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea'. 2023-07-21T22:30:06.786 INFO:tasks.workunit.client.0.smithi107.stderr: 2023-07-21T22:30:06.786 INFO:tasks.workunit.client.0.smithi107.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-07-21T22:30:06.786 INFO:tasks.workunit.client.0.smithi107.stderr:changes and commit them, and you can discard any commits you make in this 2023-07-21T22:30:06.787 INFO:tasks.workunit.client.0.smithi107.stderr:state without impacting any branches by switching back to a branch. 2023-07-21T22:30:06.787 INFO:tasks.workunit.client.0.smithi107.stderr: 2023-07-21T22:30:06.787 INFO:tasks.workunit.client.0.smithi107.stderr:If you want to create a new branch to retain commits you create, you may 2023-07-21T22:30:06.787 INFO:tasks.workunit.client.0.smithi107.stderr:do so (now or later) by using -c with the switch command. Example: 2023-07-21T22:30:06.787 INFO:tasks.workunit.client.0.smithi107.stderr: 2023-07-21T22:30:06.787 INFO:tasks.workunit.client.0.smithi107.stderr: git switch -c 2023-07-21T22:30:06.788 INFO:tasks.workunit.client.0.smithi107.stderr: 2023-07-21T22:30:06.788 INFO:tasks.workunit.client.0.smithi107.stderr:Or undo this operation with: 2023-07-21T22:30:06.788 INFO:tasks.workunit.client.0.smithi107.stderr: 2023-07-21T22:30:06.788 INFO:tasks.workunit.client.0.smithi107.stderr: git switch - 2023-07-21T22:30:06.788 INFO:tasks.workunit.client.0.smithi107.stderr: 2023-07-21T22:30:06.788 INFO:tasks.workunit.client.0.smithi107.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-07-21T22:30:06.788 INFO:tasks.workunit.client.0.smithi107.stderr: 2023-07-21T22:30:06.789 INFO:tasks.workunit.client.0.smithi107.stderr:HEAD is now at bffe049d41f temp: trigger builds on shaman 2023-07-21T22:30:06.791 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-07-21T22:30:06.821 INFO:tasks.workunit.client.0.smithi107.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-07-21T22:30:06.823 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-21T22:30:06.823 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-07-21T22:30:07.361 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-07-21T22:30:07.424 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-07-21T22:30:07.460 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-21T22:30:07.461 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-21T22:30:07.462 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-07-21T22:30:07.497 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-21T22:30:07.500 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:30:07.501 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-07-21T22:30:07.557 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2023-07-21T22:30:07.558 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2023-07-21T22:30:07.559 DEBUG:teuthology.orchestra.run.smithi107:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2023-07-21T22:30:07.624 INFO:tasks.workunit.client.0.smithi107.stdout:getting blogbench 2023-07-21T22:30:07.624 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'getting blogbench' 2023-07-21T22:30:07.624 INFO:tasks.workunit.client.0.smithi107.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-07-21T22:30:07.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:07 smithi190 ceph-mon[98334]: pgmap v324: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2023-07-21T22:30:07.655 INFO:tasks.workunit.client.0.smithi107.stderr:--2023-07-21 22:30:07-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-07-21T22:30:07.656 INFO:tasks.workunit.client.0.smithi107.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2023-07-21T22:30:07.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:07 smithi132 ceph-mon[97863]: pgmap v324: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2023-07-21T22:30:07.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:07 smithi107 ceph-mon[96543]: pgmap v324: 97 pgs: 97 active+clean; 678 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2023-07-21T22:30:07.746 INFO:tasks.workunit.client.0.smithi107.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2023-07-21T22:30:07.838 INFO:tasks.workunit.client.0.smithi107.stderr:HTTP request sent, awaiting response... 200 OK 2023-07-21T22:30:07.839 INFO:tasks.workunit.client.0.smithi107.stderr:Length: 103355 (101K) [application/octet-stream] 2023-07-21T22:30:07.840 INFO:tasks.workunit.client.0.smithi107.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2023-07-21T22:30:07.842 INFO:tasks.workunit.client.0.smithi107.stderr: 2023-07-21T22:30:08.013 INFO:tasks.workunit.client.0.smithi107.stderr: 0K .......... .......... .......... .......... .......... 49% 291K 0s 2023-07-21T22:30:08.098 INFO:tasks.workunit.client.0.smithi107.stderr: 50K .......... .......... .......... .......... .......... 99% 579K 0s 2023-07-21T22:30:08.099 INFO:tasks.workunit.client.0.smithi107.stderr: 100K 100% 1779G=0.3s 2023-07-21T22:30:08.099 INFO:tasks.workunit.client.0.smithi107.stderr: 2023-07-21T22:30:08.099 INFO:tasks.workunit.client.0.smithi107.stderr:2023-07-21 22:30:08 (391 KB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2023-07-21T22:30:08.100 INFO:tasks.workunit.client.0.smithi107.stderr: 2023-07-21T22:30:08.100 INFO:tasks.workunit.client.0.smithi107.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2023-07-21T22:30:08.134 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/ 2023-07-21T22:30:08.135 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/m4/ 2023-07-21T22:30:08.136 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/m4/acx_pthread.m4 2023-07-21T22:30:08.137 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/man/ 2023-07-21T22:30:08.138 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/man/blogbench.8 2023-07-21T22:30:08.139 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/man/Makefile.am 2023-07-21T22:30:08.140 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/man/Makefile.in 2023-07-21T22:30:08.140 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/ 2023-07-21T22:30:08.141 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/mysnprintf.c 2023-07-21T22:30:08.145 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/mysnprintf.h 2023-07-21T22:30:08.145 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/blogbench.c 2023-07-21T22:30:08.146 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/blogbench.h 2023-07-21T22:30:08.146 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/fakesnprintf.c 2023-07-21T22:30:08.146 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/fakesnprintf.h 2023-07-21T22:30:08.146 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/blogbench_p.h 2023-07-21T22:30:08.147 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/bsd-getopt_long.c 2023-07-21T22:30:08.147 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/bsd-getopt_long.h 2023-07-21T22:30:08.147 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/Makefile.am 2023-07-21T22:30:08.147 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/Makefile.in 2023-07-21T22:30:08.147 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/process.c 2023-07-21T22:30:08.147 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/globals.h 2023-07-21T22:30:08.147 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/commenter.c 2023-07-21T22:30:08.148 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/rewriter.c 2023-07-21T22:30:08.148 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/reader.c 2023-07-21T22:30:08.148 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/writer.c 2023-07-21T22:30:08.148 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/helpers.c 2023-07-21T22:30:08.148 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/NEWS 2023-07-21T22:30:08.154 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/compile 2023-07-21T22:30:08.154 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/depcomp 2023-07-21T22:30:08.155 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/aclocal.m4 2023-07-21T22:30:08.155 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/README 2023-07-21T22:30:08.155 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/configure 2023-07-21T22:30:08.157 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/configure.ac 2023-07-21T22:30:08.158 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/config.guess 2023-07-21T22:30:08.159 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/install-sh 2023-07-21T22:30:08.160 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/config.sub 2023-07-21T22:30:08.160 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/missing 2023-07-21T22:30:08.160 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/mkinstalldirs 2023-07-21T22:30:08.160 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/Makefile.am 2023-07-21T22:30:08.161 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/Makefile.in 2023-07-21T22:30:08.161 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/config.h.in 2023-07-21T22:30:08.161 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/AUTHORS 2023-07-21T22:30:08.162 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/INSTALL 2023-07-21T22:30:08.162 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/ChangeLog 2023-07-21T22:30:08.162 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/COPYING 2023-07-21T22:30:08.162 INFO:tasks.workunit.client.0.smithi107.stderr:+ cd blogbench-1.0/ 2023-07-21T22:30:08.163 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'making blogbench' 2023-07-21T22:30:08.163 INFO:tasks.workunit.client.0.smithi107.stdout:making blogbench 2023-07-21T22:30:08.163 INFO:tasks.workunit.client.0.smithi107.stderr:+ ./configure 2023-07-21T22:30:08.261 INFO:tasks.workunit.client.0.smithi107.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-07-21T22:30:09.267 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether build environment is sane... yes 2023-07-21T22:30:09.275 INFO:tasks.workunit.client.0.smithi107.stdout:checking for gawk... gawk 2023-07-21T22:30:09.302 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether make sets $(MAKE)... yes 2023-07-21T22:30:09.317 INFO:tasks.workunit.client.0.smithi107.stdout:checking for gcc... gcc 2023-07-21T22:30:09.369 INFO:tasks.workunit.client.0.smithi107.stdout:checking for C compiler default output file name... a.out 2023-07-21T22:30:09.371 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether the C compiler works... yes 2023-07-21T22:30:09.374 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether we are cross compiling... no 2023-07-21T22:30:09.410 INFO:tasks.workunit.client.0.smithi107.stdout:checking for suffix of executables... 2023-07-21T22:30:09.444 INFO:tasks.workunit.client.0.smithi107.stdout:checking for suffix of object files... o 2023-07-21T22:30:09.472 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether we are using the GNU C compiler... yes 2023-07-21T22:30:09.499 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether gcc accepts -g... yes 2023-07-21T22:30:09.536 INFO:tasks.workunit.client.0.smithi107.stdout:checking for gcc option to accept ANSI C... none needed 2023-07-21T22:30:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:checking for style of include used by make... GNU 2023-07-21T22:30:09.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:09 smithi190 ceph-mon[98334]: pgmap v325: 97 pgs: 97 active+clean; 682 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2023-07-21T22:30:09.647 INFO:tasks.workunit.client.0.smithi107.stdout:checking dependency style of gcc... gcc3 2023-07-21T22:30:09.647 INFO:tasks.workunit.client.0.smithi107.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-07-21T22:30:09.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:09 smithi132 ceph-mon[97863]: pgmap v325: 97 pgs: 97 active+clean; 682 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2023-07-21T22:30:09.684 INFO:tasks.workunit.client.0.smithi107.stdout:checking how to run the C preprocessor... gcc -E 2023-07-21T22:30:09.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:09 smithi107 ceph-mon[96543]: pgmap v325: 97 pgs: 97 active+clean; 682 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2023-07-21T22:30:09.712 INFO:tasks.workunit.client.0.smithi107.stdout:checking for egrep... grep -E 2023-07-21T22:30:09.718 INFO:tasks.workunit.client.0.smithi107.stdout:checking for AIX... no 2023-07-21T22:30:09.774 INFO:tasks.workunit.client.0.smithi107.stdout:checking for library containing strerror... none required 2023-07-21T22:30:09.906 INFO:tasks.workunit.client.0.smithi107.stdout:checking for ANSI C header files... yes 2023-07-21T22:30:09.938 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-07-21T22:30:09.979 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/types.h... yes 2023-07-21T22:30:10.020 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/stat.h... yes 2023-07-21T22:30:10.062 INFO:tasks.workunit.client.0.smithi107.stdout:checking for stdlib.h... yes 2023-07-21T22:30:10.104 INFO:tasks.workunit.client.0.smithi107.stdout:checking for string.h... yes 2023-07-21T22:30:10.146 INFO:tasks.workunit.client.0.smithi107.stdout:checking for memory.h... yes 2023-07-21T22:30:10.189 INFO:tasks.workunit.client.0.smithi107.stdout:checking for strings.h... yes 2023-07-21T22:30:10.231 INFO:tasks.workunit.client.0.smithi107.stdout:checking for inttypes.h... yes 2023-07-21T22:30:10.287 INFO:tasks.workunit.client.0.smithi107.stdout:checking for stdint.h... yes 2023-07-21T22:30:10.378 INFO:tasks.workunit.client.0.smithi107.stdout:checking for unistd.h... yes 2023-07-21T22:30:10.391 INFO:tasks.workunit.client.0.smithi107.stdout:checking for memory.h... (cached) yes 2023-07-21T22:30:10.396 INFO:tasks.workunit.client.0.smithi107.stdout:checking for strings.h... (cached) yes 2023-07-21T22:30:10.400 INFO:tasks.workunit.client.0.smithi107.stdout:checking for unistd.h... (cached) yes 2023-07-21T22:30:10.438 INFO:tasks.workunit.client.0.smithi107.stdout:checking locale.h usability... yes 2023-07-21T22:30:10.452 INFO:tasks.workunit.client.0.smithi107.stdout:checking locale.h presence... yes 2023-07-21T22:30:10.453 INFO:tasks.workunit.client.0.smithi107.stdout:checking for locale.h... yes 2023-07-21T22:30:10.467 INFO:tasks.workunit.client.0.smithi107.stdout:checking for stdlib.h... (cached) yes 2023-07-21T22:30:10.472 INFO:tasks.workunit.client.0.smithi107.stdout:checking for string.h... (cached) yes 2023-07-21T22:30:10.524 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/socket.h usability... yes 2023-07-21T22:30:10.539 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/socket.h presence... yes 2023-07-21T22:30:10.539 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/socket.h... yes 2023-07-21T22:30:10.586 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/time.h usability... yes 2023-07-21T22:30:10.600 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/time.h presence... yes 2023-07-21T22:30:10.600 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/time.h... yes 2023-07-21T22:30:10.647 INFO:tasks.workunit.client.0.smithi107.stdout:checking fcntl.h usability... yes 2023-07-21T22:30:10.661 INFO:tasks.workunit.client.0.smithi107.stdout:checking fcntl.h presence... yes 2023-07-21T22:30:10.661 INFO:tasks.workunit.client.0.smithi107.stdout:checking for fcntl.h... yes 2023-07-21T22:30:10.708 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/fcntl.h usability... yes 2023-07-21T22:30:10.722 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/fcntl.h presence... yes 2023-07-21T22:30:10.723 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/fcntl.h... yes 2023-07-21T22:30:10.782 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/param.h usability... yes 2023-07-21T22:30:10.799 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/param.h presence... yes 2023-07-21T22:30:10.799 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/param.h... yes 2023-07-21T22:30:10.840 INFO:tasks.workunit.client.0.smithi107.stdout:checking ioctl.h usability... no 2023-07-21T22:30:10.854 INFO:tasks.workunit.client.0.smithi107.stdout:checking ioctl.h presence... no 2023-07-21T22:30:10.855 INFO:tasks.workunit.client.0.smithi107.stdout:checking for ioctl.h... no 2023-07-21T22:30:10.898 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/ioctl.h usability... yes 2023-07-21T22:30:10.912 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/ioctl.h presence... yes 2023-07-21T22:30:10.912 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/ioctl.h... yes 2023-07-21T22:30:10.957 INFO:tasks.workunit.client.0.smithi107.stdout:checking return type of signal handlers... void 2023-07-21T22:30:10.994 INFO:tasks.workunit.client.0.smithi107.stdout:checking for size_t... yes 2023-07-21T22:30:11.026 INFO:tasks.workunit.client.0.smithi107.stdout:checking for dev_t... yes 2023-07-21T22:30:11.064 INFO:tasks.workunit.client.0.smithi107.stdout:checking for short... yes 2023-07-21T22:30:11.131 INFO:tasks.workunit.client.0.smithi107.stdout:checking size of short... 2 2023-07-21T22:30:11.170 INFO:tasks.workunit.client.0.smithi107.stdout:checking for int... yes 2023-07-21T22:30:11.229 INFO:tasks.workunit.client.0.smithi107.stdout:checking size of int... 4 2023-07-21T22:30:11.268 INFO:tasks.workunit.client.0.smithi107.stdout:checking for long... yes 2023-07-21T22:30:11.328 INFO:tasks.workunit.client.0.smithi107.stdout:checking size of long... 8 2023-07-21T22:30:11.368 INFO:tasks.workunit.client.0.smithi107.stdout:checking for long long... yes 2023-07-21T22:30:11.427 INFO:tasks.workunit.client.0.smithi107.stdout:checking size of long long... 8 2023-07-21T22:30:11.464 INFO:tasks.workunit.client.0.smithi107.stdout:checking for ssize_t... yes 2023-07-21T22:30:11.497 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sig_atomic_t... yes 2023-07-21T22:30:11.534 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether gcc needs -traditional... no 2023-07-21T22:30:11.563 INFO:tasks.workunit.client.0.smithi107.stdout:checking for an ANSI C-conforming const... yes 2023-07-21T22:30:11.582 INFO:tasks.workunit.client.0.smithi107.stdout:checking for inline... inline 2023-07-21T22:30:11.634 INFO:tasks.workunit.client.0.smithi107.stdout:checking for working alloca.h... yes 2023-07-21T22:30:11.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:11 smithi190 ceph-mon[98334]: pgmap v326: 97 pgs: 97 active+clean; 682 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-07-21T22:30:11.674 INFO:tasks.workunit.client.0.smithi107.stdout:checking for alloca... yes 2023-07-21T22:30:11.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:11 smithi132 ceph-mon[97863]: pgmap v326: 97 pgs: 97 active+clean; 682 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-07-21T22:30:11.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:11 smithi107 ceph-mon[96543]: pgmap v326: 97 pgs: 97 active+clean; 682 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2023-07-21T22:30:11.733 INFO:tasks.workunit.client.0.smithi107.stdout:checking for working memcmp... yes 2023-07-21T22:30:11.781 INFO:tasks.workunit.client.0.smithi107.stdout:checking for strftime... yes 2023-07-21T22:30:11.848 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2023-07-21T22:30:11.904 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether stat accepts an empty string... no 2023-07-21T22:30:11.955 INFO:tasks.workunit.client.0.smithi107.stdout:checking for vprintf... yes 2023-07-21T22:30:12.008 INFO:tasks.workunit.client.0.smithi107.stdout:checking for _doprnt... no 2023-07-21T22:30:12.032 INFO:tasks.workunit.client.0.smithi107.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-07-21T22:30:12.044 INFO:tasks.workunit.client.0.smithi107.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-07-21T22:30:12.103 INFO:tasks.workunit.client.0.smithi107.stdout:checking for the pthreads library -lpthreads... no 2023-07-21T22:30:12.151 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether pthreads work without any flags... no 2023-07-21T22:30:12.165 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether pthreads work with -Kthread... no 2023-07-21T22:30:12.179 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether pthreads work with -kthread... no 2023-07-21T22:30:12.218 INFO:tasks.workunit.client.0.smithi107.stdout:checking for the pthreads library -llthread... no 2023-07-21T22:30:12.265 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether pthreads work with -pthread... yes 2023-07-21T22:30:12.306 INFO:tasks.workunit.client.0.smithi107.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2023-07-21T22:30:12.307 INFO:tasks.workunit.client.0.smithi107.stdout:checking if more special flags are required for pthreads... no 2023-07-21T22:30:12.307 INFO:tasks.workunit.client.0.smithi107.stdout:checking for cc_r... gcc 2023-07-21T22:30:12.385 INFO:tasks.workunit.client.0.smithi107.stdout:checking for setrlimit... yes 2023-07-21T22:30:12.438 INFO:tasks.workunit.client.0.smithi107.stdout:checking for setproctitle... no 2023-07-21T22:30:12.488 INFO:tasks.workunit.client.0.smithi107.stdout:checking for getopt_long... yes 2023-07-21T22:30:12.542 INFO:tasks.workunit.client.0.smithi107.stdout:checking for strtoull... yes 2023-07-21T22:30:12.596 INFO:tasks.workunit.client.0.smithi107.stdout:checking for strtoq... yes 2023-07-21T22:30:12.651 INFO:tasks.workunit.client.0.smithi107.stdout:checking for snprintf... yes 2023-07-21T22:30:12.704 INFO:tasks.workunit.client.0.smithi107.stdout:checking for vsnprintf... yes 2023-07-21T22:30:12.758 INFO:tasks.workunit.client.0.smithi107.stdout:checking for vfprintf... yes 2023-07-21T22:30:12.812 INFO:tasks.workunit.client.0.smithi107.stdout:checking for gethostname... yes 2023-07-21T22:30:12.866 INFO:tasks.workunit.client.0.smithi107.stdout:checking for setlocale... yes 2023-07-21T22:30:12.921 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sync... yes 2023-07-21T22:30:12.982 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether snprintf is C99 conformant... done 2023-07-21T22:30:12.992 INFO:tasks.workunit.client.0.smithi107.stdout:configure: creating ./config.status 2023-07-21T22:30:13.128 INFO:tasks.workunit.client.0.smithi107.stdout:config.status: creating Makefile 2023-07-21T22:30:13.142 INFO:tasks.workunit.client.0.smithi107.stdout:config.status: creating src/Makefile 2023-07-21T22:30:13.157 INFO:tasks.workunit.client.0.smithi107.stdout:config.status: creating man/Makefile 2023-07-21T22:30:13.165 INFO:tasks.workunit.client.0.smithi107.stdout:config.status: creating config.h 2023-07-21T22:30:13.206 INFO:tasks.workunit.client.0.smithi107.stdout:config.status: executing depfiles commands 2023-07-21T22:30:13.315 INFO:tasks.workunit.client.0.smithi107.stderr:+ make 2023-07-21T22:30:13.325 INFO:tasks.workunit.client.0.smithi107.stdout:make all-recursive 2023-07-21T22:30:13.326 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-07-21T22:30:13.328 INFO:tasks.workunit.client.0.smithi107.stdout:Making all in src 2023-07-21T22:30:13.334 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-07-21T22:30:13.335 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2023-07-21T22:30:13.335 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2023-07-21T22:30:13.481 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:30:13.524 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:13 smithi107 ceph-mon[96543]: pgmap v327: 97 pgs: 97 active+clean; 682 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2023-07-21T22:30:13.532 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2023-07-21T22:30:13.532 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2023-07-21T22:30:13.592 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2023-07-21T22:30:13.592 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2023-07-21T22:30:13.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.581+0000 7f948ad06700 1 -- 172.21.15.107:0/2217485615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94840ffbd0 msgr2=0x7f9484100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:13.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.581+0000 7f948ad06700 1 --2- 172.21.15.107:0/2217485615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94840ffbd0 0x7f9484100020 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9470009990 tx=0x7f9470009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.581+0000 7f948ad06700 1 -- 172.21.15.107:0/2217485615 shutdown_connections 2023-07-21T22:30:13.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.581+0000 7f948ad06700 1 --2- 172.21.15.107:0/2217485615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9484100560 0x7f9484106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.581+0000 7f948ad06700 1 --2- 172.21.15.107:0/2217485615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94840fe9e0 0x7f94840fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.581+0000 7f948ad06700 1 --2- 172.21.15.107:0/2217485615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94840ffbd0 0x7f9484100020 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.581+0000 7f948ad06700 1 -- 172.21.15.107:0/2217485615 >> 172.21.15.107:0/2217485615 conn(0x7f94840fa000 msgr2=0x7f94840fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:13.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.581+0000 7f948ad06700 1 -- 172.21.15.107:0/2217485615 shutdown_connections 2023-07-21T22:30:13.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.581+0000 7f948ad06700 1 -- 172.21.15.107:0/2217485615 wait complete. 2023-07-21T22:30:13.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.582+0000 7f948ad06700 1 Processor -- start 2023-07-21T22:30:13.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.582+0000 7f948ad06700 1 -- start start 2023-07-21T22:30:13.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.582+0000 7f948ad06700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94840fe9e0 0x7f9484068240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.582+0000 7f948ad06700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94840ffbd0 0x7f948419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.582+0000 7f948ad06700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9484100560 0x7f948419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.582+0000 7f948ad06700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948419d3c0 con 0x7f94840fe9e0 2023-07-21T22:30:13.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.582+0000 7f948ad06700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948419d530 con 0x7f9484100560 2023-07-21T22:30:13.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.582+0000 7f948ad06700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948419d6f0 con 0x7f94840ffbd0 2023-07-21T22:30:13.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.582+0000 7f9488aa2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94840fe9e0 0x7f9484068240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.582+0000 7f9488aa2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94840fe9e0 0x7f9484068240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48806/0 (socket says 172.21.15.107:48806) 2023-07-21T22:30:13.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.582+0000 7f9488aa2700 1 -- 172.21.15.107:0/2672190068 learned_addr learned my addr 172.21.15.107:0/2672190068 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:13.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.582+0000 7f9483fff700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94840ffbd0 0x7f948419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.582+0000 7f94892a3700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9484100560 0x7f948419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.583+0000 7f9488aa2700 1 -- 172.21.15.107:0/2672190068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94840ffbd0 msgr2=0x7f948419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:13.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.583+0000 7f9488aa2700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94840ffbd0 0x7f948419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.583+0000 7f9488aa2700 1 -- 172.21.15.107:0/2672190068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9484100560 msgr2=0x7f948419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:13.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.583+0000 7f9488aa2700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9484100560 0x7f948419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.583+0000 7f9488aa2700 1 -- 172.21.15.107:0/2672190068 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700095f0 con 0x7f94840fe9e0 2023-07-21T22:30:13.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.583+0000 7f94892a3700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9484100560 0x7f948419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:13.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.583+0000 7f9488aa2700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94840fe9e0 0x7f9484068240 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f947800d980 tx=0x7f947800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:13.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.583+0000 7f9481ffb700 1 -- 172.21.15.107:0/2672190068 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478004020 con 0x7f94840fe9e0 2023-07-21T22:30:13.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.583+0000 7f948ad06700 1 -- 172.21.15.107:0/2672190068 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948419d9d0 con 0x7f94840fe9e0 2023-07-21T22:30:13.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.583+0000 7f9481ffb700 1 -- 172.21.15.107:0/2672190068 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9478009730 con 0x7f94840fe9e0 2023-07-21T22:30:13.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.583+0000 7f9481ffb700 1 -- 172.21.15.107:0/2672190068 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478003a30 con 0x7f94840fe9e0 2023-07-21T22:30:13.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.583+0000 7f948ad06700 1 -- 172.21.15.107:0/2672190068 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948419dfb0 con 0x7f94840fe9e0 2023-07-21T22:30:13.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.585+0000 7f9481ffb700 1 -- 172.21.15.107:0/2672190068 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f94780098a0 con 0x7f94840fe9e0 2023-07-21T22:30:13.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.586+0000 7f9481ffb700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f94740a1150 0x7f94740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.586+0000 7f9483fff700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f94740a1150 0x7f94740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.586+0000 7f9481ffb700 1 -- 172.21.15.107:0/2672190068 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f94780cc680 con 0x7f94840fe9e0 2023-07-21T22:30:13.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.586+0000 7f9483fff700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f94740a1150 0x7f94740a3600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f947000b5c0 tx=0x7f9470005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:13.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9496e8bb80 1 Processor -- start 2023-07-21T22:30:13.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9496e8bb80 1 -- start start 2023-07-21T22:30:13.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9496e8bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb24830 0x56361eb24c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9496e8bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb251a0 0x56361eb23430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9496e8bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb23a10 0x56361eb23e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9496e8bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361ea28cb0 con 0x56361eb24830 2023-07-21T22:30:13.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9496e8bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361ea7ba40 con 0x56361eb251a0 2023-07-21T22:30:13.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9496e8bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361ea74ed0 con 0x56361eb23a10 2023-07-21T22:30:13.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9488aa2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb24830 0x56361eb24c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9488aa2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb24830 0x56361eb24c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48814/0 (socket says 172.21.15.107:48814) 2023-07-21T22:30:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9488aa2700 1 -- 172.21.15.107:0/2367047704 learned_addr learned my addr 172.21.15.107:0/2367047704 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9483fff700 1 --2- 172.21.15.107:0/2367047704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb251a0 0x56361eb23430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f94892a3700 1 --2- 172.21.15.107:0/2367047704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb23a10 0x56361eb23e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9488aa2700 1 -- 172.21.15.107:0/2367047704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb23a10 msgr2=0x56361eb23e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9488aa2700 1 --2- 172.21.15.107:0/2367047704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb23a10 0x56361eb23e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.646 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9488aa2700 1 -- 172.21.15.107:0/2367047704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb251a0 msgr2=0x56361eb23430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9488aa2700 1 --2- 172.21.15.107:0/2367047704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb251a0 0x56361eb23430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9488aa2700 1 -- 172.21.15.107:0/2367047704 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361e8a2de0 con 0x56361eb24830 2023-07-21T22:30:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9483fff700 1 --2- 172.21.15.107:0/2367047704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb251a0 0x56361eb23430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.588+0000 7f9488aa2700 1 --2- 172.21.15.107:0/2367047704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb24830 0x56361eb24c40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f94780a1990 tx=0x7f9478099450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c013e215b3b065 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.589+0000 7f946effd700 1 -- 172.21.15.107:0/2367047704 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947809daa0 con 0x56361eb24830 2023-07-21T22:30:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.589+0000 7f946effd700 1 -- 172.21.15.107:0/2367047704 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94780a1ec0 con 0x56361eb24830 2023-07-21T22:30:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.589+0000 7f946effd700 1 -- 172.21.15.107:0/2367047704 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94780a1a30 con 0x56361eb24830 2023-07-21T22:30:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.589+0000 7f9496e8bb80 1 -- 172.21.15.107:0/2367047704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb24830 msgr2=0x56361eb24c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:13.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.589+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/2367047704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb24830 0x56361eb24c40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f94780a1990 tx=0x7f9478099450 comp rx=0 tx=0).stop 2023-07-21T22:30:13.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.589+0000 7f9496e8bb80 1 -- 172.21.15.107:0/2367047704 shutdown_connections 2023-07-21T22:30:13.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.589+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/2367047704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb23a10 0x56361eb23e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.589+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/2367047704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb24830 0x56361eb24c40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.589+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/2367047704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb251a0 0x56361eb23430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.589+0000 7f9496e8bb80 1 -- 172.21.15.107:0/2367047704 >> 172.21.15.107:0/2367047704 conn(0x56361eb18ef0 msgr2=0x56361eb1e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:13.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.589+0000 7f9496e8bb80 1 -- 172.21.15.107:0/2367047704 shutdown_connections 2023-07-21T22:30:13.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.589+0000 7f9496e8bb80 1 -- 172.21.15.107:0/2367047704 wait complete. 2023-07-21T22:30:13.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f9496e8bb80 1 -- start start 2023-07-21T22:30:13.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f9496e8bb80 1 Processor -- start 2023-07-21T22:30:13.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f9496e8bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb23a10 0x56361eb2d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f9496e8bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb24830 0x56361eb2d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f9496e8bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 0x56361eb34650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f9496e8bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361ea14ac0 con 0x56361eb251a0 2023-07-21T22:30:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f9496e8bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361ea147c0 con 0x56361eb23a10 2023-07-21T22:30:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f9496e8bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361e8f8920 con 0x56361eb24830 2023-07-21T22:30:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f9496e8bb80 10 client.0 _ll_register_callbacks cb 0x56361ea12760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f94892a3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 0x56361eb34650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f94892a3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 0x56361eb34650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48824/0 (socket says 172.21.15.107:48824) 2023-07-21T22:30:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f94892a3700 1 -- 172.21.15.107:0/1118244133 learned_addr learned my addr 172.21.15.107:0/1118244133 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f9488aa2700 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb23a10 0x56361eb2d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.590+0000 7f9483fff700 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb24830 0x56361eb2d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f94892a3700 1 -- 172.21.15.107:0/1118244133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb24830 msgr2=0x56361eb2d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f94892a3700 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb24830 0x56361eb2d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f94892a3700 1 -- 172.21.15.107:0/1118244133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb23a10 msgr2=0x56361eb2d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f94892a3700 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb23a10 0x56361eb2d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f94892a3700 1 -- 172.21.15.107:0/1118244133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9484107750 con 0x56361eb251a0 2023-07-21T22:30:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f9488aa2700 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb23a10 0x56361eb2d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f94892a3700 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 0x56361eb34650 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f947c00ce90 tx=0x7f947c00cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393feb3a41d9ac6f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f9463fff700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f9463fff700 1 -- 172.21.15.107:0/1118244133 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c011050 con 0x56361eb251a0 2023-07-21T22:30:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f9463fff700 1 -- 172.21.15.107:0/1118244133 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f947c006e40 con 0x56361eb251a0 2023-07-21T22:30:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f9496e8bb80 1 -- 172.21.15.107:0/1118244133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56361e9ce4f0 con 0x56361eb251a0 2023-07-21T22:30:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f9463fff700 1 -- 172.21.15.107:0/1118244133 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f947c009e60 con 0x56361eb251a0 2023-07-21T22:30:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f9496e8bb80 10 client.15732 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f9496e8bb80 1 -- 172.21.15.107:0/1118244133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f948419dfb0 con 0x56361eb251a0 2023-07-21T22:30:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f9463fff700 1 -- 172.21.15.107:0/1118244133 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f947c025020 con 0x56361eb251a0 2023-07-21T22:30:13.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f9463fff700 1 -- 172.21.15.107:0/1118244133 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f947c016850 con 0x56361eb251a0 2023-07-21T22:30:13.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f9496e8bb80 10 client.15732 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:13.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.591+0000 7f9496e8bb80 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:13.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.592+0000 7f9496e8bb80 20 client.15732 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:13.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.592+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56361eb38f00 0x56361eb3b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.592+0000 7f9496e8bb80 4 client.15732 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:13.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.592+0000 7f9496e8bb80 1 -- 172.21.15.107:0/1118244133 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56361ea6deb0 con 0x56361eb38f00 2023-07-21T22:30:13.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.592+0000 7f9488aa2700 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56361eb38f00 0x56361eb3b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.592+0000 7f9488aa2700 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56361eb38f00 0x56361eb3b310 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8b3e6a9732e4f7 server_cookie=c7a19424e3871ef5 in_seq=0 out_seq=0 2023-07-21T22:30:13.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.592+0000 7f9463fff700 10 client.15732 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:13.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9463fff700 1 -- 172.21.15.107:0/1118244133 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56361ea6deb0 con 0x56361eb38f00 2023-07-21T22:30:13.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9463fff700 10 client.15732 handle_command_reply: tid=0 2023-07-21T22:30:13.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9496e8bb80 1 client.15732 shutdown 2023-07-21T22:30:13.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9496e8bb80 1 -- 172.21.15.107:0/1118244133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 msgr2=0x56361eb34650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:13.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 0x56361eb34650 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f947c00ce90 tx=0x7f947c00cec0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9496e8bb80 1 -- 172.21.15.107:0/1118244133 shutdown_connections 2023-07-21T22:30:13.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56361eb38f00 0x56361eb3b310 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 0x56361eb34650 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.657 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9463fff700 0 client.15732 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:13.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb23a10 0x56361eb2d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/1118244133 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb24830 0x56361eb2d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.658 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9496e8bb80 1 -- 172.21.15.107:0/1118244133 >> 172.21.15.107:0/1118244133 conn(0x56361eb18ef0 msgr2=0x56361eb1da00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:13.659 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9496e8bb80 1 -- 172.21.15.107:0/1118244133 shutdown_connections 2023-07-21T22:30:13.659 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9496e8bb80 1 -- 172.21.15.107:0/1118244133 wait complete. 2023-07-21T22:30:13.660 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.593+0000 7f9496e8bb80 20 client.15732 trim_cache size 0 max 16384 2023-07-21T22:30:13.661 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:13 smithi190 ceph-mon[98334]: pgmap v327: 97 pgs: 97 active+clean; 682 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2023-07-21T22:30:13.661 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2023-07-21T22:30:13.662 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2023-07-21T22:30:13.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.610+0000 7f9496e8bb80 1 Processor -- start 2023-07-21T22:30:13.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.610+0000 7f9496e8bb80 1 -- start start 2023-07-21T22:30:13.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.610+0000 7f9496e8bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb23a10 0x56361eb2af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.669 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.610+0000 7f9496e8bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb24830 0x56361eb34570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.610+0000 7f9496e8bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 0x56361eb27aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.610+0000 7f9496e8bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361eb2c580 con 0x56361eb251a0 2023-07-21T22:30:13.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.610+0000 7f9496e8bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361eb2cc40 con 0x56361eb24830 2023-07-21T22:30:13.670 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.610+0000 7f9496e8bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361eb1f060 con 0x56361eb23a10 2023-07-21T22:30:13.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.610+0000 7f94892a3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 0x56361eb27aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.610+0000 7f94892a3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 0x56361eb27aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48832/0 (socket says 172.21.15.107:48832) 2023-07-21T22:30:13.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.610+0000 7f9483fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb24830 0x56361eb34570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.671 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.610+0000 7f94892a3700 1 -- 172.21.15.107:0/1781758765 learned_addr learned my addr 172.21.15.107:0/1781758765 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.610+0000 7f9488aa2700 1 --2- 172.21.15.107:0/1781758765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb23a10 0x56361eb2af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.611+0000 7f94892a3700 1 -- 172.21.15.107:0/1781758765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb23a10 msgr2=0x56361eb2af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.611+0000 7f94892a3700 1 --2- 172.21.15.107:0/1781758765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb23a10 0x56361eb2af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.611+0000 7f94892a3700 1 -- 172.21.15.107:0/1781758765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb24830 msgr2=0x56361eb34570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:13.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.611+0000 7f94892a3700 1 --2- 172.21.15.107:0/1781758765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb24830 0x56361eb34570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.611+0000 7f94892a3700 1 -- 172.21.15.107:0/1781758765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361e88bc60 con 0x56361eb251a0 2023-07-21T22:30:13.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.611+0000 7f9483fff700 1 --2- 172.21.15.107:0/1781758765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb24830 0x56361eb34570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:13.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.611+0000 7f94892a3700 1 --2- 172.21.15.107:0/1781758765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 0x56361eb27aa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f947c00e010 tx=0x7f947c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc75f675773e997 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:13.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.611+0000 7f946f7fe700 1 -- 172.21.15.107:0/1781758765 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c011040 con 0x56361eb251a0 2023-07-21T22:30:13.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.611+0000 7f946f7fe700 1 -- 172.21.15.107:0/1781758765 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f947c025030 con 0x56361eb251a0 2023-07-21T22:30:13.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.611+0000 7f946f7fe700 1 -- 172.21.15.107:0/1781758765 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c016680 con 0x56361eb251a0 2023-07-21T22:30:13.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.612+0000 7f9496e8bb80 1 -- 172.21.15.107:0/1781758765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 msgr2=0x56361eb27aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:13.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.612+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/1781758765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 0x56361eb27aa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f947c00e010 tx=0x7f947c004ad0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.612+0000 7f9496e8bb80 1 -- 172.21.15.107:0/1781758765 shutdown_connections 2023-07-21T22:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.612+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/1781758765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb251a0 0x56361eb27aa0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.612+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/1781758765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb23a10 0x56361eb2af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.612+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/1781758765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb24830 0x56361eb34570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.612+0000 7f9496e8bb80 1 -- 172.21.15.107:0/1781758765 >> 172.21.15.107:0/1781758765 conn(0x56361eb18ef0 msgr2=0x56361eb23de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:13.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.612+0000 7f9496e8bb80 1 -- 172.21.15.107:0/1781758765 shutdown_connections 2023-07-21T22:30:13.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.612+0000 7f9496e8bb80 1 -- 172.21.15.107:0/1781758765 wait complete. 2023-07-21T22:30:13.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.612+0000 7f9496e8bb80 1 -- start start 2023-07-21T22:30:13.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.612+0000 7f9496e8bb80 1 Processor -- start 2023-07-21T22:30:13.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.612+0000 7f9496e8bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb23a10 0x56361eb33fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9496e8bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb24830 0x56361eb34510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9496e8bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb251a0 0x56361eb30b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9496e8bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361eb31240 con 0x56361eb23a10 2023-07-21T22:30:13.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9496e8bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361eb313b0 con 0x56361eb24830 2023-07-21T22:30:13.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9496e8bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361eb31570 con 0x56361eb251a0 2023-07-21T22:30:13.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9496e8bb80 10 client.0 _ll_register_callbacks cb 0x7f947c00c7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:13.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9488aa2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb23a10 0x56361eb33fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9488aa2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb23a10 0x56361eb33fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48834/0 (socket says 172.21.15.107:48834) 2023-07-21T22:30:13.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9488aa2700 1 -- 172.21.15.107:0/885648543 learned_addr learned my addr 172.21.15.107:0/885648543 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:13.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9483fff700 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb24830 0x56361eb34510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f94892a3700 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb251a0 0x56361eb30b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9488aa2700 1 -- 172.21.15.107:0/885648543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb251a0 msgr2=0x56361eb30b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:13.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9488aa2700 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb251a0 0x56361eb30b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9488aa2700 1 -- 172.21.15.107:0/885648543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb24830 msgr2=0x56361eb34510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:13.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9488aa2700 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb24830 0x56361eb34510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9488aa2700 1 -- 172.21.15.107:0/885648543 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947000b770 con 0x56361eb23a10 2023-07-21T22:30:13.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9483fff700 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb24830 0x56361eb34510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:13.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9488aa2700 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb23a10 0x56361eb33fd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f94780964e0 tx=0x7f947809a030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1c3499e512b209 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:13.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9463fff700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:13.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9463fff700 1 -- 172.21.15.107:0/885648543 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478097e10 con 0x56361eb23a10 2023-07-21T22:30:13.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9463fff700 1 -- 172.21.15.107:0/885648543 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9478009b80 con 0x56361eb23a10 2023-07-21T22:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9496e8bb80 1 -- 172.21.15.107:0/885648543 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94780a1ec0 con 0x56361eb23a10 2023-07-21T22:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9463fff700 1 -- 172.21.15.107:0/885648543 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9478098f60 con 0x56361eb23a10 2023-07-21T22:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9496e8bb80 10 client.15738 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:13.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.613+0000 7f9496e8bb80 1 -- 172.21.15.107:0/885648543 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f948419dfb0 con 0x56361eb23a10 2023-07-21T22:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.614+0000 7f9463fff700 1 -- 172.21.15.107:0/885648543 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94780a1ec0 con 0x56361eb23a10 2023-07-21T22:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.614+0000 7f9463fff700 1 -- 172.21.15.107:0/885648543 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9478003b90 con 0x56361eb23a10 2023-07-21T22:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.614+0000 7f9496e8bb80 10 client.15738 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.614+0000 7f9496e8bb80 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.614+0000 7f9496e8bb80 20 client.15738 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.614+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56361eb8e110 0x56361eb90520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.614+0000 7f9496e8bb80 4 client.15738 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.614+0000 7f9496e8bb80 1 -- 172.21.15.107:0/885648543 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56361ea30050 con 0x56361eb8e110 2023-07-21T22:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.614+0000 7f9483fff700 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56361eb8e110 0x56361eb90520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.614+0000 7f9463fff700 10 client.15738 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.614+0000 7f9483fff700 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56361eb8e110 0x56361eb90520 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6abd2c31a88741d server_cookie=8860754bc7e9e1c1 in_seq=0 out_seq=0 2023-07-21T22:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9463fff700 1 -- 172.21.15.107:0/885648543 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56361ea30050 con 0x56361eb8e110 2023-07-21T22:30:13.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9463fff700 10 client.15738 handle_command_reply: tid=0 2023-07-21T22:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9496e8bb80 1 client.15738 shutdown 2023-07-21T22:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9496e8bb80 1 -- 172.21.15.107:0/885648543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb23a10 msgr2=0x56361eb33fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb23a10 0x56361eb33fd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f94780964e0 tx=0x7f947809a030 comp rx=0 tx=0).stop 2023-07-21T22:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9496e8bb80 1 -- 172.21.15.107:0/885648543 shutdown_connections 2023-07-21T22:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56361eb8e110 0x56361eb90520 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56361eb251a0 0x56361eb30b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9463fff700 0 client.15738 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361eb23a10 0x56361eb33fd0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9496e8bb80 1 --2- 172.21.15.107:0/885648543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56361eb24830 0x56361eb34510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9496e8bb80 1 -- 172.21.15.107:0/885648543 >> 172.21.15.107:0/885648543 conn(0x56361eb18ef0 msgr2=0x56361eb2af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9496e8bb80 1 -- 172.21.15.107:0/885648543 shutdown_connections 2023-07-21T22:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9496e8bb80 1 -- 172.21.15.107:0/885648543 wait complete. 2023-07-21T22:30:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f9496e8bb80 20 client.15738 trim_cache size 0 max 16384 2023-07-21T22:30:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:30:13.687 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2023-07-21T22:30:13.687 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2023-07-21T22:30:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:30:13.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f948ad06700 1 -- 172.21.15.107:0/2672190068 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f94740a1150 msgr2=0x7f94740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:13.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f948ad06700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f94740a1150 0x7f94740a3600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f947000b5c0 tx=0x7f9470005960 comp rx=0 tx=0).stop 2023-07-21T22:30:13.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f948ad06700 1 -- 172.21.15.107:0/2672190068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94840fe9e0 msgr2=0x7f9484068240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:13.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.615+0000 7f948ad06700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94840fe9e0 0x7f9484068240 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f947800d980 tx=0x7f947800dc90 comp rx=0 tx=0).stop 2023-07-21T22:30:13.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.616+0000 7f948ad06700 1 -- 172.21.15.107:0/2672190068 shutdown_connections 2023-07-21T22:30:13.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.616+0000 7f948ad06700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9484100560 0x7f948419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.616+0000 7f948ad06700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94840fe9e0 0x7f9484068240 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.616+0000 7f948ad06700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f94740a1150 0x7f94740a3600 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.616+0000 7f948ad06700 1 --2- 172.21.15.107:0/2672190068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94840ffbd0 0x7f948419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.616+0000 7f948ad06700 1 -- 172.21.15.107:0/2672190068 >> 172.21.15.107:0/2672190068 conn(0x7f94840fa000 msgr2=0x7f94840fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:13.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.616+0000 7f948ad06700 1 -- 172.21.15.107:0/2672190068 shutdown_connections 2023-07-21T22:30:13.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.616+0000 7f948ad06700 1 -- 172.21.15.107:0/2672190068 wait complete. 2023-07-21T22:30:13.690 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:30:13.691 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:30:13.691 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:13 smithi132 ceph-mon[97863]: pgmap v327: 97 pgs: 97 active+clean; 682 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2023-07-21T22:30:13.695 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2023-07-21T22:30:13.695 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2023-07-21T22:30:13.747 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2023-07-21T22:30:13.748 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2023-07-21T22:30:13.791 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2023-07-21T22:30:13.791 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2023-07-21T22:30:13.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.774+0000 7fb6de69b700 1 -- 172.21.15.107:0/168432666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d8102ed0 msgr2=0x7fb6d81052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:13.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.774+0000 7fb6de69b700 1 --2- 172.21.15.107:0/168432666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d8102ed0 0x7fb6d81052b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0098f0 tx=0x7fb6cc009c00 comp rx=0 tx=0).stop 2023-07-21T22:30:13.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.774+0000 7fb6de69b700 1 -- 172.21.15.107:0/168432666 shutdown_connections 2023-07-21T22:30:13.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.774+0000 7fb6de69b700 1 --2- 172.21.15.107:0/168432666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d81057f0 0x7fb6d8107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.774+0000 7fb6de69b700 1 --2- 172.21.15.107:0/168432666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6d81005b0 0x7fb6d8102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.774+0000 7fb6de69b700 1 --2- 172.21.15.107:0/168432666 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d8102ed0 0x7fb6d81052b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.774+0000 7fb6de69b700 1 -- 172.21.15.107:0/168432666 >> 172.21.15.107:0/168432666 conn(0x7fb6d80f9fd0 msgr2=0x7fb6d80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:13.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.774+0000 7fb6de69b700 1 -- 172.21.15.107:0/168432666 shutdown_connections 2023-07-21T22:30:13.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.774+0000 7fb6de69b700 1 -- 172.21.15.107:0/168432666 wait complete. 2023-07-21T22:30:13.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.775+0000 7fb6de69b700 1 Processor -- start 2023-07-21T22:30:13.826 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.775+0000 7fb6de69b700 1 -- start start 2023-07-21T22:30:13.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.775+0000 7fb6de69b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d81005b0 0x7fb6d81957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.775+0000 7fb6de69b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d8102ed0 0x7fb6d8195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.775+0000 7fb6de69b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6d81057f0 0x7fb6d819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.827 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.775+0000 7fb6de69b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d819d350 con 0x7fb6d81005b0 2023-07-21T22:30:13.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6de69b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d819d490 con 0x7fb6d81057f0 2023-07-21T22:30:13.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6de69b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d819d620 con 0x7fb6d8102ed0 2023-07-21T22:30:13.828 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d81005b0 0x7fb6d81957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d81005b0 0x7fb6d81957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48846/0 (socket says 172.21.15.107:48846) 2023-07-21T22:30:13.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6d7fff700 1 -- 172.21.15.107:0/2374028356 learned_addr learned my addr 172.21.15.107:0/2374028356 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:13.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6dcc38700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6d81057f0 0x7fb6d819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6d77fe700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d8102ed0 0x7fb6d8195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6d7fff700 1 -- 172.21.15.107:0/2374028356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d8102ed0 msgr2=0x7fb6d8195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:13.829 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6d7fff700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d8102ed0 0x7fb6d8195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6d7fff700 1 -- 172.21.15.107:0/2374028356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6d81057f0 msgr2=0x7fb6d819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6d7fff700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6d81057f0 0x7fb6d819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6d7fff700 1 -- 172.21.15.107:0/2374028356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8009520 con 0x7fb6d81005b0 2023-07-21T22:30:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6d7fff700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d81005b0 0x7fb6d81957d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800eb30 tx=0x7fb6c800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6d57fa700 1 -- 172.21.15.107:0/2374028356 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8009880 con 0x7fb6d81005b0 2023-07-21T22:30:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6d57fa700 1 -- 172.21.15.107:0/2374028356 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6c80044e0 con 0x7fb6d81005b0 2023-07-21T22:30:13.830 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6de69b700 1 -- 172.21.15.107:0/2374028356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6cc009520 con 0x7fb6d81005b0 2023-07-21T22:30:13.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6d57fa700 1 -- 172.21.15.107:0/2374028356 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8010430 con 0x7fb6d81005b0 2023-07-21T22:30:13.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.776+0000 7fb6de69b700 1 -- 172.21.15.107:0/2374028356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d819dc80 con 0x7fb6d81005b0 2023-07-21T22:30:13.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.777+0000 7fb6de69b700 1 -- 172.21.15.107:0/2374028356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d80fbb40 con 0x7fb6d81005b0 2023-07-21T22:30:13.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.778+0000 7fb6d57fa700 1 -- 172.21.15.107:0/2374028356 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb6c8003ee0 con 0x7fb6d81005b0 2023-07-21T22:30:13.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.779+0000 7fb6d57fa700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb6c00a11a0 0x7fb6c00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:13.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.779+0000 7fb6d57fa700 1 -- 172.21.15.107:0/2374028356 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb6c8014070 con 0x7fb6d81005b0 2023-07-21T22:30:13.831 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.779+0000 7fb6d77fe700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb6c00a11a0 0x7fb6c00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:13.832 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.779+0000 7fb6d77fe700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb6c00a11a0 0x7fb6c00a3650 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00fa40 tx=0x7fb6cc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:13.832 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:13.782+0000 7fb6d57fa700 1 -- 172.21.15.107:0/2374028356 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb6c80966b0 con 0x7fb6d81005b0 2023-07-21T22:30:13.859 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2023-07-21T22:30:13.859 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2023-07-21T22:30:13.889 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2023-07-21T22:30:13.889 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2023-07-21T22:30:13.919 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2023-07-21T22:30:13.919 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2023-07-21T22:30:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:gcc -g -O2 -pthread -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2023-07-21T22:30:13.969 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-07-21T22:30:13.969 INFO:tasks.workunit.client.0.smithi107.stdout:Making all in man 2023-07-21T22:30:14.039 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-07-21T22:30:14.040 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Nothing to be done for 'all'. 2023-07-21T22:30:14.040 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-07-21T22:30:14.041 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-07-21T22:30:14.042 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-07-21T22:30:14.042 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-07-21T22:30:14.043 INFO:tasks.workunit.client.0.smithi107.stderr:+ cd src 2023-07-21T22:30:14.043 INFO:tasks.workunit.client.0.smithi107.stderr:+ mkdir blogtest_in 2023-07-21T22:30:14.044 INFO:tasks.workunit.client.0.smithi107.stdout:running blogbench 2023-07-21T22:30:14.045 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'running blogbench' 2023-07-21T22:30:14.045 INFO:tasks.workunit.client.0.smithi107.stderr:+ ./blogbench -d blogtest_in 2023-07-21T22:30:14.045 INFO:tasks.workunit.client.0.smithi107.stdout: 2023-07-21T22:30:14.046 INFO:tasks.workunit.client.0.smithi107.stdout:Frequency = 10 secs 2023-07-21T22:30:14.046 INFO:tasks.workunit.client.0.smithi107.stdout:Scratch dir = [blogtest_in] 2023-07-21T22:30:14.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.028+0000 7fb6de69b700 1 -- 172.21.15.107:0/2374028356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6d802cb00 con 0x7fb6d81005b0 2023-07-21T22:30:14.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.028+0000 7fb6d57fa700 1 -- 172.21.15.107:0/2374028356 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10869 (secure 0 0 0) 0x7fb6c8004190 con 0x7fb6d81005b0 2023-07-21T22:30:14.075 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:30:14.076 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:29:21.093850+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:30:14.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.030+0000 7fb6de69b700 1 -- 172.21.15.107:0/2374028356 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb6c00a11a0 msgr2=0x7fb6c00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.030+0000 7fb6de69b700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb6c00a11a0 0x7fb6c00a3650 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00fa40 tx=0x7fb6cc019040 comp rx=0 tx=0).stop 2023-07-21T22:30:14.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.030+0000 7fb6de69b700 1 -- 172.21.15.107:0/2374028356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d81005b0 msgr2=0x7fb6d81957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.030+0000 7fb6de69b700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d81005b0 0x7fb6d81957d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb6c800eb30 tx=0x7fb6c800eef0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.030+0000 7fb6de69b700 1 -- 172.21.15.107:0/2374028356 shutdown_connections 2023-07-21T22:30:14.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.030+0000 7fb6de69b700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6d81057f0 0x7fb6d819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.030+0000 7fb6de69b700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6d81005b0 0x7fb6d81957d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.030+0000 7fb6de69b700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb6c00a11a0 0x7fb6c00a3650 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.030+0000 7fb6de69b700 1 --2- 172.21.15.107:0/2374028356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb6d8102ed0 0x7fb6d8195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.030+0000 7fb6de69b700 1 -- 172.21.15.107:0/2374028356 >> 172.21.15.107:0/2374028356 conn(0x7fb6d80f9fd0 msgr2=0x7fb6d8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.031+0000 7fb6de69b700 1 -- 172.21.15.107:0/2374028356 shutdown_connections 2023-07-21T22:30:14.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.031+0000 7fb6de69b700 1 -- 172.21.15.107:0/2374028356 wait complete. 2023-07-21T22:30:14.082 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2023-07-21T22:30:14.089 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-21T22:30:14.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.194+0000 7f2297110700 1 -- 172.21.15.107:0/3340726966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900fea10 msgr2=0x7f22900fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.194+0000 7f2297110700 1 --2- 172.21.15.107:0/3340726966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900fea10 0x7f22900fee20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2284009990 tx=0x7f2284009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.194+0000 7f2297110700 1 -- 172.21.15.107:0/3340726966 shutdown_connections 2023-07-21T22:30:14.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.194+0000 7f2297110700 1 --2- 172.21.15.107:0/3340726966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2290100590 0x7f2290106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.194+0000 7f2297110700 1 --2- 172.21.15.107:0/3340726966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900fea10 0x7f22900fee20 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.194+0000 7f2297110700 1 --2- 172.21.15.107:0/3340726966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22900ffc00 0x7f2290100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.194+0000 7f2297110700 1 -- 172.21.15.107:0/3340726966 >> 172.21.15.107:0/3340726966 conn(0x7f22900fa030 msgr2=0x7f22900fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.194+0000 7f2297110700 1 -- 172.21.15.107:0/3340726966 shutdown_connections 2023-07-21T22:30:14.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.194+0000 7f2297110700 1 -- 172.21.15.107:0/3340726966 wait complete. 2023-07-21T22:30:14.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.195+0000 7f2297110700 1 Processor -- start 2023-07-21T22:30:14.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.195+0000 7f2297110700 1 -- start start 2023-07-21T22:30:14.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.195+0000 7f2297110700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22900fea10 0x7f22901947f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.195+0000 7f2297110700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900ffc00 0x7f2290194d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.195+0000 7f2297110700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2290100590 0x7f229019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.195+0000 7f2297110700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229019c370 con 0x7f22900ffc00 2023-07-21T22:30:14.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.195+0000 7f2297110700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229019c4b0 con 0x7f22900fea10 2023-07-21T22:30:14.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.195+0000 7f2297110700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229019c640 con 0x7f2290100590 2023-07-21T22:30:14.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.195+0000 7f228ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900ffc00 0x7f2290194d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.195+0000 7f228ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900ffc00 0x7f2290194d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48862/0 (socket says 172.21.15.107:48862) 2023-07-21T22:30:14.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.195+0000 7f228ffff700 1 -- 172.21.15.107:0/902760592 learned_addr learned my addr 172.21.15.107:0/902760592 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.195+0000 7f2294eac700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22900fea10 0x7f22901947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.195+0000 7f22956ad700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2290100590 0x7f229019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.196+0000 7f228ffff700 1 -- 172.21.15.107:0/902760592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2290100590 msgr2=0x7f229019bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.196+0000 7f228ffff700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2290100590 0x7f229019bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.196+0000 7f228ffff700 1 -- 172.21.15.107:0/902760592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22900fea10 msgr2=0x7f22901947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.196+0000 7f228ffff700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22900fea10 0x7f22901947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.196+0000 7f228ffff700 1 -- 172.21.15.107:0/902760592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22840095f0 con 0x7f22900ffc00 2023-07-21T22:30:14.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.196+0000 7f2294eac700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22900fea10 0x7f22901947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:14.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.196+0000 7f228ffff700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900ffc00 0x7f2290194d30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f227c00e9a0 tx=0x7f227c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.196+0000 7f228dffb700 1 -- 172.21.15.107:0/902760592 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c009790 con 0x7f22900ffc00 2023-07-21T22:30:14.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.196+0000 7f2297110700 1 -- 172.21.15.107:0/902760592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22900688e0 con 0x7f22900ffc00 2023-07-21T22:30:14.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.196+0000 7f228dffb700 1 -- 172.21.15.107:0/902760592 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f227c010430 con 0x7f22900ffc00 2023-07-21T22:30:14.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.196+0000 7f228dffb700 1 -- 172.21.15.107:0/902760592 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c018680 con 0x7f22900ffc00 2023-07-21T22:30:14.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.196+0000 7f2297110700 1 -- 172.21.15.107:0/902760592 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2290068df0 con 0x7f22900ffc00 2023-07-21T22:30:14.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.199+0000 7f228dffb700 1 -- 172.21.15.107:0/902760592 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f227c0105a0 con 0x7f22900ffc00 2023-07-21T22:30:14.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.199+0000 7f228dffb700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f22800a1150 0x7f22800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.199+0000 7f228dffb700 1 -- 172.21.15.107:0/902760592 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f227c014070 con 0x7f22900ffc00 2023-07-21T22:30:14.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.199+0000 7f2294eac700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f22800a1150 0x7f22800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.200+0000 7f2294eac700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f22800a1150 0x7f22800a3600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2284006010 tx=0x7f2284005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.200+0000 7f22a3295b80 1 Processor -- start 2023-07-21T22:30:14.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.200+0000 7f22a3295b80 1 -- start start 2023-07-21T22:30:14.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.200+0000 7f22a3295b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d38721830 0x555d38721c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.200+0000 7f22a3295b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d387221a0 0x555d38720430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.200+0000 7f22a3295b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38720a10 0x555d38720e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.200+0000 7f22a3295b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d38625cb0 con 0x555d38721830 2023-07-21T22:30:14.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.200+0000 7f22a3295b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d38678a40 con 0x555d387221a0 2023-07-21T22:30:14.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.200+0000 7f22a3295b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d38671ed0 con 0x555d38720a10 2023-07-21T22:30:14.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f2294eac700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d38721830 0x555d38721c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f2294eac700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d38721830 0x555d38721c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48868/0 (socket says 172.21.15.107:48868) 2023-07-21T22:30:14.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f2294eac700 1 -- 172.21.15.107:0/1898692697 learned_addr learned my addr 172.21.15.107:0/1898692697 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f22956ad700 1 --2- 172.21.15.107:0/1898692697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38720a10 0x555d38720e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f228ffff700 1 --2- 172.21.15.107:0/1898692697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d387221a0 0x555d38720430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f2294eac700 1 -- 172.21.15.107:0/1898692697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38720a10 msgr2=0x555d38720e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f2294eac700 1 --2- 172.21.15.107:0/1898692697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38720a10 0x555d38720e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f2294eac700 1 -- 172.21.15.107:0/1898692697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d387221a0 msgr2=0x555d38720430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f2294eac700 1 --2- 172.21.15.107:0/1898692697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d387221a0 0x555d38720430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f2294eac700 1 -- 172.21.15.107:0/1898692697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d3849fde0 con 0x555d38721830 2023-07-21T22:30:14.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f2294eac700 1 --2- 172.21.15.107:0/1898692697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d38721830 0x555d38721c40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2284015000 tx=0x7f2284020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b0459870949f92 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f227affd700 1 -- 172.21.15.107:0/1898692697 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228402b070 con 0x555d38721830 2023-07-21T22:30:14.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f227affd700 1 -- 172.21.15.107:0/1898692697 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f228401e040 con 0x555d38721830 2023-07-21T22:30:14.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.201+0000 7f227affd700 1 -- 172.21.15.107:0/1898692697 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228402f400 con 0x555d38721830 2023-07-21T22:30:14.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.202+0000 7f22a3295b80 1 -- 172.21.15.107:0/1898692697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d38721830 msgr2=0x555d38721c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.202+0000 7f22a3295b80 1 --2- 172.21.15.107:0/1898692697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d38721830 0x555d38721c40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2284015000 tx=0x7f2284020000 comp rx=0 tx=0).stop 2023-07-21T22:30:14.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.202+0000 7f22a3295b80 1 -- 172.21.15.107:0/1898692697 shutdown_connections 2023-07-21T22:30:14.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.202+0000 7f22a3295b80 1 --2- 172.21.15.107:0/1898692697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38720a10 0x555d38720e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.202+0000 7f22a3295b80 1 --2- 172.21.15.107:0/1898692697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d38721830 0x555d38721c40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.202+0000 7f22a3295b80 1 --2- 172.21.15.107:0/1898692697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d387221a0 0x555d38720430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.202+0000 7f22a3295b80 1 -- 172.21.15.107:0/1898692697 >> 172.21.15.107:0/1898692697 conn(0x555d38715ef0 msgr2=0x555d3871b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.202+0000 7f22a3295b80 1 -- 172.21.15.107:0/1898692697 shutdown_connections 2023-07-21T22:30:14.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.202+0000 7f22a3295b80 1 -- 172.21.15.107:0/1898692697 wait complete. 2023-07-21T22:30:14.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.202+0000 7f22a3295b80 1 -- start start 2023-07-21T22:30:14.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.202+0000 7f22a3295b80 1 Processor -- start 2023-07-21T22:30:14.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f22a3295b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38720a10 0x555d3872a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f22a3295b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38721830 0x555d3872a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f22a3295b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38731580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f22a3295b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d38611ac0 con 0x555d387221a0 2023-07-21T22:30:14.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f22a3295b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d386117c0 con 0x555d38720a10 2023-07-21T22:30:14.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f22a3295b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d384f5920 con 0x555d38721830 2023-07-21T22:30:14.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f22a3295b80 10 client.0 _ll_register_callbacks cb 0x555d3860f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:14.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f22956ad700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38731580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f228ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38721830 0x555d3872a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f2294eac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38720a10 0x555d3872a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f22956ad700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38731580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48878/0 (socket says 172.21.15.107:48878) 2023-07-21T22:30:14.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f22956ad700 1 -- 172.21.15.107:0/3767508927 learned_addr learned my addr 172.21.15.107:0/3767508927 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f228ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38721830 0x555d3872a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37434/0 (socket says 172.21.15.107:37434) 2023-07-21T22:30:14.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f2294eac700 1 -- 172.21.15.107:0/3767508927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38721830 msgr2=0x555d3872a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f2294eac700 1 --2- 172.21.15.107:0/3767508927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38721830 0x555d3872a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f2294eac700 1 -- 172.21.15.107:0/3767508927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 msgr2=0x555d38731580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f2294eac700 1 --2- 172.21.15.107:0/3767508927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38731580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f2294eac700 1 -- 172.21.15.107:0/3767508927 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2290107780 con 0x555d38720a10 2023-07-21T22:30:14.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f2294eac700 1 --2- 172.21.15.107:0/3767508927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38720a10 0x555d3872a000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2284024af0 tx=0x7f2284005640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=277161856bae2cd9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.203+0000 7f226ffff700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:30:14.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f226ffff700 1 -- 172.21.15.107:0/3767508927 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228402b040 con 0x555d38720a10 2023-07-21T22:30:14.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f22a3295b80 1 -- 172.21.15.107:0/3767508927 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d385cb4f0 con 0x555d38720a10 2023-07-21T22:30:14.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f226ffff700 1 -- 172.21.15.107:0/3767508927 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f228400fde0 con 0x555d38720a10 2023-07-21T22:30:14.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f226ffff700 1 -- 172.21.15.107:0/3767508927 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f228402f6c0 con 0x555d38720a10 2023-07-21T22:30:14.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f22a3295b80 10 client.25268 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:14.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f22a3295b80 1 -- 172.21.15.107:0/3767508927 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555d38488c60 con 0x555d38720a10 2023-07-21T22:30:14.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f226ffff700 1 -- 172.21.15.107:0/3767508927 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d385cb4f0 con 0x555d38720a10 2023-07-21T22:30:14.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f226ffff700 1 -- 172.21.15.107:0/3767508927 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2284039940 con 0x555d38720a10 2023-07-21T22:30:14.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f22a3295b80 10 client.25268 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:14.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f22a3295b80 10 client.25268 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-21T22:30:14.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f22a3295b80 20 client.25268 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:14.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f22a3295b80 1 --2- 172.21.15.107:0/3767508927 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x555d387358b0 0x555d38737c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f22a3295b80 4 client.25268 mds_command: new command op to 24437 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f22a3295b80 1 -- 172.21.15.107:0/3767508927 --> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d3866aeb0 con 0x555d387358b0 2023-07-21T22:30:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.204+0000 7f228ffff700 1 --2- 172.21.15.107:0/3767508927 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x555d387358b0 0x555d38737c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f226ffff700 10 client.25268 ms_handle_connect on v2:172.21.15.132:6832/3293513462 2023-07-21T22:30:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f228ffff700 1 --2- 172.21.15.107:0/3767508927 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x555d387358b0 0x555d38737c90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7ba98ac481067164 server_cookie=4df7504e9796b8c3 in_seq=0 out_seq=0 2023-07-21T22:30:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f226ffff700 1 -- 172.21.15.107:0/3767508927 <== mds.4 v2:172.21.15.132:6832/3293513462 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555d3866aeb0 con 0x555d387358b0 2023-07-21T22:30:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f226ffff700 10 client.25268 handle_command_reply: tid=0 2023-07-21T22:30:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f22a3295b80 1 client.25268 shutdown 2023-07-21T22:30:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f22a3295b80 1 -- 172.21.15.107:0/3767508927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38720a10 msgr2=0x555d3872a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f22a3295b80 1 --2- 172.21.15.107:0/3767508927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38720a10 0x555d3872a000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2284024af0 tx=0x7f2284005640 comp rx=0 tx=0).stop 2023-07-21T22:30:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f22a3295b80 1 -- 172.21.15.107:0/3767508927 shutdown_connections 2023-07-21T22:30:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f22a3295b80 1 --2- 172.21.15.107:0/3767508927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38731580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f22a3295b80 1 --2- 172.21.15.107:0/3767508927 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x555d387358b0 0x555d38737c90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f22a3295b80 1 --2- 172.21.15.107:0/3767508927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38720a10 0x555d3872a000 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f22a3295b80 1 --2- 172.21.15.107:0/3767508927 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38721830 0x555d3872a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f226ffff700 0 client.25268 ms_handle_reset on v2:172.21.15.132:6832/3293513462 2023-07-21T22:30:14.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.205+0000 7f22a3295b80 1 -- 172.21.15.107:0/3767508927 >> 172.21.15.107:0/3767508927 conn(0x555d38715ef0 msgr2=0x555d3871aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.206+0000 7f22a3295b80 1 -- 172.21.15.107:0/3767508927 shutdown_connections 2023-07-21T22:30:14.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.206+0000 7f22a3295b80 1 -- 172.21.15.107:0/3767508927 wait complete. 2023-07-21T22:30:14.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.206+0000 7f22a3295b80 20 client.25268 trim_cache size 0 max 16384 2023-07-21T22:30:14.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f22a3295b80 1 Processor -- start 2023-07-21T22:30:14.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f22a3295b80 1 -- start start 2023-07-21T22:30:14.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f22a3295b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38720a10 0x555d38731e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f22a3295b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38721830 0x555d38731540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f22a3295b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38727fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f22a3295b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d387294b0 con 0x555d387221a0 2023-07-21T22:30:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f22a3295b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d3871cbd0 con 0x555d38721830 2023-07-21T22:30:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f22a3295b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d38725640 con 0x555d38720a10 2023-07-21T22:30:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f22956ad700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38727fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f2294eac700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38720a10 0x555d38731e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f2294eac700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38720a10 0x555d38731e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37440/0 (socket says 172.21.15.107:37440) 2023-07-21T22:30:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f22956ad700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38727fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48884/0 (socket says 172.21.15.107:48884) 2023-07-21T22:30:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f2294eac700 1 -- 172.21.15.107:0/1317816968 learned_addr learned my addr 172.21.15.107:0/1317816968 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f228ffff700 1 --2- 172.21.15.107:0/1317816968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38721830 0x555d38731540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f2294eac700 1 -- 172.21.15.107:0/1317816968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38721830 msgr2=0x555d38731540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f2294eac700 1 --2- 172.21.15.107:0/1317816968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38721830 0x555d38731540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f2294eac700 1 -- 172.21.15.107:0/1317816968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 msgr2=0x555d38727fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f2294eac700 1 --2- 172.21.15.107:0/1317816968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38727fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.219+0000 7f2294eac700 1 -- 172.21.15.107:0/1317816968 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d384c6250 con 0x555d38720a10 2023-07-21T22:30:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.220+0000 7f2294eac700 1 --2- 172.21.15.107:0/1317816968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38720a10 0x555d38731e70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f22840059d0 tx=0x7f228400fb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=656aa421e2d24674 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.220+0000 7f227b7fe700 1 -- 172.21.15.107:0/1317816968 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228402b040 con 0x555d38720a10 2023-07-21T22:30:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.220+0000 7f227b7fe700 1 -- 172.21.15.107:0/1317816968 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2284024910 con 0x555d38720a10 2023-07-21T22:30:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.220+0000 7f227b7fe700 1 -- 172.21.15.107:0/1317816968 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284017710 con 0x555d38720a10 2023-07-21T22:30:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 -- 172.21.15.107:0/1317816968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38720a10 msgr2=0x555d38731e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 --2- 172.21.15.107:0/1317816968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38720a10 0x555d38731e70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f22840059d0 tx=0x7f228400fb10 comp rx=0 tx=0).stop 2023-07-21T22:30:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 -- 172.21.15.107:0/1317816968 shutdown_connections 2023-07-21T22:30:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 --2- 172.21.15.107:0/1317816968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38727fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 --2- 172.21.15.107:0/1317816968 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38720a10 0x555d38731e70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 --2- 172.21.15.107:0/1317816968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38721830 0x555d38731540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 -- 172.21.15.107:0/1317816968 >> 172.21.15.107:0/1317816968 conn(0x555d38715ef0 msgr2=0x555d38720de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 -- 172.21.15.107:0/1317816968 shutdown_connections 2023-07-21T22:30:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 -- 172.21.15.107:0/1317816968 wait complete. 2023-07-21T22:30:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 -- start start 2023-07-21T22:30:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 Processor -- start 2023-07-21T22:30:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38720a10 0x555d3872d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38721830 0x555d3872da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38730a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d38730f70 con 0x555d387221a0 2023-07-21T22:30:14.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d387310e0 con 0x555d38720a10 2023-07-21T22:30:14.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d387312a0 con 0x555d38721830 2023-07-21T22:30:14.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22a3295b80 10 client.0 _ll_register_callbacks cb 0x7f227c005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:14.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.221+0000 7f22956ad700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38730a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f228ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38721830 0x555d3872da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f22956ad700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38730a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48890/0 (socket says 172.21.15.107:48890) 2023-07-21T22:30:14.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f22956ad700 1 -- 172.21.15.107:0/2980843137 learned_addr learned my addr 172.21.15.107:0/2980843137 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f2294eac700 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38720a10 0x555d3872d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f22956ad700 1 -- 172.21.15.107:0/2980843137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38721830 msgr2=0x555d3872da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f22956ad700 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38721830 0x555d3872da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f22956ad700 1 -- 172.21.15.107:0/2980843137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38720a10 msgr2=0x555d3872d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f22956ad700 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38720a10 0x555d3872d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f22956ad700 1 -- 172.21.15.107:0/2980843137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d38488c60 con 0x555d387221a0 2023-07-21T22:30:14.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f228ffff700 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38721830 0x555d3872da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:14.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f2294eac700 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38720a10 0x555d3872d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:14.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f22956ad700 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38730a30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2288003d70 tx=0x7f2288014730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f74e2a707c5ad40 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f226ffff700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:14.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f226ffff700 1 -- 172.21.15.107:0/2980843137 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288023070 con 0x555d387221a0 2023-07-21T22:30:14.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f226ffff700 1 -- 172.21.15.107:0/2980843137 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2288003e00 con 0x555d387221a0 2023-07-21T22:30:14.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f22a3295b80 1 -- 172.21.15.107:0/2980843137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f228401e040 con 0x555d387221a0 2023-07-21T22:30:14.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f226ffff700 1 -- 172.21.15.107:0/2980843137 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22880140d0 con 0x555d387221a0 2023-07-21T22:30:14.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f22a3295b80 10 client.15762 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:14.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f22a3295b80 1 -- 172.21.15.107:0/2980843137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555d3849e2b0 con 0x555d387221a0 2023-07-21T22:30:14.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.222+0000 7f226ffff700 1 -- 172.21.15.107:0/2980843137 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f228802e020 con 0x555d387221a0 2023-07-21T22:30:14.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f226ffff700 1 -- 172.21.15.107:0/2980843137 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2288026020 con 0x555d387221a0 2023-07-21T22:30:14.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f22a3295b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:14.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f22a3295b80 10 client.15762 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-21T22:30:14.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f22a3295b80 20 client.15762 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:14.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f22a3295b80 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x555d3878a990 0x555d3878cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f22a3295b80 4 client.15762 mds_command: new command op to 24437 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:30:14.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f22a3295b80 1 -- 172.21.15.107:0/2980843137 --> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555d3862d050 con 0x555d3878a990 2023-07-21T22:30:14.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f2294eac700 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x555d3878a990 0x555d3878cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f226ffff700 10 client.15762 ms_handle_connect on v2:172.21.15.132:6832/3293513462 2023-07-21T22:30:14.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f2294eac700 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x555d3878a990 0x555d3878cda0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea23a6586c712163 server_cookie=950e0751c3cf7824 in_seq=0 out_seq=0 2023-07-21T22:30:14.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f226ffff700 1 -- 172.21.15.107:0/2980843137 <== mds.4 v2:172.21.15.132:6832/3293513462 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555d3862d050 con 0x555d3878a990 2023-07-21T22:30:14.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f226ffff700 10 client.15762 handle_command_reply: tid=0 2023-07-21T22:30:14.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f22a3295b80 1 client.15762 shutdown 2023-07-21T22:30:14.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f22a3295b80 1 -- 172.21.15.107:0/2980843137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 msgr2=0x555d38730a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.223+0000 7f22a3295b80 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38730a30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2288003d70 tx=0x7f2288014730 comp rx=0 tx=0).stop 2023-07-21T22:30:14.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f22a3295b80 1 -- 172.21.15.107:0/2980843137 shutdown_connections 2023-07-21T22:30:14.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f22a3295b80 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d387221a0 0x555d38730a30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f22a3295b80 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x555d3878a990 0x555d3878cda0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f22a3295b80 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d38720a10 0x555d3872d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f226ffff700 0 client.15762 ms_handle_reset on v2:172.21.15.132:6832/3293513462 2023-07-21T22:30:14.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f22a3295b80 1 --2- 172.21.15.107:0/2980843137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d38721830 0x555d3872da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f22a3295b80 1 -- 172.21.15.107:0/2980843137 >> 172.21.15.107:0/2980843137 conn(0x555d38715ef0 msgr2=0x555d38731e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f22a3295b80 1 -- 172.21.15.107:0/2980843137 shutdown_connections 2023-07-21T22:30:14.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f22a3295b80 1 -- 172.21.15.107:0/2980843137 wait complete. 2023-07-21T22:30:14.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f22a3295b80 20 client.15762 trim_cache size 0 max 16384 2023-07-21T22:30:14.298 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:30:14.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f2297110700 1 -- 172.21.15.107:0/902760592 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f22800a1150 msgr2=0x7f22800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f2297110700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f22800a1150 0x7f22800a3600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2284006010 tx=0x7f2284005960 comp rx=0 tx=0).stop 2023-07-21T22:30:14.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f2297110700 1 -- 172.21.15.107:0/902760592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900ffc00 msgr2=0x7f2290194d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f2297110700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900ffc00 0x7f2290194d30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f227c00e9a0 tx=0x7f227c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f2297110700 1 -- 172.21.15.107:0/902760592 shutdown_connections 2023-07-21T22:30:14.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f2297110700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f22800a1150 0x7f22800a3600 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f2297110700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2290100590 0x7f229019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f2297110700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22900fea10 0x7f22901947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f2297110700 1 --2- 172.21.15.107:0/902760592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22900ffc00 0x7f2290194d30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f2297110700 1 -- 172.21.15.107:0/902760592 >> 172.21.15.107:0/902760592 conn(0x7f22900fa030 msgr2=0x7f22901030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f2297110700 1 -- 172.21.15.107:0/902760592 shutdown_connections 2023-07-21T22:30:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.224+0000 7f2297110700 1 -- 172.21.15.107:0/902760592 wait complete. 2023-07-21T22:30:14.302 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-21T22:30:14.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.387+0000 7fe6b4143700 1 -- 172.21.15.107:0/292944329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6ac0fef10 msgr2=0x7fe6ac0ff320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.387+0000 7fe6b4143700 1 --2- 172.21.15.107:0/292944329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6ac0fef10 0x7fe6ac0ff320 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009990 tx=0x7fe6a8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.387+0000 7fe6b4143700 1 -- 172.21.15.107:0/292944329 shutdown_connections 2023-07-21T22:30:14.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 --2- 172.21.15.107:0/292944329 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6ac100670 0x7fe6ac106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 --2- 172.21.15.107:0/292944329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6ac0fef10 0x7fe6ac0ff320 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 --2- 172.21.15.107:0/292944329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6ac0ffcc0 0x7fe6ac100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 -- 172.21.15.107:0/292944329 >> 172.21.15.107:0/292944329 conn(0x7fe6ac0f9770 msgr2=0x7fe6ac0fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 -- 172.21.15.107:0/292944329 shutdown_connections 2023-07-21T22:30:14.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 -- 172.21.15.107:0/292944329 wait complete. 2023-07-21T22:30:14.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 Processor -- start 2023-07-21T22:30:14.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 -- start start 2023-07-21T22:30:14.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6ac0fef10 0x7fe6ac0fe670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6ac0ffcc0 0x7fe6ac19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6ac100670 0x7fe6ac19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac19f570 con 0x7fe6ac100670 2023-07-21T22:30:14.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac19f6e0 con 0x7fe6ac0fef10 2023-07-21T22:30:14.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b4143700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac19f8a0 con 0x7fe6ac0ffcc0 2023-07-21T22:30:14.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.388+0000 7fe6b26e0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6ac100670 0x7fe6ac19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6b26e0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6ac100670 0x7fe6ac19eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48904/0 (socket says 172.21.15.107:48904) 2023-07-21T22:30:14.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6b26e0700 1 -- 172.21.15.107:0/2837311684 learned_addr learned my addr 172.21.15.107:0/2837311684 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6b16de700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6ac0ffcc0 0x7fe6ac19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6b1edf700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6ac0fef10 0x7fe6ac0fe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6b26e0700 1 -- 172.21.15.107:0/2837311684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6ac0ffcc0 msgr2=0x7fe6ac19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6b26e0700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6ac0ffcc0 0x7fe6ac19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6b26e0700 1 -- 172.21.15.107:0/2837311684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6ac0fef10 msgr2=0x7fe6ac0fe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6b26e0700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6ac0fef10 0x7fe6ac0fe670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6b26e0700 1 -- 172.21.15.107:0/2837311684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a80095f0 con 0x7fe6ac100670 2023-07-21T22:30:14.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6b26e0700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6ac100670 0x7fe6ac19eec0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400ef30 tx=0x7fe6a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6a2ffd700 1 -- 172.21.15.107:0/2837311684 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4018070 con 0x7fe6ac100670 2023-07-21T22:30:14.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6a2ffd700 1 -- 172.21.15.107:0/2837311684 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6a400f040 con 0x7fe6ac100670 2023-07-21T22:30:14.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6a2ffd700 1 -- 172.21.15.107:0/2837311684 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a400cd00 con 0x7fe6ac100670 2023-07-21T22:30:14.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6b4143700 1 -- 172.21.15.107:0/2837311684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ac19fb20 con 0x7fe6ac100670 2023-07-21T22:30:14.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.389+0000 7fe6b4143700 1 -- 172.21.15.107:0/2837311684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ac19ffe0 con 0x7fe6ac100670 2023-07-21T22:30:14.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.391+0000 7fe6a2ffd700 1 -- 172.21.15.107:0/2837311684 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe6a4003a00 con 0x7fe6ac100670 2023-07-21T22:30:14.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:14 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2374028356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:30:14.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.392+0000 7fe6a2ffd700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe6980a1250 0x7fe6980a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.392+0000 7fe6a2ffd700 1 -- 172.21.15.107:0/2837311684 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe6a40cc440 con 0x7fe6ac100670 2023-07-21T22:30:14.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.392+0000 7fe6b1edf700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe6980a1250 0x7fe6980a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.392+0000 7fe6b1edf700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe6980a1250 0x7fe6980a3700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800b2e0 tx=0x7fe6a8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.393+0000 7fe6c02c8b80 1 Processor -- start 2023-07-21T22:30:14.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.393+0000 7fe6c02c8b80 1 -- start start 2023-07-21T22:30:14.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.393+0000 7fe6c02c8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 0x55fe3dec5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.393+0000 7fe6c02c8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec61a0 0x55fe3dec4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.393+0000 7fe6c02c8b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 0x55fe3dec4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.393+0000 7fe6c02c8b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3ddc9cb0 con 0x55fe3dec4a10 2023-07-21T22:30:14.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.393+0000 7fe6c02c8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3de1ca40 con 0x55fe3dec61a0 2023-07-21T22:30:14.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.393+0000 7fe6c02c8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3de15ed0 con 0x55fe3dec5830 2023-07-21T22:30:14.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.393+0000 7fe6b26e0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 0x55fe3dec4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.393+0000 7fe6b26e0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 0x55fe3dec4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48912/0 (socket says 172.21.15.107:48912) 2023-07-21T22:30:14.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.393+0000 7fe6b26e0700 1 -- 172.21.15.107:0/3948235612 learned_addr learned my addr 172.21.15.107:0/3948235612 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.393+0000 7fe6b1edf700 1 --2- 172.21.15.107:0/3948235612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 0x55fe3dec5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.394+0000 7fe6b16de700 1 --2- 172.21.15.107:0/3948235612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec61a0 0x55fe3dec4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.394+0000 7fe6b26e0700 1 -- 172.21.15.107:0/3948235612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 msgr2=0x55fe3dec5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.394+0000 7fe6b26e0700 1 --2- 172.21.15.107:0/3948235612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 0x55fe3dec5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.394+0000 7fe6b26e0700 1 -- 172.21.15.107:0/3948235612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec61a0 msgr2=0x55fe3dec4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.394+0000 7fe6b26e0700 1 --2- 172.21.15.107:0/3948235612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec61a0 0x55fe3dec4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.394+0000 7fe6b26e0700 1 -- 172.21.15.107:0/3948235612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe3dc43de0 con 0x55fe3dec4a10 2023-07-21T22:30:14.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.394+0000 7fe6b26e0700 1 --2- 172.21.15.107:0/3948235612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 0x55fe3dec4e90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4095d60 tx=0x7fe6a40993a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a47a706a35b032 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.394+0000 7fe697fff700 1 -- 172.21.15.107:0/3948235612 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4004ad0 con 0x55fe3dec4a10 2023-07-21T22:30:14.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.394+0000 7fe697fff700 1 -- 172.21.15.107:0/3948235612 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6a4099860 con 0x55fe3dec4a10 2023-07-21T22:30:14.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.394+0000 7fe697fff700 1 -- 172.21.15.107:0/3948235612 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40b9070 con 0x55fe3dec4a10 2023-07-21T22:30:14.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.395+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/3948235612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 msgr2=0x55fe3dec4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.395+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/3948235612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 0x55fe3dec4e90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4095d60 tx=0x7fe6a40993a0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.395+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/3948235612 shutdown_connections 2023-07-21T22:30:14.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.395+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/3948235612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 0x55fe3dec4e90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.395+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/3948235612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 0x55fe3dec5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.395+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/3948235612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec61a0 0x55fe3dec4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.395+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/3948235612 >> 172.21.15.107:0/3948235612 conn(0x55fe3deb9ef0 msgr2=0x55fe3debf360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.395+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/3948235612 shutdown_connections 2023-07-21T22:30:14.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.395+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/3948235612 wait complete. 2023-07-21T22:30:14.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.395+0000 7fe6c02c8b80 1 -- start start 2023-07-21T22:30:14.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.395+0000 7fe6c02c8b80 1 Processor -- start 2023-07-21T22:30:14.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6c02c8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec4a10 0x55fe3dece010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6c02c8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec5830 0x55fe3dece550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6c02c8b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 0x55fe3ded5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6c02c8b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3ddb5ac0 con 0x55fe3dec61a0 2023-07-21T22:30:14.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6c02c8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3ddb57c0 con 0x55fe3dec5830 2023-07-21T22:30:14.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6c02c8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3dc99920 con 0x55fe3dec4a10 2023-07-21T22:30:14.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6c02c8b80 10 client.0 _ll_register_callbacks cb 0x55fe3ddb3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:14.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6b26e0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 0x55fe3ded5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.452 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6b26e0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 0x55fe3ded5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48918/0 (socket says 172.21.15.107:48918) 2023-07-21T22:30:14.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6b26e0700 1 -- 172.21.15.107:0/1422324862 learned_addr learned my addr 172.21.15.107:0/1422324862 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6b1edf700 1 --2- 172.21.15.107:0/1422324862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec4a10 0x55fe3dece010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6b16de700 1 --2- 172.21.15.107:0/1422324862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec5830 0x55fe3dece550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6b26e0700 1 -- 172.21.15.107:0/1422324862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec4a10 msgr2=0x55fe3dece010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6b26e0700 1 --2- 172.21.15.107:0/1422324862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec4a10 0x55fe3dece010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6b26e0700 1 -- 172.21.15.107:0/1422324862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec5830 msgr2=0x55fe3dece550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6b26e0700 1 --2- 172.21.15.107:0/1422324862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec5830 0x55fe3dece550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6b26e0700 1 -- 172.21.15.107:0/1422324862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ac19ffe0 con 0x55fe3dec61a0 2023-07-21T22:30:14.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6b26e0700 1 --2- 172.21.15.107:0/1422324862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 0x55fe3ded5590 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40a16f0 tx=0x7fe6a40b93d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb325dc3e28e3a75 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6957fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:14.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6957fa700 1 -- 172.21.15.107:0/1422324862 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40a1c20 con 0x55fe3dec61a0 2023-07-21T22:30:14.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6957fa700 1 -- 172.21.15.107:0/1422324862 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe6a4099cd0 con 0x55fe3dec61a0 2023-07-21T22:30:14.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.396+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/1422324862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe3dd6f4f0 con 0x55fe3dec61a0 2023-07-21T22:30:14.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6957fa700 1 -- 172.21.15.107:0/1422324862 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6a4098d50 con 0x55fe3dec61a0 2023-07-21T22:30:14.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6c02c8b80 10 client.15774 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:14.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/1422324862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fe3dc2cc60 con 0x55fe3dec61a0 2023-07-21T22:30:14.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6957fa700 1 -- 172.21.15.107:0/1422324862 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe3dd6f4f0 con 0x55fe3dec61a0 2023-07-21T22:30:14.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6957fa700 1 -- 172.21.15.107:0/1422324862 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe6a409dc10 con 0x55fe3dec61a0 2023-07-21T22:30:14.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6c02c8b80 10 client.15774 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:14.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6c02c8b80 10 client.15774 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T22:30:14.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6c02c8b80 20 client.15774 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:14.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/1422324862 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55fe3ded9960 0x55fe3dedbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6c02c8b80 4 client.15774 mds_command: new command op to 24455 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:14.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/1422324862 --> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe3de0eeb0 con 0x55fe3ded9960 2023-07-21T22:30:14.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6b1edf700 1 --2- 172.21.15.107:0/1422324862 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55fe3ded9960 0x55fe3dedbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6957fa700 10 client.15774 ms_handle_connect on v2:172.21.15.132:6834/1022724305 2023-07-21T22:30:14.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.397+0000 7fe6b1edf700 1 --2- 172.21.15.107:0/1422324862 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55fe3ded9960 0x55fe3dedbd70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c984225addc9de80 server_cookie=6a53d5c522a14f41 in_seq=0 out_seq=0 2023-07-21T22:30:14.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6957fa700 1 -- 172.21.15.107:0/1422324862 <== mds.3 v2:172.21.15.132:6834/1022724305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fe3de0eeb0 con 0x55fe3ded9960 2023-07-21T22:30:14.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6957fa700 10 client.15774 handle_command_reply: tid=0 2023-07-21T22:30:14.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6c02c8b80 1 client.15774 shutdown 2023-07-21T22:30:14.457 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/1422324862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 msgr2=0x55fe3ded5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.457 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/1422324862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 0x55fe3ded5590 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40a16f0 tx=0x7fe6a40b93d0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.457 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/1422324862 shutdown_connections 2023-07-21T22:30:14.457 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/1422324862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 0x55fe3ded5590 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.457 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/1422324862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec4a10 0x55fe3dece010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.457 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/1422324862 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55fe3ded9960 0x55fe3dedbd70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.457 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/1422324862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec5830 0x55fe3dece550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.458 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6957fa700 0 client.15774 ms_handle_reset on v2:172.21.15.132:6834/1022724305 2023-07-21T22:30:14.458 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/1422324862 >> 172.21.15.107:0/1422324862 conn(0x55fe3deb9ef0 msgr2=0x55fe3debea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.458 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/1422324862 shutdown_connections 2023-07-21T22:30:14.458 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.398+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/1422324862 wait complete. 2023-07-21T22:30:14.458 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.399+0000 7fe6c02c8b80 20 client.15774 trim_cache size 0 max 16384 2023-07-21T22:30:14.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.411+0000 7fe6c02c8b80 1 Processor -- start 2023-07-21T22:30:14.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.411+0000 7fe6c02c8b80 1 -- start start 2023-07-21T22:30:14.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.411+0000 7fe6c02c8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec4a10 0x55fe3dec4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.411+0000 7fe6c02c8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 0x55fe3ded5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.411+0000 7fe6c02c8b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 0x55fe3dec8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.411+0000 7fe6c02c8b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3decdbe0 con 0x55fe3dec61a0 2023-07-21T22:30:14.463 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.411+0000 7fe6c02c8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3dec0060 con 0x55fe3dec4a10 2023-07-21T22:30:14.464 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.411+0000 7fe6c02c8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3dec9640 con 0x55fe3dec5830 2023-07-21T22:30:14.464 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.411+0000 7fe6b26e0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 0x55fe3dec8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.464 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6b26e0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 0x55fe3dec8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48926/0 (socket says 172.21.15.107:48926) 2023-07-21T22:30:14.464 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6b26e0700 1 -- 172.21.15.107:0/2756158553 learned_addr learned my addr 172.21.15.107:0/2756158553 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.464 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6b1edf700 1 --2- 172.21.15.107:0/2756158553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec4a10 0x55fe3dec4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.464 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6b26e0700 1 -- 172.21.15.107:0/2756158553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 msgr2=0x55fe3ded5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.464 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6b26e0700 1 --2- 172.21.15.107:0/2756158553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 0x55fe3ded5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6b26e0700 1 -- 172.21.15.107:0/2756158553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec4a10 msgr2=0x55fe3dec4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6b26e0700 1 --2- 172.21.15.107:0/2756158553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec4a10 0x55fe3dec4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6b26e0700 1 -- 172.21.15.107:0/2756158553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe3dc6a250 con 0x55fe3dec61a0 2023-07-21T22:30:14.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6b26e0700 1 --2- 172.21.15.107:0/2756158553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 0x55fe3dec8ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4099330 tx=0x7fe6a4004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f054a49cca04d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6a0ff9700 1 -- 172.21.15.107:0/2756158553 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40956a0 con 0x55fe3dec61a0 2023-07-21T22:30:14.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6a0ff9700 1 -- 172.21.15.107:0/2756158553 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6a40a1c20 con 0x55fe3dec61a0 2023-07-21T22:30:14.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6a0ff9700 1 -- 172.21.15.107:0/2756158553 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4098630 con 0x55fe3dec61a0 2023-07-21T22:30:14.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/2756158553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 msgr2=0x55fe3dec8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.412+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/2756158553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 0x55fe3dec8ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4099330 tx=0x7fe6a4004dc0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/2756158553 shutdown_connections 2023-07-21T22:30:14.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/2756158553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec61a0 0x55fe3dec8ad0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/2756158553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec4a10 0x55fe3dec4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/2756158553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 0x55fe3ded5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/2756158553 >> 172.21.15.107:0/2756158553 conn(0x55fe3deb9ef0 msgr2=0x55fe3decccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/2756158553 shutdown_connections 2023-07-21T22:30:14.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/2756158553 wait complete. 2023-07-21T22:30:14.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 -- start start 2023-07-21T22:30:14.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 Processor -- start 2023-07-21T22:30:14.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 0x55fe3ded36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 0x55fe3ded3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec61a0 0x55fe3ded4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3ded4970 con 0x55fe3dec4a10 2023-07-21T22:30:14.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3ded4ae0 con 0x55fe3dec61a0 2023-07-21T22:30:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3ded4ca0 con 0x55fe3dec5830 2023-07-21T22:30:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6c02c8b80 10 client.0 _ll_register_callbacks cb 0x7fe6a800b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6b1edf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 0x55fe3ded36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6b1edf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 0x55fe3ded36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48928/0 (socket says 172.21.15.107:48928) 2023-07-21T22:30:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6b1edf700 1 -- 172.21.15.107:0/415206517 learned_addr learned my addr 172.21.15.107:0/415206517 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.413+0000 7fe6b26e0700 1 --2- 172.21.15.107:0/415206517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec61a0 0x55fe3ded4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6b16de700 1 --2- 172.21.15.107:0/415206517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 0x55fe3ded3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6b1edf700 1 -- 172.21.15.107:0/415206517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 msgr2=0x55fe3ded3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6b1edf700 1 --2- 172.21.15.107:0/415206517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 0x55fe3ded3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6b1edf700 1 -- 172.21.15.107:0/415206517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec61a0 msgr2=0x55fe3ded4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6b1edf700 1 --2- 172.21.15.107:0/415206517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec61a0 0x55fe3ded4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6b1edf700 1 -- 172.21.15.107:0/415206517 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe3dc2cc60 con 0x55fe3dec4a10 2023-07-21T22:30:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6b1edf700 1 --2- 172.21.15.107:0/415206517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 0x55fe3ded36f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009990 tx=0x7fe6a801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efcd87957dd210f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6957fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6957fa700 1 -- 172.21.15.107:0/415206517 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a802d040 con 0x55fe3dec4a10 2023-07-21T22:30:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/415206517 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6a4099860 con 0x55fe3dec4a10 2023-07-21T22:30:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6957fa700 1 -- 172.21.15.107:0/415206517 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe6a8034bd0 con 0x55fe3dec4a10 2023-07-21T22:30:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6957fa700 1 -- 172.21.15.107:0/415206517 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6a8034dd0 con 0x55fe3dec4a10 2023-07-21T22:30:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6c02c8b80 10 client.15780 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/415206517 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55fe3dc422b0 con 0x55fe3dec4a10 2023-07-21T22:30:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6957fa700 1 -- 172.21.15.107:0/415206517 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6a8043020 con 0x55fe3dec4a10 2023-07-21T22:30:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.414+0000 7fe6957fa700 1 -- 172.21.15.107:0/415206517 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe6a800f730 con 0x55fe3dec4a10 2023-07-21T22:30:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.415+0000 7fe6c02c8b80 10 client.15780 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.415+0000 7fe6c02c8b80 10 client.15780 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T22:30:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.415+0000 7fe6c02c8b80 20 client.15780 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.415+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/415206517 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55fe3df2e2c0 0x55fe3df306d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.415+0000 7fe6c02c8b80 4 client.15780 mds_command: new command op to 24455 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:30:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.415+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/415206517 --> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fe3ddd1050 con 0x55fe3df2e2c0 2023-07-21T22:30:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.415+0000 7fe6b16de700 1 --2- 172.21.15.107:0/415206517 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55fe3df2e2c0 0x55fe3df306d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.415+0000 7fe6b16de700 1 --2- 172.21.15.107:0/415206517 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55fe3df2e2c0 0x55fe3df306d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ac89664f2f14be01 server_cookie=66a633df3897ccff in_seq=0 out_seq=0 2023-07-21T22:30:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.415+0000 7fe6957fa700 10 client.15780 ms_handle_connect on v2:172.21.15.132:6834/1022724305 2023-07-21T22:30:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.415+0000 7fe6957fa700 1 -- 172.21.15.107:0/415206517 <== mds.3 v2:172.21.15.132:6834/1022724305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fe3ddd1050 con 0x55fe3df2e2c0 2023-07-21T22:30:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.415+0000 7fe6957fa700 10 client.15780 handle_command_reply: tid=0 2023-07-21T22:30:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.415+0000 7fe6c02c8b80 1 client.15780 shutdown 2023-07-21T22:30:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/415206517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 msgr2=0x55fe3ded36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/415206517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 0x55fe3ded36f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009990 tx=0x7fe6a801a050 comp rx=0 tx=0).stop 2023-07-21T22:30:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/415206517 shutdown_connections 2023-07-21T22:30:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/415206517 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55fe3df2e2c0 0x55fe3df306d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/415206517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55fe3dec61a0 0x55fe3ded4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/415206517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fe3dec4a10 0x55fe3ded36f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6c02c8b80 1 --2- 172.21.15.107:0/415206517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fe3dec5830 0x55fe3ded3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6957fa700 0 client.15780 ms_handle_reset on v2:172.21.15.132:6834/1022724305 2023-07-21T22:30:14.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/415206517 >> 172.21.15.107:0/415206517 conn(0x55fe3deb9ef0 msgr2=0x55fe3dec8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/415206517 shutdown_connections 2023-07-21T22:30:14.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6c02c8b80 1 -- 172.21.15.107:0/415206517 wait complete. 2023-07-21T22:30:14.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6c02c8b80 20 client.15780 trim_cache size 0 max 16384 2023-07-21T22:30:14.474 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:30:14.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6b4143700 1 -- 172.21.15.107:0/2837311684 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe6980a1250 msgr2=0x7fe6980a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6b4143700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe6980a1250 0x7fe6980a3700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800b2e0 tx=0x7fe6a8005fb0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6b4143700 1 -- 172.21.15.107:0/2837311684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6ac100670 msgr2=0x7fe6ac19eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6b4143700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6ac100670 0x7fe6ac19eec0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400ef30 tx=0x7fe6a400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6b4143700 1 -- 172.21.15.107:0/2837311684 shutdown_connections 2023-07-21T22:30:14.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6b4143700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6ac100670 0x7fe6ac19eec0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6b4143700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6ac0fef10 0x7fe6ac0fe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6b4143700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe6980a1250 0x7fe6980a3700 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6b4143700 1 --2- 172.21.15.107:0/2837311684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6ac0ffcc0 0x7fe6ac19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6b4143700 1 -- 172.21.15.107:0/2837311684 >> 172.21.15.107:0/2837311684 conn(0x7fe6ac0f9770 msgr2=0x7fe6ac1031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6b4143700 1 -- 172.21.15.107:0/2837311684 shutdown_connections 2023-07-21T22:30:14.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.416+0000 7fe6b4143700 1 -- 172.21.15.107:0/2837311684 wait complete. 2023-07-21T22:30:14.477 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:30:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.553+0000 7f4169cf2700 1 -- 172.21.15.107:0/2726887628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41640fe9e0 msgr2=0x7f41640fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.553+0000 7f4169cf2700 1 --2- 172.21.15.107:0/2726887628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41640fe9e0 0x7f41640fedf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4158009990 tx=0x7f4158009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.553+0000 7f4169cf2700 1 -- 172.21.15.107:0/2726887628 shutdown_connections 2023-07-21T22:30:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.553+0000 7f4169cf2700 1 --2- 172.21.15.107:0/2726887628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4164100560 0x7f4164106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.553+0000 7f4169cf2700 1 --2- 172.21.15.107:0/2726887628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41640fe9e0 0x7f41640fedf0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.553+0000 7f4169cf2700 1 --2- 172.21.15.107:0/2726887628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41640ffbd0 0x7f4164100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.553+0000 7f4169cf2700 1 -- 172.21.15.107:0/2726887628 >> 172.21.15.107:0/2726887628 conn(0x7f41640fa000 msgr2=0x7f41640fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.553+0000 7f4169cf2700 1 -- 172.21.15.107:0/2726887628 shutdown_connections 2023-07-21T22:30:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.553+0000 7f4169cf2700 1 -- 172.21.15.107:0/2726887628 wait complete. 2023-07-21T22:30:14.603 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.554+0000 7f4169cf2700 1 Processor -- start 2023-07-21T22:30:14.603 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.554+0000 7f4169cf2700 1 -- start start 2023-07-21T22:30:14.603 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.554+0000 7f4169cf2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41640fe9e0 0x7f41641092d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.603 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.554+0000 7f4169cf2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41640ffbd0 0x7f4164107910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.603 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.554+0000 7f4169cf2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4164100560 0x7f4164107e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.554+0000 7f4169cf2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41641085f0 con 0x7f41640fe9e0 2023-07-21T22:30:14.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.554+0000 7f4169cf2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4164108760 con 0x7f41640ffbd0 2023-07-21T22:30:14.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.554+0000 7f4169cf2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416419f8e0 con 0x7f4164100560 2023-07-21T22:30:14.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.554+0000 7f41637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41640fe9e0 0x7f41641092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.554+0000 7f41637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41640fe9e0 0x7f41641092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48942/0 (socket says 172.21.15.107:48942) 2023-07-21T22:30:14.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.554+0000 7f41637fe700 1 -- 172.21.15.107:0/2816369887 learned_addr learned my addr 172.21.15.107:0/2816369887 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.554+0000 7f4162ffd700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41640ffbd0 0x7f4164107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.555+0000 7f41637fe700 1 -- 172.21.15.107:0/2816369887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4164100560 msgr2=0x7f4164107e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.555+0000 7f4163fff700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4164100560 0x7f4164107e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.555+0000 7f41637fe700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4164100560 0x7f4164107e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.555+0000 7f41637fe700 1 -- 172.21.15.107:0/2816369887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41640ffbd0 msgr2=0x7f4164107910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.555+0000 7f41637fe700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41640ffbd0 0x7f4164107910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.555+0000 7f41637fe700 1 -- 172.21.15.107:0/2816369887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580095f0 con 0x7f41640fe9e0 2023-07-21T22:30:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.555+0000 7f4162ffd700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41640ffbd0 0x7f4164107910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:14.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.555+0000 7f41637fe700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41640fe9e0 0x7f41641092d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4158000c00 tx=0x7f4158009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.555+0000 7f4160ff9700 1 -- 172.21.15.107:0/2816369887 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415801d070 con 0x7f41640fe9e0 2023-07-21T22:30:14.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.555+0000 7f4160ff9700 1 -- 172.21.15.107:0/2816369887 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4158004180 con 0x7f41640fe9e0 2023-07-21T22:30:14.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.555+0000 7f4169cf2700 1 -- 172.21.15.107:0/2816369887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416419fb60 con 0x7f41640fe9e0 2023-07-21T22:30:14.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.555+0000 7f4160ff9700 1 -- 172.21.15.107:0/2816369887 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158022e00 con 0x7f41640fe9e0 2023-07-21T22:30:14.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.555+0000 7f4169cf2700 1 -- 172.21.15.107:0/2816369887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41641a0010 con 0x7f41640fe9e0 2023-07-21T22:30:14.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.557+0000 7f4160ff9700 1 -- 172.21.15.107:0/2816369887 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4158022500 con 0x7f41640fe9e0 2023-07-21T22:30:14.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.558+0000 7f4160ff9700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f414c0a1150 0x7f414c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.558+0000 7f4162ffd700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f414c0a1150 0x7f414c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.558+0000 7f4160ff9700 1 -- 172.21.15.107:0/2816369887 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f41580cdb90 con 0x7f41640fe9e0 2023-07-21T22:30:14.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.558+0000 7f4162ffd700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f414c0a1150 0x7f414c0a3600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4154005950 tx=0x7f41540058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4175e77b80 1 Processor -- start 2023-07-21T22:30:14.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4175e77b80 1 -- start start 2023-07-21T22:30:14.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4175e77b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016b830 0x55d9c016bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4175e77b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016c1a0 0x55d9c016a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4175e77b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c016ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4175e77b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c006fcb0 con 0x55d9c016aa10 2023-07-21T22:30:14.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4175e77b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c00c2a40 con 0x55d9c016c1a0 2023-07-21T22:30:14.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4175e77b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c00bbed0 con 0x55d9c016b830 2023-07-21T22:30:14.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4163fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c016ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4163fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c016ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48954/0 (socket says 172.21.15.107:48954) 2023-07-21T22:30:14.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4163fff700 1 -- 172.21.15.107:0/3381684564 learned_addr learned my addr 172.21.15.107:0/3381684564 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f41637fe700 1 --2- 172.21.15.107:0/3381684564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016b830 0x55d9c016bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4162ffd700 1 --2- 172.21.15.107:0/3381684564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016c1a0 0x55d9c016a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4163fff700 1 -- 172.21.15.107:0/3381684564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016b830 msgr2=0x55d9c016bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4163fff700 1 --2- 172.21.15.107:0/3381684564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016b830 0x55d9c016bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4163fff700 1 -- 172.21.15.107:0/3381684564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016c1a0 msgr2=0x55d9c016a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4163fff700 1 --2- 172.21.15.107:0/3381684564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016c1a0 0x55d9c016a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4163fff700 1 -- 172.21.15.107:0/3381684564 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9bfee9de0 con 0x55d9c016aa10 2023-07-21T22:30:14.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4162ffd700 1 --2- 172.21.15.107:0/3381684564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016c1a0 0x55d9c016a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:30:14.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.559+0000 7f4163fff700 1 --2- 172.21.15.107:0/3381684564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c016ae90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f415c009790 tx=0x7f415c009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b86665d1b42eb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.560+0000 7f4149ffb700 1 -- 172.21.15.107:0/3381684564 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c004bb0 con 0x55d9c016aa10 2023-07-21T22:30:14.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.560+0000 7f4149ffb700 1 -- 172.21.15.107:0/3381684564 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f415c004d10 con 0x55d9c016aa10 2023-07-21T22:30:14.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.560+0000 7f4149ffb700 1 -- 172.21.15.107:0/3381684564 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c014660 con 0x55d9c016aa10 2023-07-21T22:30:14.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.560+0000 7f4175e77b80 1 -- 172.21.15.107:0/3381684564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 msgr2=0x55d9c016ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.560+0000 7f4175e77b80 1 --2- 172.21.15.107:0/3381684564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c016ae90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f415c009790 tx=0x7f415c009f20 comp rx=0 tx=0).stop 2023-07-21T22:30:14.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.560+0000 7f4175e77b80 1 -- 172.21.15.107:0/3381684564 shutdown_connections 2023-07-21T22:30:14.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.560+0000 7f4175e77b80 1 --2- 172.21.15.107:0/3381684564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c016ae90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.560+0000 7f4175e77b80 1 --2- 172.21.15.107:0/3381684564 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016b830 0x55d9c016bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.560+0000 7f4175e77b80 1 --2- 172.21.15.107:0/3381684564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016c1a0 0x55d9c016a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.560+0000 7f4175e77b80 1 -- 172.21.15.107:0/3381684564 >> 172.21.15.107:0/3381684564 conn(0x55d9c015fef0 msgr2=0x55d9c0165360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.560+0000 7f4175e77b80 1 -- 172.21.15.107:0/3381684564 shutdown_connections 2023-07-21T22:30:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.560+0000 7f4175e77b80 1 -- 172.21.15.107:0/3381684564 wait complete. 2023-07-21T22:30:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f4175e77b80 1 -- start start 2023-07-21T22:30:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f4175e77b80 1 Processor -- start 2023-07-21T22:30:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f4175e77b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016aa10 0x55d9c0174040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f4175e77b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016b830 0x55d9c0174580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f4175e77b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016c1a0 0x55d9c017b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f4175e77b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c005bac0 con 0x55d9c016b830 2023-07-21T22:30:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f4175e77b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c005b7c0 con 0x55d9c016c1a0 2023-07-21T22:30:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f4175e77b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9bff3f920 con 0x55d9c016aa10 2023-07-21T22:30:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f4175e77b80 10 client.0 _ll_register_callbacks cb 0x55d9c0059760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f4162ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016b830 0x55d9c0174580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f4163fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016c1a0 0x55d9c017b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f4162ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016b830 0x55d9c0174580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48958/0 (socket says 172.21.15.107:48958) 2023-07-21T22:30:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f4162ffd700 1 -- 172.21.15.107:0/497211021 learned_addr learned my addr 172.21.15.107:0/497211021 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.561+0000 7f41637fe700 1 --2- 172.21.15.107:0/497211021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016aa10 0x55d9c0174040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4163fff700 1 -- 172.21.15.107:0/497211021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016aa10 msgr2=0x55d9c0174040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4163fff700 1 --2- 172.21.15.107:0/497211021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016aa10 0x55d9c0174040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4163fff700 1 -- 172.21.15.107:0/497211021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016b830 msgr2=0x55d9c0174580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4163fff700 1 --2- 172.21.15.107:0/497211021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016b830 0x55d9c0174580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4163fff700 1 -- 172.21.15.107:0/497211021 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415c009880 con 0x55d9c016c1a0 2023-07-21T22:30:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4163fff700 1 --2- 172.21.15.107:0/497211021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016c1a0 0x55d9c017b5c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f415c009550 tx=0x7f415c005710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad46008ae8a23d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4142ffd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:30:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4142ffd700 1 -- 172.21.15.107:0/497211021 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c01e660 con 0x55d9c016c1a0 2023-07-21T22:30:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4175e77b80 1 -- 172.21.15.107:0/497211021 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9c00154f0 con 0x55d9c016c1a0 2023-07-21T22:30:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4142ffd700 1 -- 172.21.15.107:0/497211021 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f415c01e7c0 con 0x55d9c016c1a0 2023-07-21T22:30:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4142ffd700 1 -- 172.21.15.107:0/497211021 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f415c01e980 con 0x55d9c016c1a0 2023-07-21T22:30:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4175e77b80 10 client.25295 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4175e77b80 1 -- 172.21.15.107:0/497211021 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d9bfed2c60 con 0x55d9c016c1a0 2023-07-21T22:30:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4142ffd700 1 -- 172.21.15.107:0/497211021 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d9c00154f0 con 0x55d9c016c1a0 2023-07-21T22:30:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4142ffd700 1 -- 172.21.15.107:0/497211021 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f415c01c040 con 0x55d9c016c1a0 2023-07-21T22:30:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4175e77b80 10 client.25295 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4175e77b80 10 client.25295 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-07-21T22:30:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4175e77b80 20 client.25295 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4175e77b80 1 --2- 172.21.15.107:0/497211021 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x55d9c017fd40 0x55d9c0182150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4175e77b80 4 client.25295 mds_command: new command op to 24466 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.562+0000 7f4175e77b80 1 -- 172.21.15.107:0/497211021 --> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9c00b4eb0 con 0x55d9c017fd40 2023-07-21T22:30:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.563+0000 7f41637fe700 1 --2- 172.21.15.107:0/497211021 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x55d9c017fd40 0x55d9c0182150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.563+0000 7f4142ffd700 10 client.25295 ms_handle_connect on v2:172.21.15.190:6834/2286851849 2023-07-21T22:30:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.563+0000 7f41637fe700 1 --2- 172.21.15.107:0/497211021 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x55d9c017fd40 0x55d9c0182150 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3578a6a232260c9f server_cookie=b82548404ec479d in_seq=0 out_seq=0 2023-07-21T22:30:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4142ffd700 1 -- 172.21.15.107:0/497211021 <== mds.2 v2:172.21.15.190:6834/2286851849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d9c00b4eb0 con 0x55d9c017fd40 2023-07-21T22:30:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4142ffd700 10 client.25295 handle_command_reply: tid=0 2023-07-21T22:30:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4175e77b80 1 client.25295 shutdown 2023-07-21T22:30:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4175e77b80 1 -- 172.21.15.107:0/497211021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016c1a0 msgr2=0x55d9c017b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4175e77b80 1 --2- 172.21.15.107:0/497211021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016c1a0 0x55d9c017b5c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f415c009550 tx=0x7f415c005710 comp rx=0 tx=0).stop 2023-07-21T22:30:14.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4175e77b80 1 -- 172.21.15.107:0/497211021 shutdown_connections 2023-07-21T22:30:14.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4175e77b80 1 --2- 172.21.15.107:0/497211021 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x55d9c017fd40 0x55d9c0182150 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4175e77b80 1 --2- 172.21.15.107:0/497211021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016c1a0 0x55d9c017b5c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4142ffd700 0 client.25295 ms_handle_reset on v2:172.21.15.190:6834/2286851849 2023-07-21T22:30:14.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4175e77b80 1 --2- 172.21.15.107:0/497211021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016aa10 0x55d9c0174040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4175e77b80 1 --2- 172.21.15.107:0/497211021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016b830 0x55d9c0174580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4175e77b80 1 -- 172.21.15.107:0/497211021 >> 172.21.15.107:0/497211021 conn(0x55d9c015fef0 msgr2=0x55d9c0164a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4175e77b80 1 -- 172.21.15.107:0/497211021 shutdown_connections 2023-07-21T22:30:14.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4175e77b80 1 -- 172.21.15.107:0/497211021 wait complete. 2023-07-21T22:30:14.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.564+0000 7f4175e77b80 20 client.25295 trim_cache size 0 max 16384 2023-07-21T22:30:14.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f4175e77b80 1 Processor -- start 2023-07-21T22:30:14.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f4175e77b80 1 -- start start 2023-07-21T22:30:14.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f4175e77b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c0171ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f4175e77b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016b830 0x55d9c017b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f4175e77b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016c1a0 0x55d9c0172430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f4175e77b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c016b460 con 0x55d9c016aa10 2023-07-21T22:30:14.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f4175e77b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c0171ca0 con 0x55d9c016b830 2023-07-21T22:30:14.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f4175e77b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c0166540 con 0x55d9c016c1a0 2023-07-21T22:30:14.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f41637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c0171ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f41637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c0171ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48964/0 (socket says 172.21.15.107:48964) 2023-07-21T22:30:14.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f41637fe700 1 -- 172.21.15.107:0/307416424 learned_addr learned my addr 172.21.15.107:0/307416424 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f4163fff700 1 --2- 172.21.15.107:0/307416424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016c1a0 0x55d9c0172430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f41637fe700 1 -- 172.21.15.107:0/307416424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016c1a0 msgr2=0x55d9c0172430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f41637fe700 1 --2- 172.21.15.107:0/307416424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016c1a0 0x55d9c0172430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f41637fe700 1 -- 172.21.15.107:0/307416424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016b830 msgr2=0x55d9c017b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f41637fe700 1 --2- 172.21.15.107:0/307416424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016b830 0x55d9c017b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.626 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f41637fe700 1 -- 172.21.15.107:0/307416424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9bff10250 con 0x55d9c016aa10 2023-07-21T22:30:14.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f41637fe700 1 --2- 172.21.15.107:0/307416424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c0171ef0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4158097a80 tx=0x7f415801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1f53b092e4414a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.577+0000 7f414a7fc700 1 -- 172.21.15.107:0/307416424 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158099150 con 0x55d9c016aa10 2023-07-21T22:30:14.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.578+0000 7f414a7fc700 1 -- 172.21.15.107:0/307416424 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41580993b0 con 0x55d9c016aa10 2023-07-21T22:30:14.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.578+0000 7f414a7fc700 1 -- 172.21.15.107:0/307416424 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41580cbb30 con 0x55d9c016aa10 2023-07-21T22:30:14.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.578+0000 7f4175e77b80 1 -- 172.21.15.107:0/307416424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 msgr2=0x55d9c0171ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.578+0000 7f4175e77b80 1 --2- 172.21.15.107:0/307416424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c0171ef0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4158097a80 tx=0x7f415801a040 comp rx=0 tx=0).stop 2023-07-21T22:30:14.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.578+0000 7f4175e77b80 1 -- 172.21.15.107:0/307416424 shutdown_connections 2023-07-21T22:30:14.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.578+0000 7f4175e77b80 1 --2- 172.21.15.107:0/307416424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016c1a0 0x55d9c0172430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.578+0000 7f4175e77b80 1 --2- 172.21.15.107:0/307416424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c0171ef0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.578+0000 7f4175e77b80 1 --2- 172.21.15.107:0/307416424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016b830 0x55d9c017b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.578+0000 7f4175e77b80 1 -- 172.21.15.107:0/307416424 >> 172.21.15.107:0/307416424 conn(0x55d9c015fef0 msgr2=0x55d9c016ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.578+0000 7f4175e77b80 1 -- 172.21.15.107:0/307416424 shutdown_connections 2023-07-21T22:30:14.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.578+0000 7f4175e77b80 1 -- 172.21.15.107:0/307416424 wait complete. 2023-07-21T22:30:14.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.578+0000 7f4175e77b80 1 -- start start 2023-07-21T22:30:14.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f4175e77b80 1 Processor -- start 2023-07-21T22:30:14.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f4175e77b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c0177760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f4175e77b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016b830 0x55d9c0177ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f4175e77b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016c1a0 0x55d9c017acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f4175e77b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c017b200 con 0x55d9c016aa10 2023-07-21T22:30:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f4175e77b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c017b370 con 0x55d9c016b830 2023-07-21T22:30:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f4175e77b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c017b530 con 0x55d9c016c1a0 2023-07-21T22:30:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f4175e77b80 10 client.0 _ll_register_callbacks cb 0x7f41580227b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f41637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c0177760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f41637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c0177760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48970/0 (socket says 172.21.15.107:48970) 2023-07-21T22:30:14.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f41637fe700 1 -- 172.21.15.107:0/3201611973 learned_addr learned my addr 172.21.15.107:0/3201611973 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f4163fff700 1 --2- 172.21.15.107:0/3201611973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016c1a0 0x55d9c017acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f4162ffd700 1 --2- 172.21.15.107:0/3201611973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016b830 0x55d9c0177ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f41637fe700 1 -- 172.21.15.107:0/3201611973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016c1a0 msgr2=0x55d9c017acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f41637fe700 1 --2- 172.21.15.107:0/3201611973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016c1a0 0x55d9c017acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f41637fe700 1 -- 172.21.15.107:0/3201611973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016b830 msgr2=0x55d9c0177ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f41637fe700 1 --2- 172.21.15.107:0/3201611973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016b830 0x55d9c0177ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f41637fe700 1 -- 172.21.15.107:0/3201611973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41641a0010 con 0x55d9c016aa10 2023-07-21T22:30:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f41637fe700 1 --2- 172.21.15.107:0/3201611973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c0177760 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f415800c010 tx=0x7f4158005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e439af117f9085 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.579+0000 7f4142ffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4142ffd700 1 -- 172.21.15.107:0/3201611973 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415809b090 con 0x55d9c016aa10 2023-07-21T22:30:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4142ffd700 1 -- 172.21.15.107:0/3201611973 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f415809a500 con 0x55d9c016aa10 2023-07-21T22:30:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4175e77b80 1 -- 172.21.15.107:0/3201611973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f415c004d10 con 0x55d9c016aa10 2023-07-21T22:30:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4142ffd700 1 -- 172.21.15.107:0/3201611973 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41580b91b0 con 0x55d9c016aa10 2023-07-21T22:30:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4175e77b80 10 client.15798 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4175e77b80 1 -- 172.21.15.107:0/3201611973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d9bfee82b0 con 0x55d9c016aa10 2023-07-21T22:30:14.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4142ffd700 1 -- 172.21.15.107:0/3201611973 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41580cb020 con 0x55d9c016aa10 2023-07-21T22:30:14.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4142ffd700 1 -- 172.21.15.107:0/3201611973 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f41580cbb30 con 0x55d9c016aa10 2023-07-21T22:30:14.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4175e77b80 10 client.15798 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:14.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4175e77b80 10 client.15798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-07-21T22:30:14.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4175e77b80 20 client.15798 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:14.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4175e77b80 1 --2- 172.21.15.107:0/3201611973 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x55d9c017fd40 0x55d9c01d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.635 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4175e77b80 4 client.15798 mds_command: new command op to 24466 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:30:14.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4175e77b80 1 -- 172.21.15.107:0/3201611973 --> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d9c0077050 con 0x55d9c017fd40 2023-07-21T22:30:14.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.580+0000 7f4162ffd700 1 --2- 172.21.15.107:0/3201611973 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x55d9c017fd40 0x55d9c01d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4142ffd700 10 client.15798 ms_handle_connect on v2:172.21.15.190:6834/2286851849 2023-07-21T22:30:14.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4162ffd700 1 --2- 172.21.15.107:0/3201611973 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x55d9c017fd40 0x55d9c01d6c00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7d425dd92b603de server_cookie=75ac6d8fbfea1c93 in_seq=0 out_seq=0 2023-07-21T22:30:14.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4142ffd700 1 -- 172.21.15.107:0/3201611973 <== mds.2 v2:172.21.15.190:6834/2286851849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d9c0077050 con 0x55d9c017fd40 2023-07-21T22:30:14.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4142ffd700 10 client.15798 handle_command_reply: tid=0 2023-07-21T22:30:14.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4175e77b80 1 client.15798 shutdown 2023-07-21T22:30:14.636 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4175e77b80 1 -- 172.21.15.107:0/3201611973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 msgr2=0x55d9c0177760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4175e77b80 1 --2- 172.21.15.107:0/3201611973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c0177760 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f415800c010 tx=0x7f4158005ad0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4175e77b80 1 -- 172.21.15.107:0/3201611973 shutdown_connections 2023-07-21T22:30:14.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4175e77b80 1 --2- 172.21.15.107:0/3201611973 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x55d9c017fd40 0x55d9c01d6c00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4175e77b80 1 --2- 172.21.15.107:0/3201611973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d9c016c1a0 0x55d9c017acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4142ffd700 0 client.15798 ms_handle_reset on v2:172.21.15.190:6834/2286851849 2023-07-21T22:30:14.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4175e77b80 1 --2- 172.21.15.107:0/3201611973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d9c016aa10 0x55d9c0177760 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4175e77b80 1 --2- 172.21.15.107:0/3201611973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9c016b830 0x55d9c0177ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.637 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4175e77b80 1 -- 172.21.15.107:0/3201611973 >> 172.21.15.107:0/3201611973 conn(0x55d9c015fef0 msgr2=0x55d9c016ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4175e77b80 1 -- 172.21.15.107:0/3201611973 shutdown_connections 2023-07-21T22:30:14.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4175e77b80 1 -- 172.21.15.107:0/3201611973 wait complete. 2023-07-21T22:30:14.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.581+0000 7f4175e77b80 20 client.15798 trim_cache size 0 max 16384 2023-07-21T22:30:14.638 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:30:14.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.582+0000 7f4169cf2700 1 -- 172.21.15.107:0/2816369887 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f414c0a1150 msgr2=0x7f414c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.582+0000 7f4169cf2700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f414c0a1150 0x7f414c0a3600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4154005950 tx=0x7f41540058e0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.582+0000 7f4169cf2700 1 -- 172.21.15.107:0/2816369887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41640fe9e0 msgr2=0x7f41641092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.582+0000 7f4169cf2700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41640fe9e0 0x7f41641092d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4158000c00 tx=0x7f4158009f90 comp rx=0 tx=0).stop 2023-07-21T22:30:14.640 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.582+0000 7f4169cf2700 1 -- 172.21.15.107:0/2816369887 shutdown_connections 2023-07-21T22:30:14.640 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.582+0000 7f4169cf2700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f414c0a1150 0x7f414c0a3600 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.640 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.582+0000 7f4169cf2700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4164100560 0x7f4164107e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.640 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.582+0000 7f4169cf2700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41640fe9e0 0x7f41641092d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.640 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.582+0000 7f4169cf2700 1 --2- 172.21.15.107:0/2816369887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41640ffbd0 0x7f4164107910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.640 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.582+0000 7f4169cf2700 1 -- 172.21.15.107:0/2816369887 >> 172.21.15.107:0/2816369887 conn(0x7f41640fa000 msgr2=0x7f41640fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.640 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.582+0000 7f4169cf2700 1 -- 172.21.15.107:0/2816369887 shutdown_connections 2023-07-21T22:30:14.640 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.582+0000 7f4169cf2700 1 -- 172.21.15.107:0/2816369887 wait complete. 2023-07-21T22:30:14.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:14 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2374028356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:30:14.642 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:30:14.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:14 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2374028356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:30:14.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.717+0000 7f79f6012700 1 -- 172.21.15.107:0/2084102092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f00fea50 msgr2=0x7f79f00fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.717+0000 7f79f6012700 1 --2- 172.21.15.107:0/2084102092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f00fea50 0x7f79f00fee60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009990 tx=0x7f79d8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.718+0000 7f79f6012700 1 -- 172.21.15.107:0/2084102092 shutdown_connections 2023-07-21T22:30:14.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.718+0000 7f79f6012700 1 --2- 172.21.15.107:0/2084102092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f01005d0 0x7f79f0106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.718+0000 7f79f6012700 1 --2- 172.21.15.107:0/2084102092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f00fea50 0x7f79f00fee60 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.718+0000 7f79f6012700 1 --2- 172.21.15.107:0/2084102092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f00ffc40 0x7f79f0100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.718+0000 7f79f6012700 1 -- 172.21.15.107:0/2084102092 >> 172.21.15.107:0/2084102092 conn(0x7f79f00fa070 msgr2=0x7f79f00fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.718+0000 7f79f6012700 1 -- 172.21.15.107:0/2084102092 shutdown_connections 2023-07-21T22:30:14.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.718+0000 7f79f6012700 1 -- 172.21.15.107:0/2084102092 wait complete. 2023-07-21T22:30:14.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.718+0000 7f79f6012700 1 Processor -- start 2023-07-21T22:30:14.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.718+0000 7f79f6012700 1 -- start start 2023-07-21T22:30:14.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.718+0000 7f79f6012700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f00fea50 0x7f79f0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.718+0000 7f79f6012700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f00ffc40 0x7f79f0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.718+0000 7f79f6012700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f01005d0 0x7f79f019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79f6012700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f019f5f0 con 0x7f79f01005d0 2023-07-21T22:30:14.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79f6012700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f019f730 con 0x7f79f00fea50 2023-07-21T22:30:14.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79f6012700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f019f8c0 con 0x7f79f00ffc40 2023-07-21T22:30:14.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79effff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f01005d0 0x7f79f019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79effff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f01005d0 0x7f79f019f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48988/0 (socket says 172.21.15.107:48988) 2023-07-21T22:30:14.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79effff700 1 -- 172.21.15.107:0/3057513186 learned_addr learned my addr 172.21.15.107:0/3057513186 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79ef7fe700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f00fea50 0x7f79f0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79eeffd700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f00ffc40 0x7f79f0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79effff700 1 -- 172.21.15.107:0/3057513186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f00ffc40 msgr2=0x7f79f0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79effff700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f00ffc40 0x7f79f0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79effff700 1 -- 172.21.15.107:0/3057513186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f00fea50 msgr2=0x7f79f0197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79effff700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f00fea50 0x7f79f0197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79effff700 1 -- 172.21.15.107:0/3057513186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80095f0 con 0x7f79f01005d0 2023-07-21T22:30:14.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79effff700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f01005d0 0x7f79f019f020 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f79e8007ee0 tx=0x7f79e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79ecff9700 1 -- 172.21.15.107:0/3057513186 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8017070 con 0x7f79f01005d0 2023-07-21T22:30:14.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79ecff9700 1 -- 172.21.15.107:0/3057513186 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79e800f040 con 0x7f79f01005d0 2023-07-21T22:30:14.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79f6012700 1 -- 172.21.15.107:0/3057513186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f019fba0 con 0x7f79f01005d0 2023-07-21T22:30:14.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79ecff9700 1 -- 172.21.15.107:0/3057513186 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8005740 con 0x7f79f01005d0 2023-07-21T22:30:14.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.719+0000 7f79f6012700 1 -- 172.21.15.107:0/3057513186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f01a0060 con 0x7f79f01005d0 2023-07-21T22:30:14.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.721+0000 7f79ecff9700 1 -- 172.21.15.107:0/3057513186 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f79e800dc60 con 0x7f79f01005d0 2023-07-21T22:30:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.722+0000 7f79ecff9700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f79dc0a1150 0x7f79dc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.722+0000 7f79ef7fe700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f79dc0a1150 0x7f79dc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.722+0000 7f79ecff9700 1 -- 172.21.15.107:0/3057513186 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f79e80cc140 con 0x7f79f01005d0 2023-07-21T22:30:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.723+0000 7f79ef7fe700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f79dc0a1150 0x7f79dc0a3600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f79d800b5c0 tx=0x7f79d8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.777 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.723+0000 7f7a02197b80 1 Processor -- start 2023-07-21T22:30:14.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.723+0000 7f7a02197b80 1 -- start start 2023-07-21T22:30:14.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.723+0000 7f7a02197b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66d830 0x55c04b66dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.723+0000 7f7a02197b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 0x55c04b66c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.723+0000 7f7a02197b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66ca10 0x55c04b66ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.778 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.723+0000 7f7a02197b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04b571cb0 con 0x55c04b66e1a0 2023-07-21T22:30:14.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.723+0000 7f7a02197b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04b5c4a40 con 0x55c04b66d830 2023-07-21T22:30:14.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.723+0000 7f7a02197b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04b5bded0 con 0x55c04b66ca10 2023-07-21T22:30:14.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.723+0000 7f79eeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 0x55c04b66c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79eeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 0x55c04b66c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48994/0 (socket says 172.21.15.107:48994) 2023-07-21T22:30:14.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79eeffd700 1 -- 172.21.15.107:0/67136370 learned_addr learned my addr 172.21.15.107:0/67136370 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79ef7fe700 1 --2- 172.21.15.107:0/67136370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66d830 0x55c04b66dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79eeffd700 1 -- 172.21.15.107:0/67136370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66ca10 msgr2=0x55c04b66ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79effff700 1 --2- 172.21.15.107:0/67136370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66ca10 0x55c04b66ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79eeffd700 1 --2- 172.21.15.107:0/67136370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66ca10 0x55c04b66ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79eeffd700 1 -- 172.21.15.107:0/67136370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66d830 msgr2=0x55c04b66dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79eeffd700 1 --2- 172.21.15.107:0/67136370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66d830 0x55c04b66dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.780 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79eeffd700 1 -- 172.21.15.107:0/67136370 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c04b3ebde0 con 0x55c04b66e1a0 2023-07-21T22:30:14.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79ef7fe700 1 --2- 172.21.15.107:0/67136370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66d830 0x55c04b66dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:14.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79eeffd700 1 --2- 172.21.15.107:0/67136370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 0x55c04b66c430 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f79e000bd70 tx=0x7f79e000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ad0f22bac60794 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79e5ffb700 1 -- 172.21.15.107:0/67136370 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000fe50 con 0x55c04b66e1a0 2023-07-21T22:30:14.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79e5ffb700 1 -- 172.21.15.107:0/67136370 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79e001a920 con 0x55c04b66e1a0 2023-07-21T22:30:14.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.724+0000 7f79e5ffb700 1 -- 172.21.15.107:0/67136370 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0011860 con 0x55c04b66e1a0 2023-07-21T22:30:14.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.725+0000 7f7a02197b80 1 -- 172.21.15.107:0/67136370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 msgr2=0x55c04b66c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.725+0000 7f7a02197b80 1 --2- 172.21.15.107:0/67136370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 0x55c04b66c430 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f79e000bd70 tx=0x7f79e000f640 comp rx=0 tx=0).stop 2023-07-21T22:30:14.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.725+0000 7f7a02197b80 1 -- 172.21.15.107:0/67136370 shutdown_connections 2023-07-21T22:30:14.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.725+0000 7f7a02197b80 1 --2- 172.21.15.107:0/67136370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66ca10 0x55c04b66ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.725+0000 7f7a02197b80 1 --2- 172.21.15.107:0/67136370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66d830 0x55c04b66dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.725+0000 7f7a02197b80 1 --2- 172.21.15.107:0/67136370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 0x55c04b66c430 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.725+0000 7f7a02197b80 1 -- 172.21.15.107:0/67136370 >> 172.21.15.107:0/67136370 conn(0x55c04b661ef0 msgr2=0x55c04b667360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.725+0000 7f7a02197b80 1 -- 172.21.15.107:0/67136370 shutdown_connections 2023-07-21T22:30:14.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.725+0000 7f7a02197b80 1 -- 172.21.15.107:0/67136370 wait complete. 2023-07-21T22:30:14.782 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.725+0000 7f7a02197b80 1 -- start start 2023-07-21T22:30:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.725+0000 7f7a02197b80 1 Processor -- start 2023-07-21T22:30:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f7a02197b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 0x55c04b6760d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f7a02197b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66d830 0x55c04b676610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f7a02197b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66e1a0 0x55c04b67d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f7a02197b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04b55dac0 con 0x55c04b66ca10 2023-07-21T22:30:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f7a02197b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04b55d7c0 con 0x55c04b66e1a0 2023-07-21T22:30:14.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f7a02197b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04b441920 con 0x55c04b66d830 2023-07-21T22:30:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f7a02197b80 10 client.0 _ll_register_callbacks cb 0x55c04b55b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79ef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 0x55c04b6760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79ef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 0x55c04b6760d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48998/0 (socket says 172.21.15.107:48998) 2023-07-21T22:30:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79ef7fe700 1 -- 172.21.15.107:0/1421603639 learned_addr learned my addr 172.21.15.107:0/1421603639 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79effff700 1 --2- 172.21.15.107:0/1421603639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66e1a0 0x55c04b67d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79eeffd700 1 --2- 172.21.15.107:0/1421603639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66d830 0x55c04b676610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79ef7fe700 1 -- 172.21.15.107:0/1421603639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66d830 msgr2=0x55c04b676610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79ef7fe700 1 --2- 172.21.15.107:0/1421603639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66d830 0x55c04b676610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79ef7fe700 1 -- 172.21.15.107:0/1421603639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66e1a0 msgr2=0x55c04b67d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79ef7fe700 1 --2- 172.21.15.107:0/1421603639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66e1a0 0x55c04b67d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79ef7fe700 1 -- 172.21.15.107:0/1421603639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e000b9a0 con 0x55c04b66ca10 2023-07-21T22:30:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79ef7fe700 1 --2- 172.21.15.107:0/1421603639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 0x55c04b6760d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f79f00ffaa0 tx=0x7f79d801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268cc951afb790e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79ceffd700 1 -- 172.21.15.107:0/1421603639 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d800bed0 con 0x55c04b66ca10 2023-07-21T22:30:14.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f79ceffd700 1 -- 172.21.15.107:0/1421603639 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f79d8003f00 con 0x55c04b66ca10 2023-07-21T22:30:14.786 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.726+0000 7f7a02197b80 1 -- 172.21.15.107:0/1421603639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c04b5174f0 con 0x55c04b66ca10 2023-07-21T22:30:14.786 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.727+0000 7f79ceffd700 1 -- 172.21.15.107:0/1421603639 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79d8004100 con 0x55c04b66ca10 2023-07-21T22:30:14.786 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.727+0000 7f7a02197b80 10 client.15810 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:14.786 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.727+0000 7f7a02197b80 1 -- 172.21.15.107:0/1421603639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c04b3d4c60 con 0x55c04b66ca10 2023-07-21T22:30:14.786 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.727+0000 7f79ceffd700 1 -- 172.21.15.107:0/1421603639 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79d803f080 con 0x55c04b66ca10 2023-07-21T22:30:14.786 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.727+0000 7f79ceffd700 1 -- 172.21.15.107:0/1421603639 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f79d8039030 con 0x55c04b66ca10 2023-07-21T22:30:14.786 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.727+0000 7f7a02197b80 10 client.15810 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:14.786 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.727+0000 7f7a02197b80 10 client.15810 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-21T22:30:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.727+0000 7f7a02197b80 20 client.15810 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.727+0000 7f7a02197b80 1 --2- 172.21.15.107:0/1421603639 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55c04b681eb0 0x55c04b6842c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.727+0000 7f7a02197b80 4 client.15810 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.727+0000 7f7a02197b80 1 -- 172.21.15.107:0/1421603639 --> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c04b5b6eb0 con 0x55c04b681eb0 2023-07-21T22:30:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.727+0000 7f79eeffd700 1 --2- 172.21.15.107:0/1421603639 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55c04b681eb0 0x55c04b6842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.728+0000 7f79eeffd700 1 --2- 172.21.15.107:0/1421603639 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55c04b681eb0 0x55c04b6842c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e82e9425f34a0548 server_cookie=ac7e6d378e71b6ab in_seq=0 out_seq=0 2023-07-21T22:30:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.728+0000 7f79ceffd700 10 client.15810 ms_handle_connect on v2:172.21.15.190:6838/2960500982 2023-07-21T22:30:14.787 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.728+0000 7f79ceffd700 1 -- 172.21.15.107:0/1421603639 <== mds.1 v2:172.21.15.190:6838/2960500982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c04b5b6eb0 con 0x55c04b681eb0 2023-07-21T22:30:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.728+0000 7f79ceffd700 10 client.15810 handle_command_reply: tid=0 2023-07-21T22:30:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.728+0000 7f7a02197b80 1 client.15810 shutdown 2023-07-21T22:30:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.729+0000 7f7a02197b80 1 -- 172.21.15.107:0/1421603639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 msgr2=0x55c04b6760d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.729+0000 7f7a02197b80 1 --2- 172.21.15.107:0/1421603639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 0x55c04b6760d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f79f00ffaa0 tx=0x7f79d801a050 comp rx=0 tx=0).stop 2023-07-21T22:30:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.729+0000 7f7a02197b80 1 -- 172.21.15.107:0/1421603639 shutdown_connections 2023-07-21T22:30:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.729+0000 7f7a02197b80 1 --2- 172.21.15.107:0/1421603639 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55c04b681eb0 0x55c04b6842c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.788 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.729+0000 7f7a02197b80 1 --2- 172.21.15.107:0/1421603639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66e1a0 0x55c04b67d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.729+0000 7f7a02197b80 1 --2- 172.21.15.107:0/1421603639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 0x55c04b6760d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.729+0000 7f79ceffd700 0 client.15810 ms_handle_reset on v2:172.21.15.190:6838/2960500982 2023-07-21T22:30:14.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.729+0000 7f7a02197b80 1 --2- 172.21.15.107:0/1421603639 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66d830 0x55c04b676610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.729+0000 7f7a02197b80 1 -- 172.21.15.107:0/1421603639 >> 172.21.15.107:0/1421603639 conn(0x55c04b661ef0 msgr2=0x55c04b666a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.729+0000 7f7a02197b80 1 -- 172.21.15.107:0/1421603639 shutdown_connections 2023-07-21T22:30:14.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.729+0000 7f7a02197b80 1 -- 172.21.15.107:0/1421603639 wait complete. 2023-07-21T22:30:14.789 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.729+0000 7f7a02197b80 20 client.15810 trim_cache size 0 max 16384 2023-07-21T22:30:14.794 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.741+0000 7f7a02197b80 1 Processor -- start 2023-07-21T22:30:14.794 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.741+0000 7f7a02197b80 1 -- start start 2023-07-21T22:30:14.794 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f7a02197b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66ca10 0x55c04b684290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.794 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f7a02197b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66d830 0x55c04b67d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.794 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f7a02197b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 0x55c04b66c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f7a02197b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04b681bb0 con 0x55c04b66e1a0 2023-07-21T22:30:14.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f7a02197b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04b66d030 con 0x55c04b66d830 2023-07-21T22:30:14.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f7a02197b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04b66d1f0 con 0x55c04b66ca10 2023-07-21T22:30:14.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79effff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 0x55c04b66c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79effff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 0x55c04b66c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49010/0 (socket says 172.21.15.107:49010) 2023-07-21T22:30:14.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79effff700 1 -- 172.21.15.107:0/2365994820 learned_addr learned my addr 172.21.15.107:0/2365994820 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.795 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79ef7fe700 1 --2- 172.21.15.107:0/2365994820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66ca10 0x55c04b684290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79effff700 1 -- 172.21.15.107:0/2365994820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66ca10 msgr2=0x55c04b684290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79eeffd700 1 --2- 172.21.15.107:0/2365994820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66d830 0x55c04b67d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79effff700 1 --2- 172.21.15.107:0/2365994820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66ca10 0x55c04b684290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79effff700 1 -- 172.21.15.107:0/2365994820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66d830 msgr2=0x55c04b67d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79effff700 1 --2- 172.21.15.107:0/2365994820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66d830 0x55c04b67d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79effff700 1 -- 172.21.15.107:0/2365994820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c04b412250 con 0x55c04b66e1a0 2023-07-21T22:30:14.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79eeffd700 1 --2- 172.21.15.107:0/2365994820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66d830 0x55c04b67d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:30:14.796 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79effff700 1 --2- 172.21.15.107:0/2365994820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 0x55c04b66c350 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f79e8095a30 tx=0x7f79e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f168abe9d83e3967 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79e67fc700 1 -- 172.21.15.107:0/2365994820 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e809d560 con 0x55c04b66e1a0 2023-07-21T22:30:14.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79e67fc700 1 -- 172.21.15.107:0/2365994820 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79e8095040 con 0x55c04b66e1a0 2023-07-21T22:30:14.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.742+0000 7f79e67fc700 1 -- 172.21.15.107:0/2365994820 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80224b0 con 0x55c04b66e1a0 2023-07-21T22:30:14.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.743+0000 7f7a02197b80 1 -- 172.21.15.107:0/2365994820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 msgr2=0x55c04b66c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.743+0000 7f7a02197b80 1 --2- 172.21.15.107:0/2365994820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 0x55c04b66c350 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f79e8095a30 tx=0x7f79e800b040 comp rx=0 tx=0).stop 2023-07-21T22:30:14.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.743+0000 7f7a02197b80 1 -- 172.21.15.107:0/2365994820 shutdown_connections 2023-07-21T22:30:14.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.743+0000 7f7a02197b80 1 --2- 172.21.15.107:0/2365994820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66e1a0 0x55c04b66c350 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.743+0000 7f7a02197b80 1 --2- 172.21.15.107:0/2365994820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66ca10 0x55c04b684290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.743+0000 7f7a02197b80 1 --2- 172.21.15.107:0/2365994820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66d830 0x55c04b67d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.743+0000 7f7a02197b80 1 -- 172.21.15.107:0/2365994820 >> 172.21.15.107:0/2365994820 conn(0x55c04b661ef0 msgr2=0x55c04b688c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.743+0000 7f7a02197b80 1 -- 172.21.15.107:0/2365994820 shutdown_connections 2023-07-21T22:30:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.743+0000 7f7a02197b80 1 -- 172.21.15.107:0/2365994820 wait complete. 2023-07-21T22:30:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.743+0000 7f7a02197b80 1 -- start start 2023-07-21T22:30:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.743+0000 7f7a02197b80 1 Processor -- start 2023-07-21T22:30:14.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f7a02197b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 0x55c04b67d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f7a02197b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66d830 0x55c04b67d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f7a02197b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66e1a0 0x55c04b679b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f7a02197b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04b67a2a0 con 0x55c04b66ca10 2023-07-21T22:30:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f7a02197b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04b67a410 con 0x55c04b66e1a0 2023-07-21T22:30:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f7a02197b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c04b67a5d0 con 0x55c04b66d830 2023-07-21T22:30:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f7a02197b80 10 client.0 _ll_register_callbacks cb 0x55c04b55b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79ef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 0x55c04b67d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.799 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79ef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 0x55c04b67d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49012/0 (socket says 172.21.15.107:49012) 2023-07-21T22:30:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79ef7fe700 1 -- 172.21.15.107:0/2171242718 learned_addr learned my addr 172.21.15.107:0/2171242718 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79eeffd700 1 --2- 172.21.15.107:0/2171242718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66d830 0x55c04b67d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79effff700 1 --2- 172.21.15.107:0/2171242718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66e1a0 0x55c04b679b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79ef7fe700 1 -- 172.21.15.107:0/2171242718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66d830 msgr2=0x55c04b67d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79ef7fe700 1 --2- 172.21.15.107:0/2171242718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66d830 0x55c04b67d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79ef7fe700 1 -- 172.21.15.107:0/2171242718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66e1a0 msgr2=0x55c04b679b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.800 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79ef7fe700 1 --2- 172.21.15.107:0/2171242718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66e1a0 0x55c04b679b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79ef7fe700 1 -- 172.21.15.107:0/2171242718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f01a0060 con 0x55c04b66ca10 2023-07-21T22:30:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79ef7fe700 1 --2- 172.21.15.107:0/2171242718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 0x55c04b67d020 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009990 tx=0x7f79d8005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e020a4bb7fedd0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79ceffd700 1 -- 172.21.15.107:0/2171242718 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8005210 con 0x55c04b66ca10 2023-07-21T22:30:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f79ceffd700 1 -- 172.21.15.107:0/2171242718 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f79d800fdb0 con 0x55c04b66ca10 2023-07-21T22:30:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.744+0000 7f7a02197b80 1 -- 172.21.15.107:0/2171242718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79e001a920 con 0x55c04b66ca10 2023-07-21T22:30:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.745+0000 7f79ceffd700 1 -- 172.21.15.107:0/2171242718 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79d800bed0 con 0x55c04b66ca10 2023-07-21T22:30:14.801 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.745+0000 7f7a02197b80 10 client.15816 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:14.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.745+0000 7f7a02197b80 1 -- 172.21.15.107:0/2171242718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c04b3ea2b0 con 0x55c04b66ca10 2023-07-21T22:30:14.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.745+0000 7f79ceffd700 1 -- 172.21.15.107:0/2171242718 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79d804c020 con 0x55c04b66ca10 2023-07-21T22:30:14.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.745+0000 7f79ceffd700 1 -- 172.21.15.107:0/2171242718 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f79d8039050 con 0x55c04b66ca10 2023-07-21T22:30:14.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.745+0000 7f7a02197b80 10 client.15816 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:14.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.745+0000 7f7a02197b80 10 client.15816 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-21T22:30:14.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.745+0000 7f7a02197b80 20 client.15816 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:14.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.745+0000 7f7a02197b80 1 --2- 172.21.15.107:0/2171242718 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55c04b6d6f90 0x55c04b6d93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.802 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.745+0000 7f7a02197b80 4 client.15816 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:30:14.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.745+0000 7f7a02197b80 1 -- 172.21.15.107:0/2171242718 --> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c04b579050 con 0x55c04b6d6f90 2023-07-21T22:30:14.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.745+0000 7f79eeffd700 1 --2- 172.21.15.107:0/2171242718 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55c04b6d6f90 0x55c04b6d93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f79ceffd700 10 client.15816 ms_handle_connect on v2:172.21.15.190:6838/2960500982 2023-07-21T22:30:14.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f79eeffd700 1 --2- 172.21.15.107:0/2171242718 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55c04b6d6f90 0x55c04b6d93a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c6c6c6b72d42172 server_cookie=906d69073c6150b6 in_seq=0 out_seq=0 2023-07-21T22:30:14.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f79ceffd700 1 -- 172.21.15.107:0/2171242718 <== mds.1 v2:172.21.15.190:6838/2960500982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c04b579050 con 0x55c04b6d6f90 2023-07-21T22:30:14.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f79ceffd700 10 client.15816 handle_command_reply: tid=0 2023-07-21T22:30:14.803 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f7a02197b80 1 client.15816 shutdown 2023-07-21T22:30:14.804 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f7a02197b80 1 -- 172.21.15.107:0/2171242718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 msgr2=0x55c04b67d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.804 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f7a02197b80 1 --2- 172.21.15.107:0/2171242718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 0x55c04b67d020 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009990 tx=0x7f79d8005a40 comp rx=0 tx=0).stop 2023-07-21T22:30:14.804 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f7a02197b80 1 -- 172.21.15.107:0/2171242718 shutdown_connections 2023-07-21T22:30:14.804 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f7a02197b80 1 --2- 172.21.15.107:0/2171242718 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55c04b6d6f90 0x55c04b6d93a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.804 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f7a02197b80 1 --2- 172.21.15.107:0/2171242718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c04b66e1a0 0x55c04b679b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.804 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f7a02197b80 1 --2- 172.21.15.107:0/2171242718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c04b66ca10 0x55c04b67d020 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.804 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f79ceffd700 0 client.15816 ms_handle_reset on v2:172.21.15.190:6838/2960500982 2023-07-21T22:30:14.804 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f7a02197b80 1 --2- 172.21.15.107:0/2171242718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c04b66d830 0x55c04b67d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.805 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f7a02197b80 1 -- 172.21.15.107:0/2171242718 >> 172.21.15.107:0/2171242718 conn(0x55c04b661ef0 msgr2=0x55c04b666a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.805 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f7a02197b80 1 -- 172.21.15.107:0/2171242718 shutdown_connections 2023-07-21T22:30:14.805 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f7a02197b80 1 -- 172.21.15.107:0/2171242718 wait complete. 2023-07-21T22:30:14.805 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.746+0000 7f7a02197b80 20 client.15816 trim_cache size 0 max 16384 2023-07-21T22:30:14.805 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:30:14.806 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.747+0000 7f79f6012700 1 -- 172.21.15.107:0/3057513186 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f79dc0a1150 msgr2=0x7f79dc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.747+0000 7f79f6012700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f79dc0a1150 0x7f79dc0a3600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f79d800b5c0 tx=0x7f79d8005960 comp rx=0 tx=0).stop 2023-07-21T22:30:14.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.747+0000 7f79f6012700 1 -- 172.21.15.107:0/3057513186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f01005d0 msgr2=0x7f79f019f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.747+0000 7f79f6012700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f01005d0 0x7f79f019f020 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f79e8007ee0 tx=0x7f79e800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.747+0000 7f79f6012700 1 -- 172.21.15.107:0/3057513186 shutdown_connections 2023-07-21T22:30:14.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.747+0000 7f79f6012700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f01005d0 0x7f79f019f020 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.747+0000 7f79f6012700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79f00fea50 0x7f79f0197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.807 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.747+0000 7f79f6012700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f79dc0a1150 0x7f79dc0a3600 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.747+0000 7f79f6012700 1 --2- 172.21.15.107:0/3057513186 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79f00ffc40 0x7f79f0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.747+0000 7f79f6012700 1 -- 172.21.15.107:0/3057513186 >> 172.21.15.107:0/3057513186 conn(0x7f79f00fa070 msgr2=0x7f79f0105510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.747+0000 7f79f6012700 1 -- 172.21.15.107:0/3057513186 shutdown_connections 2023-07-21T22:30:14.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.747+0000 7f79f6012700 1 -- 172.21.15.107:0/3057513186 wait complete. 2023-07-21T22:30:14.808 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:30:14.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.884+0000 7f2d545ac700 1 -- 172.21.15.107:0/1158449586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d4c101660 msgr2=0x7f2d4c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.884+0000 7f2d545ac700 1 --2- 172.21.15.107:0/1158449586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d4c101660 0x7f2d4c107f20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400b0a0 tx=0x7f2d4400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.884+0000 7f2d545ac700 1 -- 172.21.15.107:0/1158449586 shutdown_connections 2023-07-21T22:30:14.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.884+0000 7f2d545ac700 1 --2- 172.21.15.107:0/1158449586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d4c101660 0x7f2d4c107f20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.884+0000 7f2d545ac700 1 --2- 172.21.15.107:0/1158449586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d4c0ffae0 0x7f2d4c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.884+0000 7f2d545ac700 1 --2- 172.21.15.107:0/1158449586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4c100cd0 0x7f2d4c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.884+0000 7f2d545ac700 1 -- 172.21.15.107:0/1158449586 >> 172.21.15.107:0/1158449586 conn(0x7f2d4c0fb120 msgr2=0x7f2d4c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.884+0000 7f2d545ac700 1 -- 172.21.15.107:0/1158449586 shutdown_connections 2023-07-21T22:30:14.932 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.884+0000 7f2d545ac700 1 -- 172.21.15.107:0/1158449586 wait complete. 2023-07-21T22:30:14.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d545ac700 1 Processor -- start 2023-07-21T22:30:14.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d545ac700 1 -- start start 2023-07-21T22:30:14.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d545ac700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4c0ffae0 0x7f2d4c195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d545ac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d4c100cd0 0x7f2d4c195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d545ac700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d4c101660 0x7f2d4c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d545ac700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c19d410 con 0x7f2d4c101660 2023-07-21T22:30:14.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d545ac700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c19d550 con 0x7f2d4c100cd0 2023-07-21T22:30:14.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d545ac700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c19d6e0 con 0x7f2d4c0ffae0 2023-07-21T22:30:14.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d52b49700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d4c101660 0x7f2d4c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d52b49700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d4c101660 0x7f2d4c19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49030/0 (socket says 172.21.15.107:49030) 2023-07-21T22:30:14.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d52b49700 1 -- 172.21.15.107:0/734768030 learned_addr learned my addr 172.21.15.107:0/734768030 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d52348700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4c0ffae0 0x7f2d4c195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d51b47700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d4c100cd0 0x7f2d4c195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d52b49700 1 -- 172.21.15.107:0/734768030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4c0ffae0 msgr2=0x7f2d4c195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d52b49700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4c0ffae0 0x7f2d4c195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d52b49700 1 -- 172.21.15.107:0/734768030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d4c100cd0 msgr2=0x7f2d4c195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d52b49700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d4c100cd0 0x7f2d4c195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d52b49700 1 -- 172.21.15.107:0/734768030 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d44009d00 con 0x7f2d4c101660 2023-07-21T22:30:14.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.885+0000 7f2d51b47700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d4c100cd0 0x7f2d4c195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:14.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.886+0000 7f2d52b49700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d4c101660 0x7f2d4c19ce40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400b830 tx=0x7f2d44004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.886+0000 7f2d437fe700 1 -- 172.21.15.107:0/734768030 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4400e040 con 0x7f2d4c101660 2023-07-21T22:30:14.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.886+0000 7f2d437fe700 1 -- 172.21.15.107:0/734768030 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d440094d0 con 0x7f2d4c101660 2023-07-21T22:30:14.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.886+0000 7f2d545ac700 1 -- 172.21.15.107:0/734768030 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c19d960 con 0x7f2d4c101660 2023-07-21T22:30:14.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.886+0000 7f2d437fe700 1 -- 172.21.15.107:0/734768030 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4401bd20 con 0x7f2d4c101660 2023-07-21T22:30:14.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.886+0000 7f2d545ac700 1 -- 172.21.15.107:0/734768030 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c19de30 con 0x7f2d4c101660 2023-07-21T22:30:14.942 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.888+0000 7f2d437fe700 1 -- 172.21.15.107:0/734768030 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2d4401b450 con 0x7f2d4c101660 2023-07-21T22:30:14.942 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.888+0000 7f2d437fe700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2d380a1250 0x7f2d380a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.942 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.889+0000 7f2d437fe700 1 -- 172.21.15.107:0/734768030 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2d440ce8a0 con 0x7f2d4c101660 2023-07-21T22:30:14.942 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.889+0000 7f2d52348700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2d380a1250 0x7f2d380a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.889+0000 7f2d52348700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2d380a1250 0x7f2d380a3700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c100b30 tx=0x7f2d4800a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.889+0000 7f2d60731b80 1 Processor -- start 2023-07-21T22:30:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d60731b80 1 -- start start 2023-07-21T22:30:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d60731b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc849830 0x55dbdc849c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d60731b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc84a1a0 0x55dbdc848430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d60731b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc848e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d60731b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdc74dcb0 con 0x55dbdc84a1a0 2023-07-21T22:30:14.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d60731b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdc7a0a40 con 0x55dbdc848a10 2023-07-21T22:30:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d60731b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdc799ed0 con 0x55dbdc849830 2023-07-21T22:30:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d51b47700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc84a1a0 0x55dbdc848430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d51b47700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc84a1a0 0x55dbdc848430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49036/0 (socket says 172.21.15.107:49036) 2023-07-21T22:30:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d51b47700 1 -- 172.21.15.107:0/1669885095 learned_addr learned my addr 172.21.15.107:0/1669885095 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d52b49700 1 --2- 172.21.15.107:0/1669885095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc848e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d52348700 1 --2- 172.21.15.107:0/1669885095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc849830 0x55dbdc849c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d51b47700 1 -- 172.21.15.107:0/1669885095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc849830 msgr2=0x55dbdc849c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d51b47700 1 --2- 172.21.15.107:0/1669885095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc849830 0x55dbdc849c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d51b47700 1 -- 172.21.15.107:0/1669885095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 msgr2=0x55dbdc848e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d51b47700 1 --2- 172.21.15.107:0/1669885095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc848e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d51b47700 1 -- 172.21.15.107:0/1669885095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbdc5c7de0 con 0x55dbdc84a1a0 2023-07-21T22:30:14.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d51b47700 1 --2- 172.21.15.107:0/1669885095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc84a1a0 0x55dbdc848430 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c008ce0 tx=0x7f2d3c00e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c0680b08dc0d2b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d40ff9700 1 -- 172.21.15.107:0/1669885095 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c00ee90 con 0x55dbdc84a1a0 2023-07-21T22:30:14.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.890+0000 7f2d40ff9700 1 -- 172.21.15.107:0/1669885095 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d3c0044e0 con 0x55dbdc84a1a0 2023-07-21T22:30:14.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.891+0000 7f2d40ff9700 1 -- 172.21.15.107:0/1669885095 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c010670 con 0x55dbdc84a1a0 2023-07-21T22:30:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.891+0000 7f2d60731b80 1 -- 172.21.15.107:0/1669885095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc84a1a0 msgr2=0x55dbdc848430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.891+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1669885095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc84a1a0 0x55dbdc848430 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c008ce0 tx=0x7f2d3c00e350 comp rx=0 tx=0).stop 2023-07-21T22:30:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.891+0000 7f2d60731b80 1 -- 172.21.15.107:0/1669885095 shutdown_connections 2023-07-21T22:30:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.891+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1669885095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc848e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.891+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1669885095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc849830 0x55dbdc849c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.891+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1669885095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc84a1a0 0x55dbdc848430 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.891+0000 7f2d60731b80 1 -- 172.21.15.107:0/1669885095 >> 172.21.15.107:0/1669885095 conn(0x55dbdc83def0 msgr2=0x55dbdc843360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.946 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.891+0000 7f2d60731b80 1 -- 172.21.15.107:0/1669885095 shutdown_connections 2023-07-21T22:30:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.891+0000 7f2d60731b80 1 -- 172.21.15.107:0/1669885095 wait complete. 2023-07-21T22:30:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d60731b80 1 -- start start 2023-07-21T22:30:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d60731b80 1 Processor -- start 2023-07-21T22:30:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d60731b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc852100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d60731b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc852640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d60731b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 0x55dbdc859680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.947 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d60731b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdc739ac0 con 0x55dbdc849830 2023-07-21T22:30:14.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d60731b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdc7397c0 con 0x55dbdc848a10 2023-07-21T22:30:14.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d60731b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdc61d920 con 0x55dbdc84a1a0 2023-07-21T22:30:14.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d60731b80 10 client.0 _ll_register_callbacks cb 0x55dbdc737760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:14.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d51b47700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc852640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d51b47700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc852640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49042/0 (socket says 172.21.15.107:49042) 2023-07-21T22:30:14.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d51b47700 1 -- 172.21.15.107:0/1412365976 learned_addr learned my addr 172.21.15.107:0/1412365976 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d52348700 1 --2- 172.21.15.107:0/1412365976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc852100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d52b49700 1 --2- 172.21.15.107:0/1412365976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 0x55dbdc859680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.949 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d51b47700 1 -- 172.21.15.107:0/1412365976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 msgr2=0x55dbdc859680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.949 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d51b47700 1 --2- 172.21.15.107:0/1412365976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 0x55dbdc859680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.949 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d51b47700 1 -- 172.21.15.107:0/1412365976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 msgr2=0x55dbdc852100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.949 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d51b47700 1 --2- 172.21.15.107:0/1412365976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc852100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.949 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d51b47700 1 -- 172.21.15.107:0/1412365976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3c008910 con 0x55dbdc849830 2023-07-21T22:30:14.949 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d51b47700 1 --2- 172.21.15.107:0/1412365976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc852640 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c005f70 tx=0x7f2d3c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea8888efa9c0ace server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.949 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.892+0000 7f2d35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d35ffb700 1 -- 172.21.15.107:0/1412365976 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c01f070 con 0x55dbdc849830 2023-07-21T22:30:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d35ffb700 1 -- 172.21.15.107:0/1412365976 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2d3c010430 con 0x55dbdc849830 2023-07-21T22:30:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d60731b80 1 -- 172.21.15.107:0/1412365976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbdc6f34f0 con 0x55dbdc849830 2023-07-21T22:30:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d35ffb700 1 -- 172.21.15.107:0/1412365976 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d3c0105f0 con 0x55dbdc849830 2023-07-21T22:30:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d60731b80 10 client.15828 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d60731b80 1 -- 172.21.15.107:0/1412365976 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dbdc5b0c60 con 0x55dbdc849830 2023-07-21T22:30:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d35ffb700 1 -- 172.21.15.107:0/1412365976 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d3c032080 con 0x55dbdc849830 2023-07-21T22:30:14.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d35ffb700 1 -- 172.21.15.107:0/1412365976 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2d3c02c030 con 0x55dbdc849830 2023-07-21T22:30:14.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d60731b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:14.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d60731b80 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:14.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d60731b80 20 client.15828 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:14.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1412365976 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55dbdc85de30 0x55dbdc860240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d60731b80 4 client.15828 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:14.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d60731b80 1 -- 172.21.15.107:0/1412365976 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbdc792eb0 con 0x55dbdc85de30 2023-07-21T22:30:14.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.893+0000 7f2d52348700 1 --2- 172.21.15.107:0/1412365976 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55dbdc85de30 0x55dbdc860240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d35ffb700 10 client.15828 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:14.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d52348700 1 --2- 172.21.15.107:0/1412365976 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55dbdc85de30 0x55dbdc860240 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a343d5a9e7f43427 server_cookie=4b198e01490ee2eb in_seq=0 out_seq=0 2023-07-21T22:30:14.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d35ffb700 1 -- 172.21.15.107:0/1412365976 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dbdc792eb0 con 0x55dbdc85de30 2023-07-21T22:30:14.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d35ffb700 10 client.15828 handle_command_reply: tid=0 2023-07-21T22:30:14.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d60731b80 1 client.15828 shutdown 2023-07-21T22:30:14.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d60731b80 1 -- 172.21.15.107:0/1412365976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 msgr2=0x55dbdc852640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1412365976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc852640 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c005f70 tx=0x7f2d3c00eec0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d60731b80 1 -- 172.21.15.107:0/1412365976 shutdown_connections 2023-07-21T22:30:14.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1412365976 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55dbdc85de30 0x55dbdc860240 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1412365976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 0x55dbdc859680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d35ffb700 0 client.15828 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:14.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1412365976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc852100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1412365976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc852640 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d60731b80 1 -- 172.21.15.107:0/1412365976 >> 172.21.15.107:0/1412365976 conn(0x55dbdc83def0 msgr2=0x55dbdc842a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.894+0000 7f2d60731b80 1 -- 172.21.15.107:0/1412365976 shutdown_connections 2023-07-21T22:30:14.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.895+0000 7f2d60731b80 1 -- 172.21.15.107:0/1412365976 wait complete. 2023-07-21T22:30:14.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.895+0000 7f2d60731b80 20 client.15828 trim_cache size 0 max 16384 2023-07-21T22:30:14.957 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.907+0000 7f2d60731b80 1 Processor -- start 2023-07-21T22:30:14.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.907+0000 7f2d60731b80 1 -- start start 2023-07-21T22:30:14.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.907+0000 7f2d60731b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc8483f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.907+0000 7f2d60731b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc859640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.907+0000 7f2d60731b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 0x55dbdc848de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.907+0000 7f2d60731b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdc84fd50 con 0x55dbdc849830 2023-07-21T22:30:14.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.907+0000 7f2d60731b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdc844540 con 0x55dbdc848a10 2023-07-21T22:30:14.958 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.907+0000 7f2d60731b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdc83ae50 con 0x55dbdc84a1a0 2023-07-21T22:30:14.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.908+0000 7f2d51b47700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc859640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.908+0000 7f2d51b47700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc859640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49050/0 (socket says 172.21.15.107:49050) 2023-07-21T22:30:14.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.908+0000 7f2d51b47700 1 -- 172.21.15.107:0/1282428376 learned_addr learned my addr 172.21.15.107:0/1282428376 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.908+0000 7f2d52348700 1 --2- 172.21.15.107:0/1282428376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc8483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.908+0000 7f2d51b47700 1 -- 172.21.15.107:0/1282428376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 msgr2=0x55dbdc848de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.908+0000 7f2d51b47700 1 --2- 172.21.15.107:0/1282428376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 0x55dbdc848de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.959 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.908+0000 7f2d51b47700 1 -- 172.21.15.107:0/1282428376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 msgr2=0x55dbdc8483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.960 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.908+0000 7f2d51b47700 1 --2- 172.21.15.107:0/1282428376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc8483f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.960 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.908+0000 7f2d51b47700 1 -- 172.21.15.107:0/1282428376 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbdc5ee250 con 0x55dbdc849830 2023-07-21T22:30:14.960 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.908+0000 7f2d51b47700 1 --2- 172.21.15.107:0/1282428376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc859640 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0044b0 tx=0x7f2d3c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b0aa9e6a61be7a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.960 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.908+0000 7f2d417fa700 1 -- 172.21.15.107:0/1282428376 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c035070 con 0x55dbdc849830 2023-07-21T22:30:14.960 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.908+0000 7f2d417fa700 1 -- 172.21.15.107:0/1282428376 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dbdc6f34f0 con 0x55dbdc849830 2023-07-21T22:30:14.960 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.908+0000 7f2d417fa700 1 -- 172.21.15.107:0/1282428376 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c01f030 con 0x55dbdc849830 2023-07-21T22:30:14.960 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 -- 172.21.15.107:0/1282428376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 msgr2=0x55dbdc859640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.960 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1282428376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc859640 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0044b0 tx=0x7f2d3c003c20 comp rx=0 tx=0).stop 2023-07-21T22:30:14.961 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 -- 172.21.15.107:0/1282428376 shutdown_connections 2023-07-21T22:30:14.961 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1282428376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 0x55dbdc848de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.961 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1282428376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc8483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.961 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 --2- 172.21.15.107:0/1282428376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc859640 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.961 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 -- 172.21.15.107:0/1282428376 >> 172.21.15.107:0/1282428376 conn(0x55dbdc83def0 msgr2=0x55dbdc850de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.961 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 -- 172.21.15.107:0/1282428376 shutdown_connections 2023-07-21T22:30:14.961 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 -- 172.21.15.107:0/1282428376 wait complete. 2023-07-21T22:30:14.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 -- start start 2023-07-21T22:30:14.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 Processor -- start 2023-07-21T22:30:14.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc8553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc8558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 0x55dbdc858900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdc858e40 con 0x55dbdc849830 2023-07-21T22:30:14.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdc858fb0 con 0x55dbdc848a10 2023-07-21T22:30:14.962 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbdc859170 con 0x55dbdc84a1a0 2023-07-21T22:30:14.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d60731b80 10 client.0 _ll_register_callbacks cb 0x7f2d3c008b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:14.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.909+0000 7f2d51b47700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc8558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d51b47700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc8558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49056/0 (socket says 172.21.15.107:49056) 2023-07-21T22:30:14.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d51b47700 1 -- 172.21.15.107:0/3615245770 learned_addr learned my addr 172.21.15.107:0/3615245770 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:14.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d52348700 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc8553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d51b47700 1 -- 172.21.15.107:0/3615245770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 msgr2=0x55dbdc858900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d52b49700 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 0x55dbdc858900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.963 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d51b47700 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 0x55dbdc858900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d51b47700 1 -- 172.21.15.107:0/3615245770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 msgr2=0x55dbdc8553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:14.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d51b47700 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc8553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d51b47700 1 -- 172.21.15.107:0/3615245770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbdc5b0c60 con 0x55dbdc849830 2023-07-21T22:30:14.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d52348700 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc8553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:14.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d51b47700 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc8558e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0187f0 tx=0x7f2d3c00a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ff8438fa87cbf5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:14.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:14.964 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d35ffb700 1 -- 172.21.15.107:0/3615245770 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c01f070 con 0x55dbdc849830 2023-07-21T22:30:14.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d35ffb700 1 -- 172.21.15.107:0/3615245770 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2d3c020070 con 0x55dbdc849830 2023-07-21T22:30:14.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d60731b80 1 -- 172.21.15.107:0/3615245770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d3c0044e0 con 0x55dbdc849830 2023-07-21T22:30:14.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d35ffb700 1 -- 172.21.15.107:0/3615245770 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d3c035040 con 0x55dbdc849830 2023-07-21T22:30:14.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d60731b80 10 client.15834 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:14.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d60731b80 1 -- 172.21.15.107:0/3615245770 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dbdc5c62b0 con 0x55dbdc849830 2023-07-21T22:30:14.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.910+0000 7f2d35ffb700 1 -- 172.21.15.107:0/3615245770 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d3c0044e0 con 0x55dbdc849830 2023-07-21T22:30:14.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d35ffb700 1 -- 172.21.15.107:0/3615245770 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2d3c00e470 con 0x55dbdc849830 2023-07-21T22:30:14.965 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d60731b80 10 client.15834 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:14.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d60731b80 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:14.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d60731b80 20 client.15834 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:14.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d60731b80 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55dbdc8b2830 0x55dbdc8b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:14.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d60731b80 4 client.15834 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:30:14.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d60731b80 1 -- 172.21.15.107:0/3615245770 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dbdc755050 con 0x55dbdc8b2830 2023-07-21T22:30:14.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d52348700 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55dbdc8b2830 0x55dbdc8b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:14.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d35ffb700 10 client.15834 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:14.966 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d52348700 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55dbdc8b2830 0x55dbdc8b4c40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=827459c3bf934e5e server_cookie=f0f752f936f73048 in_seq=0 out_seq=0 2023-07-21T22:30:14.967 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d35ffb700 1 -- 172.21.15.107:0/3615245770 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dbdc755050 con 0x55dbdc8b2830 2023-07-21T22:30:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d35ffb700 10 client.15834 handle_command_reply: tid=0 2023-07-21T22:30:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d60731b80 1 client.15834 shutdown 2023-07-21T22:30:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d60731b80 1 -- 172.21.15.107:0/3615245770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 msgr2=0x55dbdc8558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.911+0000 7f2d60731b80 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc8558e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0187f0 tx=0x7f2d3c00a5e0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d60731b80 1 -- 172.21.15.107:0/3615245770 shutdown_connections 2023-07-21T22:30:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d60731b80 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55dbdc8b2830 0x55dbdc8b4c40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d60731b80 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55dbdc84a1a0 0x55dbdc858900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d60731b80 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55dbdc848a10 0x55dbdc8553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d60731b80 1 --2- 172.21.15.107:0/3615245770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dbdc849830 0x55dbdc8558e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d35ffb700 0 client.15834 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:14.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d60731b80 1 -- 172.21.15.107:0/3615245770 >> 172.21.15.107:0/3615245770 conn(0x55dbdc83def0 msgr2=0x55dbdc848de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d60731b80 1 -- 172.21.15.107:0/3615245770 shutdown_connections 2023-07-21T22:30:14.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d60731b80 1 -- 172.21.15.107:0/3615245770 wait complete. 2023-07-21T22:30:14.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d60731b80 20 client.15834 trim_cache size 0 max 16384 2023-07-21T22:30:14.970 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:30:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d545ac700 1 -- 172.21.15.107:0/734768030 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2d380a1250 msgr2=0x7f2d380a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d545ac700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2d380a1250 0x7f2d380a3700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c100b30 tx=0x7f2d4800a3b0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d545ac700 1 -- 172.21.15.107:0/734768030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d4c101660 msgr2=0x7f2d4c19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d545ac700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d4c101660 0x7f2d4c19ce40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400b830 tx=0x7f2d44004d00 comp rx=0 tx=0).stop 2023-07-21T22:30:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d545ac700 1 -- 172.21.15.107:0/734768030 shutdown_connections 2023-07-21T22:30:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d545ac700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d4c101660 0x7f2d4c19ce40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d545ac700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f2d380a1250 0x7f2d380a3700 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d545ac700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d4c0ffae0 0x7f2d4c195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d545ac700 1 --2- 172.21.15.107:0/734768030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d4c100cd0 0x7f2d4c195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d545ac700 1 -- 172.21.15.107:0/734768030 >> 172.21.15.107:0/734768030 conn(0x7f2d4c0fb120 msgr2=0x7f2d4c0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d545ac700 1 -- 172.21.15.107:0/734768030 shutdown_connections 2023-07-21T22:30:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:14.912+0000 7f2d545ac700 1 -- 172.21.15.107:0/734768030 wait complete. 2023-07-21T22:30:15.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:15 smithi190 ceph-mon[98334]: pgmap v328: 97 pgs: 97 active+clean; 2.1 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-07-21T22:30:15.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:15 smithi132 ceph-mon[97863]: pgmap v328: 97 pgs: 97 active+clean; 2.1 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-07-21T22:30:15.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:15 smithi107 ceph-mon[96543]: pgmap v328: 97 pgs: 97 active+clean; 2.1 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 123 KiB/s wr, 13 op/s 2023-07-21T22:30:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.750+0000 7f83c0dbe700 1 -- 172.21.15.107:0/3496634717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83bc0ffc90 msgr2=0x7f83bc100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.750+0000 7f83c0dbe700 1 --2- 172.21.15.107:0/3496634717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83bc0ffc90 0x7f83bc100100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009990 tx=0x7f83ac009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.750+0000 7f83c0dbe700 1 -- 172.21.15.107:0/3496634717 shutdown_connections 2023-07-21T22:30:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.750+0000 7f83c0dbe700 1 --2- 172.21.15.107:0/3496634717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83bc100640 0x7f83bc106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.750+0000 7f83c0dbe700 1 --2- 172.21.15.107:0/3496634717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83bc0fea00 0x7f83bc0fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.750+0000 7f83c0dbe700 1 --2- 172.21.15.107:0/3496634717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83bc0ffc90 0x7f83bc100100 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.750+0000 7f83c0dbe700 1 -- 172.21.15.107:0/3496634717 >> 172.21.15.107:0/3496634717 conn(0x7f83bc0fa000 msgr2=0x7f83bc0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.750+0000 7f83c0dbe700 1 -- 172.21.15.107:0/3496634717 shutdown_connections 2023-07-21T22:30:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.750+0000 7f83c0dbe700 1 -- 172.21.15.107:0/3496634717 wait complete. 2023-07-21T22:30:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83c0dbe700 1 Processor -- start 2023-07-21T22:30:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83c0dbe700 1 -- start start 2023-07-21T22:30:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83c0dbe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83bc0fea00 0x7f83bc195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83c0dbe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83bc0ffc90 0x7f83bc195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83c0dbe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83bc100640 0x7f83bc196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83c0dbe700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1973a0 con 0x7f83bc0fea00 2023-07-21T22:30:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83c0dbe700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc197510 con 0x7f83bc100640 2023-07-21T22:30:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83c0dbe700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1976d0 con 0x7f83bc0ffc90 2023-07-21T22:30:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83ba59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83bc0fea00 0x7f83bc195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83ba59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83bc0fea00 0x7f83bc195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49068/0 (socket says 172.21.15.107:49068) 2023-07-21T22:30:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83ba59c700 1 -- 172.21.15.107:0/4111792393 learned_addr learned my addr 172.21.15.107:0/4111792393 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83b9d9b700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83bc0ffc90 0x7f83bc195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83bad9d700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83bc100640 0x7f83bc196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83ba59c700 1 -- 172.21.15.107:0/4111792393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83bc0ffc90 msgr2=0x7f83bc195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83ba59c700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83bc0ffc90 0x7f83bc195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83ba59c700 1 -- 172.21.15.107:0/4111792393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83bc100640 msgr2=0x7f83bc196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83ba59c700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83bc100640 0x7f83bc196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83ba59c700 1 -- 172.21.15.107:0/4111792393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac0095f0 con 0x7f83bc0fea00 2023-07-21T22:30:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83bad9d700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83bc100640 0x7f83bc196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83ba59c700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83bc0fea00 0x7f83bc195980 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f83b000e9a0 tx=0x7f83b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.751+0000 7f83ab7fe700 1 -- 172.21.15.107:0/4111792393 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0009790 con 0x7f83bc0fea00 2023-07-21T22:30:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.752+0000 7f83ab7fe700 1 -- 172.21.15.107:0/4111792393 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b000cbd0 con 0x7f83bc0fea00 2023-07-21T22:30:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.752+0000 7f83c0dbe700 1 -- 172.21.15.107:0/4111792393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83bc1979b0 con 0x7f83bc0fea00 2023-07-21T22:30:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.752+0000 7f83ab7fe700 1 -- 172.21.15.107:0/4111792393 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0018680 con 0x7f83bc0fea00 2023-07-21T22:30:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.752+0000 7f83c0dbe700 1 -- 172.21.15.107:0/4111792393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc19e170 con 0x7f83bc0fea00 2023-07-21T22:30:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.754+0000 7f83ab7fe700 1 -- 172.21.15.107:0/4111792393 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f83b000cd40 con 0x7f83bc0fea00 2023-07-21T22:30:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.754+0000 7f83ab7fe700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f83a40a11d0 0x7f83a40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.755+0000 7f83b9d9b700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f83a40a11d0 0x7f83a40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.755+0000 7f83ab7fe700 1 -- 172.21.15.107:0/4111792393 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f83b0014070 con 0x7f83bc0fea00 2023-07-21T22:30:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.755+0000 7f83b9d9b700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f83a40a11d0 0x7f83a40a3680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00b2e0 tx=0x7f83ac005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.755+0000 7f83ccf43b80 1 Processor -- start 2023-07-21T22:30:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.755+0000 7f83ccf43b80 1 -- start start 2023-07-21T22:30:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ccf43b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 0x563c2d6cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ccf43b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cb830 0x563c2d6cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ccf43b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ccf43b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d5d0c50 con 0x563c2d5276b0 2023-07-21T22:30:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ccf43b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d6239e0 con 0x563c2d6cb830 2023-07-21T22:30:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ccf43b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d61ce70 con 0x563c2d6cc580 2023-07-21T22:30:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ba59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 0x563c2d6cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ba59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 0x563c2d6cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49076/0 (socket says 172.21.15.107:49076) 2023-07-21T22:30:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ba59c700 1 -- 172.21.15.107:0/2253537059 learned_addr learned my addr 172.21.15.107:0/2253537059 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83bad9d700 1 --2- 172.21.15.107:0/2253537059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ba59c700 1 -- 172.21.15.107:0/2253537059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 msgr2=0x563c2d6cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ba59c700 1 --2- 172.21.15.107:0/2253537059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ba59c700 1 -- 172.21.15.107:0/2253537059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cb830 msgr2=0x563c2d6cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ba59c700 1 --2- 172.21.15.107:0/2253537059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cb830 0x563c2d6cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ba59c700 1 -- 172.21.15.107:0/2253537059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c2d44ad80 con 0x563c2d5276b0 2023-07-21T22:30:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83ba59c700 1 --2- 172.21.15.107:0/2253537059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 0x563c2d6cb2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f83b0099ed0 tx=0x7f83b00998a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad8afd7c49eca0a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83a8ff9700 1 -- 172.21.15.107:0/2253537059 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b009df70 con 0x563c2d5276b0 2023-07-21T22:30:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83a8ff9700 1 -- 172.21.15.107:0/2253537059 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b00a2390 con 0x563c2d5276b0 2023-07-21T22:30:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.756+0000 7f83a8ff9700 1 -- 172.21.15.107:0/2253537059 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0094b50 con 0x563c2d5276b0 2023-07-21T22:30:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.757+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2253537059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 msgr2=0x563c2d6cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.757+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2253537059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 0x563c2d6cb2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f83b0099ed0 tx=0x7f83b00998a0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.757+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2253537059 shutdown_connections 2023-07-21T22:30:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.757+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2253537059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.757+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2253537059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 0x563c2d6cb2f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.757+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2253537059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cb830 0x563c2d6cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.757+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2253537059 >> 172.21.15.107:0/2253537059 conn(0x563c2d6c0ae0 msgr2=0x563c2d6c5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.757+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2253537059 shutdown_connections 2023-07-21T22:30:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.757+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2253537059 wait complete. 2023-07-21T22:30:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.757+0000 7f83ccf43b80 1 -- start start 2023-07-21T22:30:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ccf43b80 1 Processor -- start 2023-07-21T22:30:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ccf43b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 0x563c2d6d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ccf43b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cb830 0x563c2d6d4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ccf43b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6dbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ccf43b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d5bca60 con 0x563c2d5276b0 2023-07-21T22:30:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ccf43b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d5bc760 con 0x563c2d6cb830 2023-07-21T22:30:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ccf43b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d4a08c0 con 0x563c2d6cc580 2023-07-21T22:30:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ccf43b80 10 client.0 _ll_register_callbacks cb 0x563c2d5ba700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ba59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 0x563c2d6d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ba59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 0x563c2d6d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49082/0 (socket says 172.21.15.107:49082) 2023-07-21T22:30:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ba59c700 1 -- 172.21.15.107:0/2646739376 learned_addr learned my addr 172.21.15.107:0/2646739376 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83b9d9b700 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cb830 0x563c2d6d4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83bad9d700 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6dbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ba59c700 1 -- 172.21.15.107:0/2646739376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 msgr2=0x563c2d6dbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ba59c700 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6dbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ba59c700 1 -- 172.21.15.107:0/2646739376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cb830 msgr2=0x563c2d6d4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ba59c700 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cb830 0x563c2d6d4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ba59c700 1 -- 172.21.15.107:0/2646739376 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc19e170 con 0x563c2d5276b0 2023-07-21T22:30:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83bad9d700 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6dbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83ba59c700 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 0x563c2d6d47f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f83b001f040 tx=0x7f83b009e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85edd672eb7f2b2a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.758+0000 7f83a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83a1ffb700 1 -- 172.21.15.107:0/2646739376 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b00b8370 con 0x563c2d5276b0 2023-07-21T22:30:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83a1ffb700 1 -- 172.21.15.107:0/2646739376 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f83b00ca2a0 con 0x563c2d5276b0 2023-07-21T22:30:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2646739376 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c2d576490 con 0x563c2d5276b0 2023-07-21T22:30:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83a1ffb700 1 -- 172.21.15.107:0/2646739376 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83b00ca460 con 0x563c2d5276b0 2023-07-21T22:30:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83ccf43b80 10 client.15846 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2646739376 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563c2d433c00 con 0x563c2d5276b0 2023-07-21T22:30:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83a1ffb700 1 -- 172.21.15.107:0/2646739376 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c2d576490 con 0x563c2d5276b0 2023-07-21T22:30:15.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83a1ffb700 1 -- 172.21.15.107:0/2646739376 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f83b0095f60 con 0x563c2d5276b0 2023-07-21T22:30:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83ccf43b80 10 client.15846 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83ccf43b80 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:15.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83ccf43b80 20 client.15846 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563c2d6e0340 0x563c2d6e2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83ccf43b80 4 client.15846 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2646739376 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c2d615e50 con 0x563c2d6e0340 2023-07-21T22:30:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.759+0000 7f83b9d9b700 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563c2d6e0340 0x563c2d6e2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.760+0000 7f83b9d9b700 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563c2d6e0340 0x563c2d6e2750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f329e626baf9838 server_cookie=27691cf72e35701f in_seq=0 out_seq=0 2023-07-21T22:30:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.760+0000 7f83a1ffb700 10 client.15846 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.760+0000 7f83a1ffb700 1 -- 172.21.15.107:0/2646739376 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563c2d615e50 con 0x563c2d6e0340 2023-07-21T22:30:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.760+0000 7f83a1ffb700 10 client.15846 handle_command_reply: tid=0 2023-07-21T22:30:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.760+0000 7f83ccf43b80 1 client.15846 shutdown 2023-07-21T22:30:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.761+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2646739376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 msgr2=0x563c2d6d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.761+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 0x563c2d6d47f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f83b001f040 tx=0x7f83b009e070 comp rx=0 tx=0).stop 2023-07-21T22:30:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.761+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2646739376 shutdown_connections 2023-07-21T22:30:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.761+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563c2d6e0340 0x563c2d6e2750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.761+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6dbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.761+0000 7f83a1ffb700 0 client.15846 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.761+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d5276b0 0x563c2d6d47f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.761+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2646739376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cb830 0x563c2d6d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.761+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2646739376 >> 172.21.15.107:0/2646739376 conn(0x563c2d6c0ae0 msgr2=0x563c2d6c53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.761+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2646739376 shutdown_connections 2023-07-21T22:30:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.761+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2646739376 wait complete. 2023-07-21T22:30:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.761+0000 7f83ccf43b80 20 client.15846 trim_cache size 0 max 16384 2023-07-21T22:30:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83ccf43b80 1 Processor -- start 2023-07-21T22:30:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83ccf43b80 1 -- start start 2023-07-21T22:30:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83ccf43b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d5276b0 0x563c2d6d4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83ccf43b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 0x563c2d6dbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83ccf43b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cc580 0x563c2d6c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83ccf43b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d615e50 con 0x563c2d6cb830 2023-07-21T22:30:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83ccf43b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d576490 con 0x563c2d6cc580 2023-07-21T22:30:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83ccf43b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d6c7720 con 0x563c2d5276b0 2023-07-21T22:30:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83b9d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 0x563c2d6dbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83b9d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 0x563c2d6dbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49092/0 (socket says 172.21.15.107:49092) 2023-07-21T22:30:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83b9d9b700 1 -- 172.21.15.107:0/2985617920 learned_addr learned my addr 172.21.15.107:0/2985617920 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83ba59c700 1 --2- 172.21.15.107:0/2985617920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d5276b0 0x563c2d6d4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.847 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83bad9d700 1 --2- 172.21.15.107:0/2985617920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cc580 0x563c2d6c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.775+0000 7f83b9d9b700 1 -- 172.21.15.107:0/2985617920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d5276b0 msgr2=0x563c2d6d4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83b9d9b700 1 --2- 172.21.15.107:0/2985617920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d5276b0 0x563c2d6d4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83b9d9b700 1 -- 172.21.15.107:0/2985617920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cc580 msgr2=0x563c2d6c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83b9d9b700 1 --2- 172.21.15.107:0/2985617920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cc580 0x563c2d6c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83b9d9b700 1 -- 172.21.15.107:0/2985617920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c2d4711f0 con 0x563c2d6cb830 2023-07-21T22:30:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83bad9d700 1 --2- 172.21.15.107:0/2985617920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cc580 0x563c2d6c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:30:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83ba59c700 1 --2- 172.21.15.107:0/2985617920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d5276b0 0x563c2d6d4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83b9d9b700 1 --2- 172.21.15.107:0/2985617920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 0x563c2d6dbd30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009990 tx=0x7f83ac020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af518a9b67826479 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83a97fa700 1 -- 172.21.15.107:0/2985617920 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac02b040 con 0x563c2d6cb830 2023-07-21T22:30:15.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83a97fa700 1 -- 172.21.15.107:0/2985617920 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83ac02c040 con 0x563c2d6cb830 2023-07-21T22:30:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83a97fa700 1 -- 172.21.15.107:0/2985617920 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac004df0 con 0x563c2d6cb830 2023-07-21T22:30:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2985617920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 msgr2=0x563c2d6dbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2985617920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 0x563c2d6dbd30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009990 tx=0x7f83ac020000 comp rx=0 tx=0).stop 2023-07-21T22:30:15.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2985617920 shutdown_connections 2023-07-21T22:30:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2985617920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d6cc580 0x563c2d6c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2985617920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d5276b0 0x563c2d6d4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/2985617920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 0x563c2d6dbd30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2985617920 >> 172.21.15.107:0/2985617920 conn(0x563c2d6c0ae0 msgr2=0x563c2d6d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.776+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2985617920 shutdown_connections 2023-07-21T22:30:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83ccf43b80 1 -- 172.21.15.107:0/2985617920 wait complete. 2023-07-21T22:30:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83ccf43b80 1 -- start start 2023-07-21T22:30:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83ccf43b80 1 Processor -- start 2023-07-21T22:30:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83ccf43b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d5276b0 0x563c2d6d7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83ccf43b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 0x563c2d6d8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83ccf43b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6db250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83ccf43b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d6db790 con 0x563c2d6cb830 2023-07-21T22:30:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83ccf43b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d6db900 con 0x563c2d5276b0 2023-07-21T22:30:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83ccf43b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c2d6dbac0 con 0x563c2d6cc580 2023-07-21T22:30:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83ccf43b80 10 client.0 _ll_register_callbacks cb 0x7f83b0099b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83b9d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 0x563c2d6d8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83b9d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 0x563c2d6d8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49098/0 (socket says 172.21.15.107:49098) 2023-07-21T22:30:15.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83b9d9b700 1 -- 172.21.15.107:0/1306454200 learned_addr learned my addr 172.21.15.107:0/1306454200 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83ba59c700 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d5276b0 0x563c2d6d7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83bad9d700 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6db250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83b9d9b700 1 -- 172.21.15.107:0/1306454200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 msgr2=0x563c2d6db250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83b9d9b700 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6db250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83b9d9b700 1 -- 172.21.15.107:0/1306454200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d5276b0 msgr2=0x563c2d6d7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.777+0000 7f83b9d9b700 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d5276b0 0x563c2d6d7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83b9d9b700 1 -- 172.21.15.107:0/1306454200 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc067900 con 0x563c2d6cb830 2023-07-21T22:30:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83bad9d700 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6db250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83b9d9b700 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 0x563c2d6d8540 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00c010 tx=0x7f83ac0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d79247202ad4c06 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83a1ffb700 1 -- 172.21.15.107:0/1306454200 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac004d90 con 0x563c2d6cb830 2023-07-21T22:30:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83a1ffb700 1 -- 172.21.15.107:0/1306454200 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f83ac0329d0 con 0x563c2d6cb830 2023-07-21T22:30:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83ccf43b80 1 -- 172.21.15.107:0/1306454200 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83b00a2390 con 0x563c2d6cb830 2023-07-21T22:30:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83a1ffb700 1 -- 172.21.15.107:0/1306454200 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83ac032b90 con 0x563c2d6cb830 2023-07-21T22:30:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83ccf43b80 10 client.15852 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83ccf43b80 1 -- 172.21.15.107:0/1306454200 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563c2d449250 con 0x563c2d6cb830 2023-07-21T22:30:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83a1ffb700 1 -- 172.21.15.107:0/1306454200 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83ac04b080 con 0x563c2d6cb830 2023-07-21T22:30:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83a1ffb700 1 -- 172.21.15.107:0/1306454200 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f83ac045030 con 0x563c2d6cb830 2023-07-21T22:30:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83ccf43b80 10 client.15852 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83ccf43b80 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83ccf43b80 20 client.15852 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563c2d73d6c0 0x563c2d73faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83ccf43b80 4 client.15852 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:30:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.778+0000 7f83ccf43b80 1 -- 172.21.15.107:0/1306454200 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563c2d5d7ff0 con 0x563c2d73d6c0 2023-07-21T22:30:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.779+0000 7f83ba59c700 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563c2d73d6c0 0x563c2d73faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.779+0000 7f83a1ffb700 10 client.15852 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.779+0000 7f83ba59c700 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563c2d73d6c0 0x563c2d73faa0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c070e61b6665aab0 server_cookie=da9f916ad98d3aac in_seq=0 out_seq=0 2023-07-21T22:30:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.786+0000 7f83a1ffb700 1 -- 172.21.15.107:0/1306454200 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563c2d5d7ff0 con 0x563c2d73d6c0 2023-07-21T22:30:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.786+0000 7f83a1ffb700 10 client.15852 handle_command_reply: tid=0 2023-07-21T22:30:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.786+0000 7f83ccf43b80 1 client.15852 shutdown 2023-07-21T22:30:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.786+0000 7f83ccf43b80 1 -- 172.21.15.107:0/1306454200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 msgr2=0x563c2d6d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.786+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 0x563c2d6d8540 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00c010 tx=0x7f83ac0056e0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83ccf43b80 1 -- 172.21.15.107:0/1306454200 shutdown_connections 2023-07-21T22:30:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563c2d73d6c0 0x563c2d73faa0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563c2d6cc580 0x563c2d6db250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83a1ffb700 0 client.15852 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563c2d5276b0 0x563c2d6d7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83ccf43b80 1 --2- 172.21.15.107:0/1306454200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c2d6cb830 0x563c2d6d8540 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83ccf43b80 1 -- 172.21.15.107:0/1306454200 >> 172.21.15.107:0/1306454200 conn(0x563c2d6c0ae0 msgr2=0x563c2d6d2450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83ccf43b80 1 -- 172.21.15.107:0/1306454200 shutdown_connections 2023-07-21T22:30:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83ccf43b80 1 -- 172.21.15.107:0/1306454200 wait complete. 2023-07-21T22:30:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83ccf43b80 20 client.15852 trim_cache size 0 max 16384 2023-07-21T22:30:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:30:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:30:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:30:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:30:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83c0dbe700 1 -- 172.21.15.107:0/4111792393 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f83a40a11d0 msgr2=0x7f83a40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83c0dbe700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f83a40a11d0 0x7f83a40a3680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00b2e0 tx=0x7f83ac005fb0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83c0dbe700 1 -- 172.21.15.107:0/4111792393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83bc0fea00 msgr2=0x7f83bc195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83c0dbe700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83bc0fea00 0x7f83bc195980 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f83b000e9a0 tx=0x7f83b000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83c0dbe700 1 -- 172.21.15.107:0/4111792393 shutdown_connections 2023-07-21T22:30:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83c0dbe700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83bc100640 0x7f83bc196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83c0dbe700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83bc0fea00 0x7f83bc195980 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83c0dbe700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f83a40a11d0 0x7f83a40a3680 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83c0dbe700 1 --2- 172.21.15.107:0/4111792393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83bc0ffc90 0x7f83bc195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83c0dbe700 1 -- 172.21.15.107:0/4111792393 >> 172.21.15.107:0/4111792393 conn(0x7f83bc0fa000 msgr2=0x7f83bc103170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83c0dbe700 1 -- 172.21.15.107:0/4111792393 shutdown_connections 2023-07-21T22:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:15.787+0000 7f83c0dbe700 1 -- 172.21.15.107:0/4111792393 wait complete. 2023-07-21T22:30:15.973 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:30:15.974 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:30:16.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 -- 172.21.15.107:0/3242632432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f081c100560 msgr2=0x7f081c102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 --2- 172.21.15.107:0/3242632432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f081c100560 0x7f081c102940 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0810009990 tx=0x7f0810009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 -- 172.21.15.107:0/3242632432 shutdown_connections 2023-07-21T22:30:16.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 --2- 172.21.15.107:0/3242632432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f081c1057a0 0x7f081c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 --2- 172.21.15.107:0/3242632432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f081c100560 0x7f081c102940 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 --2- 172.21.15.107:0/3242632432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f081c102e80 0x7f081c105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 -- 172.21.15.107:0/3242632432 >> 172.21.15.107:0/3242632432 conn(0x7f081c0fa000 msgr2=0x7f081c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 -- 172.21.15.107:0/3242632432 shutdown_connections 2023-07-21T22:30:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 -- 172.21.15.107:0/3242632432 wait complete. 2023-07-21T22:30:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 Processor -- start 2023-07-21T22:30:16.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 -- start start 2023-07-21T22:30:16.104 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f081c100560 0x7f081c195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.104 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f081c102e80 0x7f081c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.104 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f081c1057a0 0x7f081c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c19d3a0 con 0x7f081c100560 2023-07-21T22:30:16.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c19d4e0 con 0x7f081c102e80 2023-07-21T22:30:16.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.054+0000 7f08243a3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c19d670 con 0x7f081c1057a0 2023-07-21T22:30:16.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f082213f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f081c100560 0x7f081c195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f082213f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f081c100560 0x7f081c195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49108/0 (socket says 172.21.15.107:49108) 2023-07-21T22:30:16.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f082213f700 1 -- 172.21.15.107:0/493417935 learned_addr learned my addr 172.21.15.107:0/493417935 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:16.105 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f0822940700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f081c1057a0 0x7f081c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f082193e700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f081c102e80 0x7f081c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f082213f700 1 -- 172.21.15.107:0/493417935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f081c1057a0 msgr2=0x7f081c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f082213f700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f081c1057a0 0x7f081c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f082213f700 1 -- 172.21.15.107:0/493417935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f081c102e80 msgr2=0x7f081c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f082213f700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f081c102e80 0x7f081c195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f082213f700 1 -- 172.21.15.107:0/493417935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08100095f0 con 0x7f081c100560 2023-07-21T22:30:16.106 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f082213f700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f081c100560 0x7f081c195820 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0810000c00 tx=0x7f081000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f080f7fe700 1 -- 172.21.15.107:0/493417935 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081001d070 con 0x7f081c100560 2023-07-21T22:30:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f080f7fe700 1 -- 172.21.15.107:0/493417935 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f081000fb50 con 0x7f081c100560 2023-07-21T22:30:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f08243a3700 1 -- 172.21.15.107:0/493417935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081c19d8f0 con 0x7f081c100560 2023-07-21T22:30:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f080f7fe700 1 -- 172.21.15.107:0/493417935 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08100173f0 con 0x7f081c100560 2023-07-21T22:30:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.055+0000 7f08243a3700 1 -- 172.21.15.107:0/493417935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c19dda0 con 0x7f081c100560 2023-07-21T22:30:16.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.057+0000 7f080f7fe700 1 -- 172.21.15.107:0/493417935 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f081000fcc0 con 0x7f081c100560 2023-07-21T22:30:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.058+0000 7f080f7fe700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f08080a1150 0x7f08080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.058+0000 7f080f7fe700 1 -- 172.21.15.107:0/493417935 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f08100cdb10 con 0x7f081c100560 2023-07-21T22:30:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.058+0000 7f082193e700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f08080a1150 0x7f08080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.058+0000 7f082193e700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f08080a1150 0x7f08080a3600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0818005950 tx=0x7f081800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f0830528b80 1 Processor -- start 2023-07-21T22:30:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f0830528b80 1 -- start start 2023-07-21T22:30:16.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f0830528b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f799518b0 0x557f79951cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f0830528b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f799504b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f0830528b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79950a90 0x557f79950f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f0830528b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f79855d30 con 0x557f79952220 2023-07-21T22:30:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f0830528b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f798a8ac0 con 0x557f79950a90 2023-07-21T22:30:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f0830528b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f798a1f50 con 0x557f799518b0 2023-07-21T22:30:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f082193e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f799504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f082193e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f799504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49118/0 (socket says 172.21.15.107:49118) 2023-07-21T22:30:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f082193e700 1 -- 172.21.15.107:0/2694828753 learned_addr learned my addr 172.21.15.107:0/2694828753 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f082213f700 1 --2- 172.21.15.107:0/2694828753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f799518b0 0x557f79951cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f082193e700 1 -- 172.21.15.107:0/2694828753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f799518b0 msgr2=0x557f79951cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f082193e700 1 --2- 172.21.15.107:0/2694828753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f799518b0 0x557f79951cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f0822940700 1 --2- 172.21.15.107:0/2694828753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79950a90 0x557f79950f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f082193e700 1 -- 172.21.15.107:0/2694828753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79950a90 msgr2=0x557f79950f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.059+0000 7f082193e700 1 --2- 172.21.15.107:0/2694828753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79950a90 0x557f79950f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.117 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.060+0000 7f082193e700 1 -- 172.21.15.107:0/2694828753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f796cfe60 con 0x557f79952220 2023-07-21T22:30:16.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.060+0000 7f0822940700 1 --2- 172.21.15.107:0/2694828753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79950a90 0x557f79950f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:30:16.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.060+0000 7f082193e700 1 --2- 172.21.15.107:0/2694828753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f799504b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0818010fa0 tx=0x7f0818013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e98114528ccf638 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:16.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.060+0000 7f080cdac700 1 -- 172.21.15.107:0/2694828753 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081801e620 con 0x557f79952220 2023-07-21T22:30:16.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.060+0000 7f080cdac700 1 -- 172.21.15.107:0/2694828753 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0818015070 con 0x557f79952220 2023-07-21T22:30:16.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.060+0000 7f080cdac700 1 -- 172.21.15.107:0/2694828753 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081801ca20 con 0x557f79952220 2023-07-21T22:30:16.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.060+0000 7f0830528b80 1 -- 172.21.15.107:0/2694828753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 msgr2=0x557f799504b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.060+0000 7f0830528b80 1 --2- 172.21.15.107:0/2694828753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f799504b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0818010fa0 tx=0x7f0818013000 comp rx=0 tx=0).stop 2023-07-21T22:30:16.118 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 -- 172.21.15.107:0/2694828753 shutdown_connections 2023-07-21T22:30:16.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 --2- 172.21.15.107:0/2694828753 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79950a90 0x557f79950f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 --2- 172.21.15.107:0/2694828753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f799518b0 0x557f79951cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 --2- 172.21.15.107:0/2694828753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f799504b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 -- 172.21.15.107:0/2694828753 >> 172.21.15.107:0/2694828753 conn(0x557f79945f70 msgr2=0x557f7994b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:16.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 -- 172.21.15.107:0/2694828753 shutdown_connections 2023-07-21T22:30:16.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 -- 172.21.15.107:0/2694828753 wait complete. 2023-07-21T22:30:16.119 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 -- start start 2023-07-21T22:30:16.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 Processor -- start 2023-07-21T22:30:16.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79950a90 0x557f7995a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f799518b0 0x557f7995a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f79961640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f79841b40 con 0x557f79952220 2023-07-21T22:30:16.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f79841840 con 0x557f79950a90 2023-07-21T22:30:16.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f797259a0 con 0x557f799518b0 2023-07-21T22:30:16.120 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0830528b80 10 client.0 _ll_register_callbacks cb 0x557f7983f7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:16.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0822940700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f79961640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0822940700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f79961640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49126/0 (socket says 172.21.15.107:49126) 2023-07-21T22:30:16.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f0822940700 1 -- 172.21.15.107:0/1368747416 learned_addr learned my addr 172.21.15.107:0/1368747416 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:16.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.061+0000 7f082213f700 1 --2- 172.21.15.107:0/1368747416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79950a90 0x557f7995a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0822940700 1 -- 172.21.15.107:0/1368747416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f799518b0 msgr2=0x557f7995a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0822940700 1 --2- 172.21.15.107:0/1368747416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f799518b0 0x557f7995a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0822940700 1 -- 172.21.15.107:0/1368747416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79950a90 msgr2=0x557f7995a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.121 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0822940700 1 --2- 172.21.15.107:0/1368747416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79950a90 0x557f7995a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0822940700 1 -- 172.21.15.107:0/1368747416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0818010c50 con 0x557f79952220 2023-07-21T22:30:16.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0822940700 1 --2- 172.21.15.107:0/1368747416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f79961640 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0814004090 tx=0x7f0814004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b992909ba4606cb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:16.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0805ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:16.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0805ffb700 1 -- 172.21.15.107:0/1368747416 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814007830 con 0x557f79952220 2023-07-21T22:30:16.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0805ffb700 1 -- 172.21.15.107:0/1368747416 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0814007990 con 0x557f79952220 2023-07-21T22:30:16.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0830528b80 1 -- 172.21.15.107:0/1368747416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f797fb570 con 0x557f79952220 2023-07-21T22:30:16.122 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0805ffb700 1 -- 172.21.15.107:0/1368747416 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0814004bb0 con 0x557f79952220 2023-07-21T22:30:16.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0830528b80 10 client.15864 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:16.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0830528b80 1 -- 172.21.15.107:0/1368747416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f796b8ce0 con 0x557f79952220 2023-07-21T22:30:16.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0805ffb700 1 -- 172.21.15.107:0/1368747416 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f797fb570 con 0x557f79952220 2023-07-21T22:30:16.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0805ffb700 1 -- 172.21.15.107:0/1368747416 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0814014070 con 0x557f79952220 2023-07-21T22:30:16.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0830528b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:16.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0830528b80 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:16.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0830528b80 20 client.15864 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:16.123 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0830528b80 1 --2- 172.21.15.107:0/1368747416 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557f79965ee0 0x557f799682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0830528b80 4 client.15864 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:16.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.062+0000 7f0830528b80 1 -- 172.21.15.107:0/1368747416 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f7989af30 con 0x557f79965ee0 2023-07-21T22:30:16.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.063+0000 7f082213f700 1 --2- 172.21.15.107:0/1368747416 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557f79965ee0 0x557f799682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.063+0000 7f0805ffb700 10 client.15864 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:16.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.063+0000 7f082213f700 1 --2- 172.21.15.107:0/1368747416 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557f79965ee0 0x557f799682f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c0b7b415b657a09 server_cookie=e33aec5955fd97f1 in_seq=0 out_seq=0 2023-07-21T22:30:16.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0805ffb700 1 -- 172.21.15.107:0/1368747416 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557f7989af30 con 0x557f79965ee0 2023-07-21T22:30:16.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0805ffb700 10 client.15864 handle_command_reply: tid=0 2023-07-21T22:30:16.124 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0830528b80 1 client.15864 shutdown 2023-07-21T22:30:16.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0830528b80 1 -- 172.21.15.107:0/1368747416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 msgr2=0x557f79961640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0830528b80 1 --2- 172.21.15.107:0/1368747416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f79961640 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0814004090 tx=0x7f0814004450 comp rx=0 tx=0).stop 2023-07-21T22:30:16.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0830528b80 1 -- 172.21.15.107:0/1368747416 shutdown_connections 2023-07-21T22:30:16.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0830528b80 1 --2- 172.21.15.107:0/1368747416 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557f79965ee0 0x557f799682f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0830528b80 1 --2- 172.21.15.107:0/1368747416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f79961640 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0830528b80 1 --2- 172.21.15.107:0/1368747416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79950a90 0x557f7995a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.125 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0805ffb700 0 client.15864 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:16.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0830528b80 1 --2- 172.21.15.107:0/1368747416 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f799518b0 0x557f7995a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0830528b80 1 -- 172.21.15.107:0/1368747416 >> 172.21.15.107:0/1368747416 conn(0x557f79945f70 msgr2=0x557f7994aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:16.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0830528b80 1 -- 172.21.15.107:0/1368747416 shutdown_connections 2023-07-21T22:30:16.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0830528b80 1 -- 172.21.15.107:0/1368747416 wait complete. 2023-07-21T22:30:16.126 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.064+0000 7f0830528b80 20 client.15864 trim_cache size 0 max 16384 2023-07-21T22:30:16.128 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f0830528b80 1 Processor -- start 2023-07-21T22:30:16.128 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f0830528b80 1 -- start start 2023-07-21T22:30:16.128 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f0830528b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79950a90 0x557f79959e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.128 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f0830528b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f799518b0 0x557f79961560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f0830528b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79952220 0x557f7995a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f0830528b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f79942ed0 con 0x557f79950a90 2023-07-21T22:30:16.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f0830528b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7994a660 con 0x557f79952220 2023-07-21T22:30:16.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f0830528b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f79955600 con 0x557f799518b0 2023-07-21T22:30:16.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f082213f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79950a90 0x557f79959e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f082213f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79950a90 0x557f79959e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49130/0 (socket says 172.21.15.107:49130) 2023-07-21T22:30:16.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f082213f700 1 -- 172.21.15.107:0/1581980872 learned_addr learned my addr 172.21.15.107:0/1581980872 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:16.129 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f082213f700 1 -- 172.21.15.107:0/1581980872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f799518b0 msgr2=0x557f79961560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f082213f700 1 --2- 172.21.15.107:0/1581980872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f799518b0 0x557f79961560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f082213f700 1 -- 172.21.15.107:0/1581980872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79952220 msgr2=0x557f7995a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f082213f700 1 --2- 172.21.15.107:0/1581980872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79952220 0x557f7995a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f082213f700 1 -- 172.21.15.107:0/1581980872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f79959130 con 0x557f79950a90 2023-07-21T22:30:16.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f082213f700 1 --2- 172.21.15.107:0/1581980872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79950a90 0x557f79959e70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f081000ff70 tx=0x7f0810005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebaf5c34dced8a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:16.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f080d5ad700 1 -- 172.21.15.107:0/1581980872 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08100a3350 con 0x557f79950a90 2023-07-21T22:30:16.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f080d5ad700 1 -- 172.21.15.107:0/1581980872 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08100990d0 con 0x557f79950a90 2023-07-21T22:30:16.131 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.081+0000 7f080d5ad700 1 -- 172.21.15.107:0/1581980872 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08100a3120 con 0x557f79950a90 2023-07-21T22:30:16.134 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.082+0000 7f0830528b80 1 -- 172.21.15.107:0/1581980872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79950a90 msgr2=0x557f79959e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.134 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.082+0000 7f0830528b80 1 --2- 172.21.15.107:0/1581980872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79950a90 0x557f79959e70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f081000ff70 tx=0x7f0810005ad0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.134 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.082+0000 7f0830528b80 1 -- 172.21.15.107:0/1581980872 shutdown_connections 2023-07-21T22:30:16.134 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.082+0000 7f0830528b80 1 --2- 172.21.15.107:0/1581980872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f79952220 0x557f7995a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.134 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.082+0000 7f0830528b80 1 --2- 172.21.15.107:0/1581980872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79950a90 0x557f79959e70 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.134 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.082+0000 7f0830528b80 1 --2- 172.21.15.107:0/1581980872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f799518b0 0x557f79961560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.135 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.082+0000 7f0830528b80 1 -- 172.21.15.107:0/1581980872 >> 172.21.15.107:0/1581980872 conn(0x557f79945f70 msgr2=0x557f79954a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:16.135 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.082+0000 7f0830528b80 1 -- 172.21.15.107:0/1581980872 shutdown_connections 2023-07-21T22:30:16.135 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.082+0000 7f0830528b80 1 -- 172.21.15.107:0/1581980872 wait complete. 2023-07-21T22:30:16.135 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.082+0000 7f0830528b80 1 -- start start 2023-07-21T22:30:16.135 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.082+0000 7f0830528b80 1 Processor -- start 2023-07-21T22:30:16.135 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.082+0000 7f0830528b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f79950a90 0x557f79961230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.135 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.082+0000 7f0830528b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f799518b0 0x557f7995dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0830528b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f7995e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0830528b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7995e750 con 0x557f79952220 2023-07-21T22:30:16.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0830528b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7995e8c0 con 0x557f799518b0 2023-07-21T22:30:16.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0830528b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f799b9db0 con 0x557f79950a90 2023-07-21T22:30:16.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0830528b80 10 client.0 _ll_register_callbacks cb 0x7f0814009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:16.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0822940700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f7995e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0822940700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f7995e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49140/0 (socket says 172.21.15.107:49140) 2023-07-21T22:30:16.136 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0822940700 1 -- 172.21.15.107:0/2048473661 learned_addr learned my addr 172.21.15.107:0/2048473661 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:16.137 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f082213f700 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f79950a90 0x557f79961230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.137 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f082193e700 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f799518b0 0x557f7995dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.137 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0822940700 1 -- 172.21.15.107:0/2048473661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f79950a90 msgr2=0x557f79961230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.137 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0822940700 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f79950a90 0x557f79961230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.137 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0822940700 1 -- 172.21.15.107:0/2048473661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f799518b0 msgr2=0x557f7995dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.137 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0822940700 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f799518b0 0x557f7995dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.137 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0822940700 1 -- 172.21.15.107:0/2048473661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c19dda0 con 0x557f79952220 2023-07-21T22:30:16.138 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f082193e700 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f799518b0 0x557f7995dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:30:16.138 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f082213f700 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f79950a90 0x557f79961230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:16.138 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0822940700 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f7995e210 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0814003fd0 tx=0x7f0814007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd60b3c459f4e775 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:16.138 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0805ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:16.138 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0805ffb700 1 -- 172.21.15.107:0/2048473661 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814028680 con 0x557f79952220 2023-07-21T22:30:16.138 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0805ffb700 1 -- 172.21.15.107:0/2048473661 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0814003ab0 con 0x557f79952220 2023-07-21T22:30:16.138 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0830528b80 1 -- 172.21.15.107:0/2048473661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0818015070 con 0x557f79952220 2023-07-21T22:30:16.138 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0805ffb700 1 -- 172.21.15.107:0/2048473661 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0814004850 con 0x557f79952220 2023-07-21T22:30:16.139 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0830528b80 10 client.15870 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:16.139 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.083+0000 7f0830528b80 1 -- 172.21.15.107:0/2048473661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f796f62d0 con 0x557f79952220 2023-07-21T22:30:16.139 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.084+0000 7f0805ffb700 1 -- 172.21.15.107:0/2048473661 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0818015070 con 0x557f79952220 2023-07-21T22:30:16.139 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.084+0000 7f0805ffb700 1 -- 172.21.15.107:0/2048473661 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0814004170 con 0x557f79952220 2023-07-21T22:30:16.139 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.084+0000 7f0830528b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:16.139 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.084+0000 7f0830528b80 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:16.139 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.084+0000 7f0830528b80 20 client.15870 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:16.139 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.084+0000 7f0830528b80 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557f799bb7e0 0x557f799bdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.140 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.084+0000 7f0830528b80 4 client.15870 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:30:16.140 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.084+0000 7f0830528b80 1 -- 172.21.15.107:0/2048473661 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557f7985d0d0 con 0x557f799bb7e0 2023-07-21T22:30:16.140 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.084+0000 7f082213f700 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557f799bb7e0 0x557f799bdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.140 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.085+0000 7f0805ffb700 10 client.15870 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:16.140 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.085+0000 7f082213f700 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557f799bb7e0 0x557f799bdbf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d87815de85ea9c59 server_cookie=87019c74d9892d33 in_seq=0 out_seq=0 2023-07-21T22:30:16.140 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.085+0000 7f0805ffb700 1 -- 172.21.15.107:0/2048473661 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557f7985d0d0 con 0x557f799bb7e0 2023-07-21T22:30:16.140 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0805ffb700 10 client.15870 handle_command_reply: tid=0 2023-07-21T22:30:16.141 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0830528b80 1 client.15870 shutdown 2023-07-21T22:30:16.141 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0830528b80 1 -- 172.21.15.107:0/2048473661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 msgr2=0x557f7995e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.141 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0830528b80 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f7995e210 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0814003fd0 tx=0x7f0814007b90 comp rx=0 tx=0).stop 2023-07-21T22:30:16.141 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0830528b80 1 -- 172.21.15.107:0/2048473661 shutdown_connections 2023-07-21T22:30:16.141 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0830528b80 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557f799bb7e0 0x557f799bdbf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.141 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0830528b80 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f79952220 0x557f7995e210 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.141 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0805ffb700 0 client.15870 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:16.141 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0830528b80 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557f79950a90 0x557f79961230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.142 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0830528b80 1 --2- 172.21.15.107:0/2048473661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f799518b0 0x557f7995dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.142 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0830528b80 1 -- 172.21.15.107:0/2048473661 >> 172.21.15.107:0/2048473661 conn(0x557f79945f70 msgr2=0x557f79959e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:16.142 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0830528b80 1 -- 172.21.15.107:0/2048473661 shutdown_connections 2023-07-21T22:30:16.142 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0830528b80 1 -- 172.21.15.107:0/2048473661 wait complete. 2023-07-21T22:30:16.142 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f0830528b80 20 client.15870 trim_cache size 0 max 16384 2023-07-21T22:30:16.142 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:30:16.143 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2023-07-21T22:30:16.143 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "9d1f9865-1cf1-4b43-8f88-ce330eb9210d", 2023-07-21T22:30:16.143 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2023-07-21T22:30:16.143 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:30:16.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f08243a3700 1 -- 172.21.15.107:0/493417935 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f08080a1150 msgr2=0x7f08080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f08243a3700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f08080a1150 0x7f08080a3600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0818005950 tx=0x7f081800b410 comp rx=0 tx=0).stop 2023-07-21T22:30:16.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f08243a3700 1 -- 172.21.15.107:0/493417935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f081c100560 msgr2=0x7f081c195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f08243a3700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f081c100560 0x7f081c195820 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0810000c00 tx=0x7f081000f710 comp rx=0 tx=0).stop 2023-07-21T22:30:16.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f08243a3700 1 -- 172.21.15.107:0/493417935 shutdown_connections 2023-07-21T22:30:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f08243a3700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f08080a1150 0x7f08080a3600 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f08243a3700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f081c1057a0 0x7f081c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f08243a3700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f081c100560 0x7f081c195820 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f08243a3700 1 --2- 172.21.15.107:0/493417935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f081c102e80 0x7f081c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.086+0000 7f08243a3700 1 -- 172.21.15.107:0/493417935 >> 172.21.15.107:0/493417935 conn(0x7f081c0fa000 msgr2=0x7f081c1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.087+0000 7f08243a3700 1 -- 172.21.15.107:0/493417935 shutdown_connections 2023-07-21T22:30:16.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.087+0000 7f08243a3700 1 -- 172.21.15.107:0/493417935 wait complete. 2023-07-21T22:30:16.146 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:30:16.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.221+0000 7f3175f5a700 1 -- 172.21.15.107:0/2360041893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31700feef0 msgr2=0x7f31700ff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.221+0000 7f3175f5a700 1 --2- 172.21.15.107:0/2360041893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31700feef0 0x7f31700ff300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3160009990 tx=0x7f3160009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.221+0000 7f3175f5a700 1 -- 172.21.15.107:0/2360041893 shutdown_connections 2023-07-21T22:30:16.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.221+0000 7f3175f5a700 1 --2- 172.21.15.107:0/2360041893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31701005b0 0x7f3170106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.221+0000 7f3175f5a700 1 --2- 172.21.15.107:0/2360041893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31700feef0 0x7f31700ff300 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.221+0000 7f3175f5a700 1 --2- 172.21.15.107:0/2360041893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31700ffc00 0x7f3170100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.221+0000 7f3175f5a700 1 -- 172.21.15.107:0/2360041893 >> 172.21.15.107:0/2360041893 conn(0x7f31700f9770 msgr2=0x7f31700fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:16.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.221+0000 7f3175f5a700 1 -- 172.21.15.107:0/2360041893 shutdown_connections 2023-07-21T22:30:16.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.221+0000 7f3175f5a700 1 -- 172.21.15.107:0/2360041893 wait complete. 2023-07-21T22:30:16.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.222+0000 7f3175f5a700 1 Processor -- start 2023-07-21T22:30:16.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.222+0000 7f3175f5a700 1 -- start start 2023-07-21T22:30:16.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.222+0000 7f3175f5a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31700feef0 0x7f31700fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.222+0000 7f3175f5a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31700ffc00 0x7f317019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.222+0000 7f3175f5a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31701005b0 0x7f317019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.222+0000 7f3175f5a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317019d3d0 con 0x7f31700ffc00 2023-07-21T22:30:16.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.222+0000 7f3175f5a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317019d540 con 0x7f31700feef0 2023-07-21T22:30:16.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.222+0000 7f3175f5a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317019d700 con 0x7f31701005b0 2023-07-21T22:30:16.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.222+0000 7f316effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31700ffc00 0x7f317019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.222+0000 7f316effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31700ffc00 0x7f317019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49152/0 (socket says 172.21.15.107:49152) 2023-07-21T22:30:16.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.222+0000 7f316effd700 1 -- 172.21.15.107:0/2540309733 learned_addr learned my addr 172.21.15.107:0/2540309733 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:16.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.223+0000 7f316f7fe700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31700feef0 0x7f31700fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.223+0000 7f316effd700 1 -- 172.21.15.107:0/2540309733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31701005b0 msgr2=0x7f317019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.223+0000 7f316effd700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31701005b0 0x7f317019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.223+0000 7f316effd700 1 -- 172.21.15.107:0/2540309733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31700feef0 msgr2=0x7f31700fe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.223+0000 7f316effd700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31700feef0 0x7f31700fe5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.223+0000 7f316effd700 1 -- 172.21.15.107:0/2540309733 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31600095f0 con 0x7f31700ffc00 2023-07-21T22:30:16.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.223+0000 7f316f7fe700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31700feef0 0x7f31700fe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:30:16.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.223+0000 7f316effd700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31700ffc00 0x7f317019a5f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f316400d980 tx=0x7f316400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:16.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.223+0000 7f316cff9700 1 -- 172.21.15.107:0/2540309733 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316400de60 con 0x7f31700ffc00 2023-07-21T22:30:16.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.223+0000 7f316cff9700 1 -- 172.21.15.107:0/2540309733 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31640044e0 con 0x7f31700ffc00 2023-07-21T22:30:16.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.223+0000 7f3175f5a700 1 -- 172.21.15.107:0/2540309733 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317019d9e0 con 0x7f31700ffc00 2023-07-21T22:30:16.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.223+0000 7f316cff9700 1 -- 172.21.15.107:0/2540309733 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164003be0 con 0x7f31700ffc00 2023-07-21T22:30:16.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.223+0000 7f3175f5a700 1 -- 172.21.15.107:0/2540309733 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317019def0 con 0x7f31700ffc00 2023-07-21T22:30:16.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.225+0000 7f316cff9700 1 -- 172.21.15.107:0/2540309733 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3164009bb0 con 0x7f31700ffc00 2023-07-21T22:30:16.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.226+0000 7f316cff9700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f31580a1150 0x7f31580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.226+0000 7f316f7fe700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f31580a1150 0x7f31580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.226+0000 7f316cff9700 1 -- 172.21.15.107:0/2540309733 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f31640cc6e0 con 0x7f31700ffc00 2023-07-21T22:30:16.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.226+0000 7f316f7fe700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f31580a1150 0x7f31580a3600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f316000b5c0 tx=0x7f3160005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:16.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f31820dfb80 1 Processor -- start 2023-07-21T22:30:16.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f31820dfb80 1 -- start start 2023-07-21T22:30:16.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f31820dfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff3580830 0x559ff3580c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f31820dfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff35811a0 0x559ff357f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f31820dfb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff357fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f31820dfb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3484cb0 con 0x559ff357fa10 2023-07-21T22:30:16.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f31820dfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff34d7a40 con 0x559ff35811a0 2023-07-21T22:30:16.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f31820dfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff34d0ed0 con 0x559ff3580830 2023-07-21T22:30:16.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f316ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff357fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f316ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff357fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49162/0 (socket says 172.21.15.107:49162) 2023-07-21T22:30:16.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f316ffff700 1 -- 172.21.15.107:0/2619191172 learned_addr learned my addr 172.21.15.107:0/2619191172 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:16.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f316f7fe700 1 --2- 172.21.15.107:0/2619191172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff3580830 0x559ff3580c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f316effd700 1 --2- 172.21.15.107:0/2619191172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff35811a0 0x559ff357f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f316ffff700 1 -- 172.21.15.107:0/2619191172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff3580830 msgr2=0x559ff3580c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f316ffff700 1 --2- 172.21.15.107:0/2619191172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff3580830 0x559ff3580c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f316ffff700 1 -- 172.21.15.107:0/2619191172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff35811a0 msgr2=0x559ff357f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f316ffff700 1 --2- 172.21.15.107:0/2619191172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff35811a0 0x559ff357f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f316ffff700 1 -- 172.21.15.107:0/2619191172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ff32fede0 con 0x559ff357fa10 2023-07-21T22:30:16.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f316ffff700 1 --2- 172.21.15.107:0/2619191172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff357fe90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3168009d00 tx=0x7f316800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26788dddcf720549 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:16.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.227+0000 7f3155ffb700 1 -- 172.21.15.107:0/2619191172 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316800e070 con 0x559ff357fa10 2023-07-21T22:30:16.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.228+0000 7f3155ffb700 1 -- 172.21.15.107:0/2619191172 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3168009830 con 0x559ff357fa10 2023-07-21T22:30:16.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.228+0000 7f3155ffb700 1 -- 172.21.15.107:0/2619191172 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168011390 con 0x559ff357fa10 2023-07-21T22:30:16.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.228+0000 7f31820dfb80 1 -- 172.21.15.107:0/2619191172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 msgr2=0x559ff357fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.228+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2619191172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff357fe90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3168009d00 tx=0x7f316800b330 comp rx=0 tx=0).stop 2023-07-21T22:30:16.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.228+0000 7f31820dfb80 1 -- 172.21.15.107:0/2619191172 shutdown_connections 2023-07-21T22:30:16.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.228+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2619191172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff357fe90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.228+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2619191172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff3580830 0x559ff3580c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.228+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2619191172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff35811a0 0x559ff357f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.228+0000 7f31820dfb80 1 -- 172.21.15.107:0/2619191172 >> 172.21.15.107:0/2619191172 conn(0x559ff3574ef0 msgr2=0x559ff357a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:16.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.228+0000 7f31820dfb80 1 -- 172.21.15.107:0/2619191172 shutdown_connections 2023-07-21T22:30:16.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.228+0000 7f31820dfb80 1 -- 172.21.15.107:0/2619191172 wait complete. 2023-07-21T22:30:16.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f31820dfb80 1 -- start start 2023-07-21T22:30:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f31820dfb80 1 Processor -- start 2023-07-21T22:30:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f31820dfb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff3589070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f31820dfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff3580830 0x559ff35895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f31820dfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff35811a0 0x559ff35905f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f31820dfb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3470ac0 con 0x559ff357fa10 2023-07-21T22:30:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f31820dfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff34707c0 con 0x559ff3580830 2023-07-21T22:30:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f31820dfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3354920 con 0x559ff35811a0 2023-07-21T22:30:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f31820dfb80 10 client.0 _ll_register_callbacks cb 0x559ff346e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:16.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f316f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff3589070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f316f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff3589070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49164/0 (socket says 172.21.15.107:49164) 2023-07-21T22:30:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f316f7fe700 1 -- 172.21.15.107:0/2453780807 learned_addr learned my addr 172.21.15.107:0/2453780807 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:16.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f316effd700 1 --2- 172.21.15.107:0/2453780807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff3580830 0x559ff35895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f316ffff700 1 --2- 172.21.15.107:0/2453780807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff35811a0 0x559ff35905f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f316f7fe700 1 -- 172.21.15.107:0/2453780807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff35811a0 msgr2=0x559ff35905f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f316f7fe700 1 --2- 172.21.15.107:0/2453780807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff35811a0 0x559ff35905f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f316f7fe700 1 -- 172.21.15.107:0/2453780807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff3580830 msgr2=0x559ff35895b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f316f7fe700 1 --2- 172.21.15.107:0/2453780807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff3580830 0x559ff35895b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.229+0000 7f316f7fe700 1 -- 172.21.15.107:0/2453780807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31680099b0 con 0x559ff357fa10 2023-07-21T22:30:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f316f7fe700 1 --2- 172.21.15.107:0/2453780807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff3589070 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f316000f860 tx=0x7f3160019050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53bf9c2f428b8a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f314effd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f314effd700 1 -- 172.21.15.107:0/2453780807 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160033070 con 0x559ff357fa10 2023-07-21T22:30:16.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f31820dfb80 1 -- 172.21.15.107:0/2453780807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ff34c9eb0 con 0x559ff357fa10 2023-07-21T22:30:16.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f314effd700 1 -- 172.21.15.107:0/2453780807 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3160004830 con 0x559ff357fa10 2023-07-21T22:30:16.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f314effd700 1 -- 172.21.15.107:0/2453780807 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f316002a050 con 0x559ff357fa10 2023-07-21T22:30:16.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f31820dfb80 10 client.15882 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:16.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f31820dfb80 1 -- 172.21.15.107:0/2453780807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559ff32e7c60 con 0x559ff357fa10 2023-07-21T22:30:16.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f314effd700 1 -- 172.21.15.107:0/2453780807 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ff34c9eb0 con 0x559ff357fa10 2023-07-21T22:30:16.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f314effd700 1 -- 172.21.15.107:0/2453780807 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3160036030 con 0x559ff357fa10 2023-07-21T22:30:16.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f31820dfb80 10 client.15882 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:16.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f31820dfb80 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:16.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f31820dfb80 20 client.15882 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:16.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2453780807 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ff3594ea0 0x559ff35972b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f31820dfb80 4 client.15882 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:16.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.230+0000 7f31820dfb80 1 -- 172.21.15.107:0/2453780807 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ff348c050 con 0x559ff3594ea0 2023-07-21T22:30:16.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.231+0000 7f316effd700 1 --2- 172.21.15.107:0/2453780807 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ff3594ea0 0x559ff35972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.231+0000 7f314effd700 10 client.15882 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:16.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.231+0000 7f316effd700 1 --2- 172.21.15.107:0/2453780807 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ff3594ea0 0x559ff35972b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09dd5e33216b527 server_cookie=454931acfb9530ad in_seq=0 out_seq=0 2023-07-21T22:30:16.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.231+0000 7f314effd700 1 -- 172.21.15.107:0/2453780807 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559ff348c050 con 0x559ff3594ea0 2023-07-21T22:30:16.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.231+0000 7f314effd700 10 client.15882 handle_command_reply: tid=0 2023-07-21T22:30:16.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.232+0000 7f31820dfb80 1 client.15882 shutdown 2023-07-21T22:30:16.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.232+0000 7f31820dfb80 1 -- 172.21.15.107:0/2453780807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 msgr2=0x559ff3589070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.232+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2453780807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff3589070 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f316000f860 tx=0x7f3160019050 comp rx=0 tx=0).stop 2023-07-21T22:30:16.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.232+0000 7f31820dfb80 1 -- 172.21.15.107:0/2453780807 shutdown_connections 2023-07-21T22:30:16.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.232+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2453780807 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ff3594ea0 0x559ff35972b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.232+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2453780807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff35811a0 0x559ff35905f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.232+0000 7f314effd700 0 client.15882 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:16.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.232+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2453780807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff3589070 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.232+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2453780807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff3580830 0x559ff35895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.232+0000 7f31820dfb80 1 -- 172.21.15.107:0/2453780807 >> 172.21.15.107:0/2453780807 conn(0x559ff3574ef0 msgr2=0x559ff3579a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:16.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.232+0000 7f31820dfb80 1 -- 172.21.15.107:0/2453780807 shutdown_connections 2023-07-21T22:30:16.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.232+0000 7f31820dfb80 1 -- 172.21.15.107:0/2453780807 wait complete. 2023-07-21T22:30:16.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.232+0000 7f31820dfb80 20 client.15882 trim_cache size 0 max 16384 2023-07-21T22:30:16.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f31820dfb80 1 Processor -- start 2023-07-21T22:30:16.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f31820dfb80 1 -- start start 2023-07-21T22:30:16.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f31820dfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff357fa10 0x559ff357f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f31820dfb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff3580830 0x559ff35905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f31820dfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff35811a0 0x559ff3587e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f31820dfb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3588c40 con 0x559ff3580830 2023-07-21T22:30:16.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f31820dfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3594c90 con 0x559ff35811a0 2023-07-21T22:30:16.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f31820dfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff3571ed0 con 0x559ff357fa10 2023-07-21T22:30:16.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f316effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff3580830 0x559ff35905b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f316effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff3580830 0x559ff35905b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49174/0 (socket says 172.21.15.107:49174) 2023-07-21T22:30:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f316effd700 1 -- 172.21.15.107:0/2581188972 learned_addr learned my addr 172.21.15.107:0/2581188972 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f316f7fe700 1 --2- 172.21.15.107:0/2581188972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff357fa10 0x559ff357f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f316ffff700 1 --2- 172.21.15.107:0/2581188972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff35811a0 0x559ff3587e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f316effd700 1 -- 172.21.15.107:0/2581188972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff357fa10 msgr2=0x559ff357f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f316effd700 1 --2- 172.21.15.107:0/2581188972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff357fa10 0x559ff357f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f316effd700 1 -- 172.21.15.107:0/2581188972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff35811a0 msgr2=0x559ff3587e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.249+0000 7f316effd700 1 --2- 172.21.15.107:0/2581188972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff35811a0 0x559ff3587e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.250+0000 7f316effd700 1 -- 172.21.15.107:0/2581188972 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ff3325250 con 0x559ff3580830 2023-07-21T22:30:16.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.250+0000 7f316ffff700 1 --2- 172.21.15.107:0/2581188972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff35811a0 0x559ff3587e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:16.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.250+0000 7f316effd700 1 --2- 172.21.15.107:0/2581188972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff3580830 0x559ff35905b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f31640997d0 tx=0x7f31640ca090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4618a8bee17b9110 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:16.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.250+0000 7f31567fc700 1 -- 172.21.15.107:0/2581188972 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316409db00 con 0x559ff3580830 2023-07-21T22:30:16.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.250+0000 7f31567fc700 1 -- 172.21.15.107:0/2581188972 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31640a1f20 con 0x559ff3580830 2023-07-21T22:30:16.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.250+0000 7f31567fc700 1 -- 172.21.15.107:0/2581188972 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164099220 con 0x559ff3580830 2023-07-21T22:30:16.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.250+0000 7f31820dfb80 1 -- 172.21.15.107:0/2581188972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff3580830 msgr2=0x559ff35905b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.250+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2581188972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff3580830 0x559ff35905b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f31640997d0 tx=0x7f31640ca090 comp rx=0 tx=0).stop 2023-07-21T22:30:16.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 -- 172.21.15.107:0/2581188972 shutdown_connections 2023-07-21T22:30:16.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2581188972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff35811a0 0x559ff3587e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2581188972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff357fa10 0x559ff357f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 --2- 172.21.15.107:0/2581188972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff3580830 0x559ff35905b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 -- 172.21.15.107:0/2581188972 >> 172.21.15.107:0/2581188972 conn(0x559ff3574ef0 msgr2=0x559ff357fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:16.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 -- 172.21.15.107:0/2581188972 shutdown_connections 2023-07-21T22:30:16.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 -- 172.21.15.107:0/2581188972 wait complete. 2023-07-21T22:30:16.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 -- start start 2023-07-21T22:30:16.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 Processor -- start 2023-07-21T22:30:16.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff3590130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff3580830 0x559ff358cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff35811a0 0x559ff358d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff358d820 con 0x559ff357fa10 2023-07-21T22:30:16.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff358d990 con 0x559ff3580830 2023-07-21T22:30:16.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ff35e8d50 con 0x559ff35811a0 2023-07-21T22:30:16.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f31820dfb80 10 client.0 _ll_register_callbacks cb 0x7f316000f460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:16.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f316f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff3590130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f316f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff3590130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49178/0 (socket says 172.21.15.107:49178) 2023-07-21T22:30:16.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.251+0000 7f316f7fe700 1 -- 172.21.15.107:0/606742046 learned_addr learned my addr 172.21.15.107:0/606742046 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:16.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f316effd700 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff3580830 0x559ff358cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f316ffff700 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff35811a0 0x559ff358d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f316f7fe700 1 -- 172.21.15.107:0/606742046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff35811a0 msgr2=0x559ff358d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f316f7fe700 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff35811a0 0x559ff358d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f316f7fe700 1 -- 172.21.15.107:0/606742046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff3580830 msgr2=0x559ff358cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:16.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f316f7fe700 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff3580830 0x559ff358cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f316f7fe700 1 -- 172.21.15.107:0/606742046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317019def0 con 0x559ff357fa10 2023-07-21T22:30:16.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f316effd700 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff3580830 0x559ff358cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:16.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f316f7fe700 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff3590130 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f316003ac50 tx=0x7f316003ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78fc83a83099012 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:16.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f314effd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:16.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f314effd700 1 -- 172.21.15.107:0/606742046 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160033070 con 0x559ff357fa10 2023-07-21T22:30:16.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f314effd700 1 -- 172.21.15.107:0/606742046 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3160005c00 con 0x559ff357fa10 2023-07-21T22:30:16.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f31820dfb80 1 -- 172.21.15.107:0/606742046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ff342a4f0 con 0x559ff357fa10 2023-07-21T22:30:16.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f314effd700 1 -- 172.21.15.107:0/606742046 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f316002a040 con 0x559ff357fa10 2023-07-21T22:30:16.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f31820dfb80 10 client.15888 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:16.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f31820dfb80 1 -- 172.21.15.107:0/606742046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559ff32fd2b0 con 0x559ff357fa10 2023-07-21T22:30:16.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f314effd700 1 -- 172.21.15.107:0/606742046 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ff342a4f0 con 0x559ff357fa10 2023-07-21T22:30:16.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f314effd700 1 -- 172.21.15.107:0/606742046 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3160036050 con 0x559ff357fa10 2023-07-21T22:30:16.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f31820dfb80 10 client.15888 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:16.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f31820dfb80 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:16.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f31820dfb80 20 client.15888 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:16.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.252+0000 7f31820dfb80 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ff35ea370 0x559ff35ec780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:16.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.253+0000 7f31820dfb80 4 client.15888 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:30:16.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.253+0000 7f31820dfb80 1 -- 172.21.15.107:0/606742046 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559ff331ff70 con 0x559ff35ea370 2023-07-21T22:30:16.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.253+0000 7f316effd700 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ff35ea370 0x559ff35ec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:16.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.253+0000 7f314effd700 10 client.15888 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:16.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.253+0000 7f316effd700 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ff35ea370 0x559ff35ec780 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6477dc82607df2e0 server_cookie=40e002cd6230ff66 in_seq=0 out_seq=0 2023-07-21T22:30:16.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.253+0000 7f314effd700 1 -- 172.21.15.107:0/606742046 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559ff331ff70 con 0x559ff35ea370 2023-07-21T22:30:16.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.253+0000 7f314effd700 10 client.15888 handle_command_reply: tid=0 2023-07-21T22:30:16.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.253+0000 7f31820dfb80 1 client.15888 shutdown 2023-07-21T22:30:16.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.253+0000 7f31820dfb80 1 -- 172.21.15.107:0/606742046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 msgr2=0x559ff3590130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.253+0000 7f31820dfb80 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff3590130 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f316003ac50 tx=0x7f316003ad30 comp rx=0 tx=0).stop 2023-07-21T22:30:16.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f31820dfb80 1 -- 172.21.15.107:0/606742046 shutdown_connections 2023-07-21T22:30:16.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f31820dfb80 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ff35ea370 0x559ff35ec780 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f31820dfb80 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ff35811a0 0x559ff358d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f31820dfb80 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff357fa10 0x559ff3590130 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f314effd700 0 client.15888 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:16.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f31820dfb80 1 --2- 172.21.15.107:0/606742046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ff3580830 0x559ff358cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f31820dfb80 1 -- 172.21.15.107:0/606742046 >> 172.21.15.107:0/606742046 conn(0x559ff3574ef0 msgr2=0x559ff3579a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:16.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f31820dfb80 1 -- 172.21.15.107:0/606742046 shutdown_connections 2023-07-21T22:30:16.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f31820dfb80 1 -- 172.21.15.107:0/606742046 wait complete. 2023-07-21T22:30:16.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f31820dfb80 20 client.15888 trim_cache size 0 max 16384 2023-07-21T22:30:16.319 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:30:16.319 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:30:16.319 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:30:16.319 INFO:teuthology.orchestra.run.smithi107.stdout: "9d1f9865-1cf1-4b43-8f88-ce330eb9210d": { 2023-07-21T22:30:16.320 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:30:16.320 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "9d1f9865-1cf1-4b43-8f88-ce330eb9210d", 2023-07-21T22:30:16.320 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:30:16.320 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:30:16.320 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:30:16.320 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:30:16.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f3175f5a700 1 -- 172.21.15.107:0/2540309733 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f31580a1150 msgr2=0x7f31580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f3175f5a700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f31580a1150 0x7f31580a3600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f316000b5c0 tx=0x7f3160005fb0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f3175f5a700 1 -- 172.21.15.107:0/2540309733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31700ffc00 msgr2=0x7f317019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:16.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f3175f5a700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31700ffc00 0x7f317019a5f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f316400d980 tx=0x7f316400dd40 comp rx=0 tx=0).stop 2023-07-21T22:30:16.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f3175f5a700 1 -- 172.21.15.107:0/2540309733 shutdown_connections 2023-07-21T22:30:16.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f3175f5a700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f31580a1150 0x7f31580a3600 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.322 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f3175f5a700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31701005b0 0x7f317019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f3175f5a700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31700feef0 0x7f31700fe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f3175f5a700 1 --2- 172.21.15.107:0/2540309733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31700ffc00 0x7f317019a5f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:16.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f3175f5a700 1 -- 172.21.15.107:0/2540309733 >> 172.21.15.107:0/2540309733 conn(0x7f31700f9770 msgr2=0x7f31700fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:16.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f3175f5a700 1 -- 172.21.15.107:0/2540309733 shutdown_connections 2023-07-21T22:30:16.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:16.254+0000 7f3175f5a700 1 -- 172.21.15.107:0/2540309733 wait complete. 2023-07-21T22:30:16.323 INFO:tasks.cephfs.filesystem:scrub status for tag:9d1f9865-1cf1-4b43-8f88-ce330eb9210d - {'path': '/', 'tag': '9d1f9865-1cf1-4b43-8f88-ce330eb9210d', 'options': 'recursive,force'} 2023-07-21T22:30:16.472 INFO:tasks.workunit.client.0.smithi107.stdout:Spawning 3 writers... 2023-07-21T22:30:16.472 INFO:tasks.workunit.client.0.smithi107.stdout:Spawning 1 rewriters... 2023-07-21T22:30:16.472 INFO:tasks.workunit.client.0.smithi107.stdout:Spawning 5 commenters... 2023-07-21T22:30:16.473 INFO:tasks.workunit.client.0.smithi107.stdout:Spawning 100 readers... 2023-07-21T22:30:16.473 INFO:tasks.workunit.client.0.smithi107.stdout:Benchmarking for 30 iterations. 2023-07-21T22:30:16.473 INFO:tasks.workunit.client.0.smithi107.stdout:The test will run during 5 minutes. 2023-07-21T22:30:16.474 INFO:tasks.workunit.client.0.smithi107.stdout: 2023-07-21T22:30:16.474 INFO:tasks.workunit.client.0.smithi107.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2023-07-21T22:30:17.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:17 smithi190 ceph-mon[98334]: pgmap v329: 97 pgs: 97 active+clean; 2.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 191 KiB/s wr, 30 op/s 2023-07-21T22:30:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:17 smithi190 ceph-mon[98334]: scrub queued for path: / 2023-07-21T22:30:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:17 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:17 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:30:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:17 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:30:17.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:17 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:17.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:17 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:17.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:17 smithi132 ceph-mon[97863]: pgmap v329: 97 pgs: 97 active+clean; 2.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 191 KiB/s wr, 30 op/s 2023-07-21T22:30:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:17 smithi132 ceph-mon[97863]: scrub queued for path: / 2023-07-21T22:30:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:17 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:17 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:30:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:17 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:30:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:17 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:17 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:17.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:17 smithi107 ceph-mon[96543]: pgmap v329: 97 pgs: 97 active+clean; 2.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 191 KiB/s wr, 30 op/s 2023-07-21T22:30:17.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:17 smithi107 ceph-mon[96543]: scrub queued for path: / 2023-07-21T22:30:17.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:17 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:17.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:17 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:30:17.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:17 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:30:17.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:17 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:17.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:17 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:19.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:19 smithi190 ceph-mon[98334]: pgmap v330: 97 pgs: 97 active+clean; 5.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 537 KiB/s wr, 66 op/s 2023-07-21T22:30:19.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:19 smithi132 ceph-mon[97863]: pgmap v330: 97 pgs: 97 active+clean; 5.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 537 KiB/s wr, 66 op/s 2023-07-21T22:30:19.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:19 smithi107 ceph-mon[96543]: pgmap v330: 97 pgs: 97 active+clean; 5.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 537 KiB/s wr, 66 op/s 2023-07-21T22:30:21.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:21 smithi190 ceph-mon[98334]: pgmap v331: 97 pgs: 97 active+clean; 5.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 539 KiB/s wr, 64 op/s 2023-07-21T22:30:21.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:21 smithi132 ceph-mon[97863]: pgmap v331: 97 pgs: 97 active+clean; 5.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 539 KiB/s wr, 64 op/s 2023-07-21T22:30:21.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:21 smithi107 ceph-mon[96543]: pgmap v331: 97 pgs: 97 active+clean; 5.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 539 KiB/s wr, 64 op/s 2023-07-21T22:30:23.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:23 smithi190 ceph-mon[98334]: pgmap v332: 97 pgs: 97 active+clean; 17 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2023-07-21T22:30:23.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:23 smithi132 ceph-mon[97863]: pgmap v332: 97 pgs: 97 active+clean; 17 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2023-07-21T22:30:23.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:23 smithi107 ceph-mon[96543]: pgmap v332: 97 pgs: 97 active+clean; 17 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2023-07-21T22:30:25.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:25 smithi190 ceph-mon[98334]: pgmap v333: 97 pgs: 97 active+clean; 22 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 2.0 MiB/s wr, 119 op/s 2023-07-21T22:30:25.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:25 smithi132 ceph-mon[97863]: pgmap v333: 97 pgs: 97 active+clean; 22 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 2.0 MiB/s wr, 119 op/s 2023-07-21T22:30:25.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:25 smithi107 ceph-mon[96543]: pgmap v333: 97 pgs: 97 active+clean; 22 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 2.0 MiB/s wr, 119 op/s 2023-07-21T22:30:26.474 INFO:tasks.workunit.client.0.smithi107.stdout: 16 215339 922 130451 934 31862 895 2023-07-21T22:30:26.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:26 smithi190 ceph-mon[98334]: scrub summary: idle 2023-07-21T22:30:26.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:26 smithi132 ceph-mon[97863]: scrub summary: idle 2023-07-21T22:30:26.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:26 smithi107 ceph-mon[96543]: scrub summary: idle 2023-07-21T22:30:27.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:27 smithi190 ceph-mon[98334]: pgmap v334: 97 pgs: 97 active+clean; 38 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2023-07-21T22:30:27.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:27 smithi190 ceph-mon[98334]: scrub summary: idle 2023-07-21T22:30:27.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:27 smithi132 ceph-mon[97863]: pgmap v334: 97 pgs: 97 active+clean; 38 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2023-07-21T22:30:27.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:27 smithi132 ceph-mon[97863]: scrub summary: idle 2023-07-21T22:30:27.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:27 smithi107 ceph-mon[96543]: pgmap v334: 97 pgs: 97 active+clean; 38 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 178 op/s 2023-07-21T22:30:27.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:27 smithi107 ceph-mon[96543]: scrub summary: idle 2023-07-21T22:30:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.899+0000 7f7a56b5f700 1 -- 172.21.15.107:0/1369133659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500fea00 msgr2=0x7f7a500fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.899+0000 7f7a56b5f700 1 --2- 172.21.15.107:0/1369133659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500fea00 0x7f7a500fee10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00b0a0 tx=0x7f7a4c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.899+0000 7f7a56b5f700 1 -- 172.21.15.107:0/1369133659 shutdown_connections 2023-07-21T22:30:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.899+0000 7f7a56b5f700 1 --2- 172.21.15.107:0/1369133659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a50100640 0x7f7a50106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.899+0000 7f7a56b5f700 1 --2- 172.21.15.107:0/1369133659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500fea00 0x7f7a500fee10 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.899+0000 7f7a56b5f700 1 --2- 172.21.15.107:0/1369133659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a500ffc90 0x7f7a50100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.899+0000 7f7a56b5f700 1 -- 172.21.15.107:0/1369133659 >> 172.21.15.107:0/1369133659 conn(0x7f7a500fa000 msgr2=0x7f7a500fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.899+0000 7f7a56b5f700 1 -- 172.21.15.107:0/1369133659 shutdown_connections 2023-07-21T22:30:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.899+0000 7f7a56b5f700 1 -- 172.21.15.107:0/1369133659 wait complete. 2023-07-21T22:30:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a56b5f700 1 Processor -- start 2023-07-21T22:30:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a56b5f700 1 -- start start 2023-07-21T22:30:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a56b5f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a500fea00 0x7f7a501948b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a56b5f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500ffc90 0x7f7a50194df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a56b5f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a50100640 0x7f7a50195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a56b5f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a501962d0 con 0x7f7a500ffc90 2023-07-21T22:30:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a56b5f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50196440 con 0x7f7a500fea00 2023-07-21T22:30:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a56b5f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50196600 con 0x7f7a50100640 2023-07-21T22:30:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a47fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500ffc90 0x7f7a50194df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a47fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500ffc90 0x7f7a50194df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49200/0 (socket says 172.21.15.107:49200) 2023-07-21T22:30:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a47fff700 1 -- 172.21.15.107:0/4205385544 learned_addr learned my addr 172.21.15.107:0/4205385544 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a548fb700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a500fea00 0x7f7a501948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a550fc700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a50100640 0x7f7a50195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a47fff700 1 -- 172.21.15.107:0/4205385544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a50100640 msgr2=0x7f7a50195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a47fff700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a50100640 0x7f7a50195c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a47fff700 1 -- 172.21.15.107:0/4205385544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a500fea00 msgr2=0x7f7a501948b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a47fff700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a500fea00 0x7f7a501948b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a47fff700 1 -- 172.21.15.107:0/4205385544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c009d00 con 0x7f7a500ffc90 2023-07-21T22:30:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a550fc700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a50100640 0x7f7a50195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.900+0000 7f7a548fb700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a500fea00 0x7f7a501948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.901+0000 7f7a47fff700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500ffc90 0x7f7a50194df0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000e9a0 tx=0x7f7a4000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.901+0000 7f7a45ffb700 1 -- 172.21.15.107:0/4205385544 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40009790 con 0x7f7a500ffc90 2023-07-21T22:30:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.901+0000 7f7a45ffb700 1 -- 172.21.15.107:0/4205385544 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a4000cbd0 con 0x7f7a500ffc90 2023-07-21T22:30:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.901+0000 7f7a56b5f700 1 -- 172.21.15.107:0/4205385544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a501968e0 con 0x7f7a500ffc90 2023-07-21T22:30:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.901+0000 7f7a45ffb700 1 -- 172.21.15.107:0/4205385544 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40018680 con 0x7f7a500ffc90 2023-07-21T22:30:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.901+0000 7f7a56b5f700 1 -- 172.21.15.107:0/4205385544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5019d0a0 con 0x7f7a500ffc90 2023-07-21T22:30:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.905+0000 7f7a45ffb700 1 -- 172.21.15.107:0/4205385544 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7a4000cd40 con 0x7f7a500ffc90 2023-07-21T22:30:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.906+0000 7f7a45ffb700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7a380a1150 0x7f7a380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.906+0000 7f7a62ce4b80 1 Processor -- start 2023-07-21T22:30:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.906+0000 7f7a45ffb700 1 -- 172.21.15.107:0/4205385544 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7a40014070 con 0x7f7a500ffc90 2023-07-21T22:30:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.906+0000 7f7a548fb700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7a380a1150 0x7f7a380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.906+0000 7f7a62ce4b80 1 -- start start 2023-07-21T22:30:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a62ce4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 0x55d0df1162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a62ce4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0df116830 0x55d0df117040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a62ce4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df117580 0x55d0df1179f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a62ce4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0df01bc50 con 0x55d0def726b0 2023-07-21T22:30:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a62ce4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0df06e9e0 con 0x55d0df116830 2023-07-21T22:30:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a62ce4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0df067e70 con 0x55d0df117580 2023-07-21T22:30:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a548fb700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7a380a1150 0x7f7a380a3600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00b780 tx=0x7f7a4c006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a548fb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 0x55d0df1162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a548fb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 0x55d0df1162f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49206/0 (socket says 172.21.15.107:49206) 2023-07-21T22:30:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a47fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0df116830 0x55d0df117040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a548fb700 1 -- 172.21.15.107:0/4007830698 learned_addr learned my addr 172.21.15.107:0/4007830698 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a548fb700 1 -- 172.21.15.107:0/4007830698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df117580 msgr2=0x55d0df1179f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:30:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a548fb700 1 --2- 172.21.15.107:0/4007830698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df117580 0x55d0df1179f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a548fb700 1 -- 172.21.15.107:0/4007830698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0df116830 msgr2=0x55d0df117040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a548fb700 1 --2- 172.21.15.107:0/4007830698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0df116830 0x55d0df117040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a548fb700 1 -- 172.21.15.107:0/4007830698 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0dee95d80 con 0x55d0def726b0 2023-07-21T22:30:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a47fff700 1 --2- 172.21.15.107:0/4007830698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0df116830 0x55d0df117040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:28.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a548fb700 1 --2- 172.21.15.107:0/4007830698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 0x55d0df1162f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c015000 tx=0x7f7a4c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449f590c60a22e74 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.907+0000 7f7a36ffd700 1 -- 172.21.15.107:0/4007830698 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00e040 con 0x55d0def726b0 2023-07-21T22:30:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.908+0000 7f7a36ffd700 1 -- 172.21.15.107:0/4007830698 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a4c007810 con 0x55d0def726b0 2023-07-21T22:30:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.908+0000 7f7a36ffd700 1 -- 172.21.15.107:0/4007830698 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c028d60 con 0x55d0def726b0 2023-07-21T22:30:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.908+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/4007830698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 msgr2=0x55d0df1162f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.908+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/4007830698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 0x55d0df1162f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c015000 tx=0x7f7a4c01d000 comp rx=0 tx=0).stop 2023-07-21T22:30:28.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.911+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/4007830698 shutdown_connections 2023-07-21T22:30:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.911+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/4007830698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df117580 0x55d0df1179f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.911+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/4007830698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 0x55d0df1162f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.911+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/4007830698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0df116830 0x55d0df117040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.911+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/4007830698 >> 172.21.15.107:0/4007830698 conn(0x55d0df10bae0 msgr2=0x55d0df110c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.911+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/4007830698 shutdown_connections 2023-07-21T22:30:28.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.911+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/4007830698 wait complete. 2023-07-21T22:30:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.911+0000 7f7a62ce4b80 1 -- start start 2023-07-21T22:30:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.911+0000 7f7a62ce4b80 1 Processor -- start 2023-07-21T22:30:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a62ce4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0def726b0 0x55d0df11dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a62ce4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df116830 0x55d0df1288d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a62ce4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0df117580 0x55d0df12aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a62ce4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0df007a60 con 0x55d0df117580 2023-07-21T22:30:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a62ce4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0df007760 con 0x55d0def726b0 2023-07-21T22:30:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a62ce4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0deeeb8c0 con 0x55d0df116830 2023-07-21T22:30:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a62ce4b80 10 client.0 _ll_register_callbacks cb 0x55d0df005700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a548fb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0def726b0 0x55d0df11dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a548fb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0def726b0 0x55d0df11dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40040/0 (socket says 172.21.15.107:40040) 2023-07-21T22:30:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a548fb700 1 -- 172.21.15.107:0/1285524400 learned_addr learned my addr 172.21.15.107:0/1285524400 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a550fc700 1 --2- 172.21.15.107:0/1285524400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0df117580 0x55d0df12aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a47fff700 1 --2- 172.21.15.107:0/1285524400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df116830 0x55d0df1288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a548fb700 1 -- 172.21.15.107:0/1285524400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df116830 msgr2=0x55d0df1288d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a548fb700 1 --2- 172.21.15.107:0/1285524400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df116830 0x55d0df1288d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a548fb700 1 -- 172.21.15.107:0/1285524400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0df117580 msgr2=0x55d0df12aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a548fb700 1 --2- 172.21.15.107:0/1285524400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0df117580 0x55d0df12aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a548fb700 1 -- 172.21.15.107:0/1285524400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5019d0a0 con 0x55d0def726b0 2023-07-21T22:30:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a548fb700 1 --2- 172.21.15.107:0/1285524400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0def726b0 0x55d0df11dd10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c007a60 tx=0x7f7a4c028680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbae07264dfa98ed server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.912+0000 7f7a347f8700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:30:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.913+0000 7f7a347f8700 1 -- 172.21.15.107:0/1285524400 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00e040 con 0x55d0def726b0 2023-07-21T22:30:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.913+0000 7f7a347f8700 1 -- 172.21.15.107:0/1285524400 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7a4c037070 con 0x55d0def726b0 2023-07-21T22:30:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.913+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/1285524400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0defc1490 con 0x55d0def726b0 2023-07-21T22:30:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.913+0000 7f7a347f8700 1 -- 172.21.15.107:0/1285524400 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a4c003c10 con 0x55d0def726b0 2023-07-21T22:30:28.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.913+0000 7f7a62ce4b80 10 client.25367 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.913+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/1285524400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f7a50107750 con 0x55d0def726b0 2023-07-21T22:30:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.913+0000 7f7a347f8700 1 -- 172.21.15.107:0/1285524400 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d0defc1490 con 0x55d0def726b0 2023-07-21T22:30:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.914+0000 7f7a347f8700 1 -- 172.21.15.107:0/1285524400 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7a4c03b420 con 0x55d0def726b0 2023-07-21T22:30:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.914+0000 7f7a62ce4b80 10 client.25367 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.914+0000 7f7a62ce4b80 10 client.25367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.914+0000 7f7a62ce4b80 20 client.25367 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.914+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/1285524400 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55d0df12f6c0 0x55d0df131ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.914+0000 7f7a62ce4b80 4 client.25367 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.914+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/1285524400 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0df060e50 con 0x55d0df12f6c0 2023-07-21T22:30:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.914+0000 7f7a47fff700 1 --2- 172.21.15.107:0/1285524400 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55d0df12f6c0 0x55d0df131ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.914+0000 7f7a347f8700 10 client.25367 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.914+0000 7f7a47fff700 1 --2- 172.21.15.107:0/1285524400 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55d0df12f6c0 0x55d0df131ad0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71ad35cafe9a2a87 server_cookie=92c38f6cfeee9e98 in_seq=0 out_seq=0 2023-07-21T22:30:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a347f8700 1 -- 172.21.15.107:0/1285524400 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d0df060e50 con 0x55d0df12f6c0 2023-07-21T22:30:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a347f8700 10 client.25367 handle_command_reply: tid=0 2023-07-21T22:30:28.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a62ce4b80 1 client.25367 shutdown 2023-07-21T22:30:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/1285524400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0def726b0 msgr2=0x55d0df11dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/1285524400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0def726b0 0x55d0df11dd10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c007a60 tx=0x7f7a4c028680 comp rx=0 tx=0).stop 2023-07-21T22:30:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/1285524400 shutdown_connections 2023-07-21T22:30:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/1285524400 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55d0df12f6c0 0x55d0df131ad0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/1285524400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0df117580 0x55d0df12aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/1285524400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0def726b0 0x55d0df11dd10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a347f8700 0 client.25367 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/1285524400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df116830 0x55d0df1288d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/1285524400 >> 172.21.15.107:0/1285524400 conn(0x55d0df10bae0 msgr2=0x55d0df110be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/1285524400 shutdown_connections 2023-07-21T22:30:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.915+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/1285524400 wait complete. 2023-07-21T22:30:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.916+0000 7f7a62ce4b80 20 client.25367 trim_cache size 0 max 16384 2023-07-21T22:30:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a62ce4b80 1 Processor -- start 2023-07-21T22:30:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a62ce4b80 1 -- start start 2023-07-21T22:30:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a62ce4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0def726b0 0x55d0df1288d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a62ce4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df116830 0x55d0df13f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a62ce4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0df117580 0x55d0df11da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a62ce4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0df108a40 con 0x55d0df117580 2023-07-21T22:30:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a62ce4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0df060e50 con 0x55d0def726b0 2023-07-21T22:30:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a62ce4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0defc1490 con 0x55d0df116830 2023-07-21T22:30:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a550fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0df117580 0x55d0df11da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a548fb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0def726b0 0x55d0df1288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a550fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0df117580 0x55d0df11da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49222/0 (socket says 172.21.15.107:49222) 2023-07-21T22:30:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a550fc700 1 -- 172.21.15.107:0/4118755194 learned_addr learned my addr 172.21.15.107:0/4118755194 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a47fff700 1 --2- 172.21.15.107:0/4118755194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df116830 0x55d0df13f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a548fb700 1 -- 172.21.15.107:0/4118755194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df116830 msgr2=0x55d0df13f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a548fb700 1 --2- 172.21.15.107:0/4118755194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df116830 0x55d0df13f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a548fb700 1 -- 172.21.15.107:0/4118755194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0df117580 msgr2=0x55d0df11da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a548fb700 1 --2- 172.21.15.107:0/4118755194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0df117580 0x55d0df11da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a548fb700 1 -- 172.21.15.107:0/4118755194 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0dee7ec00 con 0x55d0def726b0 2023-07-21T22:30:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a550fc700 1 --2- 172.21.15.107:0/4118755194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0df117580 0x55d0df11da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a47fff700 1 --2- 172.21.15.107:0/4118755194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df116830 0x55d0df13f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.931+0000 7f7a548fb700 1 --2- 172.21.15.107:0/4118755194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0def726b0 0x55d0df1288d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c03b7e0 tx=0x7f7a4c03b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b222ef73dea45772 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.932+0000 7f7a377fe700 1 -- 172.21.15.107:0/4118755194 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c009b80 con 0x55d0def726b0 2023-07-21T22:30:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.932+0000 7f7a377fe700 1 -- 172.21.15.107:0/4118755194 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a4c035070 con 0x55d0def726b0 2023-07-21T22:30:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.932+0000 7f7a377fe700 1 -- 172.21.15.107:0/4118755194 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c01b710 con 0x55d0def726b0 2023-07-21T22:30:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.932+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/4118755194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0def726b0 msgr2=0x55d0df1288d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.932+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/4118755194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0def726b0 0x55d0df1288d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c03b7e0 tx=0x7f7a4c03b900 comp rx=0 tx=0).stop 2023-07-21T22:30:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.932+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/4118755194 shutdown_connections 2023-07-21T22:30:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.932+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/4118755194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0df117580 0x55d0df11da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.932+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/4118755194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0def726b0 0x55d0df1288d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.932+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/4118755194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df116830 0x55d0df13f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.932+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/4118755194 >> 172.21.15.107:0/4118755194 conn(0x55d0df10bae0 msgr2=0x55d0df136430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.932+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/4118755194 shutdown_connections 2023-07-21T22:30:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/4118755194 wait complete. 2023-07-21T22:30:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a62ce4b80 1 -- start start 2023-07-21T22:30:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a62ce4b80 1 Processor -- start 2023-07-21T22:30:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a62ce4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 0x55d0df13c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a62ce4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0df116830 0x55d0df13cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a62ce4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df117580 0x55d0df13db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a62ce4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0df13e0c0 con 0x55d0def726b0 2023-07-21T22:30:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a62ce4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0df13e230 con 0x55d0df116830 2023-07-21T22:30:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a62ce4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0df13e3f0 con 0x55d0df117580 2023-07-21T22:30:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a62ce4b80 10 client.0 _ll_register_callbacks cb 0x7f7a40010aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a548fb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 0x55d0df13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a548fb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 0x55d0df13c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49226/0 (socket says 172.21.15.107:49226) 2023-07-21T22:30:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a548fb700 1 -- 172.21.15.107:0/3779844604 learned_addr learned my addr 172.21.15.107:0/3779844604 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a47fff700 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0df116830 0x55d0df13cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.933+0000 7f7a550fc700 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df117580 0x55d0df13db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a548fb700 1 -- 172.21.15.107:0/3779844604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df117580 msgr2=0x55d0df13db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a548fb700 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df117580 0x55d0df13db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a548fb700 1 -- 172.21.15.107:0/3779844604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0df116830 msgr2=0x55d0df13cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a548fb700 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0df116830 0x55d0df13cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a548fb700 1 -- 172.21.15.107:0/3779844604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a48003ba0 con 0x55d0def726b0 2023-07-21T22:30:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a550fc700 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df117580 0x55d0df13db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a548fb700 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 0x55d0df13c630 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c000f80 tx=0x7f7a4c028c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08743f2bee2848 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a347f8700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a347f8700 1 -- 172.21.15.107:0/3779844604 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00e070 con 0x55d0def726b0 2023-07-21T22:30:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a347f8700 1 -- 172.21.15.107:0/3779844604 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7a4c037030 con 0x55d0def726b0 2023-07-21T22:30:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/3779844604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a4c007810 con 0x55d0def726b0 2023-07-21T22:30:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a347f8700 1 -- 172.21.15.107:0/3779844604 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a4c03ba50 con 0x55d0def726b0 2023-07-21T22:30:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a62ce4b80 10 client.15906 fetch_fsmap learned FSMap version 28 2023-07-21T22:30:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/3779844604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f7a50107750 con 0x55d0def726b0 2023-07-21T22:30:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a347f8700 1 -- 172.21.15.107:0/3779844604 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a4c007810 con 0x55d0def726b0 2023-07-21T22:30:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a347f8700 1 -- 172.21.15.107:0/3779844604 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f7a4c005320 con 0x55d0def726b0 2023-07-21T22:30:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a62ce4b80 10 client.15906 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:30:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a62ce4b80 10 client.15906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a62ce4b80 20 client.15906 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55d0df183d00 0x55d0df1860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a62ce4b80 4 client.15906 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:30:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.934+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/3779844604 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d0df022ff0 con 0x55d0df183d00 2023-07-21T22:30:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.935+0000 7f7a47fff700 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55d0df183d00 0x55d0df1860e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.935+0000 7f7a47fff700 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55d0df183d00 0x55d0df1860e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d801b52f53da31 server_cookie=f791a81399206037 in_seq=0 out_seq=0 2023-07-21T22:30:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.935+0000 7f7a347f8700 10 client.15906 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a347f8700 1 -- 172.21.15.107:0/3779844604 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d0df022ff0 con 0x55d0df183d00 2023-07-21T22:30:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a347f8700 10 client.15906 handle_command_reply: tid=0 2023-07-21T22:30:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a62ce4b80 1 client.15906 shutdown 2023-07-21T22:30:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/3779844604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 msgr2=0x55d0df13c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 0x55d0df13c630 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c000f80 tx=0x7f7a4c028c70 comp rx=0 tx=0).stop 2023-07-21T22:30:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/3779844604 shutdown_connections 2023-07-21T22:30:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55d0df183d00 0x55d0df1860e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d0df117580 0x55d0df13db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a347f8700 0 client.15906 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0def726b0 0x55d0df13c630 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a62ce4b80 1 --2- 172.21.15.107:0/3779844604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0df116830 0x55d0df13cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/3779844604 >> 172.21.15.107:0/3779844604 conn(0x55d0df10bae0 msgr2=0x55d0df11d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/3779844604 shutdown_connections 2023-07-21T22:30:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a62ce4b80 1 -- 172.21.15.107:0/3779844604 wait complete. 2023-07-21T22:30:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.945+0000 7f7a62ce4b80 20 client.15906 trim_cache size 0 max 16384 2023-07-21T22:30:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:30:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:30:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:30:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:30:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.946+0000 7f7a56b5f700 1 -- 172.21.15.107:0/4205385544 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7a380a1150 msgr2=0x7f7a380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.946+0000 7f7a56b5f700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7a380a1150 0x7f7a380a3600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00b780 tx=0x7f7a4c006210 comp rx=0 tx=0).stop 2023-07-21T22:30:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.946+0000 7f7a56b5f700 1 -- 172.21.15.107:0/4205385544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500ffc90 msgr2=0x7f7a50194df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.946+0000 7f7a56b5f700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500ffc90 0x7f7a50194df0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000e9a0 tx=0x7f7a4000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.946+0000 7f7a56b5f700 1 -- 172.21.15.107:0/4205385544 shutdown_connections 2023-07-21T22:30:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.946+0000 7f7a56b5f700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7a380a1150 0x7f7a380a3600 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.946+0000 7f7a56b5f700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a50100640 0x7f7a50195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.946+0000 7f7a56b5f700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a500fea00 0x7f7a501948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.946+0000 7f7a56b5f700 1 --2- 172.21.15.107:0/4205385544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500ffc90 0x7f7a50194df0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.946+0000 7f7a56b5f700 1 -- 172.21.15.107:0/4205385544 >> 172.21.15.107:0/4205385544 conn(0x7f7a500fa000 msgr2=0x7f7a500fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.946+0000 7f7a56b5f700 1 -- 172.21.15.107:0/4205385544 shutdown_connections 2023-07-21T22:30:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:28.946+0000 7f7a56b5f700 1 -- 172.21.15.107:0/4205385544 wait complete. 2023-07-21T22:30:29.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:29 smithi190 ceph-mon[98334]: pgmap v335: 97 pgs: 97 active+clean; 52 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 232 op/s 2023-07-21T22:30:29.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:29 smithi132 ceph-mon[97863]: pgmap v335: 97 pgs: 97 active+clean; 52 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 232 op/s 2023-07-21T22:30:29.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:29 smithi107 ceph-mon[96543]: pgmap v335: 97 pgs: 97 active+clean; 52 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 232 op/s 2023-07-21T22:30:31.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:31 smithi190 ceph-mon[98334]: pgmap v336: 97 pgs: 97 active+clean; 52 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 196 op/s 2023-07-21T22:30:31.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:31 smithi132 ceph-mon[97863]: pgmap v336: 97 pgs: 97 active+clean; 52 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 196 op/s 2023-07-21T22:30:31.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:31 smithi107 ceph-mon[96543]: pgmap v336: 97 pgs: 97 active+clean; 52 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 196 op/s 2023-07-21T22:30:33.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:33 smithi190 ceph-mon[98334]: pgmap v337: 97 pgs: 97 active+clean; 53 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 221 op/s 2023-07-21T22:30:33.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:33 smithi132 ceph-mon[97863]: pgmap v337: 97 pgs: 97 active+clean; 53 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 221 op/s 2023-07-21T22:30:33.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:33 smithi107 ceph-mon[96543]: pgmap v337: 97 pgs: 97 active+clean; 53 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 221 op/s 2023-07-21T22:30:34.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:34 smithi190 ceph-mon[98334]: pgmap v338: 97 pgs: 97 active+clean; 57 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 195 op/s 2023-07-21T22:30:34.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:34 smithi132 ceph-mon[97863]: pgmap v338: 97 pgs: 97 active+clean; 57 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 195 op/s 2023-07-21T22:30:34.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:34 smithi107 ceph-mon[96543]: pgmap v338: 97 pgs: 97 active+clean; 57 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 195 op/s 2023-07-21T22:30:36.474 INFO:tasks.workunit.client.0.smithi107.stdout: 25 838728 630 546394 444 199734 1345 2023-07-21T22:30:36.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:36 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:36.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:36 smithi190 ceph-mon[98334]: pgmap v339: 97 pgs: 97 active+clean; 63 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 175 op/s 2023-07-21T22:30:36.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:36 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:36.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:36 smithi132 ceph-mon[97863]: pgmap v339: 97 pgs: 97 active+clean; 63 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 175 op/s 2023-07-21T22:30:36.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:36 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:36.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:36 smithi107 ceph-mon[96543]: pgmap v339: 97 pgs: 97 active+clean; 63 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 175 op/s 2023-07-21T22:30:39.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:38 smithi190 ceph-mon[98334]: pgmap v340: 97 pgs: 97 active+clean; 68 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 129 op/s 2023-07-21T22:30:39.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:38 smithi132 ceph-mon[97863]: pgmap v340: 97 pgs: 97 active+clean; 68 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 129 op/s 2023-07-21T22:30:39.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:38 smithi107 ceph-mon[96543]: pgmap v340: 97 pgs: 97 active+clean; 68 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 129 op/s 2023-07-21T22:30:41.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:40 smithi190 ceph-mon[98334]: pgmap v341: 97 pgs: 97 active+clean; 68 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 60 op/s 2023-07-21T22:30:41.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:40 smithi132 ceph-mon[97863]: pgmap v341: 97 pgs: 97 active+clean; 68 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 60 op/s 2023-07-21T22:30:41.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:40 smithi107 ceph-mon[96543]: pgmap v341: 97 pgs: 97 active+clean; 68 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 60 op/s 2023-07-21T22:30:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.128+0000 7f4c62dc1700 1 -- 172.21.15.107:0/1635971977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c10bd60 msgr2=0x7f4c5c10e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:42.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.128+0000 7f4c62dc1700 1 --2- 172.21.15.107:0/1635971977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c10bd60 0x7f4c5c10e210 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009e00 tx=0x7f4c5400b330 comp rx=0 tx=0).stop 2023-07-21T22:30:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.130+0000 7f4c62dc1700 1 -- 172.21.15.107:0/1635971977 shutdown_connections 2023-07-21T22:30:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.130+0000 7f4c62dc1700 1 --2- 172.21.15.107:0/1635971977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c10bd60 0x7f4c5c10e210 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.130+0000 7f4c62dc1700 1 --2- 172.21.15.107:0/1635971977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c070f00 0x7f4c5c071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.130+0000 7f4c62dc1700 1 --2- 172.21.15.107:0/1635971977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c5c071850 0x7f4c5c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.130+0000 7f4c62dc1700 1 -- 172.21.15.107:0/1635971977 >> 172.21.15.107:0/1635971977 conn(0x7f4c5c06ca50 msgr2=0x7f4c5c06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.133+0000 7f4c62dc1700 1 -- 172.21.15.107:0/1635971977 shutdown_connections 2023-07-21T22:30:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.137+0000 7f4c62dc1700 1 -- 172.21.15.107:0/1635971977 wait complete. 2023-07-21T22:30:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.137+0000 7f4c62dc1700 1 Processor -- start 2023-07-21T22:30:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.137+0000 7f4c62dc1700 1 -- start start 2023-07-21T22:30:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.138+0000 7f4c62dc1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c070f00 0x7f4c5c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.138+0000 7f4c62dc1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c5c071850 0x7f4c5c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.138+0000 7f4c62dc1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c10bd60 0x7f4c5c1c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.138+0000 7f4c62dc1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c1c5c40 con 0x7f4c5c10bd60 2023-07-21T22:30:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.138+0000 7f4c62dc1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c1c5db0 con 0x7f4c5c070f00 2023-07-21T22:30:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.138+0000 7f4c62dc1700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c1c5f70 con 0x7f4c5c071850 2023-07-21T22:30:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.138+0000 7f4c60b5d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c070f00 0x7f4c5c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.138+0000 7f4c60b5d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c070f00 0x7f4c5c19c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40072/0 (socket says 172.21.15.107:40072) 2023-07-21T22:30:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.138+0000 7f4c60b5d700 1 -- 172.21.15.107:0/964328946 learned_addr learned my addr 172.21.15.107:0/964328946 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.139+0000 7f4c6135e700 1 --2- 172.21.15.107:0/964328946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c10bd60 0x7f4c5c1c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.140+0000 7f4c6135e700 1 -- 172.21.15.107:0/964328946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c5c071850 msgr2=0x7f4c5c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.140+0000 7f4c6135e700 1 --2- 172.21.15.107:0/964328946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c5c071850 0x7f4c5c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.140+0000 7f4c6135e700 1 -- 172.21.15.107:0/964328946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c070f00 msgr2=0x7f4c5c19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.140+0000 7f4c6135e700 1 --2- 172.21.15.107:0/964328946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c070f00 0x7f4c5c19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.140+0000 7f4c6135e700 1 -- 172.21.15.107:0/964328946 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c54005e70 con 0x7f4c5c10bd60 2023-07-21T22:30:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.140+0000 7f4c6135e700 1 --2- 172.21.15.107:0/964328946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c10bd60 0x7f4c5c1c5630 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009fd0 tx=0x7f4c540077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.141+0000 7f4c59ffb700 1 -- 172.21.15.107:0/964328946 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5400e040 con 0x7f4c5c10bd60 2023-07-21T22:30:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.141+0000 7f4c59ffb700 1 -- 172.21.15.107:0/964328946 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c5401d940 con 0x7f4c5c10bd60 2023-07-21T22:30:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.141+0000 7f4c59ffb700 1 -- 172.21.15.107:0/964328946 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5401b9b0 con 0x7f4c5c10bd60 2023-07-21T22:30:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.142+0000 7f4c62dc1700 1 -- 172.21.15.107:0/964328946 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5c1c6190 con 0x7f4c5c10bd60 2023-07-21T22:30:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.142+0000 7f4c62dc1700 1 -- 172.21.15.107:0/964328946 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5c1c6640 con 0x7f4c5c10bd60 2023-07-21T22:30:42.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.148+0000 7f4c6ef46b80 1 Processor -- start 2023-07-21T22:30:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.150+0000 7f4c59ffb700 1 -- 172.21.15.107:0/964328946 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4c5401d480 con 0x7f4c5c10bd60 2023-07-21T22:30:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.151+0000 7f4c59ffb700 1 --2- 172.21.15.107:0/964328946 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4c440a1350 0x7f4c440a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.152+0000 7f4c59ffb700 1 -- 172.21.15.107:0/964328946 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4c540ce2e0 con 0x7f4c5c10bd60 2023-07-21T22:30:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.152+0000 7f4c6ef46b80 1 -- start start 2023-07-21T22:30:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.152+0000 7f4c6ef46b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 0x55ace0a212f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.152+0000 7f4c6ef46b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 0x55ace0a22040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.152+0000 7f4c6ef46b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace0a22580 0x55ace0a229f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.152+0000 7f4c6ef46b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace0926c50 con 0x55ace087d6b0 2023-07-21T22:30:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.152+0000 7f4c6ef46b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace09799e0 con 0x55ace0a21830 2023-07-21T22:30:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.152+0000 7f4c6ef46b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace0972e70 con 0x55ace0a22580 2023-07-21T22:30:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.153+0000 7f4c60b5d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 0x55ace0a212f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.153+0000 7f4c60b5d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 0x55ace0a212f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49254/0 (socket says 172.21.15.107:49254) 2023-07-21T22:30:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.153+0000 7f4c60b5d700 1 -- 172.21.15.107:0/2602466788 learned_addr learned my addr 172.21.15.107:0/2602466788 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.154+0000 7f4c6135e700 1 --2- 172.21.15.107:0/2602466788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace0a22580 0x55ace0a229f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.154+0000 7f4c60b5d700 1 -- 172.21.15.107:0/2602466788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace0a22580 msgr2=0x55ace0a229f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.154+0000 7f4c60b5d700 1 --2- 172.21.15.107:0/2602466788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace0a22580 0x55ace0a229f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.154+0000 7f4c60b5d700 1 -- 172.21.15.107:0/2602466788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 msgr2=0x55ace0a22040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.154+0000 7f4c60b5d700 1 --2- 172.21.15.107:0/2602466788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 0x55ace0a22040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.154+0000 7f4c60b5d700 1 -- 172.21.15.107:0/2602466788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ace07a0d80 con 0x55ace087d6b0 2023-07-21T22:30:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.155+0000 7f4c60b5d700 1 --2- 172.21.15.107:0/2602466788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 0x55ace0a212f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c01ada0 tx=0x7f4c4c00d5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2767ba9378b4657 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.155+0000 7f4c42ffd700 1 -- 172.21.15.107:0/2602466788 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c01a870 con 0x55ace087d6b0 2023-07-21T22:30:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.155+0000 7f4c42ffd700 1 -- 172.21.15.107:0/2602466788 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c4c00dbd0 con 0x55ace087d6b0 2023-07-21T22:30:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.155+0000 7f4c42ffd700 1 -- 172.21.15.107:0/2602466788 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c0185c0 con 0x55ace087d6b0 2023-07-21T22:30:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.156+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2602466788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 msgr2=0x55ace0a212f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.156+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2602466788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 0x55ace0a212f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c01ada0 tx=0x7f4c4c00d5c0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.160+0000 7f4c5bfff700 1 --2- 172.21.15.107:0/964328946 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4c440a1350 0x7f4c440a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.160+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2602466788 shutdown_connections 2023-07-21T22:30:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.160+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2602466788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace0a22580 0x55ace0a229f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.160+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2602466788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 0x55ace0a212f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.160+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2602466788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 0x55ace0a22040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.160+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2602466788 >> 172.21.15.107:0/2602466788 conn(0x55ace0a16ae0 msgr2=0x55ace0a1bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.160+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2602466788 shutdown_connections 2023-07-21T22:30:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.160+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2602466788 wait complete. 2023-07-21T22:30:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.160+0000 7f4c6ef46b80 1 -- start start 2023-07-21T22:30:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.161+0000 7f4c6ef46b80 1 Processor -- start 2023-07-21T22:30:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.161+0000 7f4c5bfff700 1 --2- 172.21.15.107:0/964328946 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4c440a1350 0x7f4c440a3800 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55ace0a1ca50 tx=0x7f4c50007b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.161+0000 7f4c6ef46b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace087d6b0 0x55ace0a2a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.161+0000 7f4c6ef46b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 0x55ace0a2ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.161+0000 7f4c6ef46b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace0a22580 0x55ace0a31d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.161+0000 7f4c6ef46b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace0912a60 con 0x55ace0a22580 2023-07-21T22:30:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.161+0000 7f4c6ef46b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace0912760 con 0x55ace0a21830 2023-07-21T22:30:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.161+0000 7f4c6ef46b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace07f68c0 con 0x55ace087d6b0 2023-07-21T22:30:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.161+0000 7f4c6ef46b80 10 client.0 _ll_register_callbacks cb 0x55ace0910700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.162+0000 7f4c6135e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace0a22580 0x55ace0a31d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.162+0000 7f4c5bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 0x55ace0a2ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.162+0000 7f4c6135e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace0a22580 0x55ace0a31d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49262/0 (socket says 172.21.15.107:49262) 2023-07-21T22:30:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.162+0000 7f4c6135e700 1 -- 172.21.15.107:0/1104952067 learned_addr learned my addr 172.21.15.107:0/1104952067 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.162+0000 7f4c6135e700 1 -- 172.21.15.107:0/1104952067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace087d6b0 msgr2=0x55ace0a2a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.162+0000 7f4c6135e700 1 --2- 172.21.15.107:0/1104952067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace087d6b0 0x55ace0a2a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.162+0000 7f4c6135e700 1 -- 172.21.15.107:0/1104952067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 msgr2=0x55ace0a2ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.162+0000 7f4c6135e700 1 --2- 172.21.15.107:0/1104952067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 0x55ace0a2ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.162+0000 7f4c6135e700 1 -- 172.21.15.107:0/1104952067 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c01a9d0 con 0x55ace0a22580 2023-07-21T22:30:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.162+0000 7f4c6135e700 1 --2- 172.21.15.107:0/1104952067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace0a22580 0x55ace0a31d80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009c10 tx=0x7f4c5401dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e45541ffa53b7d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.162+0000 7f4c3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.162+0000 7f4c3bfff700 1 -- 172.21.15.107:0/1104952067 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c540a3ac0 con 0x55ace0a22580 2023-07-21T22:30:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.162+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/1104952067 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ace08cc490 con 0x55ace0a22580 2023-07-21T22:30:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.164+0000 7f4c3bfff700 1 -- 172.21.15.107:0/1104952067 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4c540bb0b0 con 0x55ace0a22580 2023-07-21T22:30:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.164+0000 7f4c3bfff700 1 -- 172.21.15.107:0/1104952067 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c540c5ea0 con 0x55ace0a22580 2023-07-21T22:30:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.164+0000 7f4c6ef46b80 10 client.15918 fetch_fsmap learned FSMap version 30 2023-07-21T22:30:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.164+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/1104952067 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ace0789c00 con 0x55ace0a22580 2023-07-21T22:30:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.165+0000 7f4c3bfff700 1 -- 172.21.15.107:0/1104952067 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ace08cc490 con 0x55ace0a22580 2023-07-21T22:30:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.165+0000 7f4c3bfff700 1 -- 172.21.15.107:0/1104952067 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4c5409aca0 con 0x55ace0a22580 2023-07-21T22:30:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.165+0000 7f4c6ef46b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:30:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.165+0000 7f4c6ef46b80 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.165+0000 7f4c6ef46b80 20 client.15918 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.165+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/1104952067 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ace0a36400 0x55ace0a38810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.165+0000 7f4c6ef46b80 4 client.15918 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.165+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/1104952067 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ace096be50 con 0x55ace0a36400 2023-07-21T22:30:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.168+0000 7f4c60b5d700 1 --2- 172.21.15.107:0/1104952067 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ace0a36400 0x55ace0a38810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.169+0000 7f4c60b5d700 1 --2- 172.21.15.107:0/1104952067 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ace0a36400 0x55ace0a38810 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62d634e7aeff6f0b server_cookie=3c57c7d15ca0b841 in_seq=0 out_seq=0 2023-07-21T22:30:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.172+0000 7f4c3bfff700 10 client.15918 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.172+0000 7f4c3bfff700 1 -- 172.21.15.107:0/1104952067 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ace096be50 con 0x55ace0a36400 2023-07-21T22:30:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.172+0000 7f4c3bfff700 10 client.15918 handle_command_reply: tid=0 2023-07-21T22:30:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.172+0000 7f4c6ef46b80 1 client.15918 shutdown 2023-07-21T22:30:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.177+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/1104952067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace0a22580 msgr2=0x55ace0a31d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.177+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/1104952067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace0a22580 0x55ace0a31d80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009c10 tx=0x7f4c5401dab0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.188+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/1104952067 shutdown_connections 2023-07-21T22:30:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.188+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/1104952067 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ace0a36400 0x55ace0a38810 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.188+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/1104952067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace0a22580 0x55ace0a31d80 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.188+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/1104952067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace087d6b0 0x55ace0a2a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.188+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/1104952067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 0x55ace0a2ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.188+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/1104952067 >> 172.21.15.107:0/1104952067 conn(0x55ace0a16ae0 msgr2=0x55ace0a1b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.189+0000 7f4c3bfff700 0 client.15918 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.191+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/1104952067 shutdown_connections 2023-07-21T22:30:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.191+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/1104952067 wait complete. 2023-07-21T22:30:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.196+0000 7f4c6ef46b80 20 client.15918 trim_cache size 0 max 16384 2023-07-21T22:30:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.226+0000 7f4c6ef46b80 1 Processor -- start 2023-07-21T22:30:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.226+0000 7f4c6ef46b80 1 -- start start 2023-07-21T22:30:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.226+0000 7f4c6ef46b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace087d6b0 0x55ace0a2a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.226+0000 7f4c6ef46b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 0x55ace0a31d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.226+0000 7f4c6ef46b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace0a22580 0x55ace0a2ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.226+0000 7f4c6ef46b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace096be50 con 0x55ace0a22580 2023-07-21T22:30:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.226+0000 7f4c6ef46b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace0a29cb0 con 0x55ace0a21830 2023-07-21T22:30:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.226+0000 7f4c6ef46b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace0a1caf0 con 0x55ace087d6b0 2023-07-21T22:30:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.228+0000 7f4c5bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 0x55ace0a31d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.228+0000 7f4c5bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 0x55ace0a31d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40094/0 (socket says 172.21.15.107:40094) 2023-07-21T22:30:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.228+0000 7f4c5bfff700 1 -- 172.21.15.107:0/2280565381 learned_addr learned my addr 172.21.15.107:0/2280565381 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.228+0000 7f4c5bfff700 1 -- 172.21.15.107:0/2280565381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace087d6b0 msgr2=0x55ace0a2a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.228+0000 7f4c5bfff700 1 --2- 172.21.15.107:0/2280565381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace087d6b0 0x55ace0a2a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.228+0000 7f4c5bfff700 1 -- 172.21.15.107:0/2280565381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace0a22580 msgr2=0x55ace0a2ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.228+0000 7f4c5bfff700 1 --2- 172.21.15.107:0/2280565381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace0a22580 0x55ace0a2ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.228+0000 7f4c5bfff700 1 -- 172.21.15.107:0/2280565381 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ace07c71f0 con 0x55ace0a21830 2023-07-21T22:30:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.229+0000 7f4c5bfff700 1 --2- 172.21.15.107:0/2280565381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 0x55ace0a31d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c50016b00 tx=0x7f4c50012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db43e594201ecbd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.231+0000 7f4c437fe700 1 -- 172.21.15.107:0/2280565381 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5001b050 con 0x55ace0a21830 2023-07-21T22:30:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.231+0000 7f4c437fe700 1 -- 172.21.15.107:0/2280565381 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c5001c040 con 0x55ace0a21830 2023-07-21T22:30:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.231+0000 7f4c437fe700 1 -- 172.21.15.107:0/2280565381 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50014ba0 con 0x55ace0a21830 2023-07-21T22:30:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.232+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2280565381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 msgr2=0x55ace0a31d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.232+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2280565381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 0x55ace0a31d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c50016b00 tx=0x7f4c50012000 comp rx=0 tx=0).stop 2023-07-21T22:30:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.233+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2280565381 shutdown_connections 2023-07-21T22:30:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.233+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2280565381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace0a22580 0x55ace0a2ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.233+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2280565381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace087d6b0 0x55ace0a2a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.233+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2280565381 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a21830 0x55ace0a31d40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.233+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2280565381 >> 172.21.15.107:0/2280565381 conn(0x55ace0a16ae0 msgr2=0x55ace0a3d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.234+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2280565381 shutdown_connections 2023-07-21T22:30:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.234+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2280565381 wait complete. 2023-07-21T22:30:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.234+0000 7f4c6ef46b80 1 -- start start 2023-07-21T22:30:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.234+0000 7f4c6ef46b80 1 Processor -- start 2023-07-21T22:30:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.235+0000 7f4c6ef46b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 0x55ace0a2df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.235+0000 7f4c6ef46b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace0a21830 0x55ace0a2e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.235+0000 7f4c6ef46b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a22580 0x55ace0a31110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.235+0000 7f4c6ef46b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace0a31650 con 0x55ace087d6b0 2023-07-21T22:30:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.235+0000 7f4c6ef46b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace0a317c0 con 0x55ace0a22580 2023-07-21T22:30:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.235+0000 7f4c6ef46b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace0a31980 con 0x55ace0a21830 2023-07-21T22:30:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.235+0000 7f4c6ef46b80 10 client.0 _ll_register_callbacks cb 0x7f4c5409b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.236+0000 7f4c5bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace0a21830 0x55ace0a2e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.236+0000 7f4c60b5d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 0x55ace0a2df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.236+0000 7f4c60b5d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 0x55ace0a2df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49272/0 (socket says 172.21.15.107:49272) 2023-07-21T22:30:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.236+0000 7f4c60b5d700 1 -- 172.21.15.107:0/2594388865 learned_addr learned my addr 172.21.15.107:0/2594388865 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.237+0000 7f4c60b5d700 1 -- 172.21.15.107:0/2594388865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace0a21830 msgr2=0x55ace0a2e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.237+0000 7f4c60b5d700 1 --2- 172.21.15.107:0/2594388865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace0a21830 0x55ace0a2e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.237+0000 7f4c60b5d700 1 -- 172.21.15.107:0/2594388865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a22580 msgr2=0x55ace0a31110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.237+0000 7f4c60b5d700 1 --2- 172.21.15.107:0/2594388865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a22580 0x55ace0a31110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.237+0000 7f4c60b5d700 1 -- 172.21.15.107:0/2594388865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c50016870 con 0x55ace087d6b0 2023-07-21T22:30:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.238+0000 7f4c60b5d700 1 --2- 172.21.15.107:0/2594388865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 0x55ace0a2df20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c010f60 tx=0x7f4c4c01a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8b5d59470e2647 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.238+0000 7f4c3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.238+0000 7f4c3bfff700 1 -- 172.21.15.107:0/2594388865 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c025040 con 0x55ace087d6b0 2023-07-21T22:30:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.238+0000 7f4c3bfff700 1 -- 172.21.15.107:0/2594388865 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4c4c0293b0 con 0x55ace087d6b0 2023-07-21T22:30:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.238+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2594388865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c4c00dbd0 con 0x55ace087d6b0 2023-07-21T22:30:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.239+0000 7f4c3bfff700 1 -- 172.21.15.107:0/2594388865 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c4c029570 con 0x55ace087d6b0 2023-07-21T22:30:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.241+0000 7f4c6ef46b80 10 client.15921 fetch_fsmap learned FSMap version 30 2023-07-21T22:30:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.242+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2594388865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ace079f250 con 0x55ace087d6b0 2023-07-21T22:30:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.242+0000 7f4c3bfff700 1 -- 172.21.15.107:0/2594388865 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c4c00dbd0 con 0x55ace087d6b0 2023-07-21T22:30:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.242+0000 7f4c3bfff700 1 -- 172.21.15.107:0/2594388865 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4c4c032070 con 0x55ace087d6b0 2023-07-21T22:30:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.242+0000 7f4c6ef46b80 10 client.15921 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:30:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.242+0000 7f4c6ef46b80 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.242+0000 7f4c6ef46b80 20 client.15921 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.242+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2594388865 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ace0a8b2d0 0x55ace0a8d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.242+0000 7f4c6ef46b80 4 client.15921 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:30:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.242+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2594388865 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ace092dff0 con 0x55ace0a8b2d0 2023-07-21T22:30:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.244+0000 7f4c5bfff700 1 --2- 172.21.15.107:0/2594388865 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ace0a8b2d0 0x55ace0a8d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.246+0000 7f4c5bfff700 1 --2- 172.21.15.107:0/2594388865 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ace0a8b2d0 0x55ace0a8d6e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d138d0748d1b4646 server_cookie=d5bbbc41d99d9cce in_seq=0 out_seq=0 2023-07-21T22:30:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.246+0000 7f4c3bfff700 10 client.15921 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:42.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:41 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:42.435 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:41 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:42.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:41 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c3bfff700 1 -- 172.21.15.107:0/2594388865 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ace092dff0 con 0x55ace0a8b2d0 2023-07-21T22:30:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c3bfff700 10 client.15921 handle_command_reply: tid=0 2023-07-21T22:30:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c6ef46b80 1 client.15921 shutdown 2023-07-21T22:30:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2594388865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 msgr2=0x55ace0a2df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2594388865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 0x55ace0a2df20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c010f60 tx=0x7f4c4c01a750 comp rx=0 tx=0).stop 2023-07-21T22:30:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2594388865 shutdown_connections 2023-07-21T22:30:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2594388865 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ace0a8b2d0 0x55ace0a8d6e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2594388865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ace0a22580 0x55ace0a31110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2594388865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ace087d6b0 0x55ace0a2df20 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c3bfff700 0 client.15921 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c6ef46b80 1 --2- 172.21.15.107:0/2594388865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace0a21830 0x55ace0a2e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2594388865 >> 172.21.15.107:0/2594388865 conn(0x55ace0a16ae0 msgr2=0x55ace0a25430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2594388865 shutdown_connections 2023-07-21T22:30:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c6ef46b80 1 -- 172.21.15.107:0/2594388865 wait complete. 2023-07-21T22:30:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.768+0000 7f4c6ef46b80 20 client.15921 trim_cache size 0 max 16384 2023-07-21T22:30:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:30:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:30:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:30:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:30:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.769+0000 7f4c62dc1700 1 -- 172.21.15.107:0/964328946 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4c440a1350 msgr2=0x7f4c440a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.769+0000 7f4c62dc1700 1 --2- 172.21.15.107:0/964328946 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4c440a1350 0x7f4c440a3800 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55ace0a1ca50 tx=0x7f4c50007b70 comp rx=0 tx=0).stop 2023-07-21T22:30:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.769+0000 7f4c62dc1700 1 -- 172.21.15.107:0/964328946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c10bd60 msgr2=0x7f4c5c1c5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.769+0000 7f4c62dc1700 1 --2- 172.21.15.107:0/964328946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c10bd60 0x7f4c5c1c5630 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009fd0 tx=0x7f4c540077e0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.769+0000 7f4c62dc1700 1 -- 172.21.15.107:0/964328946 shutdown_connections 2023-07-21T22:30:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.769+0000 7f4c62dc1700 1 --2- 172.21.15.107:0/964328946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c5c10bd60 0x7f4c5c1c5630 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.769+0000 7f4c62dc1700 1 --2- 172.21.15.107:0/964328946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c070f00 0x7f4c5c19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.769+0000 7f4c62dc1700 1 --2- 172.21.15.107:0/964328946 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4c440a1350 0x7f4c440a3800 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.769+0000 7f4c62dc1700 1 --2- 172.21.15.107:0/964328946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c5c071850 0x7f4c5c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.769+0000 7f4c62dc1700 1 -- 172.21.15.107:0/964328946 >> 172.21.15.107:0/964328946 conn(0x7f4c5c06ca50 msgr2=0x7f4c5c06e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.769+0000 7f4c62dc1700 1 -- 172.21.15.107:0/964328946 shutdown_connections 2023-07-21T22:30:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:42.769+0000 7f4c62dc1700 1 -- 172.21.15.107:0/964328946 wait complete. 2023-07-21T22:30:43.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:43 smithi132 ceph-mon[97863]: pgmap v342: 97 pgs: 97 active+clean; 74 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 74 op/s 2023-07-21T22:30:43.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:43 smithi107 ceph-mon[96543]: pgmap v342: 97 pgs: 97 active+clean; 74 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 74 op/s 2023-07-21T22:30:43.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:43 smithi190 ceph-mon[98334]: pgmap v342: 97 pgs: 97 active+clean; 74 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 74 op/s 2023-07-21T22:30:45.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:45 smithi132 ceph-mon[97863]: pgmap v343: 97 pgs: 97 active+clean; 79 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 73 op/s 2023-07-21T22:30:45.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:45 smithi107 ceph-mon[96543]: pgmap v343: 97 pgs: 97 active+clean; 79 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 73 op/s 2023-07-21T22:30:45.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:45 smithi190 ceph-mon[98334]: pgmap v343: 97 pgs: 97 active+clean; 79 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 73 op/s 2023-07-21T22:30:46.324 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:30:46.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.418+0000 7f36a4235700 1 -- 172.21.15.107:0/1102954727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f369c0fea10 msgr2=0x7f369c0fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.418+0000 7f36a4235700 1 --2- 172.21.15.107:0/1102954727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f369c0fea10 0x7f369c0fee20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f368c00b0a0 tx=0x7f368c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.418+0000 7f36a4235700 1 -- 172.21.15.107:0/1102954727 shutdown_connections 2023-07-21T22:30:46.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.418+0000 7f36a4235700 1 --2- 172.21.15.107:0/1102954727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f369c100590 0x7f369c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.418+0000 7f36a4235700 1 --2- 172.21.15.107:0/1102954727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f369c0fea10 0x7f369c0fee20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.418+0000 7f36a4235700 1 --2- 172.21.15.107:0/1102954727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f369c0ffc00 0x7f369c100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.418+0000 7f36a4235700 1 -- 172.21.15.107:0/1102954727 >> 172.21.15.107:0/1102954727 conn(0x7f369c0fa030 msgr2=0x7f369c0fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:46.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.418+0000 7f36a4235700 1 -- 172.21.15.107:0/1102954727 shutdown_connections 2023-07-21T22:30:46.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.418+0000 7f36a4235700 1 -- 172.21.15.107:0/1102954727 wait complete. 2023-07-21T22:30:46.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.419+0000 7f36a4235700 1 Processor -- start 2023-07-21T22:30:46.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.419+0000 7f36a4235700 1 -- start start 2023-07-21T22:30:46.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.419+0000 7f36a4235700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f369c0fea10 0x7f369c069210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.419+0000 7f36a4235700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f369c0ffc00 0x7f369c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.419+0000 7f36a4235700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f369c100590 0x7f369c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.419+0000 7f36a4235700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c19d4d0 con 0x7f369c100590 2023-07-21T22:30:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.419+0000 7f36a4235700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c19d640 con 0x7f369c0fea10 2023-07-21T22:30:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.419+0000 7f36a4235700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c19d800 con 0x7f369c0ffc00 2023-07-21T22:30:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.419+0000 7f36a27d2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f369c100590 0x7f369c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.419+0000 7f36a27d2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f369c100590 0x7f369c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49290/0 (socket says 172.21.15.107:49290) 2023-07-21T22:30:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.419+0000 7f36a1fd1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f369c0fea10 0x7f369c069210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.419+0000 7f36a27d2700 1 -- 172.21.15.107:0/2803146257 learned_addr learned my addr 172.21.15.107:0/2803146257 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:46.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.419+0000 7f36a17d0700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f369c0ffc00 0x7f369c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.420+0000 7f36a27d2700 1 -- 172.21.15.107:0/2803146257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f369c0ffc00 msgr2=0x7f369c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.420+0000 7f36a27d2700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f369c0ffc00 0x7f369c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.420+0000 7f36a27d2700 1 -- 172.21.15.107:0/2803146257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f369c0fea10 msgr2=0x7f369c069210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.420+0000 7f36a27d2700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f369c0fea10 0x7f369c069210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.420+0000 7f36a27d2700 1 -- 172.21.15.107:0/2803146257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c009d00 con 0x7f369c100590 2023-07-21T22:30:46.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.420+0000 7f36a1fd1700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f369c0fea10 0x7f369c069210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:46.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.420+0000 7f36a27d2700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f369c100590 0x7f369c19cd30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f369400e940 tx=0x7f369400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:46.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.420+0000 7f369affd700 1 -- 172.21.15.107:0/2803146257 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36940097b0 con 0x7f369c100590 2023-07-21T22:30:46.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.420+0000 7f369affd700 1 -- 172.21.15.107:0/2803146257 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3694005020 con 0x7f369c100590 2023-07-21T22:30:46.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.420+0000 7f36a4235700 1 -- 172.21.15.107:0/2803146257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369c19dae0 con 0x7f369c100590 2023-07-21T22:30:46.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.420+0000 7f369affd700 1 -- 172.21.15.107:0/2803146257 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369400cc40 con 0x7f369c100590 2023-07-21T22:30:46.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.420+0000 7f36a4235700 1 -- 172.21.15.107:0/2803146257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369c19dff0 con 0x7f369c100590 2023-07-21T22:30:46.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.422+0000 7f369affd700 1 -- 172.21.15.107:0/2803146257 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3694005400 con 0x7f369c100590 2023-07-21T22:30:46.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.423+0000 7f369affd700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f36840a1150 0x7f36840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.423+0000 7f36a1fd1700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f36840a1150 0x7f36840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.423+0000 7f369affd700 1 -- 172.21.15.107:0/2803146257 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3694014070 con 0x7f369c100590 2023-07-21T22:30:46.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.423+0000 7f36a1fd1700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f36840a1150 0x7f36840a3600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f368c0079f0 tx=0x7f368c00b700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:46.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.424+0000 7f36b03bab80 1 Processor -- start 2023-07-21T22:30:46.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.424+0000 7f36b03bab80 1 -- start start 2023-07-21T22:30:46.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.424+0000 7f36b03bab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b852830 0x55584b852c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.424+0000 7f36b03bab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b851430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.424+0000 7f36b03bab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b851a10 0x55584b851e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.424+0000 7f36b03bab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584b756cb0 con 0x55584b8531a0 2023-07-21T22:30:46.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.424+0000 7f36b03bab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584b7a9a40 con 0x55584b851a10 2023-07-21T22:30:46.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.424+0000 7f36b03bab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584b7a2ed0 con 0x55584b852830 2023-07-21T22:30:46.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.424+0000 7f36a17d0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b851430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.424+0000 7f36a17d0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b851430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49296/0 (socket says 172.21.15.107:49296) 2023-07-21T22:30:46.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.424+0000 7f36a17d0700 1 -- 172.21.15.107:0/1901725484 learned_addr learned my addr 172.21.15.107:0/1901725484 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:46.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.424+0000 7f36a1fd1700 1 --2- 172.21.15.107:0/1901725484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b852830 0x55584b852c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.424+0000 7f36a27d2700 1 --2- 172.21.15.107:0/1901725484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b851a10 0x55584b851e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.425+0000 7f36a17d0700 1 -- 172.21.15.107:0/1901725484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b852830 msgr2=0x55584b852c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:46.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.425+0000 7f36a17d0700 1 --2- 172.21.15.107:0/1901725484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b852830 0x55584b852c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.425+0000 7f36a17d0700 1 -- 172.21.15.107:0/1901725484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b851a10 msgr2=0x55584b851e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:46.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.425+0000 7f36a17d0700 1 --2- 172.21.15.107:0/1901725484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b851a10 0x55584b851e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.425+0000 7f36a17d0700 1 -- 172.21.15.107:0/1901725484 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55584b5d0de0 con 0x55584b8531a0 2023-07-21T22:30:46.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.425+0000 7f36a1fd1700 1 --2- 172.21.15.107:0/1901725484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b852830 0x55584b852c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:46.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.425+0000 7f36a17d0700 1 --2- 172.21.15.107:0/1901725484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b851430 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f369000ad70 tx=0x7f369000f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19ffabe15916e7e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:46.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.425+0000 7f3683fff700 1 -- 172.21.15.107:0/1901725484 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369000a870 con 0x55584b8531a0 2023-07-21T22:30:46.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.425+0000 7f3683fff700 1 -- 172.21.15.107:0/1901725484 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36900044e0 con 0x55584b8531a0 2023-07-21T22:30:46.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.425+0000 7f3683fff700 1 -- 172.21.15.107:0/1901725484 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36900186a0 con 0x55584b8531a0 2023-07-21T22:30:46.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.426+0000 7f36b03bab80 1 -- 172.21.15.107:0/1901725484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 msgr2=0x55584b851430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.426+0000 7f36b03bab80 1 --2- 172.21.15.107:0/1901725484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b851430 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f369000ad70 tx=0x7f369000f6a0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.426+0000 7f36b03bab80 1 -- 172.21.15.107:0/1901725484 shutdown_connections 2023-07-21T22:30:46.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.426+0000 7f36b03bab80 1 --2- 172.21.15.107:0/1901725484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b851a10 0x55584b851e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.426+0000 7f36b03bab80 1 --2- 172.21.15.107:0/1901725484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b852830 0x55584b852c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.426+0000 7f36b03bab80 1 --2- 172.21.15.107:0/1901725484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b851430 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.479 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.426+0000 7f36b03bab80 1 -- 172.21.15.107:0/1901725484 >> 172.21.15.107:0/1901725484 conn(0x55584b846ef0 msgr2=0x55584b84c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:46.479 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.426+0000 7f36b03bab80 1 -- 172.21.15.107:0/1901725484 shutdown_connections 2023-07-21T22:30:46.479 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.426+0000 7f36b03bab80 1 -- 172.21.15.107:0/1901725484 wait complete. 2023-07-21T22:30:46.479 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.426+0000 7f36b03bab80 1 -- start start 2023-07-21T22:30:46.479 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.426+0000 7f36b03bab80 1 Processor -- start 2023-07-21T22:30:46.479 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36b03bab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b851a10 0x55584b85b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.479 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36b03bab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b852830 0x55584b85b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.480 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36b03bab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b8531a0 0x55584b862600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.480 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36b03bab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584b742ac0 con 0x55584b852830 2023-07-21T22:30:46.480 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36b03bab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584b7427c0 con 0x55584b8531a0 2023-07-21T22:30:46.480 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36b03bab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584b626920 con 0x55584b851a10 2023-07-21T22:30:46.480 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36b03bab80 10 client.0 _ll_register_callbacks cb 0x55584b740760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:46.480 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36a17d0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b852830 0x55584b85b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.480 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36a1fd1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b851a10 0x55584b85b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.480 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36a1fd1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b851a10 0x55584b85b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37858/0 (socket says 172.21.15.107:37858) 2023-07-21T22:30:46.481 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36a17d0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b852830 0x55584b85b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49302/0 (socket says 172.21.15.107:49302) 2023-07-21T22:30:46.481 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36a1fd1700 1 -- 172.21.15.107:0/1616340000 learned_addr learned my addr 172.21.15.107:0/1616340000 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:46.481 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36a27d2700 1 --2- 172.21.15.107:0/1616340000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b8531a0 0x55584b862600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.481 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36a1fd1700 1 -- 172.21.15.107:0/1616340000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b8531a0 msgr2=0x55584b862600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:46.481 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36a1fd1700 1 --2- 172.21.15.107:0/1616340000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b8531a0 0x55584b862600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.481 INFO:tasks.workunit.client.0.smithi107.stdout: 28 212369 290 137394 190 90064 828 2023-07-21T22:30:46.483 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36a1fd1700 1 -- 172.21.15.107:0/1616340000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b852830 msgr2=0x55584b85b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:46.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36a1fd1700 1 --2- 172.21.15.107:0/1616340000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b852830 0x55584b85b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36a1fd1700 1 -- 172.21.15.107:0/1616340000 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369c107780 con 0x55584b851a10 2023-07-21T22:30:46.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36a17d0700 1 --2- 172.21.15.107:0/1616340000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b852830 0x55584b85b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:46.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36a1fd1700 1 --2- 172.21.15.107:0/1616340000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b851a10 0x55584b85b080 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f369c0ffa60 tx=0x7f368c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=135fd54d23da64c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:46.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36817fa700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:30:46.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36817fa700 1 -- 172.21.15.107:0/1616340000 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c023040 con 0x55584b851a10 2023-07-21T22:30:46.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.427+0000 7f36b03bab80 1 -- 172.21.15.107:0/1616340000 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55584b6fc4f0 con 0x55584b851a10 2023-07-21T22:30:46.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.428+0000 7f36817fa700 1 -- 172.21.15.107:0/1616340000 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f368c01ccb0 con 0x55584b851a10 2023-07-21T22:30:46.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.428+0000 7f36817fa700 1 -- 172.21.15.107:0/1616340000 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f368c00e040 con 0x55584b851a10 2023-07-21T22:30:46.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.428+0000 7f36b03bab80 10 client.25459 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:46.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.428+0000 7f36b03bab80 1 -- 172.21.15.107:0/1616340000 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f369000a9d0 con 0x55584b851a10 2023-07-21T22:30:46.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.428+0000 7f36817fa700 1 -- 172.21.15.107:0/1616340000 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55584b6fc4f0 con 0x55584b851a10 2023-07-21T22:30:46.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.428+0000 7f36817fa700 1 -- 172.21.15.107:0/1616340000 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f368c02a420 con 0x55584b851a10 2023-07-21T22:30:46.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.428+0000 7f36b03bab80 10 client.25459 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:46.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.428+0000 7f36b03bab80 10 client.25459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:46.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.428+0000 7f36b03bab80 20 client.25459 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:46.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.428+0000 7f36b03bab80 1 --2- 172.21.15.107:0/1616340000 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55584b866c90 0x55584b8690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.428+0000 7f36b03bab80 4 client.25459 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:46.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.428+0000 7f36b03bab80 1 -- 172.21.15.107:0/1616340000 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55584b79beb0 con 0x55584b866c90 2023-07-21T22:30:46.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.428+0000 7f36a17d0700 1 --2- 172.21.15.107:0/1616340000 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55584b866c90 0x55584b8690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.429+0000 7f36817fa700 10 client.25459 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:46.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.429+0000 7f36a17d0700 1 --2- 172.21.15.107:0/1616340000 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55584b866c90 0x55584b8690a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=748277b870fbc496 server_cookie=e6e2cc644c29ffdf in_seq=0 out_seq=0 2023-07-21T22:30:46.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.429+0000 7f36817fa700 1 -- 172.21.15.107:0/1616340000 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55584b79beb0 con 0x55584b866c90 2023-07-21T22:30:46.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.429+0000 7f36817fa700 10 client.25459 handle_command_reply: tid=0 2023-07-21T22:30:46.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.429+0000 7f36b03bab80 1 client.25459 shutdown 2023-07-21T22:30:46.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.430+0000 7f36b03bab80 1 -- 172.21.15.107:0/1616340000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b851a10 msgr2=0x55584b85b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.430+0000 7f36b03bab80 1 --2- 172.21.15.107:0/1616340000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b851a10 0x55584b85b080 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f369c0ffa60 tx=0x7f368c0099e0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.430+0000 7f36b03bab80 1 -- 172.21.15.107:0/1616340000 shutdown_connections 2023-07-21T22:30:46.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.430+0000 7f36b03bab80 1 --2- 172.21.15.107:0/1616340000 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55584b866c90 0x55584b8690a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.430+0000 7f36b03bab80 1 --2- 172.21.15.107:0/1616340000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b8531a0 0x55584b862600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.430+0000 7f36b03bab80 1 --2- 172.21.15.107:0/1616340000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b851a10 0x55584b85b080 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.430+0000 7f36b03bab80 1 --2- 172.21.15.107:0/1616340000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b852830 0x55584b85b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.430+0000 7f36817fa700 0 client.25459 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:46.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.430+0000 7f36b03bab80 1 -- 172.21.15.107:0/1616340000 >> 172.21.15.107:0/1616340000 conn(0x55584b846ef0 msgr2=0x55584b84ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:46.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.430+0000 7f36b03bab80 1 -- 172.21.15.107:0/1616340000 shutdown_connections 2023-07-21T22:30:46.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.430+0000 7f36b03bab80 1 -- 172.21.15.107:0/1616340000 wait complete. 2023-07-21T22:30:46.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.430+0000 7f36b03bab80 20 client.25459 trim_cache size 0 max 16384 2023-07-21T22:30:46.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36b03bab80 1 Processor -- start 2023-07-21T22:30:46.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36b03bab80 1 -- start start 2023-07-21T22:30:46.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36b03bab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b851a10 0x55584b855a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36b03bab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b852830 0x55584b855f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36b03bab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b8625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36b03bab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584b8597b0 con 0x55584b8531a0 2023-07-21T22:30:46.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36b03bab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584b84d2b0 con 0x55584b852830 2023-07-21T22:30:46.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36b03bab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584b8669e0 con 0x55584b851a10 2023-07-21T22:30:46.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36a27d2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b8625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36a1fd1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b851a10 0x55584b855a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36a17d0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b852830 0x55584b855f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36a1fd1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b851a10 0x55584b855a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37866/0 (socket says 172.21.15.107:37866) 2023-07-21T22:30:46.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36a1fd1700 1 -- 172.21.15.107:0/402616147 learned_addr learned my addr 172.21.15.107:0/402616147 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:46.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36a27d2700 1 -- 172.21.15.107:0/402616147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b851a10 msgr2=0x55584b855a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:46.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36a27d2700 1 --2- 172.21.15.107:0/402616147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b851a10 0x55584b855a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36a27d2700 1 -- 172.21.15.107:0/402616147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b852830 msgr2=0x55584b855f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:46.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36a27d2700 1 --2- 172.21.15.107:0/402616147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b852830 0x55584b855f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36a27d2700 1 -- 172.21.15.107:0/402616147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55584b5b9c60 con 0x55584b8531a0 2023-07-21T22:30:46.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.447+0000 7f36a17d0700 1 --2- 172.21.15.107:0/402616147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b852830 0x55584b855f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:46.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.448+0000 7f36a27d2700 1 --2- 172.21.15.107:0/402616147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b8625c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f36940ca540 tx=0x7f36940962c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba605b602e9f97c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:46.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.448+0000 7f3698ff9700 1 -- 172.21.15.107:0/402616147 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36940a22a0 con 0x55584b8531a0 2023-07-21T22:30:46.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.448+0000 7f3698ff9700 1 -- 172.21.15.107:0/402616147 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36940981f0 con 0x55584b8531a0 2023-07-21T22:30:46.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.448+0000 7f3698ff9700 1 -- 172.21.15.107:0/402616147 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694099340 con 0x55584b8531a0 2023-07-21T22:30:46.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.448+0000 7f36b03bab80 1 -- 172.21.15.107:0/402616147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 msgr2=0x55584b8625c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.448+0000 7f36b03bab80 1 --2- 172.21.15.107:0/402616147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b8625c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f36940ca540 tx=0x7f36940962c0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.448+0000 7f36b03bab80 1 -- 172.21.15.107:0/402616147 shutdown_connections 2023-07-21T22:30:46.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.448+0000 7f36b03bab80 1 --2- 172.21.15.107:0/402616147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b8625c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.448+0000 7f36b03bab80 1 --2- 172.21.15.107:0/402616147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b851a10 0x55584b855a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.448+0000 7f36b03bab80 1 --2- 172.21.15.107:0/402616147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b852830 0x55584b855f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.448+0000 7f36b03bab80 1 -- 172.21.15.107:0/402616147 >> 172.21.15.107:0/402616147 conn(0x55584b846ef0 msgr2=0x55584b851de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:46.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.448+0000 7f36b03bab80 1 -- 172.21.15.107:0/402616147 shutdown_connections 2023-07-21T22:30:46.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.448+0000 7f36b03bab80 1 -- 172.21.15.107:0/402616147 wait complete. 2023-07-21T22:30:46.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.449+0000 7f36b03bab80 1 -- start start 2023-07-21T22:30:46.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.449+0000 7f36b03bab80 1 Processor -- start 2023-07-21T22:30:46.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.449+0000 7f36b03bab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b851a10 0x55584b8620e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.449+0000 7f36b03bab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b852830 0x55584b862620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.449+0000 7f36b03bab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b85ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.449+0000 7f36b03bab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584b85f380 con 0x55584b8531a0 2023-07-21T22:30:46.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.449+0000 7f36b03bab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584b85f4f0 con 0x55584b851a10 2023-07-21T22:30:46.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.449+0000 7f36b03bab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55584b85f6b0 con 0x55584b852830 2023-07-21T22:30:46.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.449+0000 7f36b03bab80 10 client.0 _ll_register_callbacks cb 0x55584b740760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:46.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.449+0000 7f36a27d2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b85ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.503 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.449+0000 7f36a1fd1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b851a10 0x55584b8620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.503 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.449+0000 7f36a27d2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b85ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49318/0 (socket says 172.21.15.107:49318) 2023-07-21T22:30:46.503 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.449+0000 7f36a27d2700 1 -- 172.21.15.107:0/3643917371 learned_addr learned my addr 172.21.15.107:0/3643917371 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:46.503 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36a17d0700 1 --2- 172.21.15.107:0/3643917371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b852830 0x55584b862620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.503 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36a27d2700 1 -- 172.21.15.107:0/3643917371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b852830 msgr2=0x55584b862620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:46.503 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36a27d2700 1 --2- 172.21.15.107:0/3643917371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b852830 0x55584b862620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.503 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36a27d2700 1 -- 172.21.15.107:0/3643917371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b851a10 msgr2=0x55584b8620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:46.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36a27d2700 1 --2- 172.21.15.107:0/3643917371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b851a10 0x55584b8620e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36a27d2700 1 -- 172.21.15.107:0/3643917371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55584b5d0de0 con 0x55584b8531a0 2023-07-21T22:30:46.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36a1fd1700 1 --2- 172.21.15.107:0/3643917371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b851a10 0x55584b8620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:46.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36a27d2700 1 --2- 172.21.15.107:0/3643917371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b85ec70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3694096350 tx=0x7f36940983d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4450a90705fc55f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:46.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36817fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:46.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36817fa700 1 -- 172.21.15.107:0/3643917371 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694099540 con 0x55584b8531a0 2023-07-21T22:30:46.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36817fa700 1 -- 172.21.15.107:0/3643917371 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f36940ca2c0 con 0x55584b8531a0 2023-07-21T22:30:46.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36b03bab80 1 -- 172.21.15.107:0/3643917371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36900044e0 con 0x55584b8531a0 2023-07-21T22:30:46.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36817fa700 1 -- 172.21.15.107:0/3643917371 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3694096560 con 0x55584b8531a0 2023-07-21T22:30:46.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36b03bab80 10 client.15939 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:46.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36b03bab80 1 -- 172.21.15.107:0/3643917371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f369c19dff0 con 0x55584b8531a0 2023-07-21T22:30:46.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36817fa700 1 -- 172.21.15.107:0/3643917371 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36900044e0 con 0x55584b8531a0 2023-07-21T22:30:46.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.450+0000 7f36817fa700 1 -- 172.21.15.107:0/3643917371 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f36940adc50 con 0x55584b8531a0 2023-07-21T22:30:46.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36b03bab80 10 client.15939 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:46.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36b03bab80 10 client.15939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:46.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36b03bab80 20 client.15939 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:46.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36b03bab80 1 --2- 172.21.15.107:0/3643917371 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55584b8bbff0 0x55584b8be400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36b03bab80 4 client.15939 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:30:46.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36b03bab80 1 -- 172.21.15.107:0/3643917371 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55584b75e050 con 0x55584b8bbff0 2023-07-21T22:30:46.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36a1fd1700 1 --2- 172.21.15.107:0/3643917371 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55584b8bbff0 0x55584b8be400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36817fa700 10 client.15939 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:46.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36a1fd1700 1 --2- 172.21.15.107:0/3643917371 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55584b8bbff0 0x55584b8be400 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5923c53a92455471 server_cookie=757610c2deb66219 in_seq=0 out_seq=0 2023-07-21T22:30:46.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36817fa700 1 -- 172.21.15.107:0/3643917371 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55584b75e050 con 0x55584b8bbff0 2023-07-21T22:30:46.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36817fa700 10 client.15939 handle_command_reply: tid=0 2023-07-21T22:30:46.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36b03bab80 1 client.15939 shutdown 2023-07-21T22:30:46.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36b03bab80 1 -- 172.21.15.107:0/3643917371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 msgr2=0x55584b85ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36b03bab80 1 --2- 172.21.15.107:0/3643917371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b85ec70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3694096350 tx=0x7f36940983d0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36b03bab80 1 -- 172.21.15.107:0/3643917371 shutdown_connections 2023-07-21T22:30:46.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36b03bab80 1 --2- 172.21.15.107:0/3643917371 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55584b8bbff0 0x55584b8be400 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.451+0000 7f36b03bab80 1 --2- 172.21.15.107:0/3643917371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55584b8531a0 0x55584b85ec70 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36b03bab80 1 --2- 172.21.15.107:0/3643917371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55584b851a10 0x55584b8620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36b03bab80 1 --2- 172.21.15.107:0/3643917371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55584b852830 0x55584b862620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36817fa700 0 client.15939 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:46.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36b03bab80 1 -- 172.21.15.107:0/3643917371 >> 172.21.15.107:0/3643917371 conn(0x55584b846ef0 msgr2=0x55584b855a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:46.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36b03bab80 1 -- 172.21.15.107:0/3643917371 shutdown_connections 2023-07-21T22:30:46.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36b03bab80 1 -- 172.21.15.107:0/3643917371 wait complete. 2023-07-21T22:30:46.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36b03bab80 20 client.15939 trim_cache size 0 max 16384 2023-07-21T22:30:46.509 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:30:46.509 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2023-07-21T22:30:46.509 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2023-07-21T22:30:46.509 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:30:46.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36a4235700 1 -- 172.21.15.107:0/2803146257 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f36840a1150 msgr2=0x7f36840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36a4235700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f36840a1150 0x7f36840a3600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f368c0079f0 tx=0x7f368c00b700 comp rx=0 tx=0).stop 2023-07-21T22:30:46.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36a4235700 1 -- 172.21.15.107:0/2803146257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f369c100590 msgr2=0x7f369c19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36a4235700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f369c100590 0x7f369c19cd30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f369400e940 tx=0x7f369400ec50 comp rx=0 tx=0).stop 2023-07-21T22:30:46.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36a4235700 1 -- 172.21.15.107:0/2803146257 shutdown_connections 2023-07-21T22:30:46.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36a4235700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f369c100590 0x7f369c19cd30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36a4235700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f369c0fea10 0x7f369c069210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36a4235700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f36840a1150 0x7f36840a3600 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36a4235700 1 --2- 172.21.15.107:0/2803146257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f369c0ffc00 0x7f369c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36a4235700 1 -- 172.21.15.107:0/2803146257 >> 172.21.15.107:0/2803146257 conn(0x7f369c0fa030 msgr2=0x7f369c103040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:46.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36a4235700 1 -- 172.21.15.107:0/2803146257 shutdown_connections 2023-07-21T22:30:46.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.452+0000 7f36a4235700 1 -- 172.21.15.107:0/2803146257 wait complete. 2023-07-21T22:30:46.512 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:30:46.512 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:30:46.641 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.591+0000 7f835779b700 1 -- 172.21.15.107:0/161187203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500fec10 msgr2=0x7f83500ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.641 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.591+0000 7f835779b700 1 --2- 172.21.15.107:0/161187203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500fec10 0x7f83500ff080 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8340009990 tx=0x7f8340009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.641 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.591+0000 7f835779b700 1 -- 172.21.15.107:0/161187203 shutdown_connections 2023-07-21T22:30:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.591+0000 7f835779b700 1 --2- 172.21.15.107:0/161187203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83500ff5c0 0x7f8350106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.591+0000 7f835779b700 1 --2- 172.21.15.107:0/161187203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8350100050 0x7f8350100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.591+0000 7f835779b700 1 --2- 172.21.15.107:0/161187203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500fec10 0x7f83500ff080 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.591+0000 7f835779b700 1 -- 172.21.15.107:0/161187203 >> 172.21.15.107:0/161187203 conn(0x7f83500f9770 msgr2=0x7f83500fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:46.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.592+0000 7f835779b700 1 -- 172.21.15.107:0/161187203 shutdown_connections 2023-07-21T22:30:46.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.592+0000 7f835779b700 1 -- 172.21.15.107:0/161187203 wait complete. 2023-07-21T22:30:46.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.592+0000 7f835779b700 1 Processor -- start 2023-07-21T22:30:46.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.592+0000 7f835779b700 1 -- start start 2023-07-21T22:30:46.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.592+0000 7f835779b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83500fec10 0x7f8350194680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.643 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.592+0000 7f835779b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500ff5c0 0x7f8350194bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.592+0000 7f835779b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8350100050 0x7f835019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.592+0000 7f835779b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835019c200 con 0x7f83500ff5c0 2023-07-21T22:30:46.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.592+0000 7f835779b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835019c340 con 0x7f83500fec10 2023-07-21T22:30:46.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.592+0000 7f835779b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835019c4d0 con 0x7f8350100050 2023-07-21T22:30:46.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.592+0000 7f8354d36700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500ff5c0 0x7f8350194bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.644 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.592+0000 7f8354d36700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500ff5c0 0x7f8350194bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49330/0 (socket says 172.21.15.107:49330) 2023-07-21T22:30:46.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.592+0000 7f8355d38700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8350100050 0x7f835019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f8355537700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83500fec10 0x7f8350194680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f8354d36700 1 -- 172.21.15.107:0/1505226923 learned_addr learned my addr 172.21.15.107:0/1505226923 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:46.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f8355d38700 1 -- 172.21.15.107:0/1505226923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83500fec10 msgr2=0x7f8350194680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f8355d38700 1 --2- 172.21.15.107:0/1505226923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83500fec10 0x7f8350194680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f8355d38700 1 -- 172.21.15.107:0/1505226923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500ff5c0 msgr2=0x7f8350194bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.645 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f8355d38700 1 --2- 172.21.15.107:0/1505226923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500ff5c0 0x7f8350194bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f8355d38700 1 -- 172.21.15.107:0/1505226923 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400095f0 con 0x7f8350100050 2023-07-21T22:30:46.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f8354d36700 1 --2- 172.21.15.107:0/1505226923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500ff5c0 0x7f8350194bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:46.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f8355d38700 1 --2- 172.21.15.107:0/1505226923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8350100050 0x7f835019bc30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f834800efc0 tx=0x7f834800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f83467fc700 1 -- 172.21.15.107:0/1505226923 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348018070 con 0x7f8350100050 2023-07-21T22:30:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f835779b700 1 -- 172.21.15.107:0/1505226923 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83500fdb00 con 0x7f8350100050 2023-07-21T22:30:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f83467fc700 1 -- 172.21.15.107:0/1505226923 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f834800f040 con 0x7f8350100050 2023-07-21T22:30:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f83467fc700 1 -- 172.21.15.107:0/1505226923 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800cd40 con 0x7f8350100050 2023-07-21T22:30:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.593+0000 7f835779b700 1 -- 172.21.15.107:0/1505226923 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83500fe010 con 0x7f8350100050 2023-07-21T22:30:46.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.594+0000 7f835779b700 1 -- 172.21.15.107:0/1505226923 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83500fea00 con 0x7f8350100050 2023-07-21T22:30:46.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.595+0000 7f83467fc700 1 -- 172.21.15.107:0/1505226923 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8348003a00 con 0x7f8350100050 2023-07-21T22:30:46.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.596+0000 7f83467fc700 1 --2- 172.21.15.107:0/1505226923 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f833c0a11a0 0x7f833c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:46.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.596+0000 7f83467fc700 1 -- 172.21.15.107:0/1505226923 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f83480ccea0 con 0x7f8350100050 2023-07-21T22:30:46.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.596+0000 7f8355537700 1 --2- 172.21.15.107:0/1505226923 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f833c0a11a0 0x7f833c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:46.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.597+0000 7f8355537700 1 --2- 172.21.15.107:0/1505226923 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f833c0a11a0 0x7f833c0a3650 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f834c009c60 tx=0x7f834c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:46.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.599+0000 7f83467fc700 1 -- 172.21.15.107:0/1505226923 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8348096660 con 0x7f8350100050 2023-07-21T22:30:46.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.849+0000 7f835779b700 1 -- 172.21.15.107:0/1505226923 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8350105d70 con 0x7f8350100050 2023-07-21T22:30:46.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.850+0000 7f83467fc700 1 -- 172.21.15.107:0/1505226923 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10874 (secure 0 0 0) 0x7f8348099c80 con 0x7f8350100050 2023-07-21T22:30:46.897 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:30:46.897 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:30:46.106914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/1492770170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":1492770170},{"type":"v1","addr":"172.21.15.132:6837","nonce":1492770170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:30:46.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.852+0000 7f835779b700 1 -- 172.21.15.107:0/1505226923 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f833c0a11a0 msgr2=0x7f833c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.852+0000 7f835779b700 1 --2- 172.21.15.107:0/1505226923 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f833c0a11a0 0x7f833c0a3650 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f834c009c60 tx=0x7f834c009260 comp rx=0 tx=0).stop 2023-07-21T22:30:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.852+0000 7f835779b700 1 -- 172.21.15.107:0/1505226923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8350100050 msgr2=0x7f835019bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.852+0000 7f835779b700 1 --2- 172.21.15.107:0/1505226923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8350100050 0x7f835019bc30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f834800efc0 tx=0x7f834800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.852+0000 7f835779b700 1 -- 172.21.15.107:0/1505226923 shutdown_connections 2023-07-21T22:30:46.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.852+0000 7f835779b700 1 --2- 172.21.15.107:0/1505226923 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f833c0a11a0 0x7f833c0a3650 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.852+0000 7f835779b700 1 --2- 172.21.15.107:0/1505226923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8350100050 0x7f835019bc30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.852+0000 7f835779b700 1 --2- 172.21.15.107:0/1505226923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83500fec10 0x7f8350194680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.852+0000 7f835779b700 1 --2- 172.21.15.107:0/1505226923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500ff5c0 0x7f8350194bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:46.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.852+0000 7f835779b700 1 -- 172.21.15.107:0/1505226923 >> 172.21.15.107:0/1505226923 conn(0x7f83500f9770 msgr2=0x7f8350103060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:46.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.852+0000 7f835779b700 1 -- 172.21.15.107:0/1505226923 shutdown_connections 2023-07-21T22:30:46.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.852+0000 7f835779b700 1 -- 172.21.15.107:0/1505226923 wait complete. 2023-07-21T22:30:46.902 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 31 2023-07-21T22:30:46.911 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-21T22:30:47.044 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.997+0000 7f8c99f05700 1 -- 172.21.15.107:0/509170019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c94100590 msgr2=0x7f8c94106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.044 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.997+0000 7f8c99f05700 1 --2- 172.21.15.107:0/509170019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c94100590 0x7f8c94106e50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00b0a0 tx=0x7f8c8c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.044 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.997+0000 7f8c99f05700 1 -- 172.21.15.107:0/509170019 shutdown_connections 2023-07-21T22:30:47.045 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.997+0000 7f8c99f05700 1 --2- 172.21.15.107:0/509170019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c94100590 0x7f8c94106e50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.045 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.997+0000 7f8c99f05700 1 --2- 172.21.15.107:0/509170019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c940fea10 0x7f8c940fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.045 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.997+0000 7f8c99f05700 1 --2- 172.21.15.107:0/509170019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c940ffc00 0x7f8c94100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.045 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.997+0000 7f8c99f05700 1 -- 172.21.15.107:0/509170019 >> 172.21.15.107:0/509170019 conn(0x7f8c940fa030 msgr2=0x7f8c940fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.045 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.997+0000 7f8c99f05700 1 -- 172.21.15.107:0/509170019 shutdown_connections 2023-07-21T22:30:47.045 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.997+0000 7f8c99f05700 1 -- 172.21.15.107:0/509170019 wait complete. 2023-07-21T22:30:47.045 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.997+0000 7f8c99f05700 1 Processor -- start 2023-07-21T22:30:47.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.997+0000 7f8c99f05700 1 -- start start 2023-07-21T22:30:47.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c99f05700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c940fea10 0x7f8c940691d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c99f05700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c940ffc00 0x7f8c9419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c99f05700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c94100590 0x7f8c9419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c99f05700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9419d4a0 con 0x7f8c940fea10 2023-07-21T22:30:47.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c99f05700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9419d610 con 0x7f8c940ffc00 2023-07-21T22:30:47.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c99f05700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9419d7d0 con 0x7f8c94100590 2023-07-21T22:30:47.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c940fea10 0x7f8c940691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c940fea10 0x7f8c940691d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49342/0 (socket says 172.21.15.107:49342) 2023-07-21T22:30:47.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c937fe700 1 -- 172.21.15.107:0/1604890877 learned_addr learned my addr 172.21.15.107:0/1604890877 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c8affd700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c940ffc00 0x7f8c9419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c93fff700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c94100590 0x7f8c9419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c937fe700 1 -- 172.21.15.107:0/1604890877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c94100590 msgr2=0x7f8c9419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c937fe700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c94100590 0x7f8c9419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c937fe700 1 -- 172.21.15.107:0/1604890877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c940ffc00 msgr2=0x7f8c9419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c937fe700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c940ffc00 0x7f8c9419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c937fe700 1 -- 172.21.15.107:0/1604890877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c009d00 con 0x7f8c940fea10 2023-07-21T22:30:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c937fe700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c940fea10 0x7f8c940691d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400e970 tx=0x7f8c8400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c917fa700 1 -- 172.21.15.107:0/1604890877 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84009790 con 0x7f8c940fea10 2023-07-21T22:30:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c99f05700 1 -- 172.21.15.107:0/1604890877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9419dab0 con 0x7f8c940fea10 2023-07-21T22:30:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c917fa700 1 -- 172.21.15.107:0/1604890877 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c8400c7e0 con 0x7f8c940fea10 2023-07-21T22:30:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.998+0000 7f8c917fa700 1 -- 172.21.15.107:0/1604890877 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84003c70 con 0x7f8c940fea10 2023-07-21T22:30:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:46.999+0000 7f8c99f05700 1 -- 172.21.15.107:0/1604890877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9419dfc0 con 0x7f8c940fea10 2023-07-21T22:30:47.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.000+0000 7f8c917fa700 1 -- 172.21.15.107:0/1604890877 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8c8400cc50 con 0x7f8c940fea10 2023-07-21T22:30:47.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.001+0000 7f8c917fa700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8c800a1150 0x7f8c800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.001+0000 7f8c917fa700 1 -- 172.21.15.107:0/1604890877 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8c84014070 con 0x7f8c940fea10 2023-07-21T22:30:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.001+0000 7f8c8affd700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8c800a1150 0x7f8c800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.001+0000 7f8c8affd700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8c800a1150 0x7f8c800a3600 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8c94068f40 tx=0x7f8c7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.002+0000 7f8ca608ab80 1 Processor -- start 2023-07-21T22:30:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.002+0000 7f8ca608ab80 1 -- start start 2023-07-21T22:30:47.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.002+0000 7f8ca608ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c67799a80 0x560c67799e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.002+0000 7f8ca608ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 0x560c6779aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.002+0000 7f8ca608ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779aff0 0x560c6779b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.002+0000 7f8ca608ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c6769acb0 con 0x560c6779a3d0 2023-07-21T22:30:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.002+0000 7f8ca608ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c676eda40 con 0x560c6779aff0 2023-07-21T22:30:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.002+0000 7f8ca608ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c676e6ed0 con 0x560c67799a80 2023-07-21T22:30:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c8affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 0x560c6779aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c937fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c67799a80 0x560c67799e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c8affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 0x560c6779aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49350/0 (socket says 172.21.15.107:49350) 2023-07-21T22:30:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c937fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c67799a80 0x560c67799e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37906/0 (socket says 172.21.15.107:37906) 2023-07-21T22:30:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c8affd700 1 -- 172.21.15.107:0/1684754391 learned_addr learned my addr 172.21.15.107:0/1684754391 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c93fff700 1 --2- 172.21.15.107:0/1684754391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779aff0 0x560c6779b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c8affd700 1 -- 172.21.15.107:0/1684754391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c67799a80 msgr2=0x560c67799e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c8affd700 1 --2- 172.21.15.107:0/1684754391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c67799a80 0x560c67799e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c8affd700 1 -- 172.21.15.107:0/1684754391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779aff0 msgr2=0x560c6779b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c8affd700 1 --2- 172.21.15.107:0/1684754391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779aff0 0x560c6779b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c8affd700 1 -- 172.21.15.107:0/1684754391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c67514de0 con 0x560c6779a3d0 2023-07-21T22:30:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c937fe700 1 --2- 172.21.15.107:0/1684754391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c67799a80 0x560c67799e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c8affd700 1 --2- 172.21.15.107:0/1684754391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 0x560c6779aa10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c011d50 tx=0x7f8c7c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9e0f7d939e0fbb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c89ffb700 1 -- 172.21.15.107:0/1684754391 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c00eb20 con 0x560c6779a3d0 2023-07-21T22:30:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c89ffb700 1 -- 172.21.15.107:0/1684754391 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c7c021070 con 0x560c6779a3d0 2023-07-21T22:30:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.003+0000 7f8c89ffb700 1 -- 172.21.15.107:0/1684754391 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c01d670 con 0x560c6779a3d0 2023-07-21T22:30:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.004+0000 7f8ca608ab80 1 -- 172.21.15.107:0/1684754391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 msgr2=0x560c6779aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.004+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/1684754391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 0x560c6779aa10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c011d50 tx=0x7f8c7c014000 comp rx=0 tx=0).stop 2023-07-21T22:30:47.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.004+0000 7f8ca608ab80 1 -- 172.21.15.107:0/1684754391 shutdown_connections 2023-07-21T22:30:47.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.004+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/1684754391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779aff0 0x560c6779b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.004+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/1684754391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c67799a80 0x560c67799e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.004+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/1684754391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 0x560c6779aa10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.004+0000 7f8ca608ab80 1 -- 172.21.15.107:0/1684754391 >> 172.21.15.107:0/1684754391 conn(0x560c6778aef0 msgr2=0x560c67790360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.004+0000 7f8ca608ab80 1 -- 172.21.15.107:0/1684754391 shutdown_connections 2023-07-21T22:30:47.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.004+0000 7f8ca608ab80 1 -- 172.21.15.107:0/1684754391 wait complete. 2023-07-21T22:30:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.004+0000 7f8ca608ab80 1 -- start start 2023-07-21T22:30:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.004+0000 7f8ca608ab80 1 Processor -- start 2023-07-21T22:30:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8ca608ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c67799a80 0x560c677a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8ca608ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 0x560c677a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8ca608ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779aff0 0x560c677aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8ca608ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67686ac0 con 0x560c6779a3d0 2023-07-21T22:30:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8ca608ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c676867c0 con 0x560c67799a80 2023-07-21T22:30:47.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8ca608ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c6756a920 con 0x560c6779aff0 2023-07-21T22:30:47.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8ca608ab80 10 client.0 _ll_register_callbacks cb 0x560c67684760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:47.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c8affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 0x560c677a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c8affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 0x560c677a3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49356/0 (socket says 172.21.15.107:49356) 2023-07-21T22:30:47.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c8affd700 1 -- 172.21.15.107:0/4026529196 learned_addr learned my addr 172.21.15.107:0/4026529196 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c937fe700 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c67799a80 0x560c677a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c93fff700 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779aff0 0x560c677aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c8affd700 1 -- 172.21.15.107:0/4026529196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779aff0 msgr2=0x560c677aaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c8affd700 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779aff0 0x560c677aaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c8affd700 1 -- 172.21.15.107:0/4026529196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c67799a80 msgr2=0x560c677a34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c8affd700 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c67799a80 0x560c677a34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c8affd700 1 -- 172.21.15.107:0/4026529196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9419dfc0 con 0x560c6779a3d0 2023-07-21T22:30:47.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c93fff700 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779aff0 0x560c677aaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c937fe700 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c67799a80 0x560c677a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c8affd700 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 0x560c677a3a00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c011d20 tx=0x7f8c7c011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b926a042c92f677a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:47.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c72ffd700 1 -- 172.21.15.107:0/4026529196 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c00eb20 con 0x560c6779a3d0 2023-07-21T22:30:47.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8ca608ab80 1 -- 172.21.15.107:0/4026529196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c676404f0 con 0x560c6779a3d0 2023-07-21T22:30:47.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.005+0000 7f8c72ffd700 1 -- 172.21.15.107:0/4026529196 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8c7c00ec80 con 0x560c6779a3d0 2023-07-21T22:30:47.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.006+0000 7f8c72ffd700 1 -- 172.21.15.107:0/4026529196 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c7c00ee40 con 0x560c6779a3d0 2023-07-21T22:30:47.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.006+0000 7f8ca608ab80 10 client.15957 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:47.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.006+0000 7f8ca608ab80 1 -- 172.21.15.107:0/4026529196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f8c7c017070 con 0x560c6779a3d0 2023-07-21T22:30:47.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.006+0000 7f8c72ffd700 1 -- 172.21.15.107:0/4026529196 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c7c03d080 con 0x560c6779a3d0 2023-07-21T22:30:47.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.006+0000 7f8c72ffd700 1 -- 172.21.15.107:0/4026529196 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8c7c037030 con 0x560c6779a3d0 2023-07-21T22:30:47.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.006+0000 7f8ca608ab80 10 client.15957 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:47.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.006+0000 7f8ca608ab80 10 client.15957 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-21T22:30:47.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.006+0000 7f8ca608ab80 20 client.15957 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:47.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.006+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x560c677af100 0x560c677b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.006+0000 7f8ca608ab80 4 client.15957 mds_command: new command op to 24437 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:47.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.006+0000 7f8ca608ab80 1 -- 172.21.15.107:0/4026529196 --> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c676dfeb0 con 0x560c677af100 2023-07-21T22:30:47.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.006+0000 7f8c937fe700 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x560c677af100 0x560c677b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.007+0000 7f8c937fe700 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x560c677af100 0x560c677b1510 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72818266df2c2479 server_cookie=4490924e4ca98179 in_seq=0 out_seq=0 2023-07-21T22:30:47.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.007+0000 7f8c72ffd700 10 client.15957 ms_handle_connect on v2:172.21.15.132:6832/3293513462 2023-07-21T22:30:47.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.007+0000 7f8c72ffd700 1 -- 172.21.15.107:0/4026529196 <== mds.4 v2:172.21.15.132:6832/3293513462 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560c676dfeb0 con 0x560c677af100 2023-07-21T22:30:47.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.007+0000 7f8c72ffd700 10 client.15957 handle_command_reply: tid=0 2023-07-21T22:30:47.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.007+0000 7f8ca608ab80 1 client.15957 shutdown 2023-07-21T22:30:47.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.008+0000 7f8ca608ab80 1 -- 172.21.15.107:0/4026529196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 msgr2=0x560c677a3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.008+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 0x560c677a3a00 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c011d20 tx=0x7f8c7c011e10 comp rx=0 tx=0).stop 2023-07-21T22:30:47.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.008+0000 7f8ca608ab80 1 -- 172.21.15.107:0/4026529196 shutdown_connections 2023-07-21T22:30:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.008+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779aff0 0x560c677aaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.008+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x560c677af100 0x560c677b1510 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.008+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c67799a80 0x560c677a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.008+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/4026529196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c6779a3d0 0x560c677a3a00 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.008+0000 7f8c72ffd700 0 client.15957 ms_handle_reset on v2:172.21.15.132:6832/3293513462 2023-07-21T22:30:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.008+0000 7f8ca608ab80 1 -- 172.21.15.107:0/4026529196 >> 172.21.15.107:0/4026529196 conn(0x560c6778aef0 msgr2=0x560c6778fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.008+0000 7f8ca608ab80 1 -- 172.21.15.107:0/4026529196 shutdown_connections 2023-07-21T22:30:47.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.008+0000 7f8ca608ab80 1 -- 172.21.15.107:0/4026529196 wait complete. 2023-07-21T22:30:47.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.008+0000 7f8ca608ab80 20 client.15957 trim_cache size 0 max 16384 2023-07-21T22:30:47.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.021+0000 7f8ca608ab80 1 Processor -- start 2023-07-21T22:30:47.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.021+0000 7f8ca608ab80 1 -- start start 2023-07-21T22:30:47.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.021+0000 7f8ca608ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 0x560c67799e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8ca608ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779a3d0 0x560c677aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8ca608ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779aff0 0x560c677b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8ca608ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67787e50 con 0x560c67799a80 2023-07-21T22:30:47.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8ca608ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c677a4140 con 0x560c6779aff0 2023-07-21T22:30:47.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8ca608ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c677a10c0 con 0x560c6779a3d0 2023-07-21T22:30:47.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 0x560c67799e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 0x560c67799e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49362/0 (socket says 172.21.15.107:49362) 2023-07-21T22:30:47.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c937fe700 1 -- 172.21.15.107:0/915784783 learned_addr learned my addr 172.21.15.107:0/915784783 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c93fff700 1 --2- 172.21.15.107:0/915784783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779aff0 0x560c677b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c8affd700 1 --2- 172.21.15.107:0/915784783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779a3d0 0x560c677aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c937fe700 1 -- 172.21.15.107:0/915784783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779a3d0 msgr2=0x560c677aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c937fe700 1 --2- 172.21.15.107:0/915784783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779a3d0 0x560c677aaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c937fe700 1 -- 172.21.15.107:0/915784783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779aff0 msgr2=0x560c677b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c937fe700 1 --2- 172.21.15.107:0/915784783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779aff0 0x560c677b5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c937fe700 1 -- 172.21.15.107:0/915784783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c674fdc60 con 0x560c67799a80 2023-07-21T22:30:47.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c8affd700 1 --2- 172.21.15.107:0/915784783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779a3d0 0x560c677aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:47.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c937fe700 1 --2- 172.21.15.107:0/915784783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 0x560c67799e50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8c840968f0 tx=0x7f8c840cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc72fb5bd01730d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c8a7fc700 1 -- 172.21.15.107:0/915784783 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c840b8300 con 0x560c67799a80 2023-07-21T22:30:47.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c8a7fc700 1 -- 172.21.15.107:0/915784783 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c8409df00 con 0x560c67799a80 2023-07-21T22:30:47.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.022+0000 7f8c8a7fc700 1 -- 172.21.15.107:0/915784783 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c840993c0 con 0x560c67799a80 2023-07-21T22:30:47.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.023+0000 7f8ca608ab80 1 -- 172.21.15.107:0/915784783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 msgr2=0x560c67799e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.023+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/915784783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 0x560c67799e50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8c840968f0 tx=0x7f8c840cc040 comp rx=0 tx=0).stop 2023-07-21T22:30:47.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.023+0000 7f8ca608ab80 1 -- 172.21.15.107:0/915784783 shutdown_connections 2023-07-21T22:30:47.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.023+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/915784783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779aff0 0x560c677b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.023+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/915784783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 0x560c67799e50 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.023+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/915784783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779a3d0 0x560c677aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.023+0000 7f8ca608ab80 1 -- 172.21.15.107:0/915784783 >> 172.21.15.107:0/915784783 conn(0x560c6778aef0 msgr2=0x560c677a2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.023+0000 7f8ca608ab80 1 -- 172.21.15.107:0/915784783 shutdown_connections 2023-07-21T22:30:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.023+0000 7f8ca608ab80 1 -- 172.21.15.107:0/915784783 wait complete. 2023-07-21T22:30:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.023+0000 7f8ca608ab80 1 -- start start 2023-07-21T22:30:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.023+0000 7f8ca608ab80 1 Processor -- start 2023-07-21T22:30:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8ca608ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 0x560c677a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8ca608ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779a3d0 0x560c677a7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8ca608ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779aff0 0x560c677aa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8ca608ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c677aa5f0 con 0x560c67799a80 2023-07-21T22:30:47.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8ca608ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c677aa760 con 0x560c6779a3d0 2023-07-21T22:30:47.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8ca608ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c677aa920 con 0x560c6779aff0 2023-07-21T22:30:47.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8ca608ab80 10 client.0 _ll_register_callbacks cb 0x7f8c84005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:47.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 0x560c677a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 0x560c677a6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49368/0 (socket says 172.21.15.107:49368) 2023-07-21T22:30:47.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c937fe700 1 -- 172.21.15.107:0/3716835347 learned_addr learned my addr 172.21.15.107:0/3716835347 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c93fff700 1 --2- 172.21.15.107:0/3716835347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779aff0 0x560c677aa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c8affd700 1 --2- 172.21.15.107:0/3716835347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779a3d0 0x560c677a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c937fe700 1 -- 172.21.15.107:0/3716835347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779aff0 msgr2=0x560c677aa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c937fe700 1 --2- 172.21.15.107:0/3716835347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779aff0 0x560c677aa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c937fe700 1 -- 172.21.15.107:0/3716835347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779a3d0 msgr2=0x560c677a7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c937fe700 1 --2- 172.21.15.107:0/3716835347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779a3d0 0x560c677a7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c937fe700 1 -- 172.21.15.107:0/3716835347 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9419dab0 con 0x560c67799a80 2023-07-21T22:30:47.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c937fe700 1 --2- 172.21.15.107:0/3716835347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 0x560c677a6c00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8c840a1df0 tx=0x7f8c840b8300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5ccfa96f1b28b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:47.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c72ffd700 1 -- 172.21.15.107:0/3716835347 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c840adbd0 con 0x560c67799a80 2023-07-21T22:30:47.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c72ffd700 1 -- 172.21.15.107:0/3716835347 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8c840ca260 con 0x560c67799a80 2023-07-21T22:30:47.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8ca608ab80 1 -- 172.21.15.107:0/3716835347 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c7c021070 con 0x560c67799a80 2023-07-21T22:30:47.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8c72ffd700 1 -- 172.21.15.107:0/3716835347 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c840add30 con 0x560c67799a80 2023-07-21T22:30:47.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8ca608ab80 10 client.15963 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:47.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.024+0000 7f8ca608ab80 1 -- 172.21.15.107:0/3716835347 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x560c6753b250 con 0x560c67799a80 2023-07-21T22:30:47.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.025+0000 7f8c72ffd700 1 -- 172.21.15.107:0/3716835347 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c7c021070 con 0x560c67799a80 2023-07-21T22:30:47.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.025+0000 7f8c72ffd700 1 -- 172.21.15.107:0/3716835347 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8c840b8020 con 0x560c67799a80 2023-07-21T22:30:47.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.025+0000 7f8ca608ab80 10 client.15963 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:47.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.025+0000 7f8ca608ab80 10 client.15963 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-21T22:30:47.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.025+0000 7f8ca608ab80 20 client.15963 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:47.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.025+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/3716835347 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x560c67803f20 0x560c67806330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.025+0000 7f8ca608ab80 4 client.15963 mds_command: new command op to 24437 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:30:47.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.025+0000 7f8ca608ab80 1 -- 172.21.15.107:0/3716835347 --> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560c676a2050 con 0x560c67803f20 2023-07-21T22:30:47.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.025+0000 7f8c8affd700 1 --2- 172.21.15.107:0/3716835347 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x560c67803f20 0x560c67806330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.025+0000 7f8c8affd700 1 --2- 172.21.15.107:0/3716835347 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x560c67803f20 0x560c67806330 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=78de0acb0fb6c3e7 server_cookie=8bd896cfc1ca2178 in_seq=0 out_seq=0 2023-07-21T22:30:47.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.025+0000 7f8c72ffd700 10 client.15963 ms_handle_connect on v2:172.21.15.132:6832/3293513462 2023-07-21T22:30:47.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8c72ffd700 1 -- 172.21.15.107:0/3716835347 <== mds.4 v2:172.21.15.132:6832/3293513462 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560c676a2050 con 0x560c67803f20 2023-07-21T22:30:47.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8c72ffd700 10 client.15963 handle_command_reply: tid=0 2023-07-21T22:30:47.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8ca608ab80 1 client.15963 shutdown 2023-07-21T22:30:47.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8ca608ab80 1 -- 172.21.15.107:0/3716835347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 msgr2=0x560c677a6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/3716835347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 0x560c677a6c00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8c840a1df0 tx=0x7f8c840b8300 comp rx=0 tx=0).stop 2023-07-21T22:30:47.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8ca608ab80 1 -- 172.21.15.107:0/3716835347 shutdown_connections 2023-07-21T22:30:47.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/3716835347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6779aff0 0x560c677aa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/3716835347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c67799a80 0x560c677a6c00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/3716835347 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x560c67803f20 0x560c67806330 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8ca608ab80 1 --2- 172.21.15.107:0/3716835347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560c6779a3d0 0x560c677a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8c72ffd700 0 client.15963 ms_handle_reset on v2:172.21.15.132:6832/3293513462 2023-07-21T22:30:47.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8ca608ab80 1 -- 172.21.15.107:0/3716835347 >> 172.21.15.107:0/3716835347 conn(0x560c6778aef0 msgr2=0x560c677a1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8ca608ab80 1 -- 172.21.15.107:0/3716835347 shutdown_connections 2023-07-21T22:30:47.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8ca608ab80 1 -- 172.21.15.107:0/3716835347 wait complete. 2023-07-21T22:30:47.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8ca608ab80 20 client.15963 trim_cache size 0 max 16384 2023-07-21T22:30:47.083 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:30:47.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8c99f05700 1 -- 172.21.15.107:0/1604890877 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8c800a1150 msgr2=0x7f8c800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8c99f05700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8c800a1150 0x7f8c800a3600 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8c94068f40 tx=0x7f8c7c009260 comp rx=0 tx=0).stop 2023-07-21T22:30:47.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8c99f05700 1 -- 172.21.15.107:0/1604890877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c940fea10 msgr2=0x7f8c940691d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.026+0000 7f8c99f05700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c940fea10 0x7f8c940691d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400e970 tx=0x7f8c8400ec80 comp rx=0 tx=0).stop 2023-07-21T22:30:47.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.027+0000 7f8c99f05700 1 -- 172.21.15.107:0/1604890877 shutdown_connections 2023-07-21T22:30:47.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.027+0000 7f8c99f05700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f8c800a1150 0x7f8c800a3600 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.027+0000 7f8c99f05700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c94100590 0x7f8c9419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.027+0000 7f8c99f05700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c940fea10 0x7f8c940691d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.027+0000 7f8c99f05700 1 --2- 172.21.15.107:0/1604890877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c940ffc00 0x7f8c9419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.027+0000 7f8c99f05700 1 -- 172.21.15.107:0/1604890877 >> 172.21.15.107:0/1604890877 conn(0x7f8c940fa030 msgr2=0x7f8c94103050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.027+0000 7f8c99f05700 1 -- 172.21.15.107:0/1604890877 shutdown_connections 2023-07-21T22:30:47.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.027+0000 7f8c99f05700 1 -- 172.21.15.107:0/1604890877 wait complete. 2023-07-21T22:30:47.086 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-21T22:30:47.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:47 smithi107 ceph-mon[96543]: pgmap v344: 97 pgs: 97 active+clean; 104 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 229 op/s 2023-07-21T22:30:47.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:47 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:47.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:47 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1505226923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:30:47.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.171+0000 7fcaae076700 1 -- 172.21.15.107:0/2214636623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa80fe9e0 msgr2=0x7fcaa80fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.171+0000 7fcaae076700 1 --2- 172.21.15.107:0/2214636623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa80fe9e0 0x7fcaa80fedf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fca98009990 tx=0x7fca98009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.171+0000 7fcaae076700 1 -- 172.21.15.107:0/2214636623 shutdown_connections 2023-07-21T22:30:47.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.171+0000 7fcaae076700 1 --2- 172.21.15.107:0/2214636623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa8100560 0x7fcaa8106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.171+0000 7fcaae076700 1 --2- 172.21.15.107:0/2214636623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa80fe9e0 0x7fcaa80fedf0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.171+0000 7fcaae076700 1 --2- 172.21.15.107:0/2214636623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa80ffbd0 0x7fcaa8100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.171+0000 7fcaae076700 1 -- 172.21.15.107:0/2214636623 >> 172.21.15.107:0/2214636623 conn(0x7fcaa80fa000 msgr2=0x7fcaa80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.171+0000 7fcaae076700 1 -- 172.21.15.107:0/2214636623 shutdown_connections 2023-07-21T22:30:47.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.171+0000 7fcaae076700 1 -- 172.21.15.107:0/2214636623 wait complete. 2023-07-21T22:30:47.219 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.171+0000 7fcaae076700 1 Processor -- start 2023-07-21T22:30:47.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaae076700 1 -- start start 2023-07-21T22:30:47.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaae076700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa80fe9e0 0x7fcaa8197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaae076700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa80ffbd0 0x7fcaa8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaae076700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa8100560 0x7fcaa819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaae076700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa819f5e0 con 0x7fcaa80fe9e0 2023-07-21T22:30:47.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaae076700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa819f720 con 0x7fcaa8100560 2023-07-21T22:30:47.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaae076700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa819f8b0 con 0x7fcaa80ffbd0 2023-07-21T22:30:47.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaa77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa80fe9e0 0x7fcaa8197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaa77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa80fe9e0 0x7fcaa8197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49382/0 (socket says 172.21.15.107:49382) 2023-07-21T22:30:47.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaa77fe700 1 -- 172.21.15.107:0/3196230530 learned_addr learned my addr 172.21.15.107:0/3196230530 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaa6ffd700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa80ffbd0 0x7fcaa8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaa7fff700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa8100560 0x7fcaa819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaa77fe700 1 -- 172.21.15.107:0/3196230530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa80ffbd0 msgr2=0x7fcaa8197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaa77fe700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa80ffbd0 0x7fcaa8197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaa77fe700 1 -- 172.21.15.107:0/3196230530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa8100560 msgr2=0x7fcaa819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaa77fe700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa8100560 0x7fcaa819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaa77fe700 1 -- 172.21.15.107:0/3196230530 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca980095f0 con 0x7fcaa80fe9e0 2023-07-21T22:30:47.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaa7fff700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa8100560 0x7fcaa819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:47.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.172+0000 7fcaa77fe700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa80fe9e0 0x7fcaa8197a60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fca9800b620 tx=0x7fca98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.223 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.173+0000 7fcaa4ff9700 1 -- 172.21.15.107:0/3196230530 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9801d070 con 0x7fcaa80fe9e0 2023-07-21T22:30:47.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.173+0000 7fcaae076700 1 -- 172.21.15.107:0/3196230530 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa819fb30 con 0x7fcaa80fe9e0 2023-07-21T22:30:47.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.173+0000 7fcaa4ff9700 1 -- 172.21.15.107:0/3196230530 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca98005030 con 0x7fcaa80fe9e0 2023-07-21T22:30:47.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.173+0000 7fcaa4ff9700 1 -- 172.21.15.107:0/3196230530 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98004bf0 con 0x7fcaa80fe9e0 2023-07-21T22:30:47.224 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.173+0000 7fcaae076700 1 -- 172.21.15.107:0/3196230530 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa819ffb0 con 0x7fcaa80fe9e0 2023-07-21T22:30:47.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.175+0000 7fcaa4ff9700 1 -- 172.21.15.107:0/3196230530 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fca98004d50 con 0x7fcaa80fe9e0 2023-07-21T22:30:47.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.175+0000 7fcaa4ff9700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fca900a1150 0x7fca900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.176+0000 7fcaa6ffd700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fca900a1150 0x7fca900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.176+0000 7fcaa4ff9700 1 -- 172.21.15.107:0/3196230530 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fca980cd9c0 con 0x7fcaa80fe9e0 2023-07-21T22:30:47.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.176+0000 7fcaa6ffd700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fca900a1150 0x7fca900a3600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009c60 tx=0x7fca9c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.176+0000 7fcaba1fbb80 1 Processor -- start 2023-07-21T22:30:47.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.176+0000 7fcaba1fbb80 1 -- start start 2023-07-21T22:30:47.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.176+0000 7fcaba1fbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 0x55f7060e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.176+0000 7fcaba1fbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e21a0 0x55f7060e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.176+0000 7fcaba1fbb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e0a10 0x55f7060e0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.176+0000 7fcaba1fbb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f705fe5cb0 con 0x55f7060e0a10 2023-07-21T22:30:47.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.176+0000 7fcaba1fbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f706038a40 con 0x55f7060e21a0 2023-07-21T22:30:47.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.176+0000 7fcaba1fbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f706031ed0 con 0x55f7060e1830 2023-07-21T22:30:47.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fcaa77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 0x55f7060e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fcaa7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e0a10 0x55f7060e0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fcaa77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 0x55f7060e1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37948/0 (socket says 172.21.15.107:37948) 2023-07-21T22:30:47.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fcaa77fe700 1 -- 172.21.15.107:0/2637821276 learned_addr learned my addr 172.21.15.107:0/2637821276 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fcaa6ffd700 1 --2- 172.21.15.107:0/2637821276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e21a0 0x55f7060e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fcaa77fe700 1 -- 172.21.15.107:0/2637821276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e21a0 msgr2=0x55f7060e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fcaa77fe700 1 --2- 172.21.15.107:0/2637821276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e21a0 0x55f7060e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fcaa77fe700 1 -- 172.21.15.107:0/2637821276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e0a10 msgr2=0x55f7060e0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.230 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fcaa77fe700 1 --2- 172.21.15.107:0/2637821276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e0a10 0x55f7060e0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fcaa77fe700 1 -- 172.21.15.107:0/2637821276 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f705e5fde0 con 0x55f7060e1830 2023-07-21T22:30:47.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fcaa77fe700 1 --2- 172.21.15.107:0/2637821276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 0x55f7060e1c40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fca980a2c70 tx=0x7fca98005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d277f8a03b0d839 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fca8dffb700 1 -- 172.21.15.107:0/2637821276 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98099180 con 0x55f7060e1830 2023-07-21T22:30:47.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fca8dffb700 1 -- 172.21.15.107:0/2637821276 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca980054a0 con 0x55f7060e1830 2023-07-21T22:30:47.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.177+0000 7fca8dffb700 1 -- 172.21.15.107:0/2637821276 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9809a5a0 con 0x55f7060e1830 2023-07-21T22:30:47.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.178+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/2637821276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 msgr2=0x55f7060e1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.231 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.178+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/2637821276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 0x55f7060e1c40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fca980a2c70 tx=0x7fca98005ef0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.178+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/2637821276 shutdown_connections 2023-07-21T22:30:47.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.178+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/2637821276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e0a10 0x55f7060e0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.178+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/2637821276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 0x55f7060e1c40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.178+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/2637821276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e21a0 0x55f7060e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.178+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/2637821276 >> 172.21.15.107:0/2637821276 conn(0x55f7060d5ef0 msgr2=0x55f7060db360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.178+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/2637821276 shutdown_connections 2023-07-21T22:30:47.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.178+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/2637821276 wait complete. 2023-07-21T22:30:47.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.178+0000 7fcaba1fbb80 1 -- start start 2023-07-21T22:30:47.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.178+0000 7fcaba1fbb80 1 Processor -- start 2023-07-21T22:30:47.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaba1fbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e0a10 0x55f7060e9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaba1fbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 0x55f7060ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaba1fbb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e21a0 0x55f7060f1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaba1fbb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f705fd1ac0 con 0x55f7060e21a0 2023-07-21T22:30:47.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaba1fbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f705fd17c0 con 0x55f7060e0a10 2023-07-21T22:30:47.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaba1fbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f705eb5920 con 0x55f7060e1830 2023-07-21T22:30:47.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaba1fbb80 10 client.0 _ll_register_callbacks cb 0x55f705fcf760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:47.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaa6ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 0x55f7060ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaa7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e21a0 0x55f7060f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaa6ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 0x55f7060ea2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37956/0 (socket says 172.21.15.107:37956) 2023-07-21T22:30:47.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaa6ffd700 1 -- 172.21.15.107:0/3508307580 learned_addr learned my addr 172.21.15.107:0/3508307580 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaa77fe700 1 --2- 172.21.15.107:0/3508307580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e0a10 0x55f7060e9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaa6ffd700 1 -- 172.21.15.107:0/3508307580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e0a10 msgr2=0x55f7060e9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaa6ffd700 1 --2- 172.21.15.107:0/3508307580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e0a10 0x55f7060e9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaa6ffd700 1 -- 172.21.15.107:0/3508307580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e21a0 msgr2=0x55f7060f1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaa6ffd700 1 --2- 172.21.15.107:0/3508307580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e21a0 0x55f7060f1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaa6ffd700 1 -- 172.21.15.107:0/3508307580 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa819ffb0 con 0x55f7060e1830 2023-07-21T22:30:47.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fcaa6ffd700 1 --2- 172.21.15.107:0/3508307580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 0x55f7060ea2c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fca9c01ebc0 tx=0x7fca9c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70bcb49ca6fa069f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fca86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:30:47.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.179+0000 7fca86ffd700 1 -- 172.21.15.107:0/3508307580 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00ebd0 con 0x55f7060e1830 2023-07-21T22:30:47.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.180+0000 7fca86ffd700 1 -- 172.21.15.107:0/3508307580 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fca9c00ed30 con 0x55f7060e1830 2023-07-21T22:30:47.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.180+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3508307580 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f705f8b4f0 con 0x55f7060e1830 2023-07-21T22:30:47.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.180+0000 7fca86ffd700 1 -- 172.21.15.107:0/3508307580 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca9c0283f0 con 0x55f7060e1830 2023-07-21T22:30:47.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.180+0000 7fcaba1fbb80 10 client.25501 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:47.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.180+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3508307580 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f705e48c60 con 0x55f7060e1830 2023-07-21T22:30:47.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.180+0000 7fca86ffd700 1 -- 172.21.15.107:0/3508307580 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f705f8b4f0 con 0x55f7060e1830 2023-07-21T22:30:47.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.180+0000 7fca86ffd700 1 -- 172.21.15.107:0/3508307580 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fca9c01c5d0 con 0x55f7060e1830 2023-07-21T22:30:47.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.180+0000 7fcaba1fbb80 10 client.25501 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:47.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.180+0000 7fcaba1fbb80 10 client.25501 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T22:30:47.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.180+0000 7fcaba1fbb80 20 client.25501 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:47.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.180+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/3508307580 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55f7060f5c90 0x55f7060f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.180+0000 7fcaba1fbb80 4 client.25501 mds_command: new command op to 24455 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:47.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.180+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3508307580 --> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f70602aeb0 con 0x55f7060f5c90 2023-07-21T22:30:47.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.200+0000 7fcaa77fe700 1 --2- 172.21.15.107:0/3508307580 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55f7060f5c90 0x55f7060f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.200+0000 7fcaa77fe700 1 --2- 172.21.15.107:0/3508307580 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55f7060f5c90 0x55f7060f80a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=98fcddac1e89c1d6 server_cookie=617fef47a66e6d6c in_seq=0 out_seq=0 2023-07-21T22:30:47.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.200+0000 7fca86ffd700 10 client.25501 ms_handle_connect on v2:172.21.15.132:6834/1022724305 2023-07-21T22:30:47.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fca86ffd700 1 -- 172.21.15.107:0/3508307580 <== mds.3 v2:172.21.15.132:6834/1022724305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f70602aeb0 con 0x55f7060f5c90 2023-07-21T22:30:47.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fca86ffd700 10 client.25501 handle_command_reply: tid=0 2023-07-21T22:30:47.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fcaba1fbb80 1 client.25501 shutdown 2023-07-21T22:30:47.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3508307580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 msgr2=0x55f7060ea2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/3508307580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 0x55f7060ea2c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fca9c01ebc0 tx=0x7fca9c0047c0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3508307580 shutdown_connections 2023-07-21T22:30:47.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/3508307580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e21a0 0x55f7060f1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/3508307580 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55f7060f5c90 0x55f7060f80a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/3508307580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e0a10 0x55f7060e9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/3508307580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e1830 0x55f7060ea2c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fca86ffd700 0 client.25501 ms_handle_reset on v2:172.21.15.132:6834/1022724305 2023-07-21T22:30:47.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3508307580 >> 172.21.15.107:0/3508307580 conn(0x55f7060d5ef0 msgr2=0x55f7060daa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3508307580 shutdown_connections 2023-07-21T22:30:47.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.233+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3508307580 wait complete. 2023-07-21T22:30:47.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.234+0000 7fcaba1fbb80 20 client.25501 trim_cache size 0 max 16384 2023-07-21T22:30:47.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.253+0000 7fcaba1fbb80 1 Processor -- start 2023-07-21T22:30:47.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.253+0000 7fcaba1fbb80 1 -- start start 2023-07-21T22:30:47.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.253+0000 7fcaba1fbb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e0a10 0x55f7060e9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.253+0000 7fcaba1fbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e1830 0x55f7060ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.253+0000 7fcaba1fbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e21a0 0x55f7060f1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.253+0000 7fcaba1fbb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7060f57d0 con 0x55f7060e0a10 2023-07-21T22:30:47.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.253+0000 7fcaba1fbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7060e9230 con 0x55f7060e1830 2023-07-21T22:30:47.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.253+0000 7fcaba1fbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7060e9920 con 0x55f7060e21a0 2023-07-21T22:30:47.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fcaa77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e0a10 0x55f7060e9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fcaa6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e1830 0x55f7060ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fcaa77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e0a10 0x55f7060e9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49404/0 (socket says 172.21.15.107:49404) 2023-07-21T22:30:47.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fcaa7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e21a0 0x55f7060f1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fcaa77fe700 1 -- 172.21.15.107:0/1790929789 learned_addr learned my addr 172.21.15.107:0/1790929789 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fcaa6ffd700 1 -- 172.21.15.107:0/1790929789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e21a0 msgr2=0x55f7060f1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fcaa6ffd700 1 --2- 172.21.15.107:0/1790929789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e21a0 0x55f7060f1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fcaa6ffd700 1 -- 172.21.15.107:0/1790929789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e0a10 msgr2=0x55f7060e9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fcaa6ffd700 1 --2- 172.21.15.107:0/1790929789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e0a10 0x55f7060e9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fcaa6ffd700 1 -- 172.21.15.107:0/1790929789 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f705e86250 con 0x55f7060e1830 2023-07-21T22:30:47.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fcaa77fe700 1 --2- 172.21.15.107:0/1790929789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e0a10 0x55f7060e9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fcaa6ffd700 1 --2- 172.21.15.107:0/1790929789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e1830 0x55f7060ea2c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fca9c01ca80 tx=0x7fca9c01cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fda5d33d2d0b5e98 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fca8e7fc700 1 -- 172.21.15.107:0/1790929789 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c01cda0 con 0x55f7060e1830 2023-07-21T22:30:47.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fca8e7fc700 1 -- 172.21.15.107:0/1790929789 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca9c00ee90 con 0x55f7060e1830 2023-07-21T22:30:47.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.254+0000 7fca8e7fc700 1 -- 172.21.15.107:0/1790929789 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c0045b0 con 0x55f7060e1830 2023-07-21T22:30:47.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.255+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/1790929789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e1830 msgr2=0x55f7060ea2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.255+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/1790929789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e1830 0x55f7060ea2c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fca9c01ca80 tx=0x7fca9c01cf50 comp rx=0 tx=0).stop 2023-07-21T22:30:47.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.255+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/1790929789 shutdown_connections 2023-07-21T22:30:47.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.255+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/1790929789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e21a0 0x55f7060f1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.255+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/1790929789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e0a10 0x55f7060e9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.255+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/1790929789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e1830 0x55f7060ea2c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.255+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/1790929789 >> 172.21.15.107:0/1790929789 conn(0x55f7060d5ef0 msgr2=0x55f7060e7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.255+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/1790929789 shutdown_connections 2023-07-21T22:30:47.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.255+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/1790929789 wait complete. 2023-07-21T22:30:47.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaba1fbb80 1 -- start start 2023-07-21T22:30:47.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaba1fbb80 1 Processor -- start 2023-07-21T22:30:47.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaba1fbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e0a10 0x55f7060f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaba1fbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e1830 0x55f7060f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaba1fbb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e21a0 0x55f7060ed5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaba1fbb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7060edca0 con 0x55f7060e21a0 2023-07-21T22:30:47.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaba1fbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7060ede10 con 0x55f7060e1830 2023-07-21T22:30:47.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaba1fbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7060edfd0 con 0x55f7060e0a10 2023-07-21T22:30:47.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaba1fbb80 10 client.0 _ll_register_callbacks cb 0x7fca9c01e7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:47.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaa77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e0a10 0x55f7060f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaa6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e1830 0x55f7060f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaa6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e1830 0x55f7060f1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40240/0 (socket says 172.21.15.107:40240) 2023-07-21T22:30:47.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaa77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e0a10 0x55f7060f0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37968/0 (socket says 172.21.15.107:37968) 2023-07-21T22:30:47.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaa6ffd700 1 -- 172.21.15.107:0/3491696337 learned_addr learned my addr 172.21.15.107:0/3491696337 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.256+0000 7fcaa7fff700 1 --2- 172.21.15.107:0/3491696337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e21a0 0x55f7060ed5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fcaa6ffd700 1 -- 172.21.15.107:0/3491696337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e0a10 msgr2=0x55f7060f0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fcaa6ffd700 1 --2- 172.21.15.107:0/3491696337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e0a10 0x55f7060f0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fcaa6ffd700 1 -- 172.21.15.107:0/3491696337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e21a0 msgr2=0x55f7060ed5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fcaa6ffd700 1 --2- 172.21.15.107:0/3491696337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e21a0 0x55f7060ed5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fcaa6ffd700 1 -- 172.21.15.107:0/3491696337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca980b2040 con 0x55f7060e1830 2023-07-21T22:30:47.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fcaa6ffd700 1 --2- 172.21.15.107:0/3491696337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e1830 0x55f7060f1060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fca9c01c980 tx=0x7fca9c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e8ff5db006e907e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fca86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:30:47.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fca86ffd700 1 -- 172.21.15.107:0/3491696337 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00ebd0 con 0x55f7060e1830 2023-07-21T22:30:47.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fca86ffd700 1 -- 172.21.15.107:0/3491696337 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fca9c01c6e0 con 0x55f7060e1830 2023-07-21T22:30:47.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3491696337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca980054a0 con 0x55f7060e1830 2023-07-21T22:30:47.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fca86ffd700 1 -- 172.21.15.107:0/3491696337 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca9c00ed30 con 0x55f7060e1830 2023-07-21T22:30:47.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fcaba1fbb80 10 client.25445 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:47.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3491696337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f705e48c60 con 0x55f7060e1830 2023-07-21T22:30:47.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.257+0000 7fca86ffd700 1 -- 172.21.15.107:0/3491696337 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca980054a0 con 0x55f7060e1830 2023-07-21T22:30:47.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.258+0000 7fca86ffd700 1 -- 172.21.15.107:0/3491696337 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fca9c039030 con 0x55f7060e1830 2023-07-21T22:30:47.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.258+0000 7fcaba1fbb80 10 client.25445 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:47.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.258+0000 7fcaba1fbb80 10 client.25445 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T22:30:47.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.258+0000 7fcaba1fbb80 20 client.25445 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:47.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.258+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/3491696337 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55f70614ae40 0x55f70614d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.258+0000 7fcaba1fbb80 4 client.25445 mds_command: new command op to 24455 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:30:47.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.258+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3491696337 --> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f705fed050 con 0x55f70614ae40 2023-07-21T22:30:47.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.258+0000 7fcaa77fe700 1 --2- 172.21.15.107:0/3491696337 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55f70614ae40 0x55f70614d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.258+0000 7fca86ffd700 10 client.25445 ms_handle_connect on v2:172.21.15.132:6834/1022724305 2023-07-21T22:30:47.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.258+0000 7fcaa77fe700 1 --2- 172.21.15.107:0/3491696337 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55f70614ae40 0x55f70614d250 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b18fef307b03b8d8 server_cookie=a0213f704d1557d1 in_seq=0 out_seq=0 2023-07-21T22:30:47.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.259+0000 7fca86ffd700 1 -- 172.21.15.107:0/3491696337 <== mds.3 v2:172.21.15.132:6834/1022724305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f705fed050 con 0x55f70614ae40 2023-07-21T22:30:47.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.259+0000 7fca86ffd700 10 client.25445 handle_command_reply: tid=0 2023-07-21T22:30:47.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.259+0000 7fcaba1fbb80 1 client.25445 shutdown 2023-07-21T22:30:47.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.259+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3491696337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e1830 msgr2=0x55f7060f1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.259+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/3491696337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e1830 0x55f7060f1060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fca9c01c980 tx=0x7fca9c0045e0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.259+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3491696337 shutdown_connections 2023-07-21T22:30:47.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.259+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/3491696337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f7060e21a0 0x55f7060ed5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.259+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/3491696337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f7060e0a10 0x55f7060f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.259+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/3491696337 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x55f70614ae40 0x55f70614d250 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.259+0000 7fcaba1fbb80 1 --2- 172.21.15.107:0/3491696337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f7060e1830 0x55f7060f1060 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.259+0000 7fca86ffd700 0 client.25445 ms_handle_reset on v2:172.21.15.132:6834/1022724305 2023-07-21T22:30:47.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.259+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3491696337 >> 172.21.15.107:0/3491696337 conn(0x55f7060d5ef0 msgr2=0x55f7060e4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3491696337 shutdown_connections 2023-07-21T22:30:47.317 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaba1fbb80 1 -- 172.21.15.107:0/3491696337 wait complete. 2023-07-21T22:30:47.318 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaba1fbb80 20 client.25445 trim_cache size 0 max 16384 2023-07-21T22:30:47.318 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:30:47.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaae076700 1 -- 172.21.15.107:0/3196230530 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fca900a1150 msgr2=0x7fca900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaae076700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fca900a1150 0x7fca900a3600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fca9c009c60 tx=0x7fca9c009260 comp rx=0 tx=0).stop 2023-07-21T22:30:47.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaae076700 1 -- 172.21.15.107:0/3196230530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa80fe9e0 msgr2=0x7fcaa8197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaae076700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa80fe9e0 0x7fcaa8197a60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fca9800b620 tx=0x7fca98004730 comp rx=0 tx=0).stop 2023-07-21T22:30:47.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaae076700 1 -- 172.21.15.107:0/3196230530 shutdown_connections 2023-07-21T22:30:47.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaae076700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaa8100560 0x7fcaa819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaae076700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaa80fe9e0 0x7fcaa8197a60 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaae076700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fca900a1150 0x7fca900a3600 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaae076700 1 --2- 172.21.15.107:0/3196230530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaa80ffbd0 0x7fcaa8197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaae076700 1 -- 172.21.15.107:0/3196230530 >> 172.21.15.107:0/3196230530 conn(0x7fcaa80fa000 msgr2=0x7fcaa8068720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaae076700 1 -- 172.21.15.107:0/3196230530 shutdown_connections 2023-07-21T22:30:47.321 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.260+0000 7fcaae076700 1 -- 172.21.15.107:0/3196230530 wait complete. 2023-07-21T22:30:47.328 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:30:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:47 smithi132 ceph-mon[97863]: pgmap v344: 97 pgs: 97 active+clean; 104 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 229 op/s 2023-07-21T22:30:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:47 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:47 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1505226923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:30:47.490 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.441+0000 7fb2a358b700 1 -- 172.21.15.107:0/2359631154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb29c1005b0 msgr2=0x7fb29c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.491 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.441+0000 7fb2a358b700 1 --2- 172.21.15.107:0/2359631154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb29c1005b0 0x7fb29c106e50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb29400b0a0 tx=0x7fb29400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.491 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.441+0000 7fb2a358b700 1 -- 172.21.15.107:0/2359631154 shutdown_connections 2023-07-21T22:30:47.492 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.441+0000 7fb2a358b700 1 --2- 172.21.15.107:0/2359631154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb29c1005b0 0x7fb29c106e50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.492 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.441+0000 7fb2a358b700 1 --2- 172.21.15.107:0/2359631154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb29c0feef0 0x7fb29c0ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.492 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.441+0000 7fb2a358b700 1 --2- 172.21.15.107:0/2359631154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb29c0ffc00 0x7fb29c100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.492 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.441+0000 7fb2a358b700 1 -- 172.21.15.107:0/2359631154 >> 172.21.15.107:0/2359631154 conn(0x7fb29c0f9770 msgr2=0x7fb29c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.441+0000 7fb2a358b700 1 -- 172.21.15.107:0/2359631154 shutdown_connections 2023-07-21T22:30:47.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.441+0000 7fb2a358b700 1 -- 172.21.15.107:0/2359631154 wait complete. 2023-07-21T22:30:47.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.441+0000 7fb2a358b700 1 Processor -- start 2023-07-21T22:30:47.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.441+0000 7fb2a358b700 1 -- start start 2023-07-21T22:30:47.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a358b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb29c0feef0 0x7fb29c0fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a358b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb29c0ffc00 0x7fb29c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a358b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb29c1005b0 0x7fb29c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a358b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c19f680 con 0x7fb29c0feef0 2023-07-21T22:30:47.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a358b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c19f7f0 con 0x7fb29c0ffc00 2023-07-21T22:30:47.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a358b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c19f9b0 con 0x7fb29c1005b0 2023-07-21T22:30:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a1327700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb29c0feef0 0x7fb29c0fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a1327700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb29c0feef0 0x7fb29c0fe5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49424/0 (socket says 172.21.15.107:49424) 2023-07-21T22:30:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a1327700 1 -- 172.21.15.107:0/145920494 learned_addr learned my addr 172.21.15.107:0/145920494 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a1b28700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb29c1005b0 0x7fb29c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a0b26700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb29c0ffc00 0x7fb29c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a1327700 1 -- 172.21.15.107:0/145920494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb29c1005b0 msgr2=0x7fb29c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a1327700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb29c1005b0 0x7fb29c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a1327700 1 -- 172.21.15.107:0/145920494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb29c0ffc00 msgr2=0x7fb29c19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a1327700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb29c0ffc00 0x7fb29c19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a1327700 1 -- 172.21.15.107:0/145920494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb294009d00 con 0x7fb29c0feef0 2023-07-21T22:30:47.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a1327700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb29c0feef0 0x7fb29c0fe5b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00e970 tx=0x7fb28c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2927fc700 1 -- 172.21.15.107:0/145920494 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c009790 con 0x7fb29c0feef0 2023-07-21T22:30:47.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a358b700 1 -- 172.21.15.107:0/145920494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29c19fc90 con 0x7fb29c0feef0 2023-07-21T22:30:47.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2927fc700 1 -- 172.21.15.107:0/145920494 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28c00c7e0 con 0x7fb29c0feef0 2023-07-21T22:30:47.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2927fc700 1 -- 172.21.15.107:0/145920494 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c003c70 con 0x7fb29c0feef0 2023-07-21T22:30:47.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.442+0000 7fb2a358b700 1 -- 172.21.15.107:0/145920494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c1a01a0 con 0x7fb29c0feef0 2023-07-21T22:30:47.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.444+0000 7fb2927fc700 1 -- 172.21.15.107:0/145920494 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb28c00cc50 con 0x7fb29c0feef0 2023-07-21T22:30:47.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.445+0000 7fb2927fc700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb2880a1150 0x7fb2880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.445+0000 7fb2a0b26700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb2880a1150 0x7fb2880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.445+0000 7fb2927fc700 1 -- 172.21.15.107:0/145920494 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb28c014070 con 0x7fb29c0feef0 2023-07-21T22:30:47.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.446+0000 7fb2a0b26700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb2880a1150 0x7fb2880a3600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0fec10 tx=0x7fb298009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.446+0000 7fb2af710b80 1 Processor -- start 2023-07-21T22:30:47.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.446+0000 7fb2af710b80 1 -- start start 2023-07-21T22:30:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.446+0000 7fb2af710b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e82830 0x560f97e82c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.446+0000 7fb2af710b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e81430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2af710b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e81a10 0x560f97e81e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2af710b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97d86cb0 con 0x560f97e831a0 2023-07-21T22:30:47.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2af710b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97dd9a40 con 0x560f97e81a10 2023-07-21T22:30:47.509 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2af710b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97dd2ed0 con 0x560f97e82830 2023-07-21T22:30:47.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2a0b26700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e81430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2a0b26700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e81430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49434/0 (socket says 172.21.15.107:49434) 2023-07-21T22:30:47.510 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2a1327700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e82830 0x560f97e82c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2a0b26700 1 -- 172.21.15.107:0/1328138610 learned_addr learned my addr 172.21.15.107:0/1328138610 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2a1b28700 1 --2- 172.21.15.107:0/1328138610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e81a10 0x560f97e81e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.511 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2a1327700 1 -- 172.21.15.107:0/1328138610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e81a10 msgr2=0x560f97e81e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.512 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2a1327700 1 --2- 172.21.15.107:0/1328138610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e81a10 0x560f97e81e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.512 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2a1327700 1 -- 172.21.15.107:0/1328138610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 msgr2=0x560f97e81430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.512 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2a1327700 1 --2- 172.21.15.107:0/1328138610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e81430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2a1327700 1 -- 172.21.15.107:0/1328138610 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f97c00de0 con 0x560f97e82830 2023-07-21T22:30:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2a1b28700 1 --2- 172.21.15.107:0/1328138610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e81a10 0x560f97e81e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2a0b26700 1 --2- 172.21.15.107:0/1328138610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e81430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2a1327700 1 --2- 172.21.15.107:0/1328138610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e82830 0x560f97e82c40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb28c005950 tx=0x7fb28c099aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffc22fb2cc8e3b39 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2877fe700 1 -- 172.21.15.107:0/1328138610 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c09df10 con 0x560f97e82830 2023-07-21T22:30:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2877fe700 1 -- 172.21.15.107:0/1328138610 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28c0a2330 con 0x560f97e82830 2023-07-21T22:30:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.447+0000 7fb2877fe700 1 -- 172.21.15.107:0/1328138610 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c09a140 con 0x560f97e82830 2023-07-21T22:30:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.448+0000 7fb2af710b80 1 -- 172.21.15.107:0/1328138610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e82830 msgr2=0x560f97e82c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.448+0000 7fb2af710b80 1 --2- 172.21.15.107:0/1328138610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e82830 0x560f97e82c40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb28c005950 tx=0x7fb28c099aa0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.448+0000 7fb2af710b80 1 -- 172.21.15.107:0/1328138610 shutdown_connections 2023-07-21T22:30:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.448+0000 7fb2af710b80 1 --2- 172.21.15.107:0/1328138610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e81a10 0x560f97e81e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.448+0000 7fb2af710b80 1 --2- 172.21.15.107:0/1328138610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e82830 0x560f97e82c40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.448+0000 7fb2af710b80 1 --2- 172.21.15.107:0/1328138610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e81430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.448+0000 7fb2af710b80 1 -- 172.21.15.107:0/1328138610 >> 172.21.15.107:0/1328138610 conn(0x560f97e76ef0 msgr2=0x560f97e7c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.448+0000 7fb2af710b80 1 -- 172.21.15.107:0/1328138610 shutdown_connections 2023-07-21T22:30:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.448+0000 7fb2af710b80 1 -- 172.21.15.107:0/1328138610 wait complete. 2023-07-21T22:30:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.448+0000 7fb2af710b80 1 -- start start 2023-07-21T22:30:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2af710b80 1 Processor -- start 2023-07-21T22:30:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2af710b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e81a10 0x560f97e8b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2af710b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e82830 0x560f97e8b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2af710b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e92680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2af710b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97d72ac0 con 0x560f97e831a0 2023-07-21T22:30:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2af710b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97d727c0 con 0x560f97e81a10 2023-07-21T22:30:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2af710b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97c56920 con 0x560f97e82830 2023-07-21T22:30:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2af710b80 10 client.0 _ll_register_callbacks cb 0x560f97d70760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2a1b28700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e92680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2a0b26700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e82830 0x560f97e8b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.519 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2a1327700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e81a10 0x560f97e8b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.519 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2a0b26700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e82830 0x560f97e8b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:37998/0 (socket says 172.21.15.107:37998) 2023-07-21T22:30:47.519 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2a1327700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e81a10 0x560f97e8b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40266/0 (socket says 172.21.15.107:40266) 2023-07-21T22:30:47.519 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2a1b28700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e92680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49442/0 (socket says 172.21.15.107:49442) 2023-07-21T22:30:47.519 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2a1327700 1 -- 172.21.15.107:0/132924121 learned_addr learned my addr 172.21.15.107:0/132924121 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.519 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2a1327700 1 -- 172.21.15.107:0/132924121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e82830 msgr2=0x560f97e8b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.520 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2a1327700 1 --2- 172.21.15.107:0/132924121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e82830 0x560f97e8b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.520 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2a1327700 1 -- 172.21.15.107:0/132924121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 msgr2=0x560f97e92680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.520 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2a1327700 1 --2- 172.21.15.107:0/132924121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e92680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.520 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2a1327700 1 -- 172.21.15.107:0/132924121 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c107750 con 0x560f97e81a10 2023-07-21T22:30:47.520 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.449+0000 7fb2a0b26700 1 --2- 172.21.15.107:0/132924121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e82830 0x560f97e8b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.520 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb2a1327700 1 --2- 172.21.15.107:0/132924121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e81a10 0x560f97e8b100 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb28c099be0 tx=0x7fb28c099790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0ab91cfdd385889 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.521 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb284ff9700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:30:47.521 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb284ff9700 1 -- 172.21.15.107:0/132924121 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0c8070 con 0x560f97e81a10 2023-07-21T22:30:47.521 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb2af710b80 1 -- 172.21.15.107:0/132924121 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f97d2c4f0 con 0x560f97e81a10 2023-07-21T22:30:47.521 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb284ff9700 1 -- 172.21.15.107:0/132924121 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb28c09df10 con 0x560f97e81a10 2023-07-21T22:30:47.521 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb284ff9700 1 -- 172.21.15.107:0/132924121 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb28c0ca2b0 con 0x560f97e81a10 2023-07-21T22:30:47.521 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb2af710b80 10 client.25457 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:47.521 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb2af710b80 1 -- 172.21.15.107:0/132924121 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fb298017070 con 0x560f97e81a10 2023-07-21T22:30:47.522 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb284ff9700 1 -- 172.21.15.107:0/132924121 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28c07c080 con 0x560f97e81a10 2023-07-21T22:30:47.522 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb284ff9700 1 -- 172.21.15.107:0/132924121 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb28c0a1fc0 con 0x560f97e81a10 2023-07-21T22:30:47.522 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb2af710b80 10 client.25457 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:47.522 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb2af710b80 10 client.25457 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-07-21T22:30:47.522 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb2af710b80 20 client.25457 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:47.522 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb2af710b80 1 --2- 172.21.15.107:0/132924121 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x560f97e96ac0 0x560f97e98ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.522 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb2af710b80 4 client.25457 mds_command: new command op to 24466 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.450+0000 7fb2af710b80 1 -- 172.21.15.107:0/132924121 --> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f97dcbeb0 con 0x560f97e96ac0 2023-07-21T22:30:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.451+0000 7fb2a0b26700 1 --2- 172.21.15.107:0/132924121 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x560f97e96ac0 0x560f97e98ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.451+0000 7fb284ff9700 10 client.25457 ms_handle_connect on v2:172.21.15.190:6834/2286851849 2023-07-21T22:30:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.451+0000 7fb2a0b26700 1 --2- 172.21.15.107:0/132924121 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x560f97e96ac0 0x560f97e98ed0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bff045096ef1df9 server_cookie=f56afcc13047e24f in_seq=0 out_seq=0 2023-07-21T22:30:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.452+0000 7fb284ff9700 1 -- 172.21.15.107:0/132924121 <== mds.2 v2:172.21.15.190:6834/2286851849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560f97dcbeb0 con 0x560f97e96ac0 2023-07-21T22:30:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.452+0000 7fb284ff9700 10 client.25457 handle_command_reply: tid=0 2023-07-21T22:30:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.452+0000 7fb2af710b80 1 client.25457 shutdown 2023-07-21T22:30:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.453+0000 7fb2af710b80 1 -- 172.21.15.107:0/132924121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e81a10 msgr2=0x560f97e8b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.453+0000 7fb2af710b80 1 --2- 172.21.15.107:0/132924121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e81a10 0x560f97e8b100 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb28c099be0 tx=0x7fb28c099790 comp rx=0 tx=0).stop 2023-07-21T22:30:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.453+0000 7fb2af710b80 1 -- 172.21.15.107:0/132924121 shutdown_connections 2023-07-21T22:30:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.453+0000 7fb2af710b80 1 --2- 172.21.15.107:0/132924121 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x560f97e96ac0 0x560f97e98ed0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.453+0000 7fb2af710b80 1 --2- 172.21.15.107:0/132924121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e92680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.453+0000 7fb2af710b80 1 --2- 172.21.15.107:0/132924121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e81a10 0x560f97e8b100 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.453+0000 7fb2af710b80 1 --2- 172.21.15.107:0/132924121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e82830 0x560f97e8b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.453+0000 7fb284ff9700 0 client.25457 ms_handle_reset on v2:172.21.15.190:6834/2286851849 2023-07-21T22:30:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.453+0000 7fb2af710b80 1 -- 172.21.15.107:0/132924121 >> 172.21.15.107:0/132924121 conn(0x560f97e76ef0 msgr2=0x560f97e7ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.453+0000 7fb2af710b80 1 -- 172.21.15.107:0/132924121 shutdown_connections 2023-07-21T22:30:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.453+0000 7fb2af710b80 1 -- 172.21.15.107:0/132924121 wait complete. 2023-07-21T22:30:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.453+0000 7fb2af710b80 20 client.25457 trim_cache size 0 max 16384 2023-07-21T22:30:47.529 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.466+0000 7fb2af710b80 1 Processor -- start 2023-07-21T22:30:47.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.466+0000 7fb2af710b80 1 -- start start 2023-07-21T22:30:47.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.466+0000 7fb2af710b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e81a10 0x560f97e81350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.466+0000 7fb2af710b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e82830 0x560f97e92640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.466+0000 7fb2af710b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e89de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.466+0000 7fb2af710b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97e76a50 con 0x560f97e831a0 2023-07-21T22:30:47.530 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.466+0000 7fb2af710b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97e81890 con 0x560f97e82830 2023-07-21T22:30:47.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.466+0000 7fb2af710b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97e8a5b0 con 0x560f97e81a10 2023-07-21T22:30:47.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.466+0000 7fb2a1b28700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e89de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.466+0000 7fb2a1327700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e81a10 0x560f97e81350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.466+0000 7fb2a1b28700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e89de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49450/0 (socket says 172.21.15.107:49450) 2023-07-21T22:30:47.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.466+0000 7fb2a1327700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e81a10 0x560f97e81350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38004/0 (socket says 172.21.15.107:38004) 2023-07-21T22:30:47.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.467+0000 7fb2a1b28700 1 -- 172.21.15.107:0/865303914 learned_addr learned my addr 172.21.15.107:0/865303914 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.467+0000 7fb2a0b26700 1 --2- 172.21.15.107:0/865303914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e82830 0x560f97e92640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.467+0000 7fb2a1b28700 1 -- 172.21.15.107:0/865303914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e81a10 msgr2=0x560f97e81350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.467+0000 7fb2a1b28700 1 --2- 172.21.15.107:0/865303914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e81a10 0x560f97e81350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.467+0000 7fb2a1b28700 1 -- 172.21.15.107:0/865303914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e82830 msgr2=0x560f97e92640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.467+0000 7fb2a1b28700 1 --2- 172.21.15.107:0/865303914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e82830 0x560f97e92640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.467+0000 7fb2a1b28700 1 -- 172.21.15.107:0/865303914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c1a01a0 con 0x560f97e831a0 2023-07-21T22:30:47.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.467+0000 7fb2a1327700 1 --2- 172.21.15.107:0/865303914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e81a10 0x560f97e81350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.467+0000 7fb2a1b28700 1 --2- 172.21.15.107:0/865303914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e89de0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb2940095d0 tx=0x7fb2940074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f14f2a4543904a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.467+0000 7fb287fff700 1 -- 172.21.15.107:0/865303914 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400e040 con 0x560f97e831a0 2023-07-21T22:30:47.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.467+0000 7fb287fff700 1 -- 172.21.15.107:0/865303914 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb294007880 con 0x560f97e831a0 2023-07-21T22:30:47.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.467+0000 7fb287fff700 1 -- 172.21.15.107:0/865303914 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294005540 con 0x560f97e831a0 2023-07-21T22:30:47.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.468+0000 7fb2af710b80 1 -- 172.21.15.107:0/865303914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 msgr2=0x560f97e89de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.468+0000 7fb2af710b80 1 --2- 172.21.15.107:0/865303914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e89de0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb2940095d0 tx=0x7fb2940074e0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.468+0000 7fb2af710b80 1 -- 172.21.15.107:0/865303914 shutdown_connections 2023-07-21T22:30:47.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.468+0000 7fb2af710b80 1 --2- 172.21.15.107:0/865303914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e831a0 0x560f97e89de0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.468+0000 7fb2af710b80 1 --2- 172.21.15.107:0/865303914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e81a10 0x560f97e81350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.468+0000 7fb2af710b80 1 --2- 172.21.15.107:0/865303914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e82830 0x560f97e92640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.468+0000 7fb2af710b80 1 -- 172.21.15.107:0/865303914 >> 172.21.15.107:0/865303914 conn(0x560f97e76ef0 msgr2=0x560f97e9d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.468+0000 7fb2af710b80 1 -- 172.21.15.107:0/865303914 shutdown_connections 2023-07-21T22:30:47.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.468+0000 7fb2af710b80 1 -- 172.21.15.107:0/865303914 wait complete. 2023-07-21T22:30:47.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.468+0000 7fb2af710b80 1 -- start start 2023-07-21T22:30:47.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.468+0000 7fb2af710b80 1 Processor -- start 2023-07-21T22:30:47.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.468+0000 7fb2af710b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e81a10 0x560f97e91dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2af710b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e82830 0x560f97e92310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2af710b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e831a0 0x560f97e8e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2af710b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97e8edb0 con 0x560f97e82830 2023-07-21T22:30:47.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2af710b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97e8ef20 con 0x560f97e831a0 2023-07-21T22:30:47.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2af710b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f97e8f090 con 0x560f97e81a10 2023-07-21T22:30:47.536 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2af710b80 10 client.0 _ll_register_callbacks cb 0x7fb298011a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:47.536 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2a0b26700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e82830 0x560f97e92310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.536 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2a1327700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e81a10 0x560f97e91dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.536 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2a0b26700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e82830 0x560f97e92310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49454/0 (socket says 172.21.15.107:49454) 2023-07-21T22:30:47.536 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2a0b26700 1 -- 172.21.15.107:0/2686394714 learned_addr learned my addr 172.21.15.107:0/2686394714 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.536 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2a1b28700 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e831a0 0x560f97e8e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.536 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2a0b26700 1 -- 172.21.15.107:0/2686394714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e81a10 msgr2=0x560f97e91dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.537 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2a0b26700 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e81a10 0x560f97e91dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.537 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2a0b26700 1 -- 172.21.15.107:0/2686394714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e831a0 msgr2=0x560f97e8e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.537 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2a0b26700 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e831a0 0x560f97e8e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.537 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2a0b26700 1 -- 172.21.15.107:0/2686394714 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb298017070 con 0x560f97e82830 2023-07-21T22:30:47.537 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2a1327700 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e81a10 0x560f97e91dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.537 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2a1b28700 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e831a0 0x560f97e8e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.537 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb284ff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:47.538 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2a0b26700 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e82830 0x560f97e92310 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x560f97e997b0 tx=0x7fb2980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb6dc3bd391c1df server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.538 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb284ff9700 1 -- 172.21.15.107:0/2686394714 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298022040 con 0x560f97e82830 2023-07-21T22:30:47.538 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb284ff9700 1 -- 172.21.15.107:0/2686394714 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb298031070 con 0x560f97e82830 2023-07-21T22:30:47.538 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.469+0000 7fb2af710b80 1 -- 172.21.15.107:0/2686394714 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb28c0a2330 con 0x560f97e82830 2023-07-21T22:30:47.538 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb284ff9700 1 -- 172.21.15.107:0/2686394714 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb29802bcb0 con 0x560f97e82830 2023-07-21T22:30:47.538 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb2af710b80 10 client.15999 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:47.538 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb2af710b80 1 -- 172.21.15.107:0/2686394714 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fb294009390 con 0x560f97e82830 2023-07-21T22:30:47.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb284ff9700 1 -- 172.21.15.107:0/2686394714 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28c0a2330 con 0x560f97e82830 2023-07-21T22:30:47.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb284ff9700 1 -- 172.21.15.107:0/2686394714 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb29803f050 con 0x560f97e82830 2023-07-21T22:30:47.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb2af710b80 10 client.15999 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:47.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb2af710b80 10 client.15999 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-07-21T22:30:47.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb2af710b80 20 client.15999 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:47.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb2af710b80 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x560f97eebe60 0x560f97eee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.539 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb2af710b80 4 client.15999 mds_command: new command op to 24466 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:30:47.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb2af710b80 1 -- 172.21.15.107:0/2686394714 --> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560f97d8e050 con 0x560f97eebe60 2023-07-21T22:30:47.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb2a1327700 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x560f97eebe60 0x560f97eee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb284ff9700 10 client.15999 ms_handle_connect on v2:172.21.15.190:6834/2286851849 2023-07-21T22:30:47.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.470+0000 7fb2a1327700 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x560f97eebe60 0x560f97eee270 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e46d53c6f97cc8f8 server_cookie=b537609d38754d1a in_seq=0 out_seq=0 2023-07-21T22:30:47.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb284ff9700 1 -- 172.21.15.107:0/2686394714 <== mds.2 v2:172.21.15.190:6834/2286851849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560f97d8e050 con 0x560f97eebe60 2023-07-21T22:30:47.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb284ff9700 10 client.15999 handle_command_reply: tid=0 2023-07-21T22:30:47.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb2af710b80 1 client.15999 shutdown 2023-07-21T22:30:47.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb2af710b80 1 -- 172.21.15.107:0/2686394714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e82830 msgr2=0x560f97e92310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb2af710b80 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e82830 0x560f97e92310 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x560f97e997b0 tx=0x7fb2980047c0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb2af710b80 1 -- 172.21.15.107:0/2686394714 shutdown_connections 2023-07-21T22:30:47.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb2af710b80 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x560f97eebe60 0x560f97eee270 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb2af710b80 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560f97e831a0 0x560f97e8e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb284ff9700 0 client.15999 ms_handle_reset on v2:172.21.15.190:6834/2286851849 2023-07-21T22:30:47.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb2af710b80 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560f97e81a10 0x560f97e91dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb2af710b80 1 --2- 172.21.15.107:0/2686394714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f97e82830 0x560f97e92310 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb2af710b80 1 -- 172.21.15.107:0/2686394714 >> 172.21.15.107:0/2686394714 conn(0x560f97e76ef0 msgr2=0x560f97e92f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb2af710b80 1 -- 172.21.15.107:0/2686394714 shutdown_connections 2023-07-21T22:30:47.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb2af710b80 1 -- 172.21.15.107:0/2686394714 wait complete. 2023-07-21T22:30:47.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.472+0000 7fb2af710b80 20 client.15999 trim_cache size 0 max 16384 2023-07-21T22:30:47.543 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:30:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.473+0000 7fb2a358b700 1 -- 172.21.15.107:0/145920494 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb2880a1150 msgr2=0x7fb2880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.473+0000 7fb2a358b700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb2880a1150 0x7fb2880a3600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0fec10 tx=0x7fb298009260 comp rx=0 tx=0).stop 2023-07-21T22:30:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.473+0000 7fb2a358b700 1 -- 172.21.15.107:0/145920494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb29c0feef0 msgr2=0x7fb29c0fe5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.473+0000 7fb2a358b700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb29c0feef0 0x7fb29c0fe5b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00e970 tx=0x7fb28c00ec80 comp rx=0 tx=0).stop 2023-07-21T22:30:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.473+0000 7fb2a358b700 1 -- 172.21.15.107:0/145920494 shutdown_connections 2023-07-21T22:30:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.473+0000 7fb2a358b700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb2880a1150 0x7fb2880a3600 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.473+0000 7fb2a358b700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb29c1005b0 0x7fb29c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.473+0000 7fb2a358b700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb29c0feef0 0x7fb29c0fe5b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.473+0000 7fb2a358b700 1 --2- 172.21.15.107:0/145920494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb29c0ffc00 0x7fb29c19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.473+0000 7fb2a358b700 1 -- 172.21.15.107:0/145920494 >> 172.21.15.107:0/145920494 conn(0x7fb29c0f9770 msgr2=0x7fb29c1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.473+0000 7fb2a358b700 1 -- 172.21.15.107:0/145920494 shutdown_connections 2023-07-21T22:30:47.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.473+0000 7fb2a358b700 1 -- 172.21.15.107:0/145920494 wait complete. 2023-07-21T22:30:47.546 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:30:47.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:47 smithi190 ceph-mon[98334]: pgmap v344: 97 pgs: 97 active+clean; 104 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 229 op/s 2023-07-21T22:30:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:47 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:47 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1505226923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:30:47.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.642+0000 7f7870530700 1 -- 172.21.15.107:0/3805022660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78680ffc40 msgr2=0x7f7868100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.642+0000 7f7870530700 1 --2- 172.21.15.107:0/3805022660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78680ffc40 0x7f7868100090 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7858009990 tx=0x7f7858009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.642+0000 7f7870530700 1 -- 172.21.15.107:0/3805022660 shutdown_connections 2023-07-21T22:30:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.642+0000 7f7870530700 1 --2- 172.21.15.107:0/3805022660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78681005d0 0x7f7868106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.642+0000 7f7870530700 1 --2- 172.21.15.107:0/3805022660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78680fea50 0x7f78680fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.642+0000 7f7870530700 1 --2- 172.21.15.107:0/3805022660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78680ffc40 0x7f7868100090 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.642+0000 7f7870530700 1 -- 172.21.15.107:0/3805022660 >> 172.21.15.107:0/3805022660 conn(0x7f78680fa070 msgr2=0x7f78680fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.642+0000 7f7870530700 1 -- 172.21.15.107:0/3805022660 shutdown_connections 2023-07-21T22:30:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.642+0000 7f7870530700 1 -- 172.21.15.107:0/3805022660 wait complete. 2023-07-21T22:30:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.642+0000 7f7870530700 1 Processor -- start 2023-07-21T22:30:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f7870530700 1 -- start start 2023-07-21T22:30:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f7870530700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78680fea50 0x7f78681957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f7870530700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78680ffc40 0x7f7868195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f7870530700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78681005d0 0x7f786819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f7870530700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786819d370 con 0x7f78680ffc40 2023-07-21T22:30:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f7870530700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786819d4b0 con 0x7f78681005d0 2023-07-21T22:30:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f7870530700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786819d640 con 0x7f78680fea50 2023-07-21T22:30:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f786dacb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78680ffc40 0x7f7868195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f786dacb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78680ffc40 0x7f7868195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49468/0 (socket says 172.21.15.107:49468) 2023-07-21T22:30:47.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f786dacb700 1 -- 172.21.15.107:0/4179606807 learned_addr learned my addr 172.21.15.107:0/4179606807 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f786e2cc700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78680fea50 0x7f78681957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f786eacd700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78681005d0 0x7f786819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f786dacb700 1 -- 172.21.15.107:0/4179606807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78680fea50 msgr2=0x7f78681957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f786dacb700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78680fea50 0x7f78681957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f786dacb700 1 -- 172.21.15.107:0/4179606807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78681005d0 msgr2=0x7f786819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f786dacb700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78681005d0 0x7f786819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f786dacb700 1 -- 172.21.15.107:0/4179606807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78580095f0 con 0x7f78680ffc40 2023-07-21T22:30:47.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f786dacb700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78680ffc40 0x7f7868195d30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f785800b620 tx=0x7f7858004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.643+0000 7f785f7fe700 1 -- 172.21.15.107:0/4179606807 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785801d070 con 0x7f78680ffc40 2023-07-21T22:30:47.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.644+0000 7f7870530700 1 -- 172.21.15.107:0/4179606807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786819d8c0 con 0x7f78680ffc40 2023-07-21T22:30:47.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.644+0000 7f785f7fe700 1 -- 172.21.15.107:0/4179606807 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7858005030 con 0x7f78680ffc40 2023-07-21T22:30:47.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.644+0000 7f785f7fe700 1 -- 172.21.15.107:0/4179606807 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858004b80 con 0x7f78680ffc40 2023-07-21T22:30:47.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.644+0000 7f7870530700 1 -- 172.21.15.107:0/4179606807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786819dce0 con 0x7f78680ffc40 2023-07-21T22:30:47.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.646+0000 7f785f7fe700 1 -- 172.21.15.107:0/4179606807 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f78580051a0 con 0x7f78680ffc40 2023-07-21T22:30:47.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.646+0000 7f785f7fe700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f78540a1150 0x7f78540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.647+0000 7f786e2cc700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f78540a1150 0x7f78540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.647+0000 7f785f7fe700 1 -- 172.21.15.107:0/4179606807 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f78580cd930 con 0x7f78680ffc40 2023-07-21T22:30:47.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.647+0000 7f786e2cc700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f78540a1150 0x7f78540a3600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f78680ffaa0 tx=0x7f7864009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.647+0000 7f787c6b5b80 1 Processor -- start 2023-07-21T22:30:47.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f787c6b5b80 1 -- start start 2023-07-21T22:30:47.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f787c6b5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c0830 0x56243a7c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f787c6b5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7c11a0 0x56243a7bf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f787c6b5b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7bfa10 0x56243a7bfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f787c6b5b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243a6c4cb0 con 0x56243a7bfa10 2023-07-21T22:30:47.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f787c6b5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243a717a40 con 0x56243a7c11a0 2023-07-21T22:30:47.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f787c6b5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243a710ed0 con 0x56243a7c0830 2023-07-21T22:30:47.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f786eacd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7bfa10 0x56243a7bfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f786e2cc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c0830 0x56243a7c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f786e2cc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c0830 0x56243a7c0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38032/0 (socket says 172.21.15.107:38032) 2023-07-21T22:30:47.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f786dacb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7c11a0 0x56243a7bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f786e2cc700 1 -- 172.21.15.107:0/1130106027 learned_addr learned my addr 172.21.15.107:0/1130106027 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f786eacd700 1 -- 172.21.15.107:0/1130106027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c0830 msgr2=0x56243a7c0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f786eacd700 1 --2- 172.21.15.107:0/1130106027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c0830 0x56243a7c0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f786eacd700 1 -- 172.21.15.107:0/1130106027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7c11a0 msgr2=0x56243a7bf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f786eacd700 1 --2- 172.21.15.107:0/1130106027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7c11a0 0x56243a7bf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f786eacd700 1 -- 172.21.15.107:0/1130106027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56243a53ede0 con 0x56243a7bfa10 2023-07-21T22:30:47.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f786e2cc700 1 --2- 172.21.15.107:0/1130106027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c0830 0x56243a7c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f786dacb700 1 --2- 172.21.15.107:0/1130106027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7c11a0 0x56243a7bf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.648+0000 7f786eacd700 1 --2- 172.21.15.107:0/1130106027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7bfa10 0x56243a7bfe90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7860011580 tx=0x7f7860009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3ca35c6721f904 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.649+0000 7f785cff9700 1 -- 172.21.15.107:0/1130106027 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786000e040 con 0x56243a7bfa10 2023-07-21T22:30:47.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.649+0000 7f785cff9700 1 -- 172.21.15.107:0/1130106027 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f786000f040 con 0x56243a7bfa10 2023-07-21T22:30:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.649+0000 7f785cff9700 1 -- 172.21.15.107:0/1130106027 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78600043f0 con 0x56243a7bfa10 2023-07-21T22:30:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.649+0000 7f787c6b5b80 1 -- 172.21.15.107:0/1130106027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7bfa10 msgr2=0x56243a7bfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.649+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/1130106027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7bfa10 0x56243a7bfe90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7860011580 tx=0x7f7860009920 comp rx=0 tx=0).stop 2023-07-21T22:30:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.649+0000 7f787c6b5b80 1 -- 172.21.15.107:0/1130106027 shutdown_connections 2023-07-21T22:30:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.649+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/1130106027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7bfa10 0x56243a7bfe90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.649+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/1130106027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c0830 0x56243a7c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.649+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/1130106027 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7c11a0 0x56243a7bf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.649+0000 7f787c6b5b80 1 -- 172.21.15.107:0/1130106027 >> 172.21.15.107:0/1130106027 conn(0x56243a7b4ef0 msgr2=0x56243a7ba360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.649+0000 7f787c6b5b80 1 -- 172.21.15.107:0/1130106027 shutdown_connections 2023-07-21T22:30:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f787c6b5b80 1 -- 172.21.15.107:0/1130106027 wait complete. 2023-07-21T22:30:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f787c6b5b80 1 -- start start 2023-07-21T22:30:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f787c6b5b80 1 Processor -- start 2023-07-21T22:30:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f787c6b5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7bfa10 0x56243a7c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f787c6b5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7c0830 0x56243a7c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f787c6b5b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c11a0 0x56243a7d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f787c6b5b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243a6b0ac0 con 0x56243a7c11a0 2023-07-21T22:30:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f787c6b5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243a6b07c0 con 0x56243a7c0830 2023-07-21T22:30:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f787c6b5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243a594920 con 0x56243a7bfa10 2023-07-21T22:30:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f787c6b5b80 10 client.0 _ll_register_callbacks cb 0x56243a6ae760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f786eacd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c11a0 0x56243a7d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f786e2cc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7bfa10 0x56243a7c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f786dacb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7c0830 0x56243a7c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f786eacd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c11a0 0x56243a7d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49484/0 (socket says 172.21.15.107:49484) 2023-07-21T22:30:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.650+0000 7f786eacd700 1 -- 172.21.15.107:0/2388542955 learned_addr learned my addr 172.21.15.107:0/2388542955 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f786e2cc700 1 -- 172.21.15.107:0/2388542955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7c0830 msgr2=0x56243a7c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f786e2cc700 1 --2- 172.21.15.107:0/2388542955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7c0830 0x56243a7c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f786e2cc700 1 -- 172.21.15.107:0/2388542955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c11a0 msgr2=0x56243a7d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f786e2cc700 1 --2- 172.21.15.107:0/2388542955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c11a0 0x56243a7d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f786e2cc700 1 -- 172.21.15.107:0/2388542955 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7864017070 con 0x56243a7bfa10 2023-07-21T22:30:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f786dacb700 1 --2- 172.21.15.107:0/2388542955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7c0830 0x56243a7c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f786eacd700 1 --2- 172.21.15.107:0/2388542955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c11a0 0x56243a7d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f786e2cc700 1 --2- 172.21.15.107:0/2388542955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7bfa10 0x56243a7c9080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x56243a7bf3c0 tx=0x7f78640047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=875ba0ab03b70dd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f7851ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:30:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f7851ffb700 1 -- 172.21.15.107:0/2388542955 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864022070 con 0x56243a7bfa10 2023-07-21T22:30:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f7851ffb700 1 -- 172.21.15.107:0/2388542955 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f786401c6e0 con 0x56243a7bfa10 2023-07-21T22:30:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2388542955 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56243a66a4f0 con 0x56243a7bfa10 2023-07-21T22:30:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f7851ffb700 1 -- 172.21.15.107:0/2388542955 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7864004020 con 0x56243a7bfa10 2023-07-21T22:30:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f787c6b5b80 10 client.25537 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.651+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2388542955 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f786819dce0 con 0x56243a7bfa10 2023-07-21T22:30:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.652+0000 7f7851ffb700 1 -- 172.21.15.107:0/2388542955 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56243a66a4f0 con 0x56243a7bfa10 2023-07-21T22:30:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.652+0000 7f7851ffb700 1 -- 172.21.15.107:0/2388542955 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f78640036f0 con 0x56243a7bfa10 2023-07-21T22:30:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.652+0000 7f787c6b5b80 10 client.25537 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.652+0000 7f787c6b5b80 10 client.25537 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-21T22:30:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.652+0000 7f787c6b5b80 20 client.25537 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.652+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/2388542955 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x56243a7d4f30 0x56243a7d7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.652+0000 7f787c6b5b80 4 client.25537 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.652+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2388542955 --> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56243a709eb0 con 0x56243a7d4f30 2023-07-21T22:30:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.652+0000 7f786dacb700 1 --2- 172.21.15.107:0/2388542955 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x56243a7d4f30 0x56243a7d7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.652+0000 7f786dacb700 1 --2- 172.21.15.107:0/2388542955 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x56243a7d4f30 0x56243a7d7340 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=730ba5e4b743e173 server_cookie=d06757099f6c6b81 in_seq=0 out_seq=0 2023-07-21T22:30:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.652+0000 7f7851ffb700 10 client.25537 ms_handle_connect on v2:172.21.15.190:6838/2960500982 2023-07-21T22:30:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f7851ffb700 1 -- 172.21.15.107:0/2388542955 <== mds.1 v2:172.21.15.190:6838/2960500982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56243a709eb0 con 0x56243a7d4f30 2023-07-21T22:30:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f7851ffb700 10 client.25537 handle_command_reply: tid=0 2023-07-21T22:30:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f787c6b5b80 1 client.25537 shutdown 2023-07-21T22:30:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2388542955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7bfa10 msgr2=0x56243a7c9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/2388542955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7bfa10 0x56243a7c9080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x56243a7bf3c0 tx=0x7f78640047c0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2388542955 shutdown_connections 2023-07-21T22:30:47.715 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/2388542955 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x56243a7d4f30 0x56243a7d7340 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.715 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/2388542955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c11a0 0x56243a7d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.715 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f7851ffb700 0 client.25537 ms_handle_reset on v2:172.21.15.190:6838/2960500982 2023-07-21T22:30:47.715 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/2388542955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7bfa10 0x56243a7c9080 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.715 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/2388542955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7c0830 0x56243a7c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.715 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2388542955 >> 172.21.15.107:0/2388542955 conn(0x56243a7b4ef0 msgr2=0x56243a7b9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2388542955 shutdown_connections 2023-07-21T22:30:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2388542955 wait complete. 2023-07-21T22:30:47.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.653+0000 7f787c6b5b80 20 client.25537 trim_cache size 0 max 16384 2023-07-21T22:30:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f787c6b5b80 1 Processor -- start 2023-07-21T22:30:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f787c6b5b80 1 -- start start 2023-07-21T22:30:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f787c6b5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7bfa10 0x56243a7dbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f787c6b5b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c0830 0x56243a7d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f787c6b5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c11a0 0x56243a7c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f787c6b5b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243a7dc200 con 0x56243a7c0830 2023-07-21T22:30:47.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f787c6b5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243a7d4c30 con 0x56243a7bfa10 2023-07-21T22:30:47.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f787c6b5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243a7c7b10 con 0x56243a7c11a0 2023-07-21T22:30:47.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f786dacb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c0830 0x56243a7d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f786e2cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7bfa10 0x56243a7dbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f786dacb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c0830 0x56243a7d0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49490/0 (socket says 172.21.15.107:49490) 2023-07-21T22:30:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f786dacb700 1 -- 172.21.15.107:0/1766596905 learned_addr learned my addr 172.21.15.107:0/1766596905 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f786eacd700 1 --2- 172.21.15.107:0/1766596905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c11a0 0x56243a7c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f786e2cc700 1 -- 172.21.15.107:0/1766596905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c11a0 msgr2=0x56243a7c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f786e2cc700 1 --2- 172.21.15.107:0/1766596905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c11a0 0x56243a7c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f786e2cc700 1 -- 172.21.15.107:0/1766596905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c0830 msgr2=0x56243a7d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f786e2cc700 1 --2- 172.21.15.107:0/1766596905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c0830 0x56243a7d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f786e2cc700 1 -- 172.21.15.107:0/1766596905 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7860011230 con 0x56243a7bfa10 2023-07-21T22:30:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.669+0000 7f786dacb700 1 --2- 172.21.15.107:0/1766596905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c0830 0x56243a7d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.670+0000 7f786e2cc700 1 --2- 172.21.15.107:0/1766596905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7bfa10 0x56243a7dbcc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7864011850 tx=0x7f78640093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e82eb43703c3f05 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.670+0000 7f785d7fa700 1 -- 172.21.15.107:0/1766596905 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864036070 con 0x56243a7bfa10 2023-07-21T22:30:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.670+0000 7f785d7fa700 1 -- 172.21.15.107:0/1766596905 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7864003680 con 0x56243a7bfa10 2023-07-21T22:30:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.670+0000 7f785d7fa700 1 -- 172.21.15.107:0/1766596905 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864022040 con 0x56243a7bfa10 2023-07-21T22:30:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.671+0000 7f787c6b5b80 1 -- 172.21.15.107:0/1766596905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7bfa10 msgr2=0x56243a7dbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.671+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/1766596905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7bfa10 0x56243a7dbcc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7864011850 tx=0x7f78640093d0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.671+0000 7f787c6b5b80 1 -- 172.21.15.107:0/1766596905 shutdown_connections 2023-07-21T22:30:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.671+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/1766596905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c11a0 0x56243a7c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.671+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/1766596905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7bfa10 0x56243a7dbcc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.671+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/1766596905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c0830 0x56243a7d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.671+0000 7f787c6b5b80 1 -- 172.21.15.107:0/1766596905 >> 172.21.15.107:0/1766596905 conn(0x56243a7b4ef0 msgr2=0x56243a7c7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.671+0000 7f787c6b5b80 1 -- 172.21.15.107:0/1766596905 shutdown_connections 2023-07-21T22:30:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.671+0000 7f787c6b5b80 1 -- 172.21.15.107:0/1766596905 wait complete. 2023-07-21T22:30:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.671+0000 7f787c6b5b80 1 -- start start 2023-07-21T22:30:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f787c6b5b80 1 Processor -- start 2023-07-21T22:30:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f787c6b5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7bfa10 0x56243a7cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f787c6b5b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c0830 0x56243a7d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f787c6b5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c11a0 0x56243a7cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f787c6b5b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243a7cd020 con 0x56243a7c0830 2023-07-21T22:30:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f787c6b5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243a7cd190 con 0x56243a7bfa10 2023-07-21T22:30:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f787c6b5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56243a7cd350 con 0x56243a7c11a0 2023-07-21T22:30:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f787c6b5b80 10 client.0 _ll_register_callbacks cb 0x56243a6ae760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f786dacb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c0830 0x56243a7d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f786e2cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7bfa10 0x56243a7cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f786e2cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7bfa10 0x56243a7cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40322/0 (socket says 172.21.15.107:40322) 2023-07-21T22:30:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f786e2cc700 1 -- 172.21.15.107:0/2747952216 learned_addr learned my addr 172.21.15.107:0/2747952216 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f786eacd700 1 --2- 172.21.15.107:0/2747952216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c11a0 0x56243a7cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f786dacb700 1 -- 172.21.15.107:0/2747952216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c11a0 msgr2=0x56243a7cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f786dacb700 1 --2- 172.21.15.107:0/2747952216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c11a0 0x56243a7cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f786dacb700 1 -- 172.21.15.107:0/2747952216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7bfa10 msgr2=0x56243a7cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f786dacb700 1 --2- 172.21.15.107:0/2747952216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7bfa10 0x56243a7cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f786dacb700 1 -- 172.21.15.107:0/2747952216 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786819d8c0 con 0x56243a7c0830 2023-07-21T22:30:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.672+0000 7f786e2cc700 1 --2- 172.21.15.107:0/2747952216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7bfa10 0x56243a7cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f786dacb700 1 --2- 172.21.15.107:0/2747952216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c0830 0x56243a7d03e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f78580cb410 tx=0x7f785801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1bbf5b30f4984b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f7851ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:47.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f7851ffb700 1 -- 172.21.15.107:0/2747952216 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78580b3040 con 0x56243a7c0830 2023-07-21T22:30:47.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f7851ffb700 1 -- 172.21.15.107:0/2747952216 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f78580a3170 con 0x56243a7c0830 2023-07-21T22:30:47.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2747952216 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f786000f040 con 0x56243a7c0830 2023-07-21T22:30:47.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f7851ffb700 1 -- 172.21.15.107:0/2747952216 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f785809a2a0 con 0x56243a7c0830 2023-07-21T22:30:47.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f787c6b5b80 10 client.16017 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:47.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2747952216 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f786819dce0 con 0x56243a7c0830 2023-07-21T22:30:47.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f7851ffb700 1 -- 172.21.15.107:0/2747952216 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f786000f040 con 0x56243a7c0830 2023-07-21T22:30:47.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f7851ffb700 1 -- 172.21.15.107:0/2747952216 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f78580b31a0 con 0x56243a7c0830 2023-07-21T22:30:47.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f787c6b5b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:47.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f787c6b5b80 10 client.16017 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-21T22:30:47.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f787c6b5b80 20 client.16017 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:47.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/2747952216 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x56243a82a0a0 0x56243a82c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f787c6b5b80 4 client.16017 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:30:47.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.673+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2747952216 --> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56243a6cc050 con 0x56243a82a0a0 2023-07-21T22:30:47.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.674+0000 7f786e2cc700 1 --2- 172.21.15.107:0/2747952216 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x56243a82a0a0 0x56243a82c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.674+0000 7f786e2cc700 1 --2- 172.21.15.107:0/2747952216 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x56243a82a0a0 0x56243a82c4b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e6a439cab51433c server_cookie=9eec97bc820bc08c in_seq=0 out_seq=0 2023-07-21T22:30:47.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.674+0000 7f7851ffb700 10 client.16017 ms_handle_connect on v2:172.21.15.190:6838/2960500982 2023-07-21T22:30:47.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.674+0000 7f7851ffb700 1 -- 172.21.15.107:0/2747952216 <== mds.1 v2:172.21.15.190:6838/2960500982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56243a6cc050 con 0x56243a82a0a0 2023-07-21T22:30:47.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.674+0000 7f7851ffb700 10 client.16017 handle_command_reply: tid=0 2023-07-21T22:30:47.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.674+0000 7f787c6b5b80 1 client.16017 shutdown 2023-07-21T22:30:47.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.674+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2747952216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c0830 msgr2=0x56243a7d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.674+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/2747952216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c0830 0x56243a7d03e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f78580cb410 tx=0x7f785801a040 comp rx=0 tx=0).stop 2023-07-21T22:30:47.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2747952216 shutdown_connections 2023-07-21T22:30:47.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/2747952216 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x56243a82a0a0 0x56243a82c4b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/2747952216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56243a7c11a0 0x56243a7cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f7851ffb700 0 client.16017 ms_handle_reset on v2:172.21.15.190:6838/2960500982 2023-07-21T22:30:47.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/2747952216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56243a7bfa10 0x56243a7cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.733 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f787c6b5b80 1 --2- 172.21.15.107:0/2747952216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56243a7c0830 0x56243a7d03e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.733 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2747952216 >> 172.21.15.107:0/2747952216 conn(0x56243a7b4ef0 msgr2=0x56243a7c6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.733 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2747952216 shutdown_connections 2023-07-21T22:30:47.733 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f787c6b5b80 1 -- 172.21.15.107:0/2747952216 wait complete. 2023-07-21T22:30:47.733 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f787c6b5b80 20 client.16017 trim_cache size 0 max 16384 2023-07-21T22:30:47.733 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:30:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f7870530700 1 -- 172.21.15.107:0/4179606807 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f78540a1150 msgr2=0x7f78540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f7870530700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f78540a1150 0x7f78540a3600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f78680ffaa0 tx=0x7f7864009260 comp rx=0 tx=0).stop 2023-07-21T22:30:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f7870530700 1 -- 172.21.15.107:0/4179606807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78680ffc40 msgr2=0x7f7868195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f7870530700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78680ffc40 0x7f7868195d30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f785800b620 tx=0x7f7858004730 comp rx=0 tx=0).stop 2023-07-21T22:30:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f7870530700 1 -- 172.21.15.107:0/4179606807 shutdown_connections 2023-07-21T22:30:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f7870530700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78681005d0 0x7f786819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f7870530700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f78540a1150 0x7f78540a3600 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f7870530700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78680fea50 0x7f78681957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f7870530700 1 --2- 172.21.15.107:0/4179606807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78680ffc40 0x7f7868195d30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f7870530700 1 -- 172.21.15.107:0/4179606807 >> 172.21.15.107:0/4179606807 conn(0x7f78680fa070 msgr2=0x7f78680688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f7870530700 1 -- 172.21.15.107:0/4179606807 shutdown_connections 2023-07-21T22:30:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.675+0000 7f7870530700 1 -- 172.21.15.107:0/4179606807 wait complete. 2023-07-21T22:30:47.737 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:30:47.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.822+0000 7fa160718700 1 -- 172.21.15.107:0/1909083328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1580ffae0 msgr2=0x7fa1580ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.822+0000 7fa160718700 1 --2- 172.21.15.107:0/1909083328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1580ffae0 0x7fa1580ffef0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa154009990 tx=0x7fa154009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.822+0000 7fa160718700 1 -- 172.21.15.107:0/1909083328 shutdown_connections 2023-07-21T22:30:47.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.822+0000 7fa160718700 1 --2- 172.21.15.107:0/1909083328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa158101660 0x7fa158107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.822+0000 7fa160718700 1 --2- 172.21.15.107:0/1909083328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1580ffae0 0x7fa1580ffef0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.822+0000 7fa160718700 1 --2- 172.21.15.107:0/1909083328 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa158100cd0 0x7fa158101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.822+0000 7fa160718700 1 -- 172.21.15.107:0/1909083328 >> 172.21.15.107:0/1909083328 conn(0x7fa1580fb120 msgr2=0x7fa1580fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.822+0000 7fa160718700 1 -- 172.21.15.107:0/1909083328 shutdown_connections 2023-07-21T22:30:47.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.822+0000 7fa160718700 1 -- 172.21.15.107:0/1909083328 wait complete. 2023-07-21T22:30:47.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa160718700 1 Processor -- start 2023-07-21T22:30:47.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa160718700 1 -- start start 2023-07-21T22:30:47.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa160718700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1580ffae0 0x7fa158195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa160718700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa158100cd0 0x7fa158195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa160718700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa158101660 0x7fa15819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa160718700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15819d380 con 0x7fa1580ffae0 2023-07-21T22:30:47.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa160718700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15819d4c0 con 0x7fa158100cd0 2023-07-21T22:30:47.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa160718700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15819d650 con 0x7fa158101660 2023-07-21T22:30:47.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa15e4b4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1580ffae0 0x7fa158195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa15e4b4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1580ffae0 0x7fa158195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49508/0 (socket says 172.21.15.107:49508) 2023-07-21T22:30:47.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa15e4b4700 1 -- 172.21.15.107:0/1801626589 learned_addr learned my addr 172.21.15.107:0/1801626589 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa15ecb5700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa158101660 0x7fa15819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa15dcb3700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa158100cd0 0x7fa158195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa15e4b4700 1 -- 172.21.15.107:0/1801626589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa158101660 msgr2=0x7fa15819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa15e4b4700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa158101660 0x7fa15819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa15e4b4700 1 -- 172.21.15.107:0/1801626589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa158100cd0 msgr2=0x7fa158195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa15e4b4700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa158100cd0 0x7fa158195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa15e4b4700 1 -- 172.21.15.107:0/1801626589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1540095f0 con 0x7fa1580ffae0 2023-07-21T22:30:47.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa15ecb5700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa158101660 0x7fa15819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.823+0000 7fa15e4b4700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1580ffae0 0x7fa158195800 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa15400b620 tx=0x7fa154004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.874 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.824+0000 7fa14f7fe700 1 -- 172.21.15.107:0/1801626589 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15401d070 con 0x7fa1580ffae0 2023-07-21T22:30:47.875 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.824+0000 7fa14f7fe700 1 -- 172.21.15.107:0/1801626589 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa154005030 con 0x7fa1580ffae0 2023-07-21T22:30:47.875 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.824+0000 7fa160718700 1 -- 172.21.15.107:0/1801626589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15819d8d0 con 0x7fa1580ffae0 2023-07-21T22:30:47.875 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.824+0000 7fa14f7fe700 1 -- 172.21.15.107:0/1801626589 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154004b80 con 0x7fa1580ffae0 2023-07-21T22:30:47.875 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.824+0000 7fa160718700 1 -- 172.21.15.107:0/1801626589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15819dcf0 con 0x7fa1580ffae0 2023-07-21T22:30:47.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.826+0000 7fa14f7fe700 1 -- 172.21.15.107:0/1801626589 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa1540051a0 con 0x7fa1580ffae0 2023-07-21T22:30:47.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.827+0000 7fa14f7fe700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa1440a1150 0x7fa1440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.827+0000 7fa15dcb3700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa1440a1150 0x7fa1440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.827+0000 7fa14f7fe700 1 -- 172.21.15.107:0/1801626589 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa1540cd910 con 0x7fa1580ffae0 2023-07-21T22:30:47.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.827+0000 7fa15dcb3700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa1440a1150 0x7fa1440a3600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa148009c60 tx=0x7fa148009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.827+0000 7fa16c89db80 1 Processor -- start 2023-07-21T22:30:47.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa16c89db80 1 -- start start 2023-07-21T22:30:47.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa16c89db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d0a80 0x558f9e8d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa16c89db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d13d0 0x558f9e8d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa16c89db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 0x558f9e8d2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa16c89db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9e7d1cb0 con 0x558f9e8d1ff0 2023-07-21T22:30:47.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa16c89db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9e824a40 con 0x558f9e8d0a80 2023-07-21T22:30:47.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa16c89db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9e81ded0 con 0x558f9e8d13d0 2023-07-21T22:30:47.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa15ecb5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 0x558f9e8d2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa15dcb3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d13d0 0x558f9e8d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa15dcb3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d13d0 0x558f9e8d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38074/0 (socket says 172.21.15.107:38074) 2023-07-21T22:30:47.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa15ecb5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 0x558f9e8d2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49520/0 (socket says 172.21.15.107:49520) 2023-07-21T22:30:47.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa15dcb3700 1 -- 172.21.15.107:0/3093536132 learned_addr learned my addr 172.21.15.107:0/3093536132 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa15e4b4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d0a80 0x558f9e8d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa15dcb3700 1 -- 172.21.15.107:0/3093536132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d0a80 msgr2=0x558f9e8d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa15dcb3700 1 --2- 172.21.15.107:0/3093536132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d0a80 0x558f9e8d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa15dcb3700 1 -- 172.21.15.107:0/3093536132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 msgr2=0x558f9e8d2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa15dcb3700 1 --2- 172.21.15.107:0/3093536132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 0x558f9e8d2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa15dcb3700 1 -- 172.21.15.107:0/3093536132 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f9e64bde0 con 0x558f9e8d13d0 2023-07-21T22:30:47.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa15ecb5700 1 --2- 172.21.15.107:0/3093536132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 0x558f9e8d2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa15dcb3700 1 --2- 172.21.15.107:0/3093536132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d13d0 0x558f9e8d1a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa14800e4e0 tx=0x7fa148014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc13c18499c01f9f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.828+0000 7fa14cff9700 1 -- 172.21.15.107:0/3093536132 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14800ee00 con 0x558f9e8d13d0 2023-07-21T22:30:47.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.829+0000 7fa14cff9700 1 -- 172.21.15.107:0/3093536132 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa148021070 con 0x558f9e8d13d0 2023-07-21T22:30:47.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.829+0000 7fa14cff9700 1 -- 172.21.15.107:0/3093536132 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14801d7f0 con 0x558f9e8d13d0 2023-07-21T22:30:47.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.829+0000 7fa16c89db80 1 -- 172.21.15.107:0/3093536132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d13d0 msgr2=0x558f9e8d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.829+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3093536132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d13d0 0x558f9e8d1a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa14800e4e0 tx=0x7fa148014000 comp rx=0 tx=0).stop 2023-07-21T22:30:47.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.829+0000 7fa16c89db80 1 -- 172.21.15.107:0/3093536132 shutdown_connections 2023-07-21T22:30:47.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.829+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3093536132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 0x558f9e8d2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.829+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3093536132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d0a80 0x558f9e8d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.829+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3093536132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d13d0 0x558f9e8d1a10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.829+0000 7fa16c89db80 1 -- 172.21.15.107:0/3093536132 >> 172.21.15.107:0/3093536132 conn(0x558f9e8c1ef0 msgr2=0x558f9e8c7220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.829+0000 7fa16c89db80 1 -- 172.21.15.107:0/3093536132 shutdown_connections 2023-07-21T22:30:47.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.829+0000 7fa16c89db80 1 -- 172.21.15.107:0/3093536132 wait complete. 2023-07-21T22:30:47.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa16c89db80 1 -- start start 2023-07-21T22:30:47.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa16c89db80 1 Processor -- start 2023-07-21T22:30:47.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa16c89db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d0a80 0x558f9e8da480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa16c89db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 0x558f9e8da9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa16c89db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d1ff0 0x558f9e8e1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa16c89db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9e7bdac0 con 0x558f9e8d13d0 2023-07-21T22:30:47.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa16c89db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9e7bd7c0 con 0x558f9e8d1ff0 2023-07-21T22:30:47.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa16c89db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9e6a1920 con 0x558f9e8d0a80 2023-07-21T22:30:47.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa16c89db80 10 client.0 _ll_register_callbacks cb 0x558f9e7bb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:47.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa15dcb3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 0x558f9e8da9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa15dcb3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 0x558f9e8da9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49524/0 (socket says 172.21.15.107:49524) 2023-07-21T22:30:47.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa15dcb3700 1 -- 172.21.15.107:0/3167583213 learned_addr learned my addr 172.21.15.107:0/3167583213 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa15e4b4700 1 --2- 172.21.15.107:0/3167583213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d0a80 0x558f9e8da480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa15ecb5700 1 --2- 172.21.15.107:0/3167583213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d1ff0 0x558f9e8e1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa15dcb3700 1 -- 172.21.15.107:0/3167583213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d0a80 msgr2=0x558f9e8da480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa15dcb3700 1 --2- 172.21.15.107:0/3167583213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d0a80 0x558f9e8da480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa15dcb3700 1 -- 172.21.15.107:0/3167583213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d1ff0 msgr2=0x558f9e8e1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa15dcb3700 1 --2- 172.21.15.107:0/3167583213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d1ff0 0x558f9e8e1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.830+0000 7fa15dcb3700 1 -- 172.21.15.107:0/3167583213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa150003a60 con 0x558f9e8d13d0 2023-07-21T22:30:47.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa15dcb3700 1 --2- 172.21.15.107:0/3167583213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 0x558f9e8da9c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa14800e4b0 tx=0x7fa14800edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f1cd36d1eb93dc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa141ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:47.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa141ffb700 1 -- 172.21.15.107:0/3167583213 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148005350 con 0x558f9e8d13d0 2023-07-21T22:30:47.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa141ffb700 1 -- 172.21.15.107:0/3167583213 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa1480054b0 con 0x558f9e8d13d0 2023-07-21T22:30:47.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa16c89db80 1 -- 172.21.15.107:0/3167583213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f9e7774f0 con 0x558f9e8d13d0 2023-07-21T22:30:47.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa141ffb700 1 -- 172.21.15.107:0/3167583213 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa148005670 con 0x558f9e8d13d0 2023-07-21T22:30:47.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa16c89db80 10 client.16029 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:47.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa16c89db80 1 -- 172.21.15.107:0/3167583213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fa148017070 con 0x558f9e8d13d0 2023-07-21T22:30:47.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa141ffb700 1 -- 172.21.15.107:0/3167583213 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa14803d080 con 0x558f9e8d13d0 2023-07-21T22:30:47.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa141ffb700 1 -- 172.21.15.107:0/3167583213 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa148037030 con 0x558f9e8d13d0 2023-07-21T22:30:47.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa16c89db80 10 client.16029 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:47.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa16c89db80 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:47.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa16c89db80 20 client.16029 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:47.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3167583213 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f9e8e6250 0x558f9e8e8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa16c89db80 4 client.16029 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:47.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.831+0000 7fa16c89db80 1 -- 172.21.15.107:0/3167583213 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f9e816eb0 con 0x558f9e8e6250 2023-07-21T22:30:47.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.832+0000 7fa15e4b4700 1 --2- 172.21.15.107:0/3167583213 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f9e8e6250 0x558f9e8e8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.832+0000 7fa141ffb700 10 client.16029 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:47.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.832+0000 7fa15e4b4700 1 --2- 172.21.15.107:0/3167583213 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f9e8e6250 0x558f9e8e8660 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4a247808182d73e server_cookie=a48836794343e9e in_seq=0 out_seq=0 2023-07-21T22:30:47.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.833+0000 7fa141ffb700 1 -- 172.21.15.107:0/3167583213 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558f9e816eb0 con 0x558f9e8e6250 2023-07-21T22:30:47.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.833+0000 7fa141ffb700 10 client.16029 handle_command_reply: tid=0 2023-07-21T22:30:47.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.833+0000 7fa16c89db80 1 client.16029 shutdown 2023-07-21T22:30:47.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.834+0000 7fa16c89db80 1 -- 172.21.15.107:0/3167583213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 msgr2=0x558f9e8da9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.834+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3167583213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 0x558f9e8da9c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa14800e4b0 tx=0x7fa14800edb0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.834+0000 7fa16c89db80 1 -- 172.21.15.107:0/3167583213 shutdown_connections 2023-07-21T22:30:47.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.834+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3167583213 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f9e8e6250 0x558f9e8e8660 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.834+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3167583213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d1ff0 0x558f9e8e1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.834+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3167583213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d0a80 0x558f9e8da480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.834+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3167583213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 0x558f9e8da9c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.834+0000 7fa16c89db80 1 -- 172.21.15.107:0/3167583213 >> 172.21.15.107:0/3167583213 conn(0x558f9e8c1ef0 msgr2=0x558f9e8cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.834+0000 7fa141ffb700 0 client.16029 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:47.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.834+0000 7fa16c89db80 1 -- 172.21.15.107:0/3167583213 shutdown_connections 2023-07-21T22:30:47.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.834+0000 7fa16c89db80 1 -- 172.21.15.107:0/3167583213 wait complete. 2023-07-21T22:30:47.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.834+0000 7fa16c89db80 20 client.16029 trim_cache size 0 max 16384 2023-07-21T22:30:47.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.847+0000 7fa16c89db80 1 Processor -- start 2023-07-21T22:30:47.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.847+0000 7fa16c89db80 1 -- start start 2023-07-21T22:30:47.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.847+0000 7fa16c89db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d0a80 0x558f9e8d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.847+0000 7fa16c89db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 0x558f9e8e19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.847+0000 7fa16c89db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d1ff0 0x558f9e8d51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.847+0000 7fa16c89db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9e8c7fd0 con 0x558f9e8d13d0 2023-07-21T22:30:47.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.847+0000 7fa16c89db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9e8c8be0 con 0x558f9e8d0a80 2023-07-21T22:30:47.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.847+0000 7fa16c89db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9e8d17e0 con 0x558f9e8d1ff0 2023-07-21T22:30:47.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.847+0000 7fa15dcb3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 0x558f9e8e19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.847+0000 7fa15dcb3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 0x558f9e8e19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49532/0 (socket says 172.21.15.107:49532) 2023-07-21T22:30:47.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.847+0000 7fa15dcb3700 1 -- 172.21.15.107:0/3612905747 learned_addr learned my addr 172.21.15.107:0/3612905747 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.847+0000 7fa15ecb5700 1 --2- 172.21.15.107:0/3612905747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d1ff0 0x558f9e8d51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.847+0000 7fa15e4b4700 1 --2- 172.21.15.107:0/3612905747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d0a80 0x558f9e8d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.848+0000 7fa15ecb5700 1 -- 172.21.15.107:0/3612905747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d0a80 msgr2=0x558f9e8d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.848+0000 7fa15ecb5700 1 --2- 172.21.15.107:0/3612905747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d0a80 0x558f9e8d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.848+0000 7fa15ecb5700 1 -- 172.21.15.107:0/3612905747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 msgr2=0x558f9e8e19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.848+0000 7fa15ecb5700 1 --2- 172.21.15.107:0/3612905747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 0x558f9e8e19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.848+0000 7fa15ecb5700 1 -- 172.21.15.107:0/3612905747 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f9e634c60 con 0x558f9e8d1ff0 2023-07-21T22:30:47.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.848+0000 7fa15dcb3700 1 --2- 172.21.15.107:0/3612905747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 0x558f9e8e19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:47.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.848+0000 7fa15ecb5700 1 --2- 172.21.15.107:0/3612905747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d1ff0 0x558f9e8d51b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa15000e8c0 tx=0x7fa150009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0a4a506500dd05b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.848+0000 7fa14d7fa700 1 -- 172.21.15.107:0/3612905747 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150004db0 con 0x558f9e8d1ff0 2023-07-21T22:30:47.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.848+0000 7fa14d7fa700 1 -- 172.21.15.107:0/3612905747 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa15000f040 con 0x558f9e8d1ff0 2023-07-21T22:30:47.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.848+0000 7fa14d7fa700 1 -- 172.21.15.107:0/3612905747 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150007880 con 0x558f9e8d1ff0 2023-07-21T22:30:47.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 -- 172.21.15.107:0/3612905747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d1ff0 msgr2=0x558f9e8d51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3612905747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d1ff0 0x558f9e8d51b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa15000e8c0 tx=0x7fa150009920 comp rx=0 tx=0).stop 2023-07-21T22:30:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 -- 172.21.15.107:0/3612905747 shutdown_connections 2023-07-21T22:30:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3612905747 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d1ff0 0x558f9e8d51b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3612905747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d0a80 0x558f9e8d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3612905747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d13d0 0x558f9e8e19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 -- 172.21.15.107:0/3612905747 >> 172.21.15.107:0/3612905747 conn(0x558f9e8c1ef0 msgr2=0x558f9e8ecf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 -- 172.21.15.107:0/3612905747 shutdown_connections 2023-07-21T22:30:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 -- 172.21.15.107:0/3612905747 wait complete. 2023-07-21T22:30:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 -- start start 2023-07-21T22:30:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 Processor -- start 2023-07-21T22:30:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d0a80 0x558f9e8df980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d13d0 0x558f9e8dfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 0x558f9e8e0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9e8e1410 con 0x558f9e8d1ff0 2023-07-21T22:30:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9e8e1580 con 0x558f9e8d13d0 2023-07-21T22:30:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.849+0000 7fa16c89db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9e8e16f0 con 0x558f9e8d0a80 2023-07-21T22:30:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa16c89db80 10 client.0 _ll_register_callbacks cb 0x7fa150009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa15ecb5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 0x558f9e8e0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa15e4b4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d0a80 0x558f9e8df980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa15ecb5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 0x558f9e8e0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49540/0 (socket says 172.21.15.107:49540) 2023-07-21T22:30:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa15ecb5700 1 -- 172.21.15.107:0/3769706191 learned_addr learned my addr 172.21.15.107:0/3769706191 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa15dcb3700 1 --2- 172.21.15.107:0/3769706191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d13d0 0x558f9e8dfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa15ecb5700 1 -- 172.21.15.107:0/3769706191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d0a80 msgr2=0x558f9e8df980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa15ecb5700 1 --2- 172.21.15.107:0/3769706191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d0a80 0x558f9e8df980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa15ecb5700 1 -- 172.21.15.107:0/3769706191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d13d0 msgr2=0x558f9e8dfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:47.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa15ecb5700 1 --2- 172.21.15.107:0/3769706191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d13d0 0x558f9e8dfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa15ecb5700 1 -- 172.21.15.107:0/3769706191 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa148017070 con 0x558f9e8d1ff0 2023-07-21T22:30:47.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa15dcb3700 1 --2- 172.21.15.107:0/3769706191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d13d0 0x558f9e8dfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:30:47.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa15ecb5700 1 --2- 172.21.15.107:0/3769706191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 0x558f9e8e0ed0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa15001f000 tx=0x7fa150017760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d826107c7573e7b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:47.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa141ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:47.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa141ffb700 1 -- 172.21.15.107:0/3769706191 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15002c070 con 0x558f9e8d1ff0 2023-07-21T22:30:47.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa16c89db80 1 -- 172.21.15.107:0/3769706191 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa148021070 con 0x558f9e8d1ff0 2023-07-21T22:30:47.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa141ffb700 1 -- 172.21.15.107:0/3769706191 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa1500093a0 con 0x558f9e8d1ff0 2023-07-21T22:30:47.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa141ffb700 1 -- 172.21.15.107:0/3769706191 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa150017cc0 con 0x558f9e8d1ff0 2023-07-21T22:30:47.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa16c89db80 10 client.16035 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:47.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.850+0000 7fa16c89db80 1 -- 172.21.15.107:0/3769706191 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fa15000e5a0 con 0x558f9e8d1ff0 2023-07-21T22:30:47.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.851+0000 7fa141ffb700 1 -- 172.21.15.107:0/3769706191 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa148021070 con 0x558f9e8d1ff0 2023-07-21T22:30:47.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.851+0000 7fa141ffb700 1 -- 172.21.15.107:0/3769706191 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa15002f030 con 0x558f9e8d1ff0 2023-07-21T22:30:47.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.851+0000 7fa16c89db80 10 client.16035 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:47.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.851+0000 7fa16c89db80 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:47.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.851+0000 7fa16c89db80 20 client.16035 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:47.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.851+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3769706191 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f9e93b1e0 0x558f9e93d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:47.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.851+0000 7fa16c89db80 4 client.16035 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:30:47.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.851+0000 7fa16c89db80 1 -- 172.21.15.107:0/3769706191 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558f9e7d9050 con 0x558f9e93b1e0 2023-07-21T22:30:47.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.851+0000 7fa15e4b4700 1 --2- 172.21.15.107:0/3769706191 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f9e93b1e0 0x558f9e93d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:47.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.851+0000 7fa141ffb700 10 client.16035 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:47.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.851+0000 7fa15e4b4700 1 --2- 172.21.15.107:0/3769706191 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f9e93b1e0 0x558f9e93d5f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec3bd124e56e91f server_cookie=5b24971d11e8e978 in_seq=0 out_seq=0 2023-07-21T22:30:47.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa141ffb700 1 -- 172.21.15.107:0/3769706191 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558f9e7d9050 con 0x558f9e93b1e0 2023-07-21T22:30:47.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa141ffb700 10 client.16035 handle_command_reply: tid=0 2023-07-21T22:30:47.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa16c89db80 1 client.16035 shutdown 2023-07-21T22:30:47.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa16c89db80 1 -- 172.21.15.107:0/3769706191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 msgr2=0x558f9e8e0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3769706191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 0x558f9e8e0ed0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa15001f000 tx=0x7fa150017760 comp rx=0 tx=0).stop 2023-07-21T22:30:47.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa16c89db80 1 -- 172.21.15.107:0/3769706191 shutdown_connections 2023-07-21T22:30:47.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3769706191 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x558f9e93b1e0 0x558f9e93d5f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3769706191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f9e8d1ff0 0x558f9e8e0ed0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa141ffb700 0 client.16035 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:47.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3769706191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558f9e8d0a80 0x558f9e8df980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa16c89db80 1 --2- 172.21.15.107:0/3769706191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558f9e8d13d0 0x558f9e8dfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa16c89db80 1 -- 172.21.15.107:0/3769706191 >> 172.21.15.107:0/3769706191 conn(0x558f9e8c1ef0 msgr2=0x558f9e8d51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa16c89db80 1 -- 172.21.15.107:0/3769706191 shutdown_connections 2023-07-21T22:30:47.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa16c89db80 1 -- 172.21.15.107:0/3769706191 wait complete. 2023-07-21T22:30:47.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa16c89db80 20 client.16035 trim_cache size 0 max 16384 2023-07-21T22:30:47.909 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:30:47.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa160718700 1 -- 172.21.15.107:0/1801626589 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa1440a1150 msgr2=0x7fa1440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa160718700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa1440a1150 0x7fa1440a3600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa148009c60 tx=0x7fa148009260 comp rx=0 tx=0).stop 2023-07-21T22:30:47.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa160718700 1 -- 172.21.15.107:0/1801626589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1580ffae0 msgr2=0x7fa158195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:47.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa160718700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1580ffae0 0x7fa158195800 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa15400b620 tx=0x7fa154004730 comp rx=0 tx=0).stop 2023-07-21T22:30:47.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa160718700 1 -- 172.21.15.107:0/1801626589 shutdown_connections 2023-07-21T22:30:47.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa160718700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa1440a1150 0x7fa1440a3600 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa160718700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa158101660 0x7fa15819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.911 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa160718700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1580ffae0 0x7fa158195800 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.911 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa160718700 1 --2- 172.21.15.107:0/1801626589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa158100cd0 0x7fa158195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:47.911 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa160718700 1 -- 172.21.15.107:0/1801626589 >> 172.21.15.107:0/1801626589 conn(0x7fa1580fb120 msgr2=0x7fa1580fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:47.911 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa160718700 1 -- 172.21.15.107:0/1801626589 shutdown_connections 2023-07-21T22:30:47.911 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:47.853+0000 7fa160718700 1 -- 172.21.15.107:0/1801626589 wait complete. 2023-07-21T22:30:48.912 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:30:48.913 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:30:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 -- 172.21.15.107:0/1943569941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82780feef0 msgr2=0x7f82780ff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 --2- 172.21.15.107:0/1943569941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82780feef0 0x7f82780ff300 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8268009990 tx=0x7f8268009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 -- 172.21.15.107:0/1943569941 shutdown_connections 2023-07-21T22:30:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 --2- 172.21.15.107:0/1943569941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82781005b0 0x7f8278106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 --2- 172.21.15.107:0/1943569941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82780feef0 0x7f82780ff300 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 --2- 172.21.15.107:0/1943569941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82780ffc00 0x7f8278100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 -- 172.21.15.107:0/1943569941 >> 172.21.15.107:0/1943569941 conn(0x7f82780f9770 msgr2=0x7f82780fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 -- 172.21.15.107:0/1943569941 shutdown_connections 2023-07-21T22:30:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 -- 172.21.15.107:0/1943569941 wait complete. 2023-07-21T22:30:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 Processor -- start 2023-07-21T22:30:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 -- start start 2023-07-21T22:30:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82780feef0 0x7f82780fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82780ffc00 0x7f827819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82781005b0 0x7f827819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827819d370 con 0x7f82781005b0 2023-07-21T22:30:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827819d4e0 con 0x7f82780feef0 2023-07-21T22:30:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.996+0000 7f827e9ba700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827819d6a0 con 0x7f82780ffc00 2023-07-21T22:30:49.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f827cf57700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82781005b0 0x7f827819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f8277fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82780feef0 0x7f82780fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f827cf57700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82781005b0 0x7f827819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49554/0 (socket says 172.21.15.107:49554) 2023-07-21T22:30:49.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f827cf57700 1 -- 172.21.15.107:0/2456306595 learned_addr learned my addr 172.21.15.107:0/2456306595 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:49.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f826ffff700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82780ffc00 0x7f827819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f827cf57700 1 -- 172.21.15.107:0/2456306595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82780ffc00 msgr2=0x7f827819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f827cf57700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82780ffc00 0x7f827819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.047 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f827cf57700 1 -- 172.21.15.107:0/2456306595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82780feef0 msgr2=0x7f82780fe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f827cf57700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82780feef0 0x7f82780fe5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f827cf57700 1 -- 172.21.15.107:0/2456306595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82680095f0 con 0x7f82781005b0 2023-07-21T22:30:49.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f8277fff700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82780feef0 0x7f82780fe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:49.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f827cf57700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82781005b0 0x7f827819cbd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f827000ef30 tx=0x7f827000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:49.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f8275ffb700 1 -- 172.21.15.107:0/2456306595 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270018070 con 0x7f82781005b0 2023-07-21T22:30:49.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f8275ffb700 1 -- 172.21.15.107:0/2456306595 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827000f040 con 0x7f82781005b0 2023-07-21T22:30:49.048 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f8275ffb700 1 -- 172.21.15.107:0/2456306595 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827000cd00 con 0x7f82781005b0 2023-07-21T22:30:49.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f827e9ba700 1 -- 172.21.15.107:0/2456306595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8278194360 con 0x7f82781005b0 2023-07-21T22:30:49.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.997+0000 7f827e9ba700 1 -- 172.21.15.107:0/2456306595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8278194820 con 0x7f82781005b0 2023-07-21T22:30:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:48.999+0000 7f8275ffb700 1 -- 172.21.15.107:0/2456306595 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8270003a00 con 0x7f82781005b0 2023-07-21T22:30:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.000+0000 7f8275ffb700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f82640a11d0 0x7f82640a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.000+0000 7f8277fff700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f82640a11d0 0x7f82640a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.000+0000 7f8275ffb700 1 -- 172.21.15.107:0/2456306595 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f82700cc3f0 con 0x7f82781005b0 2023-07-21T22:30:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.000+0000 7f8277fff700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f82640a11d0 0x7f82640a3680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f826800b2e0 tx=0x7f826801e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f828ab3fb80 1 Processor -- start 2023-07-21T22:30:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f828ab3fb80 1 -- start start 2023-07-21T22:30:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f828ab3fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d208fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f828ab3fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 0x555d208fb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f828ab3fb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d208fbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f828ab3fb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d20800d30 con 0x555d208fba90 2023-07-21T22:30:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f828ab3fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d20853ac0 con 0x555d208fc8b0 2023-07-21T22:30:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f828ab3fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d2084cf50 con 0x555d208fd220 2023-07-21T22:30:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f8277fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d208fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f8277fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d208fccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40386/0 (socket says 172.21.15.107:40386) 2023-07-21T22:30:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f827cf57700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d208fbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f8277fff700 1 -- 172.21.15.107:0/2639401254 learned_addr learned my addr 172.21.15.107:0/2639401254 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f826ffff700 1 --2- 172.21.15.107:0/2639401254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 0x555d208fb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f8277fff700 1 -- 172.21.15.107:0/2639401254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 msgr2=0x555d208fb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f8277fff700 1 --2- 172.21.15.107:0/2639401254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 0x555d208fb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f8277fff700 1 -- 172.21.15.107:0/2639401254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 msgr2=0x555d208fbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f8277fff700 1 --2- 172.21.15.107:0/2639401254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d208fbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.001+0000 7f8277fff700 1 -- 172.21.15.107:0/2639401254 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d2067ae60 con 0x555d208fc8b0 2023-07-21T22:30:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.002+0000 7f8277fff700 1 --2- 172.21.15.107:0/2639401254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d208fccc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8268015000 tx=0x7f8268024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81022de5681fc311 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.002+0000 7f826e7fc700 1 -- 172.21.15.107:0/2639401254 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82680054d0 con 0x555d208fc8b0 2023-07-21T22:30:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.002+0000 7f826e7fc700 1 -- 172.21.15.107:0/2639401254 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8268022040 con 0x555d208fc8b0 2023-07-21T22:30:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.002+0000 7f826e7fc700 1 -- 172.21.15.107:0/2639401254 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826800f6c0 con 0x555d208fc8b0 2023-07-21T22:30:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.002+0000 7f828ab3fb80 1 -- 172.21.15.107:0/2639401254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 msgr2=0x555d208fccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.002+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/2639401254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d208fccc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8268015000 tx=0x7f8268024000 comp rx=0 tx=0).stop 2023-07-21T22:30:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.002+0000 7f828ab3fb80 1 -- 172.21.15.107:0/2639401254 shutdown_connections 2023-07-21T22:30:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.002+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/2639401254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d208fbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.002+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/2639401254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d208fccc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.003+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/2639401254 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 0x555d208fb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.003+0000 7f828ab3fb80 1 -- 172.21.15.107:0/2639401254 >> 172.21.15.107:0/2639401254 conn(0x555d208f0f70 msgr2=0x555d208f63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.003+0000 7f828ab3fb80 1 -- 172.21.15.107:0/2639401254 shutdown_connections 2023-07-21T22:30:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.003+0000 7f828ab3fb80 1 -- 172.21.15.107:0/2639401254 wait complete. 2023-07-21T22:30:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.003+0000 7f828ab3fb80 1 -- start start 2023-07-21T22:30:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.003+0000 7f828ab3fb80 1 Processor -- start 2023-07-21T22:30:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.003+0000 7f828ab3fb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d20905010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.003+0000 7f828ab3fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d20905550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f828ab3fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 0x555d2090c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f828ab3fb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d207ecb40 con 0x555d208fba90 2023-07-21T22:30:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f828ab3fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d207ec840 con 0x555d208fc8b0 2023-07-21T22:30:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f828ab3fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d206d09a0 con 0x555d208fd220 2023-07-21T22:30:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f828ab3fb80 10 client.0 _ll_register_callbacks cb 0x555d207ea7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f8277fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d20905010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f8277fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d20905010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49564/0 (socket says 172.21.15.107:49564) 2023-07-21T22:30:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f826ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d20905550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f8277fff700 1 -- 172.21.15.107:0/756279806 learned_addr learned my addr 172.21.15.107:0/756279806 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f827cf57700 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 0x555d2090c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f8277fff700 1 -- 172.21.15.107:0/756279806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 msgr2=0x555d2090c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f8277fff700 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 0x555d2090c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f8277fff700 1 -- 172.21.15.107:0/756279806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 msgr2=0x555d20905550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f8277fff700 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d20905550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f8277fff700 1 -- 172.21.15.107:0/756279806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8278107750 con 0x555d208fba90 2023-07-21T22:30:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f826ffff700 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d20905550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f827cf57700 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 0x555d2090c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:30:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f8277fff700 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d20905010 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8268036040 tx=0x7f8268005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827e3d4651dad9a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f82577fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f82577fe700 1 -- 172.21.15.107:0/756279806 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826800f460 con 0x555d208fba90 2023-07-21T22:30:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f82577fe700 1 -- 172.21.15.107:0/756279806 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f826800f5c0 con 0x555d208fba90 2023-07-21T22:30:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f828ab3fb80 1 -- 172.21.15.107:0/756279806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d207a6570 con 0x555d208fba90 2023-07-21T22:30:49.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f82577fe700 1 -- 172.21.15.107:0/756279806 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f826800f7a0 con 0x555d208fba90 2023-07-21T22:30:49.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f828ab3fb80 10 client.16047 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:49.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.004+0000 7f828ab3fb80 1 -- 172.21.15.107:0/756279806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x555d20663ce0 con 0x555d208fba90 2023-07-21T22:30:49.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.005+0000 7f82577fe700 1 -- 172.21.15.107:0/756279806 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d207a6570 con 0x555d208fba90 2023-07-21T22:30:49.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.005+0000 7f82577fe700 1 -- 172.21.15.107:0/756279806 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8268046830 con 0x555d208fba90 2023-07-21T22:30:49.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.005+0000 7f828ab3fb80 10 client.16047 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:49.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.005+0000 7f828ab3fb80 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:49.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.005+0000 7f828ab3fb80 20 client.16047 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:49.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.005+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d209108c0 0x555d20912cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.005+0000 7f828ab3fb80 4 client.16047 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:49.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.005+0000 7f828ab3fb80 1 -- 172.21.15.107:0/756279806 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d20845f30 con 0x555d209108c0 2023-07-21T22:30:49.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.005+0000 7f826ffff700 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d209108c0 0x555d20912cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.005+0000 7f826ffff700 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d209108c0 0x555d20912cd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80794936b3885c94 server_cookie=c39427fc594cdbf2 in_seq=0 out_seq=0 2023-07-21T22:30:49.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.005+0000 7f82577fe700 10 client.16047 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:49.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.006+0000 7f82577fe700 1 -- 172.21.15.107:0/756279806 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555d20845f30 con 0x555d209108c0 2023-07-21T22:30:49.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.006+0000 7f82577fe700 10 client.16047 handle_command_reply: tid=0 2023-07-21T22:30:49.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.006+0000 7f828ab3fb80 1 client.16047 shutdown 2023-07-21T22:30:49.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.006+0000 7f828ab3fb80 1 -- 172.21.15.107:0/756279806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 msgr2=0x555d20905010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.006+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d20905010 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8268036040 tx=0x7f8268005a00 comp rx=0 tx=0).stop 2023-07-21T22:30:49.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.006+0000 7f828ab3fb80 1 -- 172.21.15.107:0/756279806 shutdown_connections 2023-07-21T22:30:49.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.006+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d209108c0 0x555d20912cd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.006+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 0x555d2090c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.006+0000 7f82577fe700 0 client.16047 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:49.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.006+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d20905010 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.006+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/756279806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d20905550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.006+0000 7f828ab3fb80 1 -- 172.21.15.107:0/756279806 >> 172.21.15.107:0/756279806 conn(0x555d208f0f70 msgr2=0x555d208f5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.006+0000 7f828ab3fb80 1 -- 172.21.15.107:0/756279806 shutdown_connections 2023-07-21T22:30:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.007+0000 7f828ab3fb80 1 -- 172.21.15.107:0/756279806 wait complete. 2023-07-21T22:30:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.007+0000 7f828ab3fb80 20 client.16047 trim_cache size 0 max 16384 2023-07-21T22:30:49.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f828ab3fb80 1 Processor -- start 2023-07-21T22:30:49.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f828ab3fb80 1 -- start start 2023-07-21T22:30:49.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f828ab3fb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d20904dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f828ab3fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d2090c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f828ab3fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 0x555d20905300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f828ab3fb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d20904be0 con 0x555d208fba90 2023-07-21T22:30:49.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f828ab3fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d209044c0 con 0x555d208fc8b0 2023-07-21T22:30:49.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f828ab3fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d208f7c80 con 0x555d208fd220 2023-07-21T22:30:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f8277fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d20904dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f826ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d2090c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f826ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d2090c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40402/0 (socket says 172.21.15.107:40402) 2023-07-21T22:30:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f826ffff700 1 -- 172.21.15.107:0/3361695373 learned_addr learned my addr 172.21.15.107:0/3361695373 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f8277fff700 1 --2- 172.21.15.107:0/3361695373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d20904dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49572/0 (socket says 172.21.15.107:49572) 2023-07-21T22:30:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f827cf57700 1 --2- 172.21.15.107:0/3361695373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 0x555d20905300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f826ffff700 1 -- 172.21.15.107:0/3361695373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 msgr2=0x555d20905300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.024+0000 7f826ffff700 1 --2- 172.21.15.107:0/3361695373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 0x555d20905300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.025+0000 7f826ffff700 1 -- 172.21.15.107:0/3361695373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 msgr2=0x555d20904dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.025+0000 7f826ffff700 1 --2- 172.21.15.107:0/3361695373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d20904dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.025+0000 7f826ffff700 1 -- 172.21.15.107:0/3361695373 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d206a12d0 con 0x555d208fc8b0 2023-07-21T22:30:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.025+0000 7f8277fff700 1 --2- 172.21.15.107:0/3361695373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d20904dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.025+0000 7f826ffff700 1 --2- 172.21.15.107:0/3361695373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d2090c550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8260005c50 tx=0x7f8260009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=929a02972fda6691 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.025+0000 7f826effd700 1 -- 172.21.15.107:0/3361695373 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260017070 con 0x555d208fc8b0 2023-07-21T22:30:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.025+0000 7f826effd700 1 -- 172.21.15.107:0/3361695373 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8260018040 con 0x555d208fc8b0 2023-07-21T22:30:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.025+0000 7f826effd700 1 -- 172.21.15.107:0/3361695373 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260004580 con 0x555d208fc8b0 2023-07-21T22:30:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 -- 172.21.15.107:0/3361695373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 msgr2=0x555d2090c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/3361695373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d2090c550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8260005c50 tx=0x7f8260009260 comp rx=0 tx=0).stop 2023-07-21T22:30:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 -- 172.21.15.107:0/3361695373 shutdown_connections 2023-07-21T22:30:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/3361695373 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fd220 0x555d20905300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/3361695373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fba90 0x555d20904dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/3361695373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fc8b0 0x555d2090c550 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 -- 172.21.15.107:0/3361695373 >> 172.21.15.107:0/3361695373 conn(0x555d208f0f70 msgr2=0x555d208ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 -- 172.21.15.107:0/3361695373 shutdown_connections 2023-07-21T22:30:49.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 -- 172.21.15.107:0/3361695373 wait complete. 2023-07-21T22:30:49.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 -- start start 2023-07-21T22:30:49.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 Processor -- start 2023-07-21T22:30:49.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fba90 0x555d20909510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fc8b0 0x555d20909a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fd220 0x555d20964770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d20964cb0 con 0x555d208fc8b0 2023-07-21T22:30:49.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d20964df0 con 0x555d208fd220 2023-07-21T22:30:49.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d20964fb0 con 0x555d208fba90 2023-07-21T22:30:49.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.026+0000 7f828ab3fb80 10 client.0 _ll_register_callbacks cb 0x7f8270095490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:49.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f826ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fc8b0 0x555d20909a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f826ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fc8b0 0x555d20909a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49580/0 (socket says 172.21.15.107:49580) 2023-07-21T22:30:49.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f826ffff700 1 -- 172.21.15.107:0/4136068026 learned_addr learned my addr 172.21.15.107:0/4136068026 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:49.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f827cf57700 1 --2- 172.21.15.107:0/4136068026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fd220 0x555d20964770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f8277fff700 1 --2- 172.21.15.107:0/4136068026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fba90 0x555d20909510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f826ffff700 1 -- 172.21.15.107:0/4136068026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fba90 msgr2=0x555d20909510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f826ffff700 1 --2- 172.21.15.107:0/4136068026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fba90 0x555d20909510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f826ffff700 1 -- 172.21.15.107:0/4136068026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fd220 msgr2=0x555d20964770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f826ffff700 1 --2- 172.21.15.107:0/4136068026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fd220 0x555d20964770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f826ffff700 1 -- 172.21.15.107:0/4136068026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d20663ce0 con 0x555d208fc8b0 2023-07-21T22:30:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f826ffff700 1 --2- 172.21.15.107:0/4136068026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fc8b0 0x555d20909a50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8260005c50 tx=0x7f8260004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8442240d139365f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f82577fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f82577fe700 1 -- 172.21.15.107:0/4136068026 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260017040 con 0x555d208fc8b0 2023-07-21T22:30:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f828ab3fb80 1 -- 172.21.15.107:0/4136068026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8268022040 con 0x555d208fc8b0 2023-07-21T22:30:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f82577fe700 1 -- 172.21.15.107:0/4136068026 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8260021c60 con 0x555d208fc8b0 2023-07-21T22:30:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f82577fe700 1 -- 172.21.15.107:0/4136068026 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8260021e20 con 0x555d208fc8b0 2023-07-21T22:30:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f828ab3fb80 10 client.16053 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f828ab3fb80 1 -- 172.21.15.107:0/4136068026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f8260009eb0 con 0x555d208fc8b0 2023-07-21T22:30:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.027+0000 7f82577fe700 1 -- 172.21.15.107:0/4136068026 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8268022040 con 0x555d208fc8b0 2023-07-21T22:30:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.028+0000 7f82577fe700 1 -- 172.21.15.107:0/4136068026 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8260010430 con 0x555d208fc8b0 2023-07-21T22:30:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.028+0000 7f828ab3fb80 10 client.16053 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.028+0000 7f828ab3fb80 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.028+0000 7f828ab3fb80 20 client.16053 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.028+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/4136068026 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d20966940 0x555d20968d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.028+0000 7f828ab3fb80 4 client.16053 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:30:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.028+0000 7f828ab3fb80 1 -- 172.21.15.107:0/4136068026 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555d208080d0 con 0x555d20966940 2023-07-21T22:30:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.028+0000 7f8277fff700 1 --2- 172.21.15.107:0/4136068026 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d20966940 0x555d20968d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.028+0000 7f82577fe700 10 client.16053 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:49.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.028+0000 7f8277fff700 1 --2- 172.21.15.107:0/4136068026 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d20966940 0x555d20968d50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab3c5c11629396b server_cookie=1ff558c5c9595dab in_seq=0 out_seq=0 2023-07-21T22:30:49.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.029+0000 7f82577fe700 1 -- 172.21.15.107:0/4136068026 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555d208080d0 con 0x555d20966940 2023-07-21T22:30:49.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.029+0000 7f82577fe700 10 client.16053 handle_command_reply: tid=0 2023-07-21T22:30:49.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.029+0000 7f828ab3fb80 1 client.16053 shutdown 2023-07-21T22:30:49.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.029+0000 7f828ab3fb80 1 -- 172.21.15.107:0/4136068026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fc8b0 msgr2=0x555d20909a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.029+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/4136068026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fc8b0 0x555d20909a50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8260005c50 tx=0x7f8260004460 comp rx=0 tx=0).stop 2023-07-21T22:30:49.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.029+0000 7f828ab3fb80 1 -- 172.21.15.107:0/4136068026 shutdown_connections 2023-07-21T22:30:49.084 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.029+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/4136068026 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d20966940 0x555d20968d50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.029+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/4136068026 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d208fd220 0x555d20964770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.029+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/4136068026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d208fba90 0x555d20909510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.029+0000 7f828ab3fb80 1 --2- 172.21.15.107:0/4136068026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d208fc8b0 0x555d20909a50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.029+0000 7f828ab3fb80 1 -- 172.21.15.107:0/4136068026 >> 172.21.15.107:0/4136068026 conn(0x555d208f0f70 msgr2=0x555d20904dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.030+0000 7f82577fe700 0 client.16053 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.030+0000 7f828ab3fb80 1 -- 172.21.15.107:0/4136068026 shutdown_connections 2023-07-21T22:30:49.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.030+0000 7f828ab3fb80 1 -- 172.21.15.107:0/4136068026 wait complete. 2023-07-21T22:30:49.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.030+0000 7f828ab3fb80 20 client.16053 trim_cache size 0 max 16384 2023-07-21T22:30:49.086 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:30:49.086 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2023-07-21T22:30:49.086 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "d037412e-e4e0-4d2c-8d6e-c7193371fe0f", 2023-07-21T22:30:49.086 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2023-07-21T22:30:49.086 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:30:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.030+0000 7f827e9ba700 1 -- 172.21.15.107:0/2456306595 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f82640a11d0 msgr2=0x7f82640a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.030+0000 7f827e9ba700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f82640a11d0 0x7f82640a3680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f826800b2e0 tx=0x7f826801e040 comp rx=0 tx=0).stop 2023-07-21T22:30:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.030+0000 7f827e9ba700 1 -- 172.21.15.107:0/2456306595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82781005b0 msgr2=0x7f827819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.030+0000 7f827e9ba700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82781005b0 0x7f827819cbd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f827000ef30 tx=0x7f827000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.031+0000 7f827e9ba700 1 -- 172.21.15.107:0/2456306595 shutdown_connections 2023-07-21T22:30:49.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.031+0000 7f827e9ba700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82781005b0 0x7f827819cbd0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.031+0000 7f827e9ba700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f82780feef0 0x7f82780fe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.031+0000 7f827e9ba700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f82640a11d0 0x7f82640a3680 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.031+0000 7f827e9ba700 1 --2- 172.21.15.107:0/2456306595 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82780ffc00 0x7f827819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.031+0000 7f827e9ba700 1 -- 172.21.15.107:0/2456306595 >> 172.21.15.107:0/2456306595 conn(0x7f82780f9770 msgr2=0x7f82780fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:49.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.031+0000 7f827e9ba700 1 -- 172.21.15.107:0/2456306595 shutdown_connections 2023-07-21T22:30:49.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.031+0000 7f827e9ba700 1 -- 172.21.15.107:0/2456306595 wait complete. 2023-07-21T22:30:49.094 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:30:49.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:49 smithi107 ceph-mon[96543]: pgmap v345: 97 pgs: 97 active+clean; 147 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 519 op/s 2023-07-21T22:30:49.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.201+0000 7f85f737f700 1 -- 172.21.15.107:0/863375837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f00ffc40 msgr2=0x7f85f0100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.201+0000 7f85f737f700 1 --2- 172.21.15.107:0/863375837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f00ffc40 0x7f85f0100090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009990 tx=0x7f85e0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.248 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.201+0000 7f85f737f700 1 -- 172.21.15.107:0/863375837 shutdown_connections 2023-07-21T22:30:49.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.201+0000 7f85f737f700 1 --2- 172.21.15.107:0/863375837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85f01005d0 0x7f85f0106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.201+0000 7f85f737f700 1 --2- 172.21.15.107:0/863375837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f00fea50 0x7f85f00fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.201+0000 7f85f737f700 1 --2- 172.21.15.107:0/863375837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f00ffc40 0x7f85f0100090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.201+0000 7f85f737f700 1 -- 172.21.15.107:0/863375837 >> 172.21.15.107:0/863375837 conn(0x7f85f00fa070 msgr2=0x7f85f00fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:49.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.201+0000 7f85f737f700 1 -- 172.21.15.107:0/863375837 shutdown_connections 2023-07-21T22:30:49.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.201+0000 7f85f737f700 1 -- 172.21.15.107:0/863375837 wait complete. 2023-07-21T22:30:49.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f737f700 1 Processor -- start 2023-07-21T22:30:49.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f737f700 1 -- start start 2023-07-21T22:30:49.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f737f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f00fea50 0x7f85f00693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f737f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f00ffc40 0x7f85f019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f737f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85f01005d0 0x7f85f019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f737f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f019d330 con 0x7f85f01005d0 2023-07-21T22:30:49.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f737f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f019d4a0 con 0x7f85f00fea50 2023-07-21T22:30:49.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f737f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f019d660 con 0x7f85f00ffc40 2023-07-21T22:30:49.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f491a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f00ffc40 0x7f85f019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f511b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f00fea50 0x7f85f00693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f491a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f00ffc40 0x7f85f019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38152/0 (socket says 172.21.15.107:38152) 2023-07-21T22:30:49.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f511b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f00fea50 0x7f85f00693b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40420/0 (socket says 172.21.15.107:40420) 2023-07-21T22:30:49.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f491a700 1 -- 172.21.15.107:0/4156966287 learned_addr learned my addr 172.21.15.107:0/4156966287 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:49.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f591c700 1 --2- 172.21.15.107:0/4156966287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85f01005d0 0x7f85f019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f491a700 1 -- 172.21.15.107:0/4156966287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f00fea50 msgr2=0x7f85f00693b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f491a700 1 --2- 172.21.15.107:0/4156966287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f00fea50 0x7f85f00693b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f491a700 1 -- 172.21.15.107:0/4156966287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85f01005d0 msgr2=0x7f85f019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f491a700 1 --2- 172.21.15.107:0/4156966287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85f01005d0 0x7f85f019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.202+0000 7f85f491a700 1 -- 172.21.15.107:0/4156966287 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e00095f0 con 0x7f85f00ffc40 2023-07-21T22:30:49.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.203+0000 7f85f491a700 1 --2- 172.21.15.107:0/4156966287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f00ffc40 0x7f85f019a510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f85e0000c00 tx=0x7f85e0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:49.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.203+0000 7f85e67fc700 1 -- 172.21.15.107:0/4156966287 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e001d070 con 0x7f85f00ffc40 2023-07-21T22:30:49.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.203+0000 7f85f737f700 1 -- 172.21.15.107:0/4156966287 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f019d8e0 con 0x7f85f00ffc40 2023-07-21T22:30:49.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.203+0000 7f85e67fc700 1 -- 172.21.15.107:0/4156966287 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85e0022470 con 0x7f85f00ffc40 2023-07-21T22:30:49.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.203+0000 7f85e67fc700 1 -- 172.21.15.107:0/4156966287 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0017650 con 0x7f85f00ffc40 2023-07-21T22:30:49.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.203+0000 7f85f737f700 1 -- 172.21.15.107:0/4156966287 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f019ddb0 con 0x7f85f00ffc40 2023-07-21T22:30:49.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.205+0000 7f85e67fc700 1 -- 172.21.15.107:0/4156966287 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f85e00225e0 con 0x7f85f00ffc40 2023-07-21T22:30:49.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.206+0000 7f85e67fc700 1 --2- 172.21.15.107:0/4156966287 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f85dc0a1150 0x7f85dc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.206+0000 7f85e67fc700 1 -- 172.21.15.107:0/4156966287 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f85e00cdc20 con 0x7f85f00ffc40 2023-07-21T22:30:49.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.206+0000 7f85f511b700 1 --2- 172.21.15.107:0/4156966287 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f85dc0a1150 0x7f85dc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.206+0000 7f85f511b700 1 --2- 172.21.15.107:0/4156966287 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f85dc0a1150 0x7f85dc0a3600 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f85f00ffaa0 tx=0x7f85ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:49.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f8603504b80 1 Processor -- start 2023-07-21T22:30:49.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f8603504b80 1 -- start start 2023-07-21T22:30:49.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f8603504b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87f830 0x55cdbf87fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f8603504b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf8801a0 0x55cdbf87e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f8603504b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87ea10 0x55cdbf87ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f8603504b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbf783cb0 con 0x55cdbf87ea10 2023-07-21T22:30:49.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f8603504b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbf7d6a40 con 0x55cdbf87f830 2023-07-21T22:30:49.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f8603504b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbf7cfed0 con 0x55cdbf8801a0 2023-07-21T22:30:49.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f85f511b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87f830 0x55cdbf87fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f85f511b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87f830 0x55cdbf87fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40428/0 (socket says 172.21.15.107:40428) 2023-07-21T22:30:49.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f85f511b700 1 -- 172.21.15.107:0/2078626250 learned_addr learned my addr 172.21.15.107:0/2078626250 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:49.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f85f491a700 1 --2- 172.21.15.107:0/2078626250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf8801a0 0x55cdbf87e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f85f591c700 1 --2- 172.21.15.107:0/2078626250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87ea10 0x55cdbf87ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f85f491a700 1 -- 172.21.15.107:0/2078626250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87f830 msgr2=0x55cdbf87fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f85f491a700 1 --2- 172.21.15.107:0/2078626250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87f830 0x55cdbf87fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f85f491a700 1 -- 172.21.15.107:0/2078626250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87ea10 msgr2=0x55cdbf87ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f85f491a700 1 --2- 172.21.15.107:0/2078626250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87ea10 0x55cdbf87ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f85f491a700 1 -- 172.21.15.107:0/2078626250 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdbf5fdde0 con 0x55cdbf8801a0 2023-07-21T22:30:49.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.207+0000 7f85f511b700 1 --2- 172.21.15.107:0/2078626250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87f830 0x55cdbf87fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:49.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.211+0000 7f85f491a700 1 --2- 172.21.15.107:0/2078626250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf8801a0 0x55cdbf87e430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f85e00cba70 tx=0x7f85e0022c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f0c486de29ef3f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:49.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.211+0000 7f85db7fe700 1 -- 172.21.15.107:0/2078626250 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e009f0d0 con 0x55cdbf8801a0 2023-07-21T22:30:49.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.211+0000 7f85db7fe700 1 -- 172.21.15.107:0/2078626250 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85e00a3460 con 0x55cdbf8801a0 2023-07-21T22:30:49.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.211+0000 7f85db7fe700 1 -- 172.21.15.107:0/2078626250 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e009a590 con 0x55cdbf8801a0 2023-07-21T22:30:49.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.212+0000 7f8603504b80 1 -- 172.21.15.107:0/2078626250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf8801a0 msgr2=0x55cdbf87e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.212+0000 7f8603504b80 1 --2- 172.21.15.107:0/2078626250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf8801a0 0x55cdbf87e430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f85e00cba70 tx=0x7f85e0022c80 comp rx=0 tx=0).stop 2023-07-21T22:30:49.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.212+0000 7f8603504b80 1 -- 172.21.15.107:0/2078626250 shutdown_connections 2023-07-21T22:30:49.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.212+0000 7f8603504b80 1 --2- 172.21.15.107:0/2078626250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87ea10 0x55cdbf87ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.212+0000 7f8603504b80 1 --2- 172.21.15.107:0/2078626250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87f830 0x55cdbf87fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.212+0000 7f8603504b80 1 --2- 172.21.15.107:0/2078626250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf8801a0 0x55cdbf87e430 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.212+0000 7f8603504b80 1 -- 172.21.15.107:0/2078626250 >> 172.21.15.107:0/2078626250 conn(0x55cdbf873ef0 msgr2=0x55cdbf879360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.212+0000 7f8603504b80 1 -- 172.21.15.107:0/2078626250 shutdown_connections 2023-07-21T22:30:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.212+0000 7f8603504b80 1 -- 172.21.15.107:0/2078626250 wait complete. 2023-07-21T22:30:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.212+0000 7f8603504b80 1 -- start start 2023-07-21T22:30:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.212+0000 7f8603504b80 1 Processor -- start 2023-07-21T22:30:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f8603504b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf87ea10 0x55cdbf887d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f8603504b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87f830 0x55cdbf888290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f8603504b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf8801a0 0x55cdbf88f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f8603504b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbf76fac0 con 0x55cdbf8801a0 2023-07-21T22:30:49.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f8603504b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbf76f7c0 con 0x55cdbf87f830 2023-07-21T22:30:49.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f8603504b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbf653920 con 0x55cdbf87ea10 2023-07-21T22:30:49.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f8603504b80 10 client.0 _ll_register_callbacks cb 0x55cdbf76d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:49.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f511b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf87ea10 0x55cdbf887d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f591c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf8801a0 0x55cdbf88f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f511b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf87ea10 0x55cdbf887d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38164/0 (socket says 172.21.15.107:38164) 2023-07-21T22:30:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f511b700 1 -- 172.21.15.107:0/2335362099 learned_addr learned my addr 172.21.15.107:0/2335362099 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f591c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf8801a0 0x55cdbf88f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49610/0 (socket says 172.21.15.107:49610) 2023-07-21T22:30:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f591c700 1 -- 172.21.15.107:0/2335362099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf87ea10 msgr2=0x55cdbf887d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f491a700 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87f830 0x55cdbf888290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f591c700 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf87ea10 0x55cdbf887d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f591c700 1 -- 172.21.15.107:0/2335362099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87f830 msgr2=0x55cdbf888290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f591c700 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87f830 0x55cdbf888290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f591c700 1 -- 172.21.15.107:0/2335362099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ec017070 con 0x55cdbf8801a0 2023-07-21T22:30:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f511b700 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf87ea10 0x55cdbf887d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f491a700 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87f830 0x55cdbf888290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:30:49.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85f591c700 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf8801a0 0x55cdbf88f2d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f85e8012dc0 tx=0x7f85e8014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cbdd7a5545127d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:49.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:49.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85d8ff9700 1 -- 172.21.15.107:0/2335362099 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800e040 con 0x55cdbf8801a0 2023-07-21T22:30:49.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f85d8ff9700 1 -- 172.21.15.107:0/2335362099 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f85e8014b30 con 0x55cdbf8801a0 2023-07-21T22:30:49.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.213+0000 7f8603504b80 1 -- 172.21.15.107:0/2335362099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdbf7294f0 con 0x55cdbf8801a0 2023-07-21T22:30:49.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.214+0000 7f85d8ff9700 1 -- 172.21.15.107:0/2335362099 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e8014cf0 con 0x55cdbf8801a0 2023-07-21T22:30:49.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.214+0000 7f8603504b80 10 client.16065 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:49.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.214+0000 7f8603504b80 1 -- 172.21.15.107:0/2335362099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f85f019ddb0 con 0x55cdbf8801a0 2023-07-21T22:30:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.214+0000 7f85d8ff9700 1 -- 172.21.15.107:0/2335362099 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cdbf7294f0 con 0x55cdbf8801a0 2023-07-21T22:30:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.214+0000 7f85d8ff9700 1 -- 172.21.15.107:0/2335362099 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f85e8027050 con 0x55cdbf8801a0 2023-07-21T22:30:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.214+0000 7f8603504b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.214+0000 7f8603504b80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.214+0000 7f8603504b80 20 client.16065 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:49.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.214+0000 7f8603504b80 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cdbf893b10 0x55cdbf895f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.214+0000 7f8603504b80 4 client.16065 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:49.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.214+0000 7f8603504b80 1 -- 172.21.15.107:0/2335362099 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdbf7c8eb0 con 0x55cdbf893b10 2023-07-21T22:30:49.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.214+0000 7f85f511b700 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cdbf893b10 0x55cdbf895f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.215+0000 7f85f511b700 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cdbf893b10 0x55cdbf895f20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4330a518d08c8e19 server_cookie=6ca8fcc0e2318a7e in_seq=0 out_seq=0 2023-07-21T22:30:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.215+0000 7f85d8ff9700 10 client.16065 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.216+0000 7f85d8ff9700 1 -- 172.21.15.107:0/2335362099 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cdbf7c8eb0 con 0x55cdbf893b10 2023-07-21T22:30:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.216+0000 7f85d8ff9700 10 client.16065 handle_command_reply: tid=0 2023-07-21T22:30:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.216+0000 7f8603504b80 1 client.16065 shutdown 2023-07-21T22:30:49.279 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.216+0000 7f8603504b80 1 -- 172.21.15.107:0/2335362099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf8801a0 msgr2=0x55cdbf88f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.279 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.216+0000 7f8603504b80 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf8801a0 0x55cdbf88f2d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f85e8012dc0 tx=0x7f85e8014600 comp rx=0 tx=0).stop 2023-07-21T22:30:49.279 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.217+0000 7f8603504b80 1 -- 172.21.15.107:0/2335362099 shutdown_connections 2023-07-21T22:30:49.279 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.217+0000 7f8603504b80 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cdbf893b10 0x55cdbf895f20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.217+0000 7f8603504b80 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf8801a0 0x55cdbf88f2d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.217+0000 7f85d8ff9700 0 client.16065 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:49.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.217+0000 7f8603504b80 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf87ea10 0x55cdbf887d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.217+0000 7f8603504b80 1 --2- 172.21.15.107:0/2335362099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87f830 0x55cdbf888290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.217+0000 7f8603504b80 1 -- 172.21.15.107:0/2335362099 >> 172.21.15.107:0/2335362099 conn(0x55cdbf873ef0 msgr2=0x55cdbf878a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:49.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.217+0000 7f8603504b80 1 -- 172.21.15.107:0/2335362099 shutdown_connections 2023-07-21T22:30:49.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.217+0000 7f8603504b80 1 -- 172.21.15.107:0/2335362099 wait complete. 2023-07-21T22:30:49.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.217+0000 7f8603504b80 20 client.16065 trim_cache size 0 max 16384 2023-07-21T22:30:49.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.237+0000 7f8603504b80 1 Processor -- start 2023-07-21T22:30:49.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.237+0000 7f8603504b80 1 -- start start 2023-07-21T22:30:49.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f8603504b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf873ef0 0x55cdbf882a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f8603504b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87ea10 0x55cdbf882fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f8603504b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87f830 0x55cdbf88f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f8603504b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbf8878c0 con 0x55cdbf87f830 2023-07-21T22:30:49.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f8603504b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbf890f90 con 0x55cdbf87ea10 2023-07-21T22:30:49.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f8603504b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbf88f730 con 0x55cdbf873ef0 2023-07-21T22:30:49.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f85f591c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87f830 0x55cdbf88f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f85f511b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf873ef0 0x55cdbf882a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f85f491a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87ea10 0x55cdbf882fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f85f591c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87f830 0x55cdbf88f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49618/0 (socket says 172.21.15.107:49618) 2023-07-21T22:30:49.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f85f591c700 1 -- 172.21.15.107:0/3700827617 learned_addr learned my addr 172.21.15.107:0/3700827617 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:49.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f85f511b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf873ef0 0x55cdbf882a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38172/0 (socket says 172.21.15.107:38172) 2023-07-21T22:30:49.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f85f591c700 1 -- 172.21.15.107:0/3700827617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf873ef0 msgr2=0x55cdbf882a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f85f591c700 1 --2- 172.21.15.107:0/3700827617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf873ef0 0x55cdbf882a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f85f591c700 1 -- 172.21.15.107:0/3700827617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87ea10 msgr2=0x55cdbf882fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f85f591c700 1 --2- 172.21.15.107:0/3700827617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87ea10 0x55cdbf882fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f85f591c700 1 -- 172.21.15.107:0/3700827617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdbf5e6c60 con 0x55cdbf87f830 2023-07-21T22:30:49.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f85f591c700 1 --2- 172.21.15.107:0/3700827617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87f830 0x55cdbf88f1f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009a00 tx=0x7f85e80092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fe562a699f73c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:49.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.238+0000 7f85dbfff700 1 -- 172.21.15.107:0/3700827617 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800e040 con 0x55cdbf87f830 2023-07-21T22:30:49.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.239+0000 7f85dbfff700 1 -- 172.21.15.107:0/3700827617 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85e800f040 con 0x55cdbf87f830 2023-07-21T22:30:49.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.239+0000 7f85dbfff700 1 -- 172.21.15.107:0/3700827617 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8014a00 con 0x55cdbf87f830 2023-07-21T22:30:49.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.239+0000 7f8603504b80 1 -- 172.21.15.107:0/3700827617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87f830 msgr2=0x55cdbf88f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.295 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.239+0000 7f8603504b80 1 --2- 172.21.15.107:0/3700827617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87f830 0x55cdbf88f1f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009a00 tx=0x7f85e80092e0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.239+0000 7f8603504b80 1 -- 172.21.15.107:0/3700827617 shutdown_connections 2023-07-21T22:30:49.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.239+0000 7f8603504b80 1 --2- 172.21.15.107:0/3700827617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87f830 0x55cdbf88f1f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.239+0000 7f8603504b80 1 --2- 172.21.15.107:0/3700827617 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf873ef0 0x55cdbf882a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.239+0000 7f8603504b80 1 --2- 172.21.15.107:0/3700827617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf87ea10 0x55cdbf882fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.239+0000 7f8603504b80 1 -- 172.21.15.107:0/3700827617 >> 172.21.15.107:0/3700827617 conn(0x55cdbf87e640 msgr2=0x55cdbf88fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:49.296 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.239+0000 7f8603504b80 1 -- 172.21.15.107:0/3700827617 shutdown_connections 2023-07-21T22:30:49.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.239+0000 7f8603504b80 1 -- 172.21.15.107:0/3700827617 wait complete. 2023-07-21T22:30:49.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f8603504b80 1 -- start start 2023-07-21T22:30:49.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f8603504b80 1 Processor -- start 2023-07-21T22:30:49.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f8603504b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf873ef0 0x55cdbf88e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f8603504b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87ea10 0x55cdbf88ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f8603504b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf87f830 0x55cdbf88b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f8603504b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbf88ba00 con 0x55cdbf87ea10 2023-07-21T22:30:49.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f8603504b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbf88bb40 con 0x55cdbf873ef0 2023-07-21T22:30:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f8603504b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdbf88bd00 con 0x55cdbf87f830 2023-07-21T22:30:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f8603504b80 10 client.0 _ll_register_callbacks cb 0x7f85e80126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f85f491a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87ea10 0x55cdbf88ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f85f491a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87ea10 0x55cdbf88ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49622/0 (socket says 172.21.15.107:49622) 2023-07-21T22:30:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f85f511b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf873ef0 0x55cdbf88e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f85f491a700 1 -- 172.21.15.107:0/1427531306 learned_addr learned my addr 172.21.15.107:0/1427531306 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:49.298 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.240+0000 7f85f591c700 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf87f830 0x55cdbf88b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85f491a700 1 -- 172.21.15.107:0/1427531306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf87f830 msgr2=0x55cdbf88b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85f491a700 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf87f830 0x55cdbf88b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85f491a700 1 -- 172.21.15.107:0/1427531306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf873ef0 msgr2=0x55cdbf88e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:49.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85f491a700 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf873ef0 0x55cdbf88e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85f491a700 1 -- 172.21.15.107:0/1427531306 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f019ddb0 con 0x55cdbf87ea10 2023-07-21T22:30:49.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85f591c700 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf87f830 0x55cdbf88b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:49.299 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85f511b700 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf873ef0 0x55cdbf88e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85f491a700 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87ea10 0x55cdbf88ef20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f85e0097b10 tx=0x7f85e0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70f1cff06389ddb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85d8ff9700 1 -- 172.21.15.107:0/1427531306 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e009a790 con 0x55cdbf87ea10 2023-07-21T22:30:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85d8ff9700 1 -- 172.21.15.107:0/1427531306 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f85e00b9210 con 0x55cdbf87ea10 2023-07-21T22:30:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f8603504b80 1 -- 172.21.15.107:0/1427531306 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85e00a3460 con 0x55cdbf87ea10 2023-07-21T22:30:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85d8ff9700 1 -- 172.21.15.107:0/1427531306 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e009f150 con 0x55cdbf87ea10 2023-07-21T22:30:49.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f8603504b80 10 client.16071 fetch_fsmap learned FSMap version 31 2023-07-21T22:30:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f8603504b80 1 -- 172.21.15.107:0/1427531306 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55cdbf624250 con 0x55cdbf87ea10 2023-07-21T22:30:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85d8ff9700 1 -- 172.21.15.107:0/1427531306 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85e00c9080 con 0x55cdbf87ea10 2023-07-21T22:30:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f85d8ff9700 1 -- 172.21.15.107:0/1427531306 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f85e00cbbc0 con 0x55cdbf87ea10 2023-07-21T22:30:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f8603504b80 10 client.16071 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:30:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f8603504b80 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f8603504b80 20 client.16071 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.241+0000 7f8603504b80 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cdbf8801a0 0x55cdbf8eab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:49.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f8603504b80 4 client.16071 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:30:49.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f8603504b80 1 -- 172.21.15.107:0/1427531306 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cdbf78b050 con 0x55cdbf8801a0 2023-07-21T22:30:49.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f85f511b700 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cdbf8801a0 0x55cdbf8eab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:49.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f85d8ff9700 10 client.16071 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:49.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f85f511b700 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cdbf8801a0 0x55cdbf8eab50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63753f633f55a9e1 server_cookie=b6ecc59a0991b557 in_seq=0 out_seq=0 2023-07-21T22:30:49.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f85d8ff9700 1 -- 172.21.15.107:0/1427531306 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cdbf78b050 con 0x55cdbf8801a0 2023-07-21T22:30:49.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f85d8ff9700 10 client.16071 handle_command_reply: tid=0 2023-07-21T22:30:49.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f8603504b80 1 client.16071 shutdown 2023-07-21T22:30:49.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f8603504b80 1 -- 172.21.15.107:0/1427531306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87ea10 msgr2=0x55cdbf88ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f8603504b80 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87ea10 0x55cdbf88ef20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f85e0097b10 tx=0x7f85e0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f8603504b80 1 -- 172.21.15.107:0/1427531306 shutdown_connections 2023-07-21T22:30:49.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f8603504b80 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cdbf8801a0 0x55cdbf8eab50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f8603504b80 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cdbf87f830 0x55cdbf88b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f8603504b80 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cdbf873ef0 0x55cdbf88e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f8603504b80 1 --2- 172.21.15.107:0/1427531306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cdbf87ea10 0x55cdbf88ef20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f8603504b80 1 -- 172.21.15.107:0/1427531306 >> 172.21.15.107:0/1427531306 conn(0x55cdbf87e640 msgr2=0x55cdbf882a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:49.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.242+0000 7f85d8ff9700 0 client.16071 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:49.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f8603504b80 1 -- 172.21.15.107:0/1427531306 shutdown_connections 2023-07-21T22:30:49.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f8603504b80 1 -- 172.21.15.107:0/1427531306 wait complete. 2023-07-21T22:30:49.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f8603504b80 20 client.16071 trim_cache size 0 max 16384 2023-07-21T22:30:49.305 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:30:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:30:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:30:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: "d037412e-e4e0-4d2c-8d6e-c7193371fe0f": { 2023-07-21T22:30:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:30:49.305 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d037412e-e4e0-4d2c-8d6e-c7193371fe0f", 2023-07-21T22:30:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:30:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:30:49.306 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:30:49.306 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:30:49.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f85f737f700 1 -- 172.21.15.107:0/4156966287 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f85dc0a1150 msgr2=0x7f85dc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f85f737f700 1 --2- 172.21.15.107:0/4156966287 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f85dc0a1150 0x7f85dc0a3600 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f85f00ffaa0 tx=0x7f85ec009260 comp rx=0 tx=0).stop 2023-07-21T22:30:49.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f85f737f700 1 -- 172.21.15.107:0/4156966287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f00ffc40 msgr2=0x7f85f019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:49.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f85f737f700 1 --2- 172.21.15.107:0/4156966287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f00ffc40 0x7f85f019a510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f85e0000c00 tx=0x7f85e0004730 comp rx=0 tx=0).stop 2023-07-21T22:30:49.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f85f737f700 1 -- 172.21.15.107:0/4156966287 shutdown_connections 2023-07-21T22:30:49.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f85f737f700 1 --2- 172.21.15.107:0/4156966287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85f01005d0 0x7f85f019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f85f737f700 1 --2- 172.21.15.107:0/4156966287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f00fea50 0x7f85f00693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f85f737f700 1 --2- 172.21.15.107:0/4156966287 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f85dc0a1150 0x7f85dc0a3600 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f85f737f700 1 --2- 172.21.15.107:0/4156966287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85f00ffc40 0x7f85f019a510 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:49.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f85f737f700 1 -- 172.21.15.107:0/4156966287 >> 172.21.15.107:0/4156966287 conn(0x7f85f00fa070 msgr2=0x7f85f00fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f85f737f700 1 -- 172.21.15.107:0/4156966287 shutdown_connections 2023-07-21T22:30:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:30:49.243+0000 7f85f737f700 1 -- 172.21.15.107:0/4156966287 wait complete. 2023-07-21T22:30:49.309 INFO:tasks.cephfs.filesystem:scrub status for tag:d037412e-e4e0-4d2c-8d6e-c7193371fe0f - {'path': '/', 'tag': 'd037412e-e4e0-4d2c-8d6e-c7193371fe0f', 'options': 'recursive,force'} 2023-07-21T22:30:49.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:49 smithi132 ceph-mon[97863]: pgmap v345: 97 pgs: 97 active+clean; 147 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 519 op/s 2023-07-21T22:30:49.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:49 smithi190 ceph-mon[98334]: pgmap v345: 97 pgs: 97 active+clean; 147 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.7 MiB/s wr, 519 op/s 2023-07-21T22:30:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub queued for path: / 2023-07-21T22:30:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:30:50.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:30:50.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:30:50.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:30:50.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:30:50.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.435 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.435 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:30:50.435 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.435 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:30:50.436 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:30:50.436 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.436 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:30:50.436 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:30:50.437 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.437 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.437 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:50 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:30:50.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub queued for path: / 2023-07-21T22:30:50.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:30:50.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:30:50.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:30:50.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:30:50.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:30:50.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:30:50.440 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:30:50.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:30:50.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:30:50.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:30:50.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:50 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:30:50.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub queued for path: / 2023-07-21T22:30:50.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:30:50.644 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:30:50.645 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.645 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:30:50.645 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:30:50.645 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.645 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:30:50.645 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.645 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.646 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:30:50.646 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.646 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:30:50.646 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:30:50.646 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.646 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:30:50.646 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:30:50.646 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.647 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:50.647 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:50 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:30:51.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:51 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:51.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:51 smithi132 ceph-mon[97863]: pgmap v346: 97 pgs: 97 active+clean; 147 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.3 MiB/s wr, 505 op/s 2023-07-21T22:30:51.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:51 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:30:51.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:51 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:30:51.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:51 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:30:51.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:51 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:51.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:51 smithi107 ceph-mon[96543]: pgmap v346: 97 pgs: 97 active+clean; 147 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.3 MiB/s wr, 505 op/s 2023-07-21T22:30:51.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:51 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:30:51.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:51 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:30:51.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:51 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:30:51.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:51 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:30:51.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:51 smithi190 ceph-mon[98334]: pgmap v346: 97 pgs: 97 active+clean; 147 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.3 MiB/s wr, 505 op/s 2023-07-21T22:30:51.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:51 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:30:51.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:51 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:30:51.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:51 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:30:52.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:52 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:52.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:52 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:52.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:52 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:30:53.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:53 smithi132 ceph-mon[97863]: pgmap v347: 97 pgs: 97 active+clean; 144 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.4 MiB/s wr, 568 op/s 2023-07-21T22:30:53.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:53 smithi107 ceph-mon[96543]: pgmap v347: 97 pgs: 97 active+clean; 144 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.4 MiB/s wr, 568 op/s 2023-07-21T22:30:53.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:53 smithi190 ceph-mon[98334]: pgmap v347: 97 pgs: 97 active+clean; 144 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.4 MiB/s wr, 568 op/s 2023-07-21T22:30:55.102 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:30:54 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-h[122867]: *** Caught signal (Segmentation fault) ** 2023-07-21T22:30:55.102 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:30:54 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-h[122867]: in thread 7f7b278a1700 thread_name:msgr-worker-0 2023-07-21T22:30:55.182 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:30:55 smithi132 systemd[1]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.h.service: Main process exited, code=exited, status=139/n/a 2023-07-21T22:30:55.184 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:55 smithi132 ceph-mon[97863]: pgmap v348: 97 pgs: 97 active+clean; 147 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.0 MiB/s wr, 565 op/s 2023-07-21T22:30:55.431 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:30:55 smithi132 systemd[1]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.h.service: Failed with result 'exit-code'. 2023-07-21T22:30:55.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:55 smithi107 ceph-mon[96543]: pgmap v348: 97 pgs: 97 active+clean; 147 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.0 MiB/s wr, 565 op/s 2023-07-21T22:30:55.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:55 smithi190 ceph-mon[98334]: pgmap v348: 97 pgs: 97 active+clean; 147 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.0 MiB/s wr, 565 op/s 2023-07-21T22:30:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.879+0000 7ff533511700 1 -- 172.21.15.107:0/3786626796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c0ffc70 msgr2=0x7ff52c1000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.879+0000 7ff533511700 1 --2- 172.21.15.107:0/3786626796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c0ffc70 0x7ff52c1000c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff528009990 tx=0x7ff528009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.879+0000 7ff533511700 1 -- 172.21.15.107:0/3786626796 shutdown_connections 2023-07-21T22:30:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.879+0000 7ff533511700 1 --2- 172.21.15.107:0/3786626796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c100600 0x7ff52c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.879+0000 7ff533511700 1 --2- 172.21.15.107:0/3786626796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff52c0fea80 0x7ff52c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.879+0000 7ff533511700 1 --2- 172.21.15.107:0/3786626796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c0ffc70 0x7ff52c1000c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.879+0000 7ff533511700 1 -- 172.21.15.107:0/3786626796 >> 172.21.15.107:0/3786626796 conn(0x7ff52c0fa0a0 msgr2=0x7ff52c0fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.879+0000 7ff533511700 1 -- 172.21.15.107:0/3786626796 shutdown_connections 2023-07-21T22:30:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff533511700 1 -- 172.21.15.107:0/3786626796 wait complete. 2023-07-21T22:30:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff533511700 1 Processor -- start 2023-07-21T22:30:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff533511700 1 -- start start 2023-07-21T22:30:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff533511700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0fea80 0x7ff52c194830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff533511700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff52c0ffc70 0x7ff52c194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff533511700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c100600 0x7ff52c195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff533511700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c196250 con 0x7ff52c100600 2023-07-21T22:30:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff533511700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c1963c0 con 0x7ff52c0ffc70 2023-07-21T22:30:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff533511700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c196580 con 0x7ff52c0fea80 2023-07-21T22:30:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff531aae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c100600 0x7ff52c195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff5312ad700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0fea80 0x7ff52c194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff531aae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c100600 0x7ff52c195bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49640/0 (socket says 172.21.15.107:49640) 2023-07-21T22:30:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff531aae700 1 -- 172.21.15.107:0/2532063135 learned_addr learned my addr 172.21.15.107:0/2532063135 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.880+0000 7ff530aac700 1 --2- 172.21.15.107:0/2532063135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff52c0ffc70 0x7ff52c194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.881+0000 7ff531aae700 1 -- 172.21.15.107:0/2532063135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0fea80 msgr2=0x7ff52c194830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.881+0000 7ff531aae700 1 --2- 172.21.15.107:0/2532063135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0fea80 0x7ff52c194830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.881+0000 7ff531aae700 1 -- 172.21.15.107:0/2532063135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff52c0ffc70 msgr2=0x7ff52c194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.881+0000 7ff531aae700 1 --2- 172.21.15.107:0/2532063135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff52c0ffc70 0x7ff52c194d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.881+0000 7ff531aae700 1 -- 172.21.15.107:0/2532063135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5280095f0 con 0x7ff52c100600 2023-07-21T22:30:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.881+0000 7ff531aae700 1 --2- 172.21.15.107:0/2532063135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c100600 0x7ff52c195bf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff52400ef30 tx=0x7ff52400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.881+0000 7ff5227fc700 1 -- 172.21.15.107:0/2532063135 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524018070 con 0x7ff52c100600 2023-07-21T22:30:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.881+0000 7ff533511700 1 -- 172.21.15.107:0/2532063135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c196800 con 0x7ff52c100600 2023-07-21T22:30:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.881+0000 7ff5227fc700 1 -- 172.21.15.107:0/2532063135 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff52400f040 con 0x7ff52c100600 2023-07-21T22:30:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.881+0000 7ff5227fc700 1 -- 172.21.15.107:0/2532063135 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52400cd00 con 0x7ff52c100600 2023-07-21T22:30:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.881+0000 7ff533511700 1 -- 172.21.15.107:0/2532063135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c19d020 con 0x7ff52c100600 2023-07-21T22:30:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.883+0000 7ff5227fc700 1 -- 172.21.15.107:0/2532063135 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff524003a00 con 0x7ff52c100600 2023-07-21T22:30:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.884+0000 7ff5227fc700 1 --2- 172.21.15.107:0/2532063135 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff5180a1150 0x7ff5180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.884+0000 7ff5312ad700 1 --2- 172.21.15.107:0/2532063135 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff5180a1150 0x7ff5180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.884+0000 7ff5227fc700 1 -- 172.21.15.107:0/2532063135 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff5240cc4e0 con 0x7ff52c100600 2023-07-21T22:30:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.884+0000 7ff5312ad700 1 --2- 172.21.15.107:0/2532063135 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff5180a1150 0x7ff5180a3600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0ffad0 tx=0x7ff51c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff53f696b80 1 Processor -- start 2023-07-21T22:30:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff53f696b80 1 -- start start 2023-07-21T22:30:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff53f696b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764d226b0 0x563764ec8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff53f696b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 0x563764ec9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff53f696b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764ec9860 0x563764ec9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff53f696b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563764dcbc50 con 0x563764ec89a0 2023-07-21T22:30:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff53f696b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563764e1e9e0 con 0x563764d226b0 2023-07-21T22:30:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff53f696b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563764e17e70 con 0x563764ec9860 2023-07-21T22:30:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff530aac700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 0x563764ec9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff5312ad700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764d226b0 0x563764ec8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff530aac700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 0x563764ec9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49646/0 (socket says 172.21.15.107:49646) 2023-07-21T22:30:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff531aae700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764ec9860 0x563764ec9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff5312ad700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764d226b0 0x563764ec8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40472/0 (socket says 172.21.15.107:40472) 2023-07-21T22:30:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff530aac700 1 -- 172.21.15.107:0/3991924980 learned_addr learned my addr 172.21.15.107:0/3991924980 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff530aac700 1 -- 172.21.15.107:0/3991924980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764ec9860 msgr2=0x563764ec9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff530aac700 1 --2- 172.21.15.107:0/3991924980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764ec9860 0x563764ec9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff530aac700 1 -- 172.21.15.107:0/3991924980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764d226b0 msgr2=0x563764ec8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff530aac700 1 --2- 172.21.15.107:0/3991924980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764d226b0 0x563764ec8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff530aac700 1 -- 172.21.15.107:0/3991924980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563764c45d80 con 0x563764ec89a0 2023-07-21T22:30:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff531aae700 1 --2- 172.21.15.107:0/3991924980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764ec9860 0x563764ec9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.885+0000 7ff530aac700 1 --2- 172.21.15.107:0/3991924980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 0x563764ec9320 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff528009990 tx=0x7ff528005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39486a7a9d6c769d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:55.941 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.886+0000 7ff5177fe700 1 -- 172.21.15.107:0/3991924980 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52801d070 con 0x563764ec89a0 2023-07-21T22:30:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.886+0000 7ff5177fe700 1 -- 172.21.15.107:0/3991924980 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff52801e040 con 0x563764ec89a0 2023-07-21T22:30:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.886+0000 7ff5177fe700 1 -- 172.21.15.107:0/3991924980 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528004920 con 0x563764ec89a0 2023-07-21T22:30:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.886+0000 7ff53f696b80 1 -- 172.21.15.107:0/3991924980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 msgr2=0x563764ec9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.886+0000 7ff53f696b80 1 --2- 172.21.15.107:0/3991924980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 0x563764ec9320 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff528009990 tx=0x7ff528005bc0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.886+0000 7ff53f696b80 1 -- 172.21.15.107:0/3991924980 shutdown_connections 2023-07-21T22:30:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.886+0000 7ff53f696b80 1 --2- 172.21.15.107:0/3991924980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764ec9860 0x563764ec9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.942 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.886+0000 7ff53f696b80 1 --2- 172.21.15.107:0/3991924980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764d226b0 0x563764ec8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.886+0000 7ff53f696b80 1 --2- 172.21.15.107:0/3991924980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 0x563764ec9320 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.886+0000 7ff53f696b80 1 -- 172.21.15.107:0/3991924980 >> 172.21.15.107:0/3991924980 conn(0x563764ebbae0 msgr2=0x563764ec0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.886+0000 7ff53f696b80 1 -- 172.21.15.107:0/3991924980 shutdown_connections 2023-07-21T22:30:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.886+0000 7ff53f696b80 1 -- 172.21.15.107:0/3991924980 wait complete. 2023-07-21T22:30:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff53f696b80 1 -- start start 2023-07-21T22:30:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff53f696b80 1 Processor -- start 2023-07-21T22:30:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff53f696b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764d226b0 0x563764ed18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff53f696b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec89a0 0x563764ed1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff53f696b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec9860 0x563764ed8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff53f696b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563764db7a60 con 0x563764ec9860 2023-07-21T22:30:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff53f696b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563764db7760 con 0x563764ec89a0 2023-07-21T22:30:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff53f696b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563764c9b8c0 con 0x563764d226b0 2023-07-21T22:30:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff53f696b80 10 client.0 _ll_register_callbacks cb 0x563764db5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff531aae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec9860 0x563764ed8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.944 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff531aae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec9860 0x563764ed8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49654/0 (socket says 172.21.15.107:49654) 2023-07-21T22:30:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff531aae700 1 -- 172.21.15.107:0/815277199 learned_addr learned my addr 172.21.15.107:0/815277199 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff5312ad700 1 --2- 172.21.15.107:0/815277199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764d226b0 0x563764ed18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.887+0000 7ff530aac700 1 --2- 172.21.15.107:0/815277199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec89a0 0x563764ed1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff531aae700 1 -- 172.21.15.107:0/815277199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764d226b0 msgr2=0x563764ed18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff531aae700 1 --2- 172.21.15.107:0/815277199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764d226b0 0x563764ed18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff531aae700 1 -- 172.21.15.107:0/815277199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec89a0 msgr2=0x563764ed1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff531aae700 1 --2- 172.21.15.107:0/815277199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec89a0 0x563764ed1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff531aae700 1 -- 172.21.15.107:0/815277199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c19d020 con 0x563764ec9860 2023-07-21T22:30:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff531aae700 1 --2- 172.21.15.107:0/815277199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec9860 0x563764ed8e60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff52400c6f0 tx=0x7ff5240997c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf3621ec9914dd8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff514ff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff514ff9700 1 -- 172.21.15.107:0/815277199 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5240b7ea0 con 0x563764ec9860 2023-07-21T22:30:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff514ff9700 1 -- 172.21.15.107:0/815277199 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff524099900 con 0x563764ec9860 2023-07-21T22:30:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff53f696b80 1 -- 172.21.15.107:0/815277199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563764d71490 con 0x563764ec9860 2023-07-21T22:30:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff514ff9700 1 -- 172.21.15.107:0/815277199 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff52409d930 con 0x563764ec9860 2023-07-21T22:30:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff53f696b80 10 client.16083 fetch_fsmap learned FSMap version 32 2023-07-21T22:30:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff53f696b80 1 -- 172.21.15.107:0/815277199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7ff52c1077f0 con 0x563764ec9860 2023-07-21T22:30:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff514ff9700 1 -- 172.21.15.107:0/815277199 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5240c6020 con 0x563764ec9860 2023-07-21T22:30:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.888+0000 7ff514ff9700 1 -- 172.21.15.107:0/815277199 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff5240b5030 con 0x563764ec9860 2023-07-21T22:30:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.889+0000 7ff53f696b80 10 client.16083 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:30:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.889+0000 7ff53f696b80 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.889+0000 7ff53f696b80 20 client.16083 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.889+0000 7ff53f696b80 1 --2- 172.21.15.107:0/815277199 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563764edd7b0 0x563764edfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.889+0000 7ff53f696b80 4 client.16083 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.889+0000 7ff53f696b80 1 -- 172.21.15.107:0/815277199 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563764e10e50 con 0x563764edd7b0 2023-07-21T22:30:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.889+0000 7ff5312ad700 1 --2- 172.21.15.107:0/815277199 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563764edd7b0 0x563764edfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.889+0000 7ff5312ad700 1 --2- 172.21.15.107:0/815277199 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563764edd7b0 0x563764edfbc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53014c81a887bdac server_cookie=31a3bcab22e0c0a5 in_seq=0 out_seq=0 2023-07-21T22:30:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.889+0000 7ff514ff9700 10 client.16083 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff514ff9700 1 -- 172.21.15.107:0/815277199 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563764e10e50 con 0x563764edd7b0 2023-07-21T22:30:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff514ff9700 10 client.16083 handle_command_reply: tid=0 2023-07-21T22:30:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff53f696b80 1 client.16083 shutdown 2023-07-21T22:30:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff53f696b80 1 -- 172.21.15.107:0/815277199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec9860 msgr2=0x563764ed8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff53f696b80 1 --2- 172.21.15.107:0/815277199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec9860 0x563764ed8e60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff52400c6f0 tx=0x7ff5240997c0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff53f696b80 1 -- 172.21.15.107:0/815277199 shutdown_connections 2023-07-21T22:30:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff53f696b80 1 --2- 172.21.15.107:0/815277199 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563764edd7b0 0x563764edfbc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff53f696b80 1 --2- 172.21.15.107:0/815277199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec9860 0x563764ed8e60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff53f696b80 1 --2- 172.21.15.107:0/815277199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764d226b0 0x563764ed18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff53f696b80 1 --2- 172.21.15.107:0/815277199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec89a0 0x563764ed1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff514ff9700 0 client.16083 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff53f696b80 1 -- 172.21.15.107:0/815277199 >> 172.21.15.107:0/815277199 conn(0x563764ebbae0 msgr2=0x563764ec03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff53f696b80 1 -- 172.21.15.107:0/815277199 shutdown_connections 2023-07-21T22:30:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff53f696b80 1 -- 172.21.15.107:0/815277199 wait complete. 2023-07-21T22:30:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.890+0000 7ff53f696b80 20 client.16083 trim_cache size 0 max 16384 2023-07-21T22:30:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.905+0000 7ff53f696b80 1 Processor -- start 2023-07-21T22:30:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.905+0000 7ff53f696b80 1 -- start start 2023-07-21T22:30:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.905+0000 7ff53f696b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764d226b0 0x563764ec9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.905+0000 7ff53f696b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764ec89a0 0x563764ed8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.905+0000 7ff53f696b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec9860 0x563764ec0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.905+0000 7ff53f696b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563764ed0d90 con 0x563764d226b0 2023-07-21T22:30:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.905+0000 7ff53f696b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563764ed1450 con 0x563764ec9860 2023-07-21T22:30:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.905+0000 7ff53f696b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563764ec91b0 con 0x563764ec89a0 2023-07-21T22:30:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.905+0000 7ff5312ad700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764d226b0 0x563764ec9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.905+0000 7ff5312ad700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764d226b0 0x563764ec9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49658/0 (socket says 172.21.15.107:49658) 2023-07-21T22:30:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.905+0000 7ff5312ad700 1 -- 172.21.15.107:0/204069560 learned_addr learned my addr 172.21.15.107:0/204069560 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.905+0000 7ff530aac700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764ec89a0 0x563764ed8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.905+0000 7ff531aae700 1 --2- 172.21.15.107:0/204069560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec9860 0x563764ec0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.906+0000 7ff5312ad700 1 -- 172.21.15.107:0/204069560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764ec89a0 msgr2=0x563764ed8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.906+0000 7ff5312ad700 1 --2- 172.21.15.107:0/204069560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764ec89a0 0x563764ed8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.906+0000 7ff5312ad700 1 -- 172.21.15.107:0/204069560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec9860 msgr2=0x563764ec0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.906+0000 7ff5312ad700 1 --2- 172.21.15.107:0/204069560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec9860 0x563764ec0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.906+0000 7ff5312ad700 1 -- 172.21.15.107:0/204069560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563764c2ec00 con 0x563764d226b0 2023-07-21T22:30:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.906+0000 7ff530aac700 1 --2- 172.21.15.107:0/204069560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764ec89a0 0x563764ed8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.906+0000 7ff5312ad700 1 --2- 172.21.15.107:0/204069560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764d226b0 0x563764ec9c30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff51c013010 tx=0x7ff51c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736d7c64e44f62d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.906+0000 7ff517fff700 1 -- 172.21.15.107:0/204069560 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c022040 con 0x563764d226b0 2023-07-21T22:30:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.906+0000 7ff517fff700 1 -- 172.21.15.107:0/204069560 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff51c023040 con 0x563764d226b0 2023-07-21T22:30:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.906+0000 7ff517fff700 1 -- 172.21.15.107:0/204069560 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c01da50 con 0x563764d226b0 2023-07-21T22:30:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.907+0000 7ff53f696b80 1 -- 172.21.15.107:0/204069560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764d226b0 msgr2=0x563764ec9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:55.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.907+0000 7ff53f696b80 1 --2- 172.21.15.107:0/204069560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764d226b0 0x563764ec9c30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff51c013010 tx=0x7ff51c014000 comp rx=0 tx=0).stop 2023-07-21T22:30:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.907+0000 7ff53f696b80 1 -- 172.21.15.107:0/204069560 shutdown_connections 2023-07-21T22:30:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.907+0000 7ff53f696b80 1 --2- 172.21.15.107:0/204069560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec9860 0x563764ec0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.907+0000 7ff53f696b80 1 --2- 172.21.15.107:0/204069560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764d226b0 0x563764ec9c30 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.907+0000 7ff53f696b80 1 --2- 172.21.15.107:0/204069560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764ec89a0 0x563764ed8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.907+0000 7ff53f696b80 1 -- 172.21.15.107:0/204069560 >> 172.21.15.107:0/204069560 conn(0x563764ebbae0 msgr2=0x563764ee44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.907+0000 7ff53f696b80 1 -- 172.21.15.107:0/204069560 shutdown_connections 2023-07-21T22:30:55.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.907+0000 7ff53f696b80 1 -- 172.21.15.107:0/204069560 wait complete. 2023-07-21T22:30:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.907+0000 7ff53f696b80 1 -- start start 2023-07-21T22:30:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.907+0000 7ff53f696b80 1 Processor -- start 2023-07-21T22:30:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff53f696b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764d226b0 0x563764ed4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff53f696b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 0x563764ed5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff53f696b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec9860 0x563764ed8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff53f696b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563764ed88d0 con 0x563764ec89a0 2023-07-21T22:30:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff53f696b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563764ed8a40 con 0x563764ec9860 2023-07-21T22:30:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff53f696b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563764ed8c00 con 0x563764d226b0 2023-07-21T22:30:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff53f696b80 10 client.0 _ll_register_callbacks cb 0x7ff524099260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff530aac700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 0x563764ed5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff530aac700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 0x563764ed5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49666/0 (socket says 172.21.15.107:49666) 2023-07-21T22:30:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff530aac700 1 -- 172.21.15.107:0/3889370948 learned_addr learned my addr 172.21.15.107:0/3889370948 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:30:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff531aae700 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec9860 0x563764ed8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff5312ad700 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764d226b0 0x563764ed4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff530aac700 1 -- 172.21.15.107:0/3889370948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764d226b0 msgr2=0x563764ed4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff530aac700 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764d226b0 0x563764ed4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff530aac700 1 -- 172.21.15.107:0/3889370948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec9860 msgr2=0x563764ed8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff530aac700 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec9860 0x563764ed8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff530aac700 1 -- 172.21.15.107:0/3889370948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563764c45d80 con 0x563764ec89a0 2023-07-21T22:30:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff5312ad700 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764d226b0 0x563764ed4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff530aac700 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 0x563764ed5370 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff528009990 tx=0x7ff52800ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3511047b0c5ffe1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:55.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff514ff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:30:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff514ff9700 1 -- 172.21.15.107:0/3889370948 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52801d040 con 0x563764ec89a0 2023-07-21T22:30:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff514ff9700 1 -- 172.21.15.107:0/3889370948 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff528004700 con 0x563764ec89a0 2023-07-21T22:30:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.908+0000 7ff53f696b80 1 -- 172.21.15.107:0/3889370948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff52801e040 con 0x563764ec89a0 2023-07-21T22:30:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff514ff9700 1 -- 172.21.15.107:0/3889370948 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5280048c0 con 0x563764ec89a0 2023-07-21T22:30:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff53f696b80 10 client.16089 fetch_fsmap learned FSMap version 32 2023-07-21T22:30:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff53f696b80 1 -- 172.21.15.107:0/3889370948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7ff51c017070 con 0x563764ec89a0 2023-07-21T22:30:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff514ff9700 1 -- 172.21.15.107:0/3889370948 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff52801e040 con 0x563764ec89a0 2023-07-21T22:30:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff514ff9700 1 -- 172.21.15.107:0/3889370948 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff52800faa0 con 0x563764ec89a0 2023-07-21T22:30:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff53f696b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:30:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff53f696b80 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff53f696b80 20 client.16089 populate_metadata read hostname 'smithi107' 2023-07-21T22:30:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff53f696b80 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563764f322f0 0x563764f346d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff53f696b80 4 client.16089 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:30:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff53f696b80 1 -- 172.21.15.107:0/3889370948 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563764dd2ff0 con 0x563764f322f0 2023-07-21T22:30:55.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff5312ad700 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563764f322f0 0x563764f346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:55.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff514ff9700 10 client.16089 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:55.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:55.909+0000 7ff5312ad700 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563764f322f0 0x563764f346d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd5a884cb0248de server_cookie=5b04866a2018be3f in_seq=0 out_seq=0 2023-07-21T22:30:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.083+0000 7ff514ff9700 1 -- 172.21.15.107:0/3889370948 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563764dd2ff0 con 0x563764f322f0 2023-07-21T22:30:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.083+0000 7ff514ff9700 10 client.16089 handle_command_reply: tid=0 2023-07-21T22:30:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.083+0000 7ff53f696b80 1 client.16089 shutdown 2023-07-21T22:30:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.084+0000 7ff53f696b80 1 -- 172.21.15.107:0/3889370948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 msgr2=0x563764ed5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.084+0000 7ff53f696b80 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 0x563764ed5370 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff528009990 tx=0x7ff52800ba20 comp rx=0 tx=0).stop 2023-07-21T22:30:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.084+0000 7ff53f696b80 1 -- 172.21.15.107:0/3889370948 shutdown_connections 2023-07-21T22:30:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.084+0000 7ff53f696b80 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563764f322f0 0x563764f346d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.084+0000 7ff53f696b80 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563764ec9860 0x563764ed8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.084+0000 7ff514ff9700 0 client.16089 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:30:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.084+0000 7ff53f696b80 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563764d226b0 0x563764ed4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.084+0000 7ff53f696b80 1 --2- 172.21.15.107:0/3889370948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563764ec89a0 0x563764ed5370 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.084+0000 7ff53f696b80 1 -- 172.21.15.107:0/3889370948 >> 172.21.15.107:0/3889370948 conn(0x563764ebbae0 msgr2=0x563764ecfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.084+0000 7ff53f696b80 1 -- 172.21.15.107:0/3889370948 shutdown_connections 2023-07-21T22:30:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.084+0000 7ff53f696b80 1 -- 172.21.15.107:0/3889370948 wait complete. 2023-07-21T22:30:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.084+0000 7ff53f696b80 20 client.16089 trim_cache size 0 max 16384 2023-07-21T22:30:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:30:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:30:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:30:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:30:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.085+0000 7ff533511700 1 -- 172.21.15.107:0/2532063135 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff5180a1150 msgr2=0x7ff5180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.085+0000 7ff533511700 1 --2- 172.21.15.107:0/2532063135 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff5180a1150 0x7ff5180a3600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0ffad0 tx=0x7ff51c009260 comp rx=0 tx=0).stop 2023-07-21T22:30:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.085+0000 7ff533511700 1 -- 172.21.15.107:0/2532063135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c100600 msgr2=0x7ff52c195bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.085+0000 7ff533511700 1 --2- 172.21.15.107:0/2532063135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c100600 0x7ff52c195bf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff52400ef30 tx=0x7ff52400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:30:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.085+0000 7ff533511700 1 -- 172.21.15.107:0/2532063135 shutdown_connections 2023-07-21T22:30:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.085+0000 7ff533511700 1 --2- 172.21.15.107:0/2532063135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c100600 0x7ff52c195bf0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.085+0000 7ff533511700 1 --2- 172.21.15.107:0/2532063135 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff5180a1150 0x7ff5180a3600 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.085+0000 7ff533511700 1 --2- 172.21.15.107:0/2532063135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0fea80 0x7ff52c194830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.085+0000 7ff533511700 1 --2- 172.21.15.107:0/2532063135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff52c0ffc70 0x7ff52c194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.085+0000 7ff533511700 1 -- 172.21.15.107:0/2532063135 >> 172.21.15.107:0/2532063135 conn(0x7ff52c0fa0a0 msgr2=0x7ff52c0fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.085+0000 7ff533511700 1 -- 172.21.15.107:0/2532063135 shutdown_connections 2023-07-21T22:30:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:30:56.085+0000 7ff533511700 1 -- 172.21.15.107:0/2532063135 wait complete. 2023-07-21T22:30:56.474 INFO:tasks.workunit.client.0.smithi107.stdout: 31 224063 170 149947 205 100611 591 2023-07-21T22:30:57.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:57 smithi190 ceph-mon[98334]: pgmap v349: 97 pgs: 97 active+clean; 147 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.6 MiB/s wr, 549 op/s 2023-07-21T22:30:57.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:57 smithi132 ceph-mon[97863]: pgmap v349: 97 pgs: 97 active+clean; 147 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.6 MiB/s wr, 549 op/s 2023-07-21T22:30:57.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:57 smithi107 ceph-mon[96543]: pgmap v349: 97 pgs: 97 active+clean; 147 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.6 MiB/s wr, 549 op/s 2023-07-21T22:30:59.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:30:59 smithi190 ceph-mon[98334]: pgmap v350: 97 pgs: 97 active+clean; 152 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 441 op/s 2023-07-21T22:30:59.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:30:59 smithi132 ceph-mon[97863]: pgmap v350: 97 pgs: 97 active+clean; 152 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 441 op/s 2023-07-21T22:30:59.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:30:59 smithi107 ceph-mon[96543]: pgmap v350: 97 pgs: 97 active+clean; 152 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 441 op/s 2023-07-21T22:31:01.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:01 smithi190 ceph-mon[98334]: pgmap v351: 97 pgs: 97 active+clean; 152 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 930 KiB/s wr, 148 op/s 2023-07-21T22:31:01.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:01 smithi132 ceph-mon[97863]: pgmap v351: 97 pgs: 97 active+clean; 152 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 930 KiB/s wr, 148 op/s 2023-07-21T22:31:01.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:01 smithi107 ceph-mon[96543]: pgmap v351: 97 pgs: 97 active+clean; 152 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 930 KiB/s wr, 148 op/s 2023-07-21T22:31:03.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:03 smithi190 ceph-mon[98334]: pgmap v352: 97 pgs: 97 active+clean; 155 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 245 op/s 2023-07-21T22:31:03.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:03 smithi132 ceph-mon[97863]: pgmap v352: 97 pgs: 97 active+clean; 155 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 245 op/s 2023-07-21T22:31:03.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:03 smithi107 ceph-mon[96543]: pgmap v352: 97 pgs: 97 active+clean; 155 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 245 op/s 2023-07-21T22:31:05.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:05 smithi190 ceph-mon[98334]: pgmap v353: 97 pgs: 97 active+clean; 157 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 190 op/s 2023-07-21T22:31:05.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:05 smithi132 ceph-mon[97863]: pgmap v353: 97 pgs: 97 active+clean; 157 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 190 op/s 2023-07-21T22:31:05.682 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:31:05 smithi132 systemd[1]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.h.service: Service RestartSec=10s expired, scheduling restart. 2023-07-21T22:31:05.682 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:31:05 smithi132 systemd[1]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.h.service: Scheduled restart job, restart counter is at 1. 2023-07-21T22:31:05.682 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:31:05 smithi132 systemd[1]: Stopped Ceph mds.h for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:31:05.682 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:31:05 smithi132 systemd[1]: Starting Ceph mds.h for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:31:05.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:05 smithi107 ceph-mon[96543]: pgmap v353: 97 pgs: 97 active+clean; 157 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 190 op/s 2023-07-21T22:31:05.999 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:31:05 smithi132 podman[125448]: 2023-07-21T22:31:06.432 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:31:05 smithi132 ceph-mds[125488]: -- 172.21.15.132:0/1268012110 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561cf13a91e0 con 0x561cf137f000 2023-07-21T22:31:06.432 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:31:05 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-h[125464]: starting mds.h at 2023-07-21T22:31:06.433 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:31:05 smithi132 bash[125448]: 95e6ac014905e24f242a07ccd0ed8e6685e5261599637c79a17aff7ae9e6e4f5 2023-07-21T22:31:06.433 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:31:05 smithi132 systemd[1]: Started Ceph mds.h for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:31:06.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:06 smithi132 ceph-mon[97863]: daemon mds.h restarted 2023-07-21T22:31:06.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:06 smithi132 ceph-mon[97863]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T22:31:06.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:06 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.132:6836/2828468744,v1:172.21.15.132:6837/2828468744] up:boot 2023-07-21T22:31:06.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:06 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:06.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:06 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:31:06.474 INFO:tasks.workunit.client.0.smithi107.stdout: 48 470815 904 325961 857 234092 909 2023-07-21T22:31:06.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:06 smithi190 ceph-mon[98334]: daemon mds.h restarted 2023-07-21T22:31:06.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:06 smithi190 ceph-mon[98334]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T22:31:06.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:06 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.132:6836/2828468744,v1:172.21.15.132:6837/2828468744] up:boot 2023-07-21T22:31:06.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:06 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:06.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:06 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:31:06.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:06 smithi107 ceph-mon[96543]: daemon mds.h restarted 2023-07-21T22:31:06.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:06 smithi107 ceph-mon[96543]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T22:31:06.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:06 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.132:6836/2828468744,v1:172.21.15.132:6837/2828468744] up:boot 2023-07-21T22:31:06.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:06 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:06.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:06 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:31:07.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:07 smithi190 ceph-mon[98334]: pgmap v354: 97 pgs: 97 active+clean; 159 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2023-07-21T22:31:07.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:07 smithi132 ceph-mon[97863]: pgmap v354: 97 pgs: 97 active+clean; 159 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2023-07-21T22:31:07.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:07 smithi107 ceph-mon[96543]: pgmap v354: 97 pgs: 97 active+clean; 159 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2023-07-21T22:31:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.209+0000 7fd52e3f9700 1 -- 172.21.15.107:0/1404539745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528100580 msgr2=0x7fd528106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.209+0000 7fd52e3f9700 1 --2- 172.21.15.107:0/1404539745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528100580 0x7fd528106e40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd52000b0a0 tx=0x7fd52000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.209+0000 7fd52e3f9700 1 -- 172.21.15.107:0/1404539745 shutdown_connections 2023-07-21T22:31:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.209+0000 7fd52e3f9700 1 --2- 172.21.15.107:0/1404539745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528100580 0x7fd528106e40 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.209+0000 7fd52e3f9700 1 --2- 172.21.15.107:0/1404539745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5280fea00 0x7fd5280fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.209+0000 7fd52e3f9700 1 --2- 172.21.15.107:0/1404539745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5280ffbf0 0x7fd528100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.209+0000 7fd52e3f9700 1 -- 172.21.15.107:0/1404539745 >> 172.21.15.107:0/1404539745 conn(0x7fd5280f9fe0 msgr2=0x7fd5280fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.209+0000 7fd52e3f9700 1 -- 172.21.15.107:0/1404539745 shutdown_connections 2023-07-21T22:31:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.209+0000 7fd52e3f9700 1 -- 172.21.15.107:0/1404539745 wait complete. 2023-07-21T22:31:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.209+0000 7fd52e3f9700 1 Processor -- start 2023-07-21T22:31:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.209+0000 7fd52e3f9700 1 -- start start 2023-07-21T22:31:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd52e3f9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5280fea00 0x7fd52819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd52e3f9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5280ffbf0 0x7fd52819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.259 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd52e3f9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528100580 0x7fd52819f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd52e3f9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5280693f0 con 0x7fd528100580 2023-07-21T22:31:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd52e3f9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52819fa70 con 0x7fd5280fea00 2023-07-21T22:31:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd52e3f9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52819fc00 con 0x7fd5280ffbf0 2023-07-21T22:31:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd527fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5280fea00 0x7fd52819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd527fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5280fea00 0x7fd52819aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40504/0 (socket says 172.21.15.107:40504) 2023-07-21T22:31:09.260 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd527fff700 1 -- 172.21.15.107:0/2148346063 learned_addr learned my addr 172.21.15.107:0/2148346063 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd52c996700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528100580 0x7fd52819f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd5277fe700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5280ffbf0 0x7fd52819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd527fff700 1 -- 172.21.15.107:0/2148346063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5280ffbf0 msgr2=0x7fd52819cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd527fff700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5280ffbf0 0x7fd52819cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd527fff700 1 -- 172.21.15.107:0/2148346063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528100580 msgr2=0x7fd52819f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd527fff700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528100580 0x7fd52819f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd527fff700 1 -- 172.21.15.107:0/2148346063 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd520009d00 con 0x7fd5280fea00 2023-07-21T22:31:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd52c996700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528100580 0x7fd52819f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd5277fe700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5280ffbf0 0x7fd52819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.210+0000 7fd527fff700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5280fea00 0x7fd52819aa90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00e950 tx=0x7fd51c00ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.211+0000 7fd5257fa700 1 -- 172.21.15.107:0/2148346063 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c009770 con 0x7fd5280fea00 2023-07-21T22:31:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.211+0000 7fd5257fa700 1 -- 172.21.15.107:0/2148346063 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd51c00c7e0 con 0x7fd5280fea00 2023-07-21T22:31:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.211+0000 7fd52e3f9700 1 -- 172.21.15.107:0/2148346063 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52819fee0 con 0x7fd5280fea00 2023-07-21T22:31:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.211+0000 7fd5257fa700 1 -- 172.21.15.107:0/2148346063 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c003cf0 con 0x7fd5280fea00 2023-07-21T22:31:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.211+0000 7fd52e3f9700 1 -- 172.21.15.107:0/2148346063 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5281a02b0 con 0x7fd5280fea00 2023-07-21T22:31:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.213+0000 7fd5257fa700 1 -- 172.21.15.107:0/2148346063 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd51c00c950 con 0x7fd5280fea00 2023-07-21T22:31:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.213+0000 7fd5257fa700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd5100a1150 0x7fd5100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd5277fe700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd5100a1150 0x7fd5100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd5257fa700 1 -- 172.21.15.107:0/2148346063 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd51c014070 con 0x7fd5280fea00 2023-07-21T22:31:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd53a57eb80 1 Processor -- start 2023-07-21T22:31:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd5277fe700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd5100a1150 0x7fd5100a3600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd528068950 tx=0x7fd518009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd53a57eb80 1 -- start start 2023-07-21T22:31:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd53a57eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 0x55705349f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd53a57eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534a0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd53a57eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570534a0580 0x5570534a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd53a57eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570533a4c50 con 0x5570532fb6b0 2023-07-21T22:31:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd53a57eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570533f79e0 con 0x55705349f830 2023-07-21T22:31:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd53a57eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570533f0e70 con 0x5570534a0580 2023-07-21T22:31:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd527fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 0x55705349f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd527fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 0x55705349f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49684/0 (socket says 172.21.15.107:49684) 2023-07-21T22:31:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd527fff700 1 -- 172.21.15.107:0/4162581693 learned_addr learned my addr 172.21.15.107:0/4162581693 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd5277fe700 1 --2- 172.21.15.107:0/4162581693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd52c996700 1 --2- 172.21.15.107:0/4162581693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570534a0580 0x5570534a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd527fff700 1 -- 172.21.15.107:0/4162581693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570534a0580 msgr2=0x5570534a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.214+0000 7fd527fff700 1 --2- 172.21.15.107:0/4162581693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570534a0580 0x5570534a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd527fff700 1 -- 172.21.15.107:0/4162581693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 msgr2=0x5570534a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd527fff700 1 --2- 172.21.15.107:0/4162581693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd527fff700 1 -- 172.21.15.107:0/4162581693 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55705321ed80 con 0x5570532fb6b0 2023-07-21T22:31:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd527fff700 1 --2- 172.21.15.107:0/4162581693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 0x55705349f2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd51c095c90 tx=0x7fd51c003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf3132aa1139f50 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd50e7fc700 1 -- 172.21.15.107:0/4162581693 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c09df40 con 0x5570532fb6b0 2023-07-21T22:31:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd50e7fc700 1 -- 172.21.15.107:0/4162581693 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd51c0a2360 con 0x5570532fb6b0 2023-07-21T22:31:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd50e7fc700 1 -- 172.21.15.107:0/4162581693 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c099400 con 0x5570532fb6b0 2023-07-21T22:31:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd53a57eb80 1 -- 172.21.15.107:0/4162581693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 msgr2=0x55705349f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/4162581693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 0x55705349f2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd51c095c90 tx=0x7fd51c003710 comp rx=0 tx=0).stop 2023-07-21T22:31:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd53a57eb80 1 -- 172.21.15.107:0/4162581693 shutdown_connections 2023-07-21T22:31:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/4162581693 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570534a0580 0x5570534a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/4162581693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 0x55705349f2f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/4162581693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd53a57eb80 1 -- 172.21.15.107:0/4162581693 >> 172.21.15.107:0/4162581693 conn(0x557053494ae0 msgr2=0x557053499c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.215+0000 7fd53a57eb80 1 -- 172.21.15.107:0/4162581693 shutdown_connections 2023-07-21T22:31:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.216+0000 7fd53a57eb80 1 -- 172.21.15.107:0/4162581693 wait complete. 2023-07-21T22:31:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.216+0000 7fd53a57eb80 1 -- start start 2023-07-21T22:31:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.216+0000 7fd53a57eb80 1 Processor -- start 2023-07-21T22:31:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.216+0000 7fd53a57eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 0x5570534a8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.216+0000 7fd53a57eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534a8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.216+0000 7fd53a57eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570534a0580 0x5570534afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.216+0000 7fd53a57eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557053390a60 con 0x5570532fb6b0 2023-07-21T22:31:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd53a57eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557053390760 con 0x55705349f830 2023-07-21T22:31:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd53a57eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570532748c0 con 0x5570534a0580 2023-07-21T22:31:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd527fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 0x5570534a8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd53a57eb80 10 client.0 _ll_register_callbacks cb 0x55705338e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd527fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 0x5570534a8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49690/0 (socket says 172.21.15.107:49690) 2023-07-21T22:31:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd527fff700 1 -- 172.21.15.107:0/2079477265 learned_addr learned my addr 172.21.15.107:0/2079477265 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd5277fe700 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534a8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd52c996700 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570534a0580 0x5570534afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd527fff700 1 -- 172.21.15.107:0/2079477265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570534a0580 msgr2=0x5570534afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd527fff700 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570534a0580 0x5570534afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd527fff700 1 -- 172.21.15.107:0/2079477265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 msgr2=0x5570534a8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd527fff700 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534a8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd527fff700 1 -- 172.21.15.107:0/2079477265 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5281a02b0 con 0x5570532fb6b0 2023-07-21T22:31:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd5277fe700 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534a8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:09.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd527fff700 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 0x5570534a8830 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd51c099ea0 tx=0x7fd51c095ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e51863f1b80431 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd5077fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:31:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd5077fe700 1 -- 172.21.15.107:0/2079477265 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0b8180 con 0x5570532fb6b0 2023-07-21T22:31:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd5077fe700 1 -- 172.21.15.107:0/2079477265 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd51c0b82e0 con 0x5570532fb6b0 2023-07-21T22:31:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd53a57eb80 1 -- 172.21.15.107:0/2079477265 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55705334a490 con 0x5570532fb6b0 2023-07-21T22:31:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd5077fe700 1 -- 172.21.15.107:0/2079477265 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd51c0ca200 con 0x5570532fb6b0 2023-07-21T22:31:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd53a57eb80 10 client.16107 fetch_fsmap learned FSMap version 33 2023-07-21T22:31:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.217+0000 7fd53a57eb80 1 -- 172.21.15.107:0/2079477265 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x557053207c00 con 0x5570532fb6b0 2023-07-21T22:31:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.218+0000 7fd5077fe700 1 -- 172.21.15.107:0/2079477265 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd51c063020 con 0x5570532fb6b0 2023-07-21T22:31:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.218+0000 7fd5077fe700 1 -- 172.21.15.107:0/2079477265 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd51c095d90 con 0x5570532fb6b0 2023-07-21T22:31:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.218+0000 7fd53a57eb80 10 client.16107 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:31:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.218+0000 7fd53a57eb80 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.218+0000 7fd53a57eb80 20 client.16107 populate_metadata read hostname 'smithi107' 2023-07-21T22:31:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.218+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5570534b42d0 0x5570534b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.218+0000 7fd53a57eb80 4 client.16107 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.218+0000 7fd53a57eb80 1 -- 172.21.15.107:0/2079477265 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570533e9e50 con 0x5570534b42d0 2023-07-21T22:31:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.218+0000 7fd5277fe700 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5570534b42d0 0x5570534b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.218+0000 7fd5077fe700 10 client.16107 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.218+0000 7fd5277fe700 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5570534b42d0 0x5570534b66e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc0837552eb7f9c server_cookie=fd7c5823670f7ae9 in_seq=0 out_seq=0 2023-07-21T22:31:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.219+0000 7fd5077fe700 1 -- 172.21.15.107:0/2079477265 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5570533e9e50 con 0x5570534b42d0 2023-07-21T22:31:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.219+0000 7fd5077fe700 10 client.16107 handle_command_reply: tid=0 2023-07-21T22:31:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.219+0000 7fd53a57eb80 1 client.16107 shutdown 2023-07-21T22:31:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.219+0000 7fd53a57eb80 1 -- 172.21.15.107:0/2079477265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 msgr2=0x5570534a8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.219+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 0x5570534a8830 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd51c099ea0 tx=0x7fd51c095ba0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.219+0000 7fd53a57eb80 1 -- 172.21.15.107:0/2079477265 shutdown_connections 2023-07-21T22:31:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.219+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5570534b42d0 0x5570534b66e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.219+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570534a0580 0x5570534afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.219+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570532fb6b0 0x5570534a8830 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.219+0000 7fd5077fe700 0 client.16107 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.219+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/2079477265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534a8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.220+0000 7fd53a57eb80 1 -- 172.21.15.107:0/2079477265 >> 172.21.15.107:0/2079477265 conn(0x557053494ae0 msgr2=0x5570534993e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.220+0000 7fd53a57eb80 1 -- 172.21.15.107:0/2079477265 shutdown_connections 2023-07-21T22:31:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.220+0000 7fd53a57eb80 1 -- 172.21.15.107:0/2079477265 wait complete. 2023-07-21T22:31:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.220+0000 7fd53a57eb80 20 client.16107 trim_cache size 0 max 16384 2023-07-21T22:31:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.236+0000 7fd53a57eb80 1 Processor -- start 2023-07-21T22:31:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.236+0000 7fd53a57eb80 1 -- start start 2023-07-21T22:31:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd53a57eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570532fb6b0 0x557053499be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd53a57eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd53a57eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570534a0580 0x5570534bb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd53a57eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55705349a880 con 0x5570534a0580 2023-07-21T22:31:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd53a57eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55705349b6a0 con 0x55705349f830 2023-07-21T22:31:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd53a57eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55705349a120 con 0x5570532fb6b0 2023-07-21T22:31:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd52c996700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570534a0580 0x5570534bb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd5277fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd52c996700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570534a0580 0x5570534bb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49700/0 (socket says 172.21.15.107:49700) 2023-07-21T22:31:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd52c996700 1 -- 172.21.15.107:0/2660940511 learned_addr learned my addr 172.21.15.107:0/2660940511 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd527fff700 1 --2- 172.21.15.107:0/2660940511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570532fb6b0 0x557053499be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd5277fe700 1 -- 172.21.15.107:0/2660940511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570532fb6b0 msgr2=0x557053499be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd5277fe700 1 --2- 172.21.15.107:0/2660940511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570532fb6b0 0x557053499be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd5277fe700 1 -- 172.21.15.107:0/2660940511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570534a0580 msgr2=0x5570534bb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd5277fe700 1 --2- 172.21.15.107:0/2660940511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570534a0580 0x5570534bb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd5277fe700 1 -- 172.21.15.107:0/2660940511 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570532451f0 con 0x55705349f830 2023-07-21T22:31:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.237+0000 7fd52c996700 1 --2- 172.21.15.107:0/2660940511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570534a0580 0x5570534bb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.238+0000 7fd5277fe700 1 --2- 172.21.15.107:0/2660940511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534afd40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5570534b6fc0 tx=0x7fd518014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=946d3450c958f635 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.238+0000 7fd50effd700 1 -- 172.21.15.107:0/2660940511 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518024040 con 0x55705349f830 2023-07-21T22:31:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.238+0000 7fd50effd700 1 -- 172.21.15.107:0/2660940511 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd518003d00 con 0x55705349f830 2023-07-21T22:31:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.238+0000 7fd50effd700 1 -- 172.21.15.107:0/2660940511 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51801ec50 con 0x55705349f830 2023-07-21T22:31:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.238+0000 7fd53a57eb80 1 -- 172.21.15.107:0/2660940511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 msgr2=0x5570534afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.238+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/2660940511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534afd40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5570534b6fc0 tx=0x7fd518014000 comp rx=0 tx=0).stop 2023-07-21T22:31:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.239+0000 7fd53a57eb80 1 -- 172.21.15.107:0/2660940511 shutdown_connections 2023-07-21T22:31:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.239+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/2660940511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570534a0580 0x5570534bb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.239+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/2660940511 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570532fb6b0 0x557053499be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.239+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/2660940511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55705349f830 0x5570534afd40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.239+0000 7fd53a57eb80 1 -- 172.21.15.107:0/2660940511 >> 172.21.15.107:0/2660940511 conn(0x557053494ae0 msgr2=0x5570534a7590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.239+0000 7fd53a57eb80 1 -- 172.21.15.107:0/2660940511 shutdown_connections 2023-07-21T22:31:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.239+0000 7fd53a57eb80 1 -- 172.21.15.107:0/2660940511 wait complete. 2023-07-21T22:31:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.239+0000 7fd53a57eb80 1 -- start start 2023-07-21T22:31:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.239+0000 7fd53a57eb80 1 Processor -- start 2023-07-21T22:31:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.239+0000 7fd53a57eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570532fb6b0 0x5570534abf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.239+0000 7fd53a57eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55705349f830 0x5570534ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd53a57eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5570534a0580 0x5570534af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd53a57eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570534af6e0 con 0x55705349f830 2023-07-21T22:31:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd53a57eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570534af850 con 0x5570534a0580 2023-07-21T22:31:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd53a57eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570534af9c0 con 0x5570532fb6b0 2023-07-21T22:31:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd53a57eb80 10 client.0 _ll_register_callbacks cb 0x7fd518004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd5277fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55705349f830 0x5570534ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd52c996700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5570534a0580 0x5570534af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd5277fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55705349f830 0x5570534ac4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49708/0 (socket says 172.21.15.107:49708) 2023-07-21T22:31:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd5277fe700 1 -- 172.21.15.107:0/1675598351 learned_addr learned my addr 172.21.15.107:0/1675598351 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd52c996700 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5570534a0580 0x5570534af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40534/0 (socket says 172.21.15.107:40534) 2023-07-21T22:31:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd527fff700 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570532fb6b0 0x5570534abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd5277fe700 1 -- 172.21.15.107:0/1675598351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570532fb6b0 msgr2=0x5570534abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd5277fe700 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570532fb6b0 0x5570534abf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd5277fe700 1 -- 172.21.15.107:0/1675598351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5570534a0580 msgr2=0x5570534af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd5277fe700 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5570534a0580 0x5570534af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd5277fe700 1 -- 172.21.15.107:0/1675598351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd528107770 con 0x55705349f830 2023-07-21T22:31:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd52c996700 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5570534a0580 0x5570534af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd5077fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:31:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd5277fe700 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55705349f830 0x5570534ac4a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd51801e8a0 tx=0x7fd518003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728a5a72e848bbfa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.240+0000 7fd5077fe700 1 -- 172.21.15.107:0/1675598351 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518022070 con 0x55705349f830 2023-07-21T22:31:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd5077fe700 1 -- 172.21.15.107:0/1675598351 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd518003a40 con 0x55705349f830 2023-07-21T22:31:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd53a57eb80 1 -- 172.21.15.107:0/1675598351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd51c0a2360 con 0x55705349f830 2023-07-21T22:31:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd5077fe700 1 -- 172.21.15.107:0/1675598351 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd518024040 con 0x55705349f830 2023-07-21T22:31:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd53a57eb80 10 client.16113 fetch_fsmap learned FSMap version 33 2023-07-21T22:31:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd53a57eb80 1 -- 172.21.15.107:0/1675598351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fd518017070 con 0x55705349f830 2023-07-21T22:31:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd5077fe700 1 -- 172.21.15.107:0/1675598351 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd518042020 con 0x55705349f830 2023-07-21T22:31:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd5077fe700 1 -- 172.21.15.107:0/1675598351 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd51803e6d0 con 0x55705349f830 2023-07-21T22:31:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd53a57eb80 10 client.16113 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:31:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd53a57eb80 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd53a57eb80 20 client.16113 populate_metadata read hostname 'smithi107' 2023-07-21T22:31:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557053509340 0x55705350b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd53a57eb80 4 client.16113 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:31:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd53a57eb80 1 -- 172.21.15.107:0/1675598351 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5570533abff0 con 0x557053509340 2023-07-21T22:31:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.241+0000 7fd527fff700 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557053509340 0x55705350b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.242+0000 7fd5077fe700 10 client.16113 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.242+0000 7fd527fff700 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557053509340 0x55705350b750 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882a9a8c2968a7af server_cookie=ce7a46edc1a76d6f in_seq=0 out_seq=0 2023-07-21T22:31:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd5077fe700 1 -- 172.21.15.107:0/1675598351 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5570533abff0 con 0x557053509340 2023-07-21T22:31:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd5077fe700 10 client.16113 handle_command_reply: tid=0 2023-07-21T22:31:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd53a57eb80 1 client.16113 shutdown 2023-07-21T22:31:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd53a57eb80 1 -- 172.21.15.107:0/1675598351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55705349f830 msgr2=0x5570534ac4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55705349f830 0x5570534ac4a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd51801e8a0 tx=0x7fd518003f00 comp rx=0 tx=0).stop 2023-07-21T22:31:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd53a57eb80 1 -- 172.21.15.107:0/1675598351 shutdown_connections 2023-07-21T22:31:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557053509340 0x55705350b750 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5570534a0580 0x5570534af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5570532fb6b0 0x5570534abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd53a57eb80 1 --2- 172.21.15.107:0/1675598351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55705349f830 0x5570534ac4a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd53a57eb80 1 -- 172.21.15.107:0/1675598351 >> 172.21.15.107:0/1675598351 conn(0x557053494ae0 msgr2=0x5570534bb040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd5077fe700 0 client.16113 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd53a57eb80 1 -- 172.21.15.107:0/1675598351 shutdown_connections 2023-07-21T22:31:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.399+0000 7fd53a57eb80 1 -- 172.21.15.107:0/1675598351 wait complete. 2023-07-21T22:31:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.400+0000 7fd53a57eb80 20 client.16113 trim_cache size 0 max 16384 2023-07-21T22:31:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:31:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:31:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:31:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:31:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.400+0000 7fd52e3f9700 1 -- 172.21.15.107:0/2148346063 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd5100a1150 msgr2=0x7fd5100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.400+0000 7fd52e3f9700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd5100a1150 0x7fd5100a3600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd528068950 tx=0x7fd518009260 comp rx=0 tx=0).stop 2023-07-21T22:31:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.400+0000 7fd52e3f9700 1 -- 172.21.15.107:0/2148346063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5280fea00 msgr2=0x7fd52819aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.400+0000 7fd52e3f9700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5280fea00 0x7fd52819aa90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00e950 tx=0x7fd51c00ec60 comp rx=0 tx=0).stop 2023-07-21T22:31:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.400+0000 7fd52e3f9700 1 -- 172.21.15.107:0/2148346063 shutdown_connections 2023-07-21T22:31:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.400+0000 7fd52e3f9700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd528100580 0x7fd52819f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.400+0000 7fd52e3f9700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5280fea00 0x7fd52819aa90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.400+0000 7fd52e3f9700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd5100a1150 0x7fd5100a3600 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.400+0000 7fd52e3f9700 1 --2- 172.21.15.107:0/2148346063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5280ffbf0 0x7fd52819cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.400+0000 7fd52e3f9700 1 -- 172.21.15.107:0/2148346063 >> 172.21.15.107:0/2148346063 conn(0x7fd5280f9fe0 msgr2=0x7fd5280fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.400+0000 7fd52e3f9700 1 -- 172.21.15.107:0/2148346063 shutdown_connections 2023-07-21T22:31:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:09.400+0000 7fd52e3f9700 1 -- 172.21.15.107:0/2148346063 wait complete. 2023-07-21T22:31:09.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:09 smithi190 ceph-mon[98334]: pgmap v356: 97 pgs: 97 active+clean; 168 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2023-07-21T22:31:09.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:09 smithi132 ceph-mon[97863]: pgmap v356: 97 pgs: 97 active+clean; 168 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2023-07-21T22:31:09.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:09 smithi107 ceph-mon[96543]: pgmap v356: 97 pgs: 97 active+clean; 168 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2023-07-21T22:31:11.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:11 smithi190 ceph-mon[98334]: pgmap v357: 97 pgs: 97 active+clean; 171 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2023-07-21T22:31:11.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:11 smithi132 ceph-mon[97863]: pgmap v357: 97 pgs: 97 active+clean; 171 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2023-07-21T22:31:11.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:11 smithi107 ceph-mon[96543]: pgmap v357: 97 pgs: 97 active+clean; 171 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2023-07-21T22:31:13.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:13 smithi190 ceph-mon[98334]: pgmap v358: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.2 MiB/s wr, 565 op/s 2023-07-21T22:31:13.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:13 smithi132 ceph-mon[97863]: pgmap v358: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.2 MiB/s wr, 565 op/s 2023-07-21T22:31:13.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:13 smithi107 ceph-mon[96543]: pgmap v358: 97 pgs: 97 active+clean; 231 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.2 MiB/s wr, 565 op/s 2023-07-21T22:31:15.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:15 smithi190 ceph-mon[98334]: pgmap v359: 97 pgs: 97 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.2 MiB/s wr, 656 op/s 2023-07-21T22:31:15.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:15 smithi132 ceph-mon[97863]: pgmap v359: 97 pgs: 97 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.2 MiB/s wr, 656 op/s 2023-07-21T22:31:15.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:15 smithi107 ceph-mon[96543]: pgmap v359: 97 pgs: 97 active+clean; 243 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.2 MiB/s wr, 656 op/s 2023-07-21T22:31:16.474 INFO:tasks.workunit.client.0.smithi107.stdout: 59 826889 696 548423 545 343992 238 2023-07-21T22:31:17.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:17 smithi190 ceph-mon[98334]: pgmap v360: 97 pgs: 97 active+clean; 242 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.9 MiB/s wr, 656 op/s 2023-07-21T22:31:17.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:17 smithi132 ceph-mon[97863]: pgmap v360: 97 pgs: 97 active+clean; 242 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.9 MiB/s wr, 656 op/s 2023-07-21T22:31:17.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:17 smithi107 ceph-mon[96543]: pgmap v360: 97 pgs: 97 active+clean; 242 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.9 MiB/s wr, 656 op/s 2023-07-21T22:31:19.310 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:31:19.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.409+0000 7fca48e5a700 1 -- 172.21.15.107:0/907672737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca44102f50 msgr2=0x7fca44105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.409+0000 7fca48e5a700 1 --2- 172.21.15.107:0/907672737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca44102f50 0x7fca44105330 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fca38009990 tx=0x7fca38009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.409+0000 7fca48e5a700 1 -- 172.21.15.107:0/907672737 shutdown_connections 2023-07-21T22:31:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.409+0000 7fca48e5a700 1 --2- 172.21.15.107:0/907672737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca44105870 0x7fca44107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.409+0000 7fca48e5a700 1 --2- 172.21.15.107:0/907672737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca44100630 0x7fca44102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.456 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.409+0000 7fca48e5a700 1 --2- 172.21.15.107:0/907672737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca44102f50 0x7fca44105330 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.457 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.409+0000 7fca48e5a700 1 -- 172.21.15.107:0/907672737 >> 172.21.15.107:0/907672737 conn(0x7fca440fa030 msgr2=0x7fca440fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:19.457 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.409+0000 7fca48e5a700 1 -- 172.21.15.107:0/907672737 shutdown_connections 2023-07-21T22:31:19.457 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.409+0000 7fca48e5a700 1 -- 172.21.15.107:0/907672737 wait complete. 2023-07-21T22:31:19.458 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.409+0000 7fca48e5a700 1 Processor -- start 2023-07-21T22:31:19.459 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.409+0000 7fca48e5a700 1 -- start start 2023-07-21T22:31:19.459 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca48e5a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca44100630 0x7fca441957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.459 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca48e5a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca44102f50 0x7fca44195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.459 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca48e5a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca44105870 0x7fca4419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.459 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca48e5a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4419d330 con 0x7fca44105870 2023-07-21T22:31:19.459 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca48e5a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4419d470 con 0x7fca44102f50 2023-07-21T22:31:19.460 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca48e5a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4419d600 con 0x7fca44100630 2023-07-21T22:31:19.460 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca42d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca44105870 0x7fca4419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.460 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca4259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca44100630 0x7fca441957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.460 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca4259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca44100630 0x7fca441957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38278/0 (socket says 172.21.15.107:38278) 2023-07-21T22:31:19.460 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca4259c700 1 -- 172.21.15.107:0/511444839 learned_addr learned my addr 172.21.15.107:0/511444839 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:19.461 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca41d9b700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca44102f50 0x7fca44195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.461 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca42d9d700 1 -- 172.21.15.107:0/511444839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca44100630 msgr2=0x7fca441957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:19.461 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca42d9d700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca44100630 0x7fca441957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.461 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca42d9d700 1 -- 172.21.15.107:0/511444839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca44102f50 msgr2=0x7fca44195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:19.461 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca42d9d700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca44102f50 0x7fca44195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.461 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca42d9d700 1 -- 172.21.15.107:0/511444839 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca380095f0 con 0x7fca44105870 2023-07-21T22:31:19.461 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca4259c700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca44100630 0x7fca441957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:19.461 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca42d9d700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca44105870 0x7fca4419cd60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00efc0 tx=0x7fca3c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:19.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca337fe700 1 -- 172.21.15.107:0/511444839 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c018070 con 0x7fca44105870 2023-07-21T22:31:19.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca48e5a700 1 -- 172.21.15.107:0/511444839 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4419d880 con 0x7fca44105870 2023-07-21T22:31:19.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca337fe700 1 -- 172.21.15.107:0/511444839 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca3c00f040 con 0x7fca44105870 2023-07-21T22:31:19.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.410+0000 7fca337fe700 1 -- 172.21.15.107:0/511444839 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00cd40 con 0x7fca44105870 2023-07-21T22:31:19.462 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.411+0000 7fca48e5a700 1 -- 172.21.15.107:0/511444839 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4419dde0 con 0x7fca44105870 2023-07-21T22:31:19.466 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.412+0000 7fca337fe700 1 -- 172.21.15.107:0/511444839 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fca3c003a00 con 0x7fca44105870 2023-07-21T22:31:19.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.413+0000 7fca337fe700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fca2c0a1150 0x7fca2c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.413+0000 7fca4259c700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fca2c0a1150 0x7fca2c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.413+0000 7fca337fe700 1 -- 172.21.15.107:0/511444839 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fca3c0cc4f0 con 0x7fca44105870 2023-07-21T22:31:19.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.414+0000 7fca4259c700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fca2c0a1150 0x7fca2c0a3600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fca34011550 tx=0x7fca34009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:19.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.414+0000 7fca54fdfb80 1 Processor -- start 2023-07-21T22:31:19.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.414+0000 7fca54fdfb80 1 -- start start 2023-07-21T22:31:19.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca54fdfb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938112050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca54fdfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9381125b0 0x55e938110840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca54fdfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e938110e20 0x55e9381112a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca54fdfb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e938011cb0 con 0x55e938111c40 2023-07-21T22:31:19.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca54fdfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e938064a40 con 0x55e938110e20 2023-07-21T22:31:19.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca54fdfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93805ded0 con 0x55e9381125b0 2023-07-21T22:31:19.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca4259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938112050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca4259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938112050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49728/0 (socket says 172.21.15.107:49728) 2023-07-21T22:31:19.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca4259c700 1 -- 172.21.15.107:0/3518981214 learned_addr learned my addr 172.21.15.107:0/3518981214 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:19.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca41d9b700 1 --2- 172.21.15.107:0/3518981214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9381125b0 0x55e938110840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca42d9d700 1 --2- 172.21.15.107:0/3518981214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e938110e20 0x55e9381112a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca4259c700 1 -- 172.21.15.107:0/3518981214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9381125b0 msgr2=0x55e938110840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:19.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca4259c700 1 --2- 172.21.15.107:0/3518981214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9381125b0 0x55e938110840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca4259c700 1 -- 172.21.15.107:0/3518981214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e938110e20 msgr2=0x55e9381112a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:19.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca4259c700 1 --2- 172.21.15.107:0/3518981214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e938110e20 0x55e9381112a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.469 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca4259c700 1 -- 172.21.15.107:0/3518981214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e937e8bde0 con 0x55e938111c40 2023-07-21T22:31:19.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca4259c700 1 --2- 172.21.15.107:0/3518981214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938112050 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fca34009880 tx=0x7fca34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be93e187c32a99fd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:19.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca30ff9700 1 -- 172.21.15.107:0/3518981214 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34026630 con 0x55e938111c40 2023-07-21T22:31:19.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca30ff9700 1 -- 172.21.15.107:0/3518981214 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca34021070 con 0x55e938111c40 2023-07-21T22:31:19.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.415+0000 7fca30ff9700 1 -- 172.21.15.107:0/3518981214 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3401d610 con 0x55e938111c40 2023-07-21T22:31:19.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.416+0000 7fca54fdfb80 1 -- 172.21.15.107:0/3518981214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 msgr2=0x55e938112050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:19.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.416+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/3518981214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938112050 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fca34009880 tx=0x7fca34014000 comp rx=0 tx=0).stop 2023-07-21T22:31:19.470 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.416+0000 7fca54fdfb80 1 -- 172.21.15.107:0/3518981214 shutdown_connections 2023-07-21T22:31:19.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.416+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/3518981214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e938110e20 0x55e9381112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.416+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/3518981214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938112050 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.416+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/3518981214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9381125b0 0x55e938110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.416+0000 7fca54fdfb80 1 -- 172.21.15.107:0/3518981214 >> 172.21.15.107:0/3518981214 conn(0x55e938101ef0 msgr2=0x55e938107360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:19.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.416+0000 7fca54fdfb80 1 -- 172.21.15.107:0/3518981214 shutdown_connections 2023-07-21T22:31:19.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.416+0000 7fca54fdfb80 1 -- 172.21.15.107:0/3518981214 wait complete. 2023-07-21T22:31:19.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.416+0000 7fca54fdfb80 1 -- start start 2023-07-21T22:31:19.471 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca54fdfb80 1 Processor -- start 2023-07-21T22:31:19.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca54fdfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e938110e20 0x55e93811a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca54fdfb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e93811aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca54fdfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e9381125b0 0x55e938121a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca54fdfb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e937ffdac0 con 0x55e938111c40 2023-07-21T22:31:19.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca54fdfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e937ffd7c0 con 0x55e9381125b0 2023-07-21T22:31:19.472 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca54fdfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e937ee1920 con 0x55e938110e20 2023-07-21T22:31:19.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca54fdfb80 10 client.0 _ll_register_callbacks cb 0x55e937ffb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:19.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca41d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e93811aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca4259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e938110e20 0x55e93811a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca41d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e93811aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49736/0 (socket says 172.21.15.107:49736) 2023-07-21T22:31:19.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca4259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e938110e20 0x55e93811a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38292/0 (socket says 172.21.15.107:38292) 2023-07-21T22:31:19.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca41d9b700 1 -- 172.21.15.107:0/1484084358 learned_addr learned my addr 172.21.15.107:0/1484084358 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:19.473 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca42d9d700 1 --2- 172.21.15.107:0/1484084358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e9381125b0 0x55e938121a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca41d9b700 1 -- 172.21.15.107:0/1484084358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e938110e20 msgr2=0x55e93811a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:19.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca41d9b700 1 --2- 172.21.15.107:0/1484084358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e938110e20 0x55e93811a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca41d9b700 1 -- 172.21.15.107:0/1484084358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e9381125b0 msgr2=0x55e938121a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:19.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca41d9b700 1 --2- 172.21.15.107:0/1484084358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e9381125b0 0x55e938121a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca41d9b700 1 -- 172.21.15.107:0/1484084358 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca34017070 con 0x55e938111c40 2023-07-21T22:31:19.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca41d9b700 1 --2- 172.21.15.107:0/1484084358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e93811aa20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fca38015000 tx=0x7fca38009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cc7c94eeddf5d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:19.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.417+0000 7fca29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:31:19.474 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca29ffb700 1 -- 172.21.15.107:0/1484084358 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3801d030 con 0x55e938111c40 2023-07-21T22:31:19.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca29ffb700 1 -- 172.21.15.107:0/1484084358 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fca38004500 con 0x55e938111c40 2023-07-21T22:31:19.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca54fdfb80 1 -- 172.21.15.107:0/1484084358 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e937fb74f0 con 0x55e938111c40 2023-07-21T22:31:19.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca29ffb700 1 -- 172.21.15.107:0/1484084358 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca380046c0 con 0x55e938111c40 2023-07-21T22:31:19.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca54fdfb80 10 client.16125 fetch_fsmap learned FSMap version 33 2023-07-21T22:31:19.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca54fdfb80 1 -- 172.21.15.107:0/1484084358 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e937e74c60 con 0x55e938111c40 2023-07-21T22:31:19.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca29ffb700 1 -- 172.21.15.107:0/1484084358 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca38033020 con 0x55e938111c40 2023-07-21T22:31:19.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca29ffb700 1 -- 172.21.15.107:0/1484084358 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fca3800f9b0 con 0x55e938111c40 2023-07-21T22:31:19.475 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca54fdfb80 10 client.16125 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:31:19.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca54fdfb80 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:19.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca54fdfb80 20 client.16125 populate_metadata read hostname 'smithi107' 2023-07-21T22:31:19.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/1484084358 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e938126390 0x55e9381287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca54fdfb80 4 client.16125 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:19.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca54fdfb80 1 -- 172.21.15.107:0/1484084358 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e938056eb0 con 0x55e938126390 2023-07-21T22:31:19.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.418+0000 7fca4259c700 1 --2- 172.21.15.107:0/1484084358 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e938126390 0x55e9381287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.476 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca29ffb700 10 client.16125 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:19.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca4259c700 1 --2- 172.21.15.107:0/1484084358 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e938126390 0x55e9381287a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=154413e47bdc0504 server_cookie=f0ae9061e65bab58 in_seq=0 out_seq=0 2023-07-21T22:31:19.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca29ffb700 1 -- 172.21.15.107:0/1484084358 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e938056eb0 con 0x55e938126390 2023-07-21T22:31:19.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca29ffb700 10 client.16125 handle_command_reply: tid=0 2023-07-21T22:31:19.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca54fdfb80 1 client.16125 shutdown 2023-07-21T22:31:19.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca54fdfb80 1 -- 172.21.15.107:0/1484084358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 msgr2=0x55e93811aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:19.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/1484084358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e93811aa20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fca38015000 tx=0x7fca38009570 comp rx=0 tx=0).stop 2023-07-21T22:31:19.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca54fdfb80 1 -- 172.21.15.107:0/1484084358 shutdown_connections 2023-07-21T22:31:19.477 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/1484084358 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e938126390 0x55e9381287a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/1484084358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e9381125b0 0x55e938121a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca29ffb700 0 client.16125 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:19.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/1484084358 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e938110e20 0x55e93811a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/1484084358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e93811aa20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca54fdfb80 1 -- 172.21.15.107:0/1484084358 >> 172.21.15.107:0/1484084358 conn(0x55e938101ef0 msgr2=0x55e938106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:19.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca54fdfb80 1 -- 172.21.15.107:0/1484084358 shutdown_connections 2023-07-21T22:31:19.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.419+0000 7fca54fdfb80 1 -- 172.21.15.107:0/1484084358 wait complete. 2023-07-21T22:31:19.478 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.420+0000 7fca54fdfb80 20 client.16125 trim_cache size 0 max 16384 2023-07-21T22:31:19.484 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.436+0000 7fca54fdfb80 1 Processor -- start 2023-07-21T22:31:19.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.436+0000 7fca54fdfb80 1 -- start start 2023-07-21T22:31:19.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.436+0000 7fca54fdfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e938110e20 0x55e938118390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.436+0000 7fca54fdfb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938121a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.436+0000 7fca54fdfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e9381125b0 0x55e93811a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.436+0000 7fca54fdfb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e938108060 con 0x55e938111c40 2023-07-21T22:31:19.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.436+0000 7fca54fdfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e938118140 con 0x55e9381125b0 2023-07-21T22:31:19.485 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.436+0000 7fca54fdfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93811a290 con 0x55e938110e20 2023-07-21T22:31:19.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.436+0000 7fca41d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.436+0000 7fca4259c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e938110e20 0x55e938118390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.436+0000 7fca41d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938121a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49744/0 (socket says 172.21.15.107:49744) 2023-07-21T22:31:19.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.437+0000 7fca41d9b700 1 -- 172.21.15.107:0/1960060998 learned_addr learned my addr 172.21.15.107:0/1960060998 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:19.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.437+0000 7fca42d9d700 1 --2- 172.21.15.107:0/1960060998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e9381125b0 0x55e93811a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.437+0000 7fca41d9b700 1 -- 172.21.15.107:0/1960060998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e938110e20 msgr2=0x55e938118390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:19.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.437+0000 7fca41d9b700 1 --2- 172.21.15.107:0/1960060998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e938110e20 0x55e938118390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.486 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.437+0000 7fca41d9b700 1 -- 172.21.15.107:0/1960060998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e9381125b0 msgr2=0x55e93811a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:19.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.437+0000 7fca41d9b700 1 --2- 172.21.15.107:0/1960060998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e9381125b0 0x55e93811a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.437+0000 7fca41d9b700 1 -- 172.21.15.107:0/1960060998 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e937eb2250 con 0x55e938111c40 2023-07-21T22:31:19.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.437+0000 7fca4259c700 1 --2- 172.21.15.107:0/1960060998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e938110e20 0x55e938118390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:19.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.437+0000 7fca41d9b700 1 --2- 172.21.15.107:0/1960060998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938121a20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fca3800fd60 tx=0x7fca380094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aef5b882509675e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:19.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.437+0000 7fca317fa700 1 -- 172.21.15.107:0/1960060998 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3801d070 con 0x55e938111c40 2023-07-21T22:31:19.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.437+0000 7fca317fa700 1 -- 172.21.15.107:0/1960060998 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca38033030 con 0x55e938111c40 2023-07-21T22:31:19.487 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.437+0000 7fca317fa700 1 -- 172.21.15.107:0/1960060998 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38024a00 con 0x55e938111c40 2023-07-21T22:31:19.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 -- 172.21.15.107:0/1960060998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 msgr2=0x55e938121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:19.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/1960060998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938121a20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fca3800fd60 tx=0x7fca380094b0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 -- 172.21.15.107:0/1960060998 shutdown_connections 2023-07-21T22:31:19.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/1960060998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e9381125b0 0x55e93811a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/1960060998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e938110e20 0x55e938118390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/1960060998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938121a20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 -- 172.21.15.107:0/1960060998 >> 172.21.15.107:0/1960060998 conn(0x55e938101ef0 msgr2=0x55e9381191f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:19.488 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 -- 172.21.15.107:0/1960060998 shutdown_connections 2023-07-21T22:31:19.489 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 -- 172.21.15.107:0/1960060998 wait complete. 2023-07-21T22:31:19.489 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 -- start start 2023-07-21T22:31:19.489 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 Processor -- start 2023-07-21T22:31:19.492 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e938110e20 0x55e9381214d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.492 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938121a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.492 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9381125b0 0x55e93811e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.492 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93811e990 con 0x55e938111c40 2023-07-21T22:31:19.492 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93811eb00 con 0x55e938110e20 2023-07-21T22:31:19.492 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93811ecc0 con 0x55e9381125b0 2023-07-21T22:31:19.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca54fdfb80 10 client.0 _ll_register_callbacks cb 0x7fca3800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:19.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca41d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938121a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca41d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938121a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49750/0 (socket says 172.21.15.107:49750) 2023-07-21T22:31:19.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca41d9b700 1 -- 172.21.15.107:0/4291241124 learned_addr learned my addr 172.21.15.107:0/4291241124 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:19.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca4259c700 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e938110e20 0x55e9381214d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.438+0000 7fca42d9d700 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9381125b0 0x55e93811e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca41d9b700 1 -- 172.21.15.107:0/4291241124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9381125b0 msgr2=0x55e93811e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:19.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca41d9b700 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9381125b0 0x55e93811e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca41d9b700 1 -- 172.21.15.107:0/4291241124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e938110e20 msgr2=0x55e9381214d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:19.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca41d9b700 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e938110e20 0x55e9381214d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca41d9b700 1 -- 172.21.15.107:0/4291241124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3402a040 con 0x55e938111c40 2023-07-21T22:31:19.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca42d9d700 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9381125b0 0x55e93811e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:19.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca41d9b700 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938121a10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fca3800fbe0 tx=0x7fca3800f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a51cb2956a4bfc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:19.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca29ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:31:19.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca29ffb700 1 -- 172.21.15.107:0/4291241124 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3801d030 con 0x55e938111c40 2023-07-21T22:31:19.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca29ffb700 1 -- 172.21.15.107:0/4291241124 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fca38000e40 con 0x55e938111c40 2023-07-21T22:31:19.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca54fdfb80 1 -- 172.21.15.107:0/4291241124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca34021070 con 0x55e938111c40 2023-07-21T22:31:19.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca29ffb700 1 -- 172.21.15.107:0/4291241124 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca38024900 con 0x55e938111c40 2023-07-21T22:31:19.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca54fdfb80 10 client.16131 fetch_fsmap learned FSMap version 33 2023-07-21T22:31:19.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca54fdfb80 1 -- 172.21.15.107:0/4291241124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e937e74c60 con 0x55e938111c40 2023-07-21T22:31:19.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca29ffb700 1 -- 172.21.15.107:0/4291241124 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca34021070 con 0x55e938111c40 2023-07-21T22:31:19.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca29ffb700 1 -- 172.21.15.107:0/4291241124 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 33) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fca3804c020 con 0x55e938111c40 2023-07-21T22:31:19.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca54fdfb80 10 client.16131 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:31:19.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca54fdfb80 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:19.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca54fdfb80 20 client.16131 populate_metadata read hostname 'smithi107' 2023-07-21T22:31:19.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e938126390 0x55e93817d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:19.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca54fdfb80 4 client.16131 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:31:19.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.439+0000 7fca54fdfb80 1 -- 172.21.15.107:0/4291241124 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e938019050 con 0x55e938126390 2023-07-21T22:31:19.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca4259c700 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e938126390 0x55e93817d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:19.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca4259c700 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e938126390 0x55e93817d620 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f496463b996a310 server_cookie=ccd6bc99189831d4 in_seq=0 out_seq=0 2023-07-21T22:31:19.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca29ffb700 10 client.16131 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:19.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca29ffb700 1 -- 172.21.15.107:0/4291241124 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e938019050 con 0x55e938126390 2023-07-21T22:31:19.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca29ffb700 10 client.16131 handle_command_reply: tid=0 2023-07-21T22:31:19.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca54fdfb80 1 client.16131 shutdown 2023-07-21T22:31:19.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca54fdfb80 1 -- 172.21.15.107:0/4291241124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 msgr2=0x55e938121a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:19.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938121a10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fca3800fbe0 tx=0x7fca3800f960 comp rx=0 tx=0).stop 2023-07-21T22:31:19.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca54fdfb80 1 -- 172.21.15.107:0/4291241124 shutdown_connections 2023-07-21T22:31:19.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e938126390 0x55e93817d620 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9381125b0 0x55e93811e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca29ffb700 0 client.16131 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:19.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e938110e20 0x55e9381214d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca54fdfb80 1 --2- 172.21.15.107:0/4291241124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e938111c40 0x55e938121a10 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca54fdfb80 1 -- 172.21.15.107:0/4291241124 >> 172.21.15.107:0/4291241124 conn(0x55e938101ef0 msgr2=0x55e938118390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:19.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca54fdfb80 1 -- 172.21.15.107:0/4291241124 shutdown_connections 2023-07-21T22:31:19.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.440+0000 7fca54fdfb80 1 -- 172.21.15.107:0/4291241124 wait complete. 2023-07-21T22:31:19.498 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.441+0000 7fca54fdfb80 20 client.16131 trim_cache size 0 max 16384 2023-07-21T22:31:19.499 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:31:19.499 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:31:19.499 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:31:19.499 INFO:teuthology.orchestra.run.smithi107.stdout: "d037412e-e4e0-4d2c-8d6e-c7193371fe0f": { 2023-07-21T22:31:19.499 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:31:19.499 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d037412e-e4e0-4d2c-8d6e-c7193371fe0f", 2023-07-21T22:31:19.500 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:31:19.500 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:31:19.500 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:31:19.500 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:31:19.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.441+0000 7fca48e5a700 1 -- 172.21.15.107:0/511444839 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fca2c0a1150 msgr2=0x7fca2c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:19.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.441+0000 7fca48e5a700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fca2c0a1150 0x7fca2c0a3600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fca34011550 tx=0x7fca34009260 comp rx=0 tx=0).stop 2023-07-21T22:31:19.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.441+0000 7fca48e5a700 1 -- 172.21.15.107:0/511444839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca44105870 msgr2=0x7fca4419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:19.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.441+0000 7fca48e5a700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca44105870 0x7fca4419cd60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00efc0 tx=0x7fca3c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.441+0000 7fca48e5a700 1 -- 172.21.15.107:0/511444839 shutdown_connections 2023-07-21T22:31:19.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.441+0000 7fca48e5a700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca44105870 0x7fca4419cd60 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.441+0000 7fca48e5a700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fca2c0a1150 0x7fca2c0a3600 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.441+0000 7fca48e5a700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca44100630 0x7fca441957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.441+0000 7fca48e5a700 1 --2- 172.21.15.107:0/511444839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca44102f50 0x7fca44195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:19.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.441+0000 7fca48e5a700 1 -- 172.21.15.107:0/511444839 >> 172.21.15.107:0/511444839 conn(0x7fca440fa030 msgr2=0x7fca44101290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:19.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.441+0000 7fca48e5a700 1 -- 172.21.15.107:0/511444839 shutdown_connections 2023-07-21T22:31:19.503 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:19.441+0000 7fca48e5a700 1 -- 172.21.15.107:0/511444839 wait complete. 2023-07-21T22:31:19.503 INFO:tasks.cephfs.filesystem:scrub status for tag:d037412e-e4e0-4d2c-8d6e-c7193371fe0f - {'path': '/', 'tag': 'd037412e-e4e0-4d2c-8d6e-c7193371fe0f', 'options': 'recursive,force'} 2023-07-21T22:31:19.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:19 smithi190 ceph-mon[98334]: pgmap v361: 97 pgs: 97 active+clean; 248 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.2 MiB/s wr, 611 op/s 2023-07-21T22:31:19.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:19 smithi132 ceph-mon[97863]: pgmap v361: 97 pgs: 97 active+clean; 248 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.2 MiB/s wr, 611 op/s 2023-07-21T22:31:19.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:19 smithi107 ceph-mon[96543]: pgmap v361: 97 pgs: 97 active+clean; 248 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.2 MiB/s wr, 611 op/s 2023-07-21T22:31:21.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:21 smithi190 ceph-mon[98334]: pgmap v362: 97 pgs: 97 active+clean; 249 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.3 MiB/s wr, 595 op/s 2023-07-21T22:31:21.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:21 smithi132 ceph-mon[97863]: pgmap v362: 97 pgs: 97 active+clean; 249 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.3 MiB/s wr, 595 op/s 2023-07-21T22:31:21.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:21 smithi107 ceph-mon[96543]: pgmap v362: 97 pgs: 97 active+clean; 249 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.3 MiB/s wr, 595 op/s 2023-07-21T22:31:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.538+0000 7f7457727700 1 -- 172.21.15.107:0/248579689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74500fea80 msgr2=0x7f74500fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.538+0000 7f7457727700 1 --2- 172.21.15.107:0/248579689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74500fea80 0x7f74500fee90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f744c009990 tx=0x7f744c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.538+0000 7f7457727700 1 -- 172.21.15.107:0/248579689 shutdown_connections 2023-07-21T22:31:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.538+0000 7f7457727700 1 --2- 172.21.15.107:0/248579689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7450100600 0x7f7450106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.538+0000 7f7457727700 1 --2- 172.21.15.107:0/248579689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74500fea80 0x7f74500fee90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.538+0000 7f7457727700 1 --2- 172.21.15.107:0/248579689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74500ffc70 0x7f74501000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.538+0000 7f7457727700 1 -- 172.21.15.107:0/248579689 >> 172.21.15.107:0/248579689 conn(0x7f74500fa0c0 msgr2=0x7f74500fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.538+0000 7f7457727700 1 -- 172.21.15.107:0/248579689 shutdown_connections 2023-07-21T22:31:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.538+0000 7f7457727700 1 -- 172.21.15.107:0/248579689 wait complete. 2023-07-21T22:31:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7457727700 1 Processor -- start 2023-07-21T22:31:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7457727700 1 -- start start 2023-07-21T22:31:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7457727700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74500fea80 0x7f74501985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7457727700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74500ffc70 0x7f745019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7457727700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7450100600 0x7f745019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7457727700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745019d610 con 0x7f7450100600 2023-07-21T22:31:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7457727700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745019d750 con 0x7f74500ffc70 2023-07-21T22:31:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7457727700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745019d910 con 0x7f74500fea80 2023-07-21T22:31:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7455cc4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7450100600 0x7f745019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f74554c3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74500fea80 0x7f74501985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7455cc4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7450100600 0x7f745019d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49766/0 (socket says 172.21.15.107:49766) 2023-07-21T22:31:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7455cc4700 1 -- 172.21.15.107:0/838379891 learned_addr learned my addr 172.21.15.107:0/838379891 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7454cc2700 1 --2- 172.21.15.107:0/838379891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74500ffc70 0x7f745019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7455cc4700 1 -- 172.21.15.107:0/838379891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74500fea80 msgr2=0x7f74501985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7455cc4700 1 --2- 172.21.15.107:0/838379891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74500fea80 0x7f74501985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7455cc4700 1 -- 172.21.15.107:0/838379891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74500ffc70 msgr2=0x7f745019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7455cc4700 1 --2- 172.21.15.107:0/838379891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74500ffc70 0x7f745019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7455cc4700 1 -- 172.21.15.107:0/838379891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c0095f0 con 0x7f7450100600 2023-07-21T22:31:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.539+0000 7f7455cc4700 1 --2- 172.21.15.107:0/838379891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7450100600 0x7f745019d040 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f744800ef30 tx=0x7f744800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.540+0000 7f74467fc700 1 -- 172.21.15.107:0/838379891 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448018070 con 0x7f7450100600 2023-07-21T22:31:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.540+0000 7f74467fc700 1 -- 172.21.15.107:0/838379891 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f744800f040 con 0x7f7450100600 2023-07-21T22:31:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.540+0000 7f7457727700 1 -- 172.21.15.107:0/838379891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7450194380 con 0x7f7450100600 2023-07-21T22:31:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.540+0000 7f74467fc700 1 -- 172.21.15.107:0/838379891 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744800cd00 con 0x7f7450100600 2023-07-21T22:31:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.540+0000 7f7457727700 1 -- 172.21.15.107:0/838379891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74501948b0 con 0x7f7450100600 2023-07-21T22:31:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.542+0000 7f74467fc700 1 -- 172.21.15.107:0/838379891 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7448003a00 con 0x7f7450100600 2023-07-21T22:31:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.542+0000 7f74467fc700 1 --2- 172.21.15.107:0/838379891 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f743c0a11d0 0x7f743c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.543+0000 7f74554c3700 1 --2- 172.21.15.107:0/838379891 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f743c0a11d0 0x7f743c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.543+0000 7f74467fc700 1 -- 172.21.15.107:0/838379891 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f74480cc3d0 con 0x7f7450100600 2023-07-21T22:31:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.543+0000 7f74554c3700 1 --2- 172.21.15.107:0/838379891 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f743c0a11d0 0x7f743c0a3680 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f744c006010 tx=0x7f744c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.543+0000 7f74638acb80 1 Processor -- start 2023-07-21T22:31:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.543+0000 7f74638acb80 1 -- start start 2023-07-21T22:31:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f74638acb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee2516b0 0x5600ee3f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f74638acb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee3f5830 0x5600ee3f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f74638acb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee3f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f74638acb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ee2fac50 con 0x5600ee2516b0 2023-07-21T22:31:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f74638acb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ee34d9e0 con 0x5600ee3f6580 2023-07-21T22:31:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f74638acb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ee346e70 con 0x5600ee3f5830 2023-07-21T22:31:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f74554c3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee2516b0 0x5600ee3f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f74554c3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee2516b0 0x5600ee3f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49772/0 (socket says 172.21.15.107:49772) 2023-07-21T22:31:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f74554c3700 1 -- 172.21.15.107:0/550950180 learned_addr learned my addr 172.21.15.107:0/550950180 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f7454cc2700 1 --2- 172.21.15.107:0/550950180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee3f5830 0x5600ee3f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f7455cc4700 1 --2- 172.21.15.107:0/550950180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee3f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f7454cc2700 1 -- 172.21.15.107:0/550950180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 msgr2=0x5600ee3f69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f7454cc2700 1 --2- 172.21.15.107:0/550950180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee3f69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f7454cc2700 1 -- 172.21.15.107:0/550950180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee2516b0 msgr2=0x5600ee3f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f7454cc2700 1 --2- 172.21.15.107:0/550950180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee2516b0 0x5600ee3f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f7454cc2700 1 -- 172.21.15.107:0/550950180 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600ee174d80 con 0x5600ee3f5830 2023-07-21T22:31:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f7455cc4700 1 --2- 172.21.15.107:0/550950180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee3f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:31:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.544+0000 7f74554c3700 1 --2- 172.21.15.107:0/550950180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee2516b0 0x5600ee3f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.545+0000 7f7454cc2700 1 --2- 172.21.15.107:0/550950180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee3f5830 0x5600ee3f6040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7440009c80 tx=0x7f7440009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=836c1ca2cfe9c0d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.545+0000 7f743b7fe700 1 -- 172.21.15.107:0/550950180 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440011cb0 con 0x5600ee3f5830 2023-07-21T22:31:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.545+0000 7f743b7fe700 1 -- 172.21.15.107:0/550950180 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7440011e10 con 0x5600ee3f5830 2023-07-21T22:31:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.545+0000 7f743b7fe700 1 -- 172.21.15.107:0/550950180 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440005280 con 0x5600ee3f5830 2023-07-21T22:31:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.545+0000 7f74638acb80 1 -- 172.21.15.107:0/550950180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee3f5830 msgr2=0x5600ee3f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.545+0000 7f74638acb80 1 --2- 172.21.15.107:0/550950180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee3f5830 0x5600ee3f6040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7440009c80 tx=0x7f7440009f90 comp rx=0 tx=0).stop 2023-07-21T22:31:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.545+0000 7f74638acb80 1 -- 172.21.15.107:0/550950180 shutdown_connections 2023-07-21T22:31:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.545+0000 7f74638acb80 1 --2- 172.21.15.107:0/550950180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee3f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.545+0000 7f74638acb80 1 --2- 172.21.15.107:0/550950180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee2516b0 0x5600ee3f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.545+0000 7f74638acb80 1 --2- 172.21.15.107:0/550950180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee3f5830 0x5600ee3f6040 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.545+0000 7f74638acb80 1 -- 172.21.15.107:0/550950180 >> 172.21.15.107:0/550950180 conn(0x5600ee3eaae0 msgr2=0x5600ee3efc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f74638acb80 1 -- 172.21.15.107:0/550950180 shutdown_connections 2023-07-21T22:31:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f74638acb80 1 -- 172.21.15.107:0/550950180 wait complete. 2023-07-21T22:31:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f74638acb80 1 -- start start 2023-07-21T22:31:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f74638acb80 1 Processor -- start 2023-07-21T22:31:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f74638acb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 0x5600ee407020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f74638acb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 0x5600ee407560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f74638acb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee40e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f74638acb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ee2e6a60 con 0x5600ee3f5830 2023-07-21T22:31:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f74638acb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ee2e6760 con 0x5600ee3f6580 2023-07-21T22:31:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f74638acb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ee1ca8c0 con 0x5600ee2516b0 2023-07-21T22:31:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f74638acb80 10 client.0 _ll_register_callbacks cb 0x5600ee2e4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f7454cc2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 0x5600ee407560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f74554c3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 0x5600ee407020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f7454cc2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 0x5600ee407560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49778/0 (socket says 172.21.15.107:49778) 2023-07-21T22:31:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f7454cc2700 1 -- 172.21.15.107:0/3160803138 learned_addr learned my addr 172.21.15.107:0/3160803138 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.546+0000 7f7455cc4700 1 --2- 172.21.15.107:0/3160803138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee40e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f7454cc2700 1 -- 172.21.15.107:0/3160803138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 msgr2=0x5600ee407020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f7454cc2700 1 --2- 172.21.15.107:0/3160803138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 0x5600ee407020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f7454cc2700 1 -- 172.21.15.107:0/3160803138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 msgr2=0x5600ee40e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f7454cc2700 1 --2- 172.21.15.107:0/3160803138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee40e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f7454cc2700 1 -- 172.21.15.107:0/3160803138 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74501077f0 con 0x5600ee3f5830 2023-07-21T22:31:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f7454cc2700 1 --2- 172.21.15.107:0/3160803138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 0x5600ee407560 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7440011720 tx=0x7f7440004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad866da1f2f85603 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f7438ff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:31:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f7438ff9700 1 -- 172.21.15.107:0/3160803138 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440004260 con 0x5600ee3f5830 2023-07-21T22:31:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f74638acb80 1 -- 172.21.15.107:0/3160803138 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600ee2a0490 con 0x5600ee3f5830 2023-07-21T22:31:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f7438ff9700 1 -- 172.21.15.107:0/3160803138 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f74400043c0 con 0x5600ee3f5830 2023-07-21T22:31:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f7438ff9700 1 -- 172.21.15.107:0/3160803138 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7440004580 con 0x5600ee3f5830 2023-07-21T22:31:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f74638acb80 10 client.16143 fetch_fsmap learned FSMap version 34 2023-07-21T22:31:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f74638acb80 1 -- 172.21.15.107:0/3160803138 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f74400098e0 con 0x5600ee3f5830 2023-07-21T22:31:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.547+0000 7f7438ff9700 1 -- 172.21.15.107:0/3160803138 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5600ee2a0490 con 0x5600ee3f5830 2023-07-21T22:31:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.548+0000 7f7438ff9700 1 -- 172.21.15.107:0/3160803138 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7440005220 con 0x5600ee3f5830 2023-07-21T22:31:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.548+0000 7f74638acb80 10 client.16143 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:31:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.548+0000 7f74638acb80 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.548+0000 7f74638acb80 20 client.16143 populate_metadata read hostname 'smithi107' 2023-07-21T22:31:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.548+0000 7f74638acb80 1 --2- 172.21.15.107:0/3160803138 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5600ee412d30 0x5600ee415140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.548+0000 7f74638acb80 4 client.16143 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:22.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.548+0000 7f74638acb80 1 -- 172.21.15.107:0/3160803138 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600ee33fe50 con 0x5600ee412d30 2023-07-21T22:31:22.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.548+0000 7f74554c3700 1 --2- 172.21.15.107:0/3160803138 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5600ee412d30 0x5600ee415140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.548+0000 7f7438ff9700 10 client.16143 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:22.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.548+0000 7f74554c3700 1 --2- 172.21.15.107:0/3160803138 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5600ee412d30 0x5600ee415140 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=793436da412b6f3b server_cookie=987906fe8192a902 in_seq=0 out_seq=0 2023-07-21T22:31:22.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f7438ff9700 1 -- 172.21.15.107:0/3160803138 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5600ee33fe50 con 0x5600ee412d30 2023-07-21T22:31:22.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f7438ff9700 10 client.16143 handle_command_reply: tid=0 2023-07-21T22:31:22.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f74638acb80 1 client.16143 shutdown 2023-07-21T22:31:22.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f74638acb80 1 -- 172.21.15.107:0/3160803138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 msgr2=0x5600ee407560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f74638acb80 1 --2- 172.21.15.107:0/3160803138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 0x5600ee407560 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7440011720 tx=0x7f7440004750 comp rx=0 tx=0).stop 2023-07-21T22:31:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f74638acb80 1 -- 172.21.15.107:0/3160803138 shutdown_connections 2023-07-21T22:31:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f74638acb80 1 --2- 172.21.15.107:0/3160803138 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5600ee412d30 0x5600ee415140 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f74638acb80 1 --2- 172.21.15.107:0/3160803138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee40e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f7438ff9700 0 client.16143 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f74638acb80 1 --2- 172.21.15.107:0/3160803138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 0x5600ee407020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f74638acb80 1 --2- 172.21.15.107:0/3160803138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 0x5600ee407560 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f74638acb80 1 -- 172.21.15.107:0/3160803138 >> 172.21.15.107:0/3160803138 conn(0x5600ee3eaae0 msgr2=0x5600ee3ef3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f74638acb80 1 -- 172.21.15.107:0/3160803138 shutdown_connections 2023-07-21T22:31:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f74638acb80 1 -- 172.21.15.107:0/3160803138 wait complete. 2023-07-21T22:31:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.550+0000 7f74638acb80 20 client.16143 trim_cache size 0 max 16384 2023-07-21T22:31:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f74638acb80 1 Processor -- start 2023-07-21T22:31:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f74638acb80 1 -- start start 2023-07-21T22:31:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f74638acb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 0x5600ee3f6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f74638acb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f5830 0x5600ee40e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f74638acb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f6580 0x5600ee405d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f74638acb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ee3f6e90 con 0x5600ee3f6580 2023-07-21T22:31:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f74638acb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ee405aa0 con 0x5600ee3f5830 2023-07-21T22:31:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f74638acb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ee407ca0 con 0x5600ee2516b0 2023-07-21T22:31:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f7455cc4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f6580 0x5600ee405d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f7455cc4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f6580 0x5600ee405d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49788/0 (socket says 172.21.15.107:49788) 2023-07-21T22:31:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f7455cc4700 1 -- 172.21.15.107:0/3960962544 learned_addr learned my addr 172.21.15.107:0/3960962544 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f7454cc2700 1 --2- 172.21.15.107:0/3960962544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f5830 0x5600ee40e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f74554c3700 1 --2- 172.21.15.107:0/3960962544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 0x5600ee3f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f7455cc4700 1 -- 172.21.15.107:0/3960962544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 msgr2=0x5600ee3f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f7455cc4700 1 --2- 172.21.15.107:0/3960962544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 0x5600ee3f6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f7455cc4700 1 -- 172.21.15.107:0/3960962544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f5830 msgr2=0x5600ee40e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f7455cc4700 1 --2- 172.21.15.107:0/3960962544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f5830 0x5600ee40e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f7455cc4700 1 -- 172.21.15.107:0/3960962544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600ee15dc00 con 0x5600ee3f6580 2023-07-21T22:31:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f74554c3700 1 --2- 172.21.15.107:0/3960962544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 0x5600ee3f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f7454cc2700 1 --2- 172.21.15.107:0/3960962544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f5830 0x5600ee40e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f7455cc4700 1 --2- 172.21.15.107:0/3960962544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f6580 0x5600ee405d10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f74480962c0 tx=0x7f74480092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833cf4ce20fe5a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f743bfff700 1 -- 172.21.15.107:0/3960962544 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448099850 con 0x5600ee3f6580 2023-07-21T22:31:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f743bfff700 1 -- 172.21.15.107:0/3960962544 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f744809d880 con 0x5600ee3f6580 2023-07-21T22:31:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.566+0000 7f743bfff700 1 -- 172.21.15.107:0/3960962544 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448099ab0 con 0x5600ee3f6580 2023-07-21T22:31:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.567+0000 7f74638acb80 1 -- 172.21.15.107:0/3960962544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f6580 msgr2=0x5600ee405d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.567+0000 7f74638acb80 1 --2- 172.21.15.107:0/3960962544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f6580 0x5600ee405d10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f74480962c0 tx=0x7f74480092e0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.567+0000 7f74638acb80 1 -- 172.21.15.107:0/3960962544 shutdown_connections 2023-07-21T22:31:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.567+0000 7f74638acb80 1 --2- 172.21.15.107:0/3960962544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f6580 0x5600ee405d10 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.567+0000 7f74638acb80 1 --2- 172.21.15.107:0/3960962544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 0x5600ee3f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.567+0000 7f74638acb80 1 --2- 172.21.15.107:0/3960962544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f5830 0x5600ee40e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.567+0000 7f74638acb80 1 -- 172.21.15.107:0/3960962544 >> 172.21.15.107:0/3960962544 conn(0x5600ee3eaae0 msgr2=0x5600ee419aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.567+0000 7f74638acb80 1 -- 172.21.15.107:0/3960962544 shutdown_connections 2023-07-21T22:31:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.567+0000 7f74638acb80 1 -- 172.21.15.107:0/3960962544 wait complete. 2023-07-21T22:31:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.567+0000 7f74638acb80 1 -- start start 2023-07-21T22:31:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.568+0000 7f74638acb80 1 Processor -- start 2023-07-21T22:31:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.568+0000 7f74638acb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 0x5600ee40a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.568+0000 7f74638acb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 0x5600ee40a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.568+0000 7f74638acb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee40d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.568+0000 7f74638acb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ee40de90 con 0x5600ee3f5830 2023-07-21T22:31:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.568+0000 7f74638acb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ee40e000 con 0x5600ee3f6580 2023-07-21T22:31:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.568+0000 7f74638acb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ee40e1c0 con 0x5600ee2516b0 2023-07-21T22:31:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.568+0000 7f74638acb80 10 client.0 _ll_register_callbacks cb 0x7f744c004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.568+0000 7f7454cc2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 0x5600ee40a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.568+0000 7f7454cc2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 0x5600ee40a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49792/0 (socket says 172.21.15.107:49792) 2023-07-21T22:31:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.568+0000 7f7454cc2700 1 -- 172.21.15.107:0/2516621745 learned_addr learned my addr 172.21.15.107:0/2516621745 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.568+0000 7f74554c3700 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 0x5600ee40a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.568+0000 7f7455cc4700 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee40d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f7455cc4700 1 -- 172.21.15.107:0/2516621745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 msgr2=0x5600ee40a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f7455cc4700 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 0x5600ee40a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f7455cc4700 1 -- 172.21.15.107:0/2516621745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 msgr2=0x5600ee40a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f7455cc4700 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 0x5600ee40a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f7455cc4700 1 -- 172.21.15.107:0/2516621745 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74501948b0 con 0x5600ee3f6580 2023-07-21T22:31:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f7454cc2700 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 0x5600ee40a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f7455cc4700 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee40d950 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f74480962c0 tx=0x7f74480c9e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4dd2fd8776f6b63 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f7438ff9700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:31:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f7438ff9700 1 -- 172.21.15.107:0/2516621745 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448099cb0 con 0x5600ee3f6580 2023-07-21T22:31:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f7438ff9700 1 -- 172.21.15.107:0/2516621745 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7448097990 con 0x5600ee3f6580 2023-07-21T22:31:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f74638acb80 1 -- 172.21.15.107:0/2516621745 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7440011e10 con 0x5600ee3f6580 2023-07-21T22:31:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f7438ff9700 1 -- 172.21.15.107:0/2516621745 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7448099850 con 0x5600ee3f6580 2023-07-21T22:31:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f74638acb80 10 client.25604 fetch_fsmap learned FSMap version 34 2023-07-21T22:31:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f74638acb80 1 -- 172.21.15.107:0/2516621745 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5600ee19b1f0 con 0x5600ee3f6580 2023-07-21T22:31:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f7438ff9700 1 -- 172.21.15.107:0/2516621745 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7440011e10 con 0x5600ee3f6580 2023-07-21T22:31:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.569+0000 7f7438ff9700 1 -- 172.21.15.107:0/2516621745 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f744801b9d0 con 0x5600ee3f6580 2023-07-21T22:31:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.570+0000 7f74638acb80 10 client.25604 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:31:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.570+0000 7f74638acb80 10 client.25604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.570+0000 7f74638acb80 20 client.25604 populate_metadata read hostname 'smithi107' 2023-07-21T22:31:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.570+0000 7f74638acb80 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5600ee4678f0 0x5600ee469d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.570+0000 7f74638acb80 4 client.25604 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:31:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.570+0000 7f74638acb80 1 -- 172.21.15.107:0/2516621745 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5600ee301ff0 con 0x5600ee4678f0 2023-07-21T22:31:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.570+0000 7f74554c3700 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5600ee4678f0 0x5600ee469d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.570+0000 7f74554c3700 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5600ee4678f0 0x5600ee469d00 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=773af841a4174ba0 server_cookie=4efd8e129501ac1 in_seq=0 out_seq=0 2023-07-21T22:31:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:22.570+0000 7f7438ff9700 10 client.25604 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:22.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:22 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:22.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:22 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:22.702 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:22 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:31:23.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:23 smithi190 ceph-mon[98334]: pgmap v363: 97 pgs: 97 active+clean; 254 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 622 op/s 2023-07-21T22:31:23.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:23 smithi132 ceph-mon[97863]: pgmap v363: 97 pgs: 97 active+clean; 254 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 622 op/s 2023-07-21T22:31:23.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:23 smithi107 ceph-mon[96543]: pgmap v363: 97 pgs: 97 active+clean; 254 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 622 op/s 2023-07-21T22:31:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.900+0000 7f7438ff9700 1 -- 172.21.15.107:0/2516621745 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5600ee301ff0 con 0x5600ee4678f0 2023-07-21T22:31:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.900+0000 7f7438ff9700 10 client.25604 handle_command_reply: tid=0 2023-07-21T22:31:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.900+0000 7f74638acb80 1 client.25604 shutdown 2023-07-21T22:31:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f74638acb80 1 -- 172.21.15.107:0/2516621745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 msgr2=0x5600ee40d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f74638acb80 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee40d950 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f74480962c0 tx=0x7f74480c9e20 comp rx=0 tx=0).stop 2023-07-21T22:31:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f74638acb80 1 -- 172.21.15.107:0/2516621745 shutdown_connections 2023-07-21T22:31:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f74638acb80 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5600ee4678f0 0x5600ee469d00 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f74638acb80 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5600ee3f6580 0x5600ee40d950 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f74638acb80 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600ee2516b0 0x5600ee40a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f7438ff9700 0 client.25604 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f74638acb80 1 --2- 172.21.15.107:0/2516621745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600ee3f5830 0x5600ee40a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f74638acb80 1 -- 172.21.15.107:0/2516621745 >> 172.21.15.107:0/2516621745 conn(0x5600ee3eaae0 msgr2=0x5600ee405d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f74638acb80 1 -- 172.21.15.107:0/2516621745 shutdown_connections 2023-07-21T22:31:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f74638acb80 1 -- 172.21.15.107:0/2516621745 wait complete. 2023-07-21T22:31:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f74638acb80 20 client.25604 trim_cache size 0 max 16384 2023-07-21T22:31:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:31:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:31:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:31:23.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:31:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f7457727700 1 -- 172.21.15.107:0/838379891 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f743c0a11d0 msgr2=0x7f743c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f7457727700 1 --2- 172.21.15.107:0/838379891 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f743c0a11d0 0x7f743c0a3680 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f744c006010 tx=0x7f744c005960 comp rx=0 tx=0).stop 2023-07-21T22:31:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f7457727700 1 -- 172.21.15.107:0/838379891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7450100600 msgr2=0x7f745019d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f7457727700 1 --2- 172.21.15.107:0/838379891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7450100600 0x7f745019d040 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f744800ef30 tx=0x7f744800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:31:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f7457727700 1 -- 172.21.15.107:0/838379891 shutdown_connections 2023-07-21T22:31:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f7457727700 1 --2- 172.21.15.107:0/838379891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7450100600 0x7f745019d040 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f7457727700 1 --2- 172.21.15.107:0/838379891 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f743c0a11d0 0x7f743c0a3680 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f7457727700 1 --2- 172.21.15.107:0/838379891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74500fea80 0x7f74501985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f7457727700 1 --2- 172.21.15.107:0/838379891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74500ffc70 0x7f745019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.901+0000 7f7457727700 1 -- 172.21.15.107:0/838379891 >> 172.21.15.107:0/838379891 conn(0x7f74500fa0c0 msgr2=0x7f74500fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.902+0000 7f7457727700 1 -- 172.21.15.107:0/838379891 shutdown_connections 2023-07-21T22:31:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:23.902+0000 7f7457727700 1 -- 172.21.15.107:0/838379891 wait complete. 2023-07-21T22:31:25.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:25 smithi190 ceph-mon[98334]: pgmap v364: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2023-07-21T22:31:25.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:25 smithi132 ceph-mon[97863]: pgmap v364: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2023-07-21T22:31:25.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:25 smithi107 ceph-mon[96543]: pgmap v364: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2023-07-21T22:31:26.478 INFO:tasks.workunit.client.0.smithi107.stdout: 84 709416 950 469183 1181 229703 172 2023-07-21T22:31:27.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:27 smithi190 ceph-mon[98334]: pgmap v365: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2023-07-21T22:31:27.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:27 smithi132 ceph-mon[97863]: pgmap v365: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2023-07-21T22:31:27.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:27 smithi107 ceph-mon[96543]: pgmap v365: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2023-07-21T22:31:29.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:29 smithi190 ceph-mon[98334]: pgmap v366: 97 pgs: 97 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2023-07-21T22:31:29.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:29 smithi132 ceph-mon[97863]: pgmap v366: 97 pgs: 97 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2023-07-21T22:31:29.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:29 smithi107 ceph-mon[96543]: pgmap v366: 97 pgs: 97 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2023-07-21T22:31:31.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:31 smithi190 ceph-mon[98334]: pgmap v367: 97 pgs: 97 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2023-07-21T22:31:31.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:31 smithi132 ceph-mon[97863]: pgmap v367: 97 pgs: 97 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2023-07-21T22:31:31.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:31 smithi107 ceph-mon[96543]: pgmap v367: 97 pgs: 97 active+clean; 264 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2023-07-21T22:31:33.643 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:33 smithi190 ceph-mon[98334]: pgmap v368: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2023-07-21T22:31:33.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:33 smithi132 ceph-mon[97863]: pgmap v368: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2023-07-21T22:31:33.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:33 smithi107 ceph-mon[96543]: pgmap v368: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2023-07-21T22:31:34.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:34 smithi190 ceph-mon[98334]: pgmap v369: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 250 op/s 2023-07-21T22:31:34.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:34 smithi132 ceph-mon[97863]: pgmap v369: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 250 op/s 2023-07-21T22:31:34.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:34 smithi107 ceph-mon[96543]: pgmap v369: 97 pgs: 97 active+clean; 271 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 250 op/s 2023-07-21T22:31:36.487 INFO:tasks.workunit.client.0.smithi107.stdout: 99 1143502 678 771806 996 309871 96 2023-07-21T22:31:37.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.037+0000 7ff3605cc700 1 -- 172.21.15.107:0/3726227618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3580ffb00 msgr2=0x7ff3580fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.037+0000 7ff3605cc700 1 --2- 172.21.15.107:0/3726227618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3580ffb00 0x7ff3580fff10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff354009990 tx=0x7ff354009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.037+0000 7ff3605cc700 1 -- 172.21.15.107:0/3726227618 shutdown_connections 2023-07-21T22:31:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.037+0000 7ff3605cc700 1 --2- 172.21.15.107:0/3726227618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff358101680 0x7ff358107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.037+0000 7ff3605cc700 1 --2- 172.21.15.107:0/3726227618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3580ffb00 0x7ff3580fff10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.037+0000 7ff3605cc700 1 --2- 172.21.15.107:0/3726227618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff358100cf0 0x7ff358101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.037+0000 7ff3605cc700 1 -- 172.21.15.107:0/3726227618 >> 172.21.15.107:0/3726227618 conn(0x7ff3580fb120 msgr2=0x7ff3580fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.037+0000 7ff3605cc700 1 -- 172.21.15.107:0/3726227618 shutdown_connections 2023-07-21T22:31:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.038+0000 7ff3605cc700 1 -- 172.21.15.107:0/3726227618 wait complete. 2023-07-21T22:31:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.038+0000 7ff3605cc700 1 Processor -- start 2023-07-21T22:31:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.038+0000 7ff3605cc700 1 -- start start 2023-07-21T22:31:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.038+0000 7ff3605cc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3580ffb00 0x7ff358197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.038+0000 7ff3605cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff358100cf0 0x7ff3581980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.038+0000 7ff3605cc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff358101680 0x7ff358198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff3605cc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff358199590 con 0x7ff3580ffb00 2023-07-21T22:31:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff3605cc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff358199700 con 0x7ff358100cf0 2023-07-21T22:31:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff3605cc700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3581998c0 con 0x7ff358101680 2023-07-21T22:31:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff35e368700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3580ffb00 0x7ff358197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff35e368700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3580ffb00 0x7ff358197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49804/0 (socket says 172.21.15.107:49804) 2023-07-21T22:31:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff35e368700 1 -- 172.21.15.107:0/3033290776 learned_addr learned my addr 172.21.15.107:0/3033290776 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff35eb69700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff358101680 0x7ff358198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff35db67700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff358100cf0 0x7ff3581980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff35eb69700 1 -- 172.21.15.107:0/3033290776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff358100cf0 msgr2=0x7ff3581980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff35eb69700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff358100cf0 0x7ff3581980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff35eb69700 1 -- 172.21.15.107:0/3033290776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3580ffb00 msgr2=0x7ff358197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff35eb69700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3580ffb00 0x7ff358197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff35eb69700 1 -- 172.21.15.107:0/3033290776 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3540095f0 con 0x7ff358101680 2023-07-21T22:31:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff35e368700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3580ffb00 0x7ff358197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff35eb69700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff358101680 0x7ff358198f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff35000efc0 tx=0x7ff35000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff34f7fe700 1 -- 172.21.15.107:0/3033290776 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff350018070 con 0x7ff358101680 2023-07-21T22:31:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff3605cc700 1 -- 172.21.15.107:0/3033290776 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff358199b40 con 0x7ff358101680 2023-07-21T22:31:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff34f7fe700 1 -- 172.21.15.107:0/3033290776 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff35000f040 con 0x7ff358101680 2023-07-21T22:31:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.039+0000 7ff34f7fe700 1 -- 172.21.15.107:0/3033290776 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35000cd40 con 0x7ff358101680 2023-07-21T22:31:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.040+0000 7ff3605cc700 1 -- 172.21.15.107:0/3033290776 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3581a0360 con 0x7ff358101680 2023-07-21T22:31:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.043+0000 7ff34f7fe700 1 -- 172.21.15.107:0/3033290776 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff350003a00 con 0x7ff358101680 2023-07-21T22:31:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.043+0000 7ff34f7fe700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff3440a1150 0x7ff3440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.043+0000 7ff34f7fe700 1 -- 172.21.15.107:0/3033290776 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff3500cc500 con 0x7ff358101680 2023-07-21T22:31:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.043+0000 7ff35e368700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff3440a1150 0x7ff3440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.044+0000 7ff36c751b80 1 Processor -- start 2023-07-21T22:31:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.044+0000 7ff36c751b80 1 -- start start 2023-07-21T22:31:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.044+0000 7ff35e368700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff3440a1150 0x7ff3440a3600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff35400b2e0 tx=0x7ff354005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.044+0000 7ff36c751b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 0x557b063ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.044+0000 7ff36c751b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 0x557b063ad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.044+0000 7ff36c751b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063ad580 0x557b063ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.044+0000 7ff36c751b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b062b1c50 con 0x557b063ac830 2023-07-21T22:31:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.044+0000 7ff36c751b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b063049e0 con 0x557b063ad580 2023-07-21T22:31:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.044+0000 7ff36c751b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b062fde70 con 0x557b062086b0 2023-07-21T22:31:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.044+0000 7ff35e368700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 0x557b063ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.044+0000 7ff35db67700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 0x557b063ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.045+0000 7ff35e368700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 0x557b063ac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38370/0 (socket says 172.21.15.107:38370) 2023-07-21T22:31:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.045+0000 7ff35db67700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 0x557b063ad040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49814/0 (socket says 172.21.15.107:49814) 2023-07-21T22:31:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.045+0000 7ff35eb69700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063ad580 0x557b063ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.045+0000 7ff35e368700 1 -- 172.21.15.107:0/1924815970 learned_addr learned my addr 172.21.15.107:0/1924815970 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.045+0000 7ff35db67700 1 -- 172.21.15.107:0/1924815970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 msgr2=0x557b063ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.045+0000 7ff35db67700 1 --2- 172.21.15.107:0/1924815970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 0x557b063ac2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.045+0000 7ff35db67700 1 -- 172.21.15.107:0/1924815970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063ad580 msgr2=0x557b063ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.045+0000 7ff35db67700 1 --2- 172.21.15.107:0/1924815970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063ad580 0x557b063ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.045+0000 7ff35db67700 1 -- 172.21.15.107:0/1924815970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b0612bd80 con 0x557b063ac830 2023-07-21T22:31:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.045+0000 7ff35db67700 1 --2- 172.21.15.107:0/1924815970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 0x557b063ad040 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff348009cb0 tx=0x7ff348011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f86ee95892eddb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:37.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.045+0000 7ff34cff9700 1 -- 172.21.15.107:0/1924815970 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348011d20 con 0x557b063ac830 2023-07-21T22:31:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.045+0000 7ff34cff9700 1 -- 172.21.15.107:0/1924815970 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff348011e80 con 0x557b063ac830 2023-07-21T22:31:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.045+0000 7ff34cff9700 1 -- 172.21.15.107:0/1924815970 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348005280 con 0x557b063ac830 2023-07-21T22:31:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.046+0000 7ff36c751b80 1 -- 172.21.15.107:0/1924815970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 msgr2=0x557b063ad040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.046+0000 7ff36c751b80 1 --2- 172.21.15.107:0/1924815970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 0x557b063ad040 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff348009cb0 tx=0x7ff348011440 comp rx=0 tx=0).stop 2023-07-21T22:31:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.046+0000 7ff36c751b80 1 -- 172.21.15.107:0/1924815970 shutdown_connections 2023-07-21T22:31:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.046+0000 7ff36c751b80 1 --2- 172.21.15.107:0/1924815970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063ad580 0x557b063ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.046+0000 7ff36c751b80 1 --2- 172.21.15.107:0/1924815970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 0x557b063ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.046+0000 7ff36c751b80 1 --2- 172.21.15.107:0/1924815970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 0x557b063ad040 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.046+0000 7ff36c751b80 1 -- 172.21.15.107:0/1924815970 >> 172.21.15.107:0/1924815970 conn(0x557b063a1ae0 msgr2=0x557b063a6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.046+0000 7ff36c751b80 1 -- 172.21.15.107:0/1924815970 shutdown_connections 2023-07-21T22:31:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.046+0000 7ff36c751b80 1 -- 172.21.15.107:0/1924815970 wait complete. 2023-07-21T22:31:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.046+0000 7ff36c751b80 1 -- start start 2023-07-21T22:31:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.046+0000 7ff36c751b80 1 Processor -- start 2023-07-21T22:31:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff36c751b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b062086b0 0x557b063b5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff36c751b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b063ac830 0x557b063b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff36c751b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 0x557b063bcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff36c751b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0629da60 con 0x557b063ad580 2023-07-21T22:31:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff36c751b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0629d760 con 0x557b062086b0 2023-07-21T22:31:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff36c751b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b061818c0 con 0x557b063ac830 2023-07-21T22:31:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff36c751b80 10 client.0 _ll_register_callbacks cb 0x557b0629b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff35eb69700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 0x557b063bcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff35db67700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b063ac830 0x557b063b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff35e368700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b062086b0 0x557b063b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff35eb69700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 0x557b063bcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49822/0 (socket says 172.21.15.107:49822) 2023-07-21T22:31:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff35eb69700 1 -- 172.21.15.107:0/1383760045 learned_addr learned my addr 172.21.15.107:0/1383760045 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff35db67700 1 -- 172.21.15.107:0/1383760045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b062086b0 msgr2=0x557b063b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff35db67700 1 --2- 172.21.15.107:0/1383760045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b062086b0 0x557b063b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff35db67700 1 -- 172.21.15.107:0/1383760045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 msgr2=0x557b063bcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff35db67700 1 --2- 172.21.15.107:0/1383760045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 0x557b063bcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff35db67700 1 -- 172.21.15.107:0/1383760045 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3480098e0 con 0x557b063ac830 2023-07-21T22:31:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff35eb69700 1 --2- 172.21.15.107:0/1383760045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 0x557b063bcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff35db67700 1 --2- 172.21.15.107:0/1383760045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b063ac830 0x557b063b5cd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff348000c00 tx=0x7ff348011b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5473f3153ee9a1f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff341ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:31:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.047+0000 7ff341ffb700 1 -- 172.21.15.107:0/1383760045 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348004260 con 0x557b063ac830 2023-07-21T22:31:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff36c751b80 1 -- 172.21.15.107:0/1383760045 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b062f6e50 con 0x557b063ac830 2023-07-21T22:31:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff341ffb700 1 -- 172.21.15.107:0/1383760045 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff3480043c0 con 0x557b063ac830 2023-07-21T22:31:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff341ffb700 1 -- 172.21.15.107:0/1383760045 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff348004580 con 0x557b063ac830 2023-07-21T22:31:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff36c751b80 10 client.25684 fetch_fsmap learned FSMap version 34 2023-07-21T22:31:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff36c751b80 1 -- 172.21.15.107:0/1383760045 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557b06114c00 con 0x557b063ac830 2023-07-21T22:31:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff341ffb700 1 -- 172.21.15.107:0/1383760045 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff348031020 con 0x557b063ac830 2023-07-21T22:31:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff341ffb700 1 -- 172.21.15.107:0/1383760045 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff348003a20 con 0x557b063ac830 2023-07-21T22:31:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff36c751b80 10 client.25684 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:31:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff36c751b80 10 client.25684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff36c751b80 20 client.25684 populate_metadata read hostname 'smithi107' 2023-07-21T22:31:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff36c751b80 1 --2- 172.21.15.107:0/1383760045 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557b063c1550 0x557b063c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff36c751b80 4 client.25684 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff36c751b80 1 -- 172.21.15.107:0/1383760045 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b062b8ff0 con 0x557b063c1550 2023-07-21T22:31:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.048+0000 7ff35e368700 1 --2- 172.21.15.107:0/1383760045 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557b063c1550 0x557b063c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.049+0000 7ff341ffb700 10 client.25684 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.049+0000 7ff35e368700 1 --2- 172.21.15.107:0/1383760045 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557b063c1550 0x557b063c3960 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb76eb3feaeecd9 server_cookie=b1f3d15c349f5c51 in_seq=0 out_seq=0 2023-07-21T22:31:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff341ffb700 1 -- 172.21.15.107:0/1383760045 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557b062b8ff0 con 0x557b063c1550 2023-07-21T22:31:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff341ffb700 10 client.25684 handle_command_reply: tid=0 2023-07-21T22:31:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff36c751b80 1 client.25684 shutdown 2023-07-21T22:31:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff36c751b80 1 -- 172.21.15.107:0/1383760045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b063ac830 msgr2=0x557b063b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff36c751b80 1 --2- 172.21.15.107:0/1383760045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b063ac830 0x557b063b5cd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff348000c00 tx=0x7ff348011b20 comp rx=0 tx=0).stop 2023-07-21T22:31:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff36c751b80 1 -- 172.21.15.107:0/1383760045 shutdown_connections 2023-07-21T22:31:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff36c751b80 1 --2- 172.21.15.107:0/1383760045 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557b063c1550 0x557b063c3960 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff36c751b80 1 --2- 172.21.15.107:0/1383760045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 0x557b063bcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff36c751b80 1 --2- 172.21.15.107:0/1383760045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b062086b0 0x557b063b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff36c751b80 1 --2- 172.21.15.107:0/1383760045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b063ac830 0x557b063b5cd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff341ffb700 0 client.25684 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff36c751b80 1 -- 172.21.15.107:0/1383760045 >> 172.21.15.107:0/1383760045 conn(0x557b063a1ae0 msgr2=0x557b063a63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff36c751b80 1 -- 172.21.15.107:0/1383760045 shutdown_connections 2023-07-21T22:31:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff36c751b80 1 -- 172.21.15.107:0/1383760045 wait complete. 2023-07-21T22:31:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.051+0000 7ff36c751b80 20 client.25684 trim_cache size 0 max 16384 2023-07-21T22:31:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.068+0000 7ff36c751b80 1 Processor -- start 2023-07-21T22:31:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.068+0000 7ff36c751b80 1 -- start start 2023-07-21T22:31:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.068+0000 7ff36c751b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 0x557b063ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.068+0000 7ff36c751b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 0x557b063bccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.068+0000 7ff36c751b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063ad580 0x557b063a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.068+0000 7ff36c751b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b063a3c30 con 0x557b063ac830 2023-07-21T22:31:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.068+0000 7ff36c751b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b063ade90 con 0x557b063ad580 2023-07-21T22:31:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.068+0000 7ff36c751b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b063ace50 con 0x557b062086b0 2023-07-21T22:31:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.068+0000 7ff35db67700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 0x557b063bccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.068+0000 7ff35db67700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 0x557b063bccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49828/0 (socket says 172.21.15.107:49828) 2023-07-21T22:31:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.068+0000 7ff35db67700 1 -- 172.21.15.107:0/663054575 learned_addr learned my addr 172.21.15.107:0/663054575 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.068+0000 7ff35eb69700 1 --2- 172.21.15.107:0/663054575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063ad580 0x557b063a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.069+0000 7ff35db67700 1 -- 172.21.15.107:0/663054575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 msgr2=0x557b063ad950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:31:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.069+0000 7ff35db67700 1 --2- 172.21.15.107:0/663054575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 0x557b063ad950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.069+0000 7ff35db67700 1 -- 172.21.15.107:0/663054575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063ad580 msgr2=0x557b063a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.069+0000 7ff35db67700 1 --2- 172.21.15.107:0/663054575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063ad580 0x557b063a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.069+0000 7ff35db67700 1 -- 172.21.15.107:0/663054575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b061521f0 con 0x557b063ac830 2023-07-21T22:31:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.069+0000 7ff35eb69700 1 --2- 172.21.15.107:0/663054575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063ad580 0x557b063a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.069+0000 7ff35db67700 1 --2- 172.21.15.107:0/663054575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 0x557b063bccd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff3480092a0 tx=0x7ff3480117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f18a283f876be6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.069+0000 7ff34d7fa700 1 -- 172.21.15.107:0/663054575 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348003a20 con 0x557b063ac830 2023-07-21T22:31:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.069+0000 7ff34d7fa700 1 -- 172.21.15.107:0/663054575 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff348031070 con 0x557b063ac830 2023-07-21T22:31:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.069+0000 7ff34d7fa700 1 -- 172.21.15.107:0/663054575 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34802bb00 con 0x557b063ac830 2023-07-21T22:31:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.070+0000 7ff36c751b80 1 -- 172.21.15.107:0/663054575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 msgr2=0x557b063bccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.070+0000 7ff36c751b80 1 --2- 172.21.15.107:0/663054575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 0x557b063bccd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff3480092a0 tx=0x7ff3480117d0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.070+0000 7ff36c751b80 1 -- 172.21.15.107:0/663054575 shutdown_connections 2023-07-21T22:31:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.070+0000 7ff36c751b80 1 --2- 172.21.15.107:0/663054575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063ad580 0x557b063a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.070+0000 7ff36c751b80 1 --2- 172.21.15.107:0/663054575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 0x557b063ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.070+0000 7ff36c751b80 1 --2- 172.21.15.107:0/663054575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ac830 0x557b063bccd0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.070+0000 7ff36c751b80 1 -- 172.21.15.107:0/663054575 >> 172.21.15.107:0/663054575 conn(0x557b063a1ae0 msgr2=0x557b063c8280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.070+0000 7ff36c751b80 1 -- 172.21.15.107:0/663054575 shutdown_connections 2023-07-21T22:31:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.070+0000 7ff36c751b80 1 -- 172.21.15.107:0/663054575 wait complete. 2023-07-21T22:31:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff36c751b80 1 -- start start 2023-07-21T22:31:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff36c751b80 1 Processor -- start 2023-07-21T22:31:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff36c751b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 0x557b063b8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff36c751b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 0x557b063b8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff36c751b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063bbfa0 0x557b063bc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff36c751b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b063bc950 con 0x557b063ad580 2023-07-21T22:31:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff36c751b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b063bcac0 con 0x557b063bbfa0 2023-07-21T22:31:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff36c751b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b063bcc80 con 0x557b062086b0 2023-07-21T22:31:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff36c751b80 10 client.0 _ll_register_callbacks cb 0x7ff35000cec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff35db67700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 0x557b063b8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff35e368700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 0x557b063b8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff35db67700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 0x557b063b8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49834/0 (socket says 172.21.15.107:49834) 2023-07-21T22:31:37.122 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff35db67700 1 -- 172.21.15.107:0/519978585 learned_addr learned my addr 172.21.15.107:0/519978585 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.071+0000 7ff35eb69700 1 --2- 172.21.15.107:0/519978585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063bbfa0 0x557b063bc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff35db67700 1 -- 172.21.15.107:0/519978585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 msgr2=0x557b063b8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff35db67700 1 --2- 172.21.15.107:0/519978585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 0x557b063b8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff35db67700 1 -- 172.21.15.107:0/519978585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063bbfa0 msgr2=0x557b063bc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff35db67700 1 --2- 172.21.15.107:0/519978585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063bbfa0 0x557b063bc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff35db67700 1 -- 172.21.15.107:0/519978585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b06114c00 con 0x557b063ad580 2023-07-21T22:31:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff35e368700 1 --2- 172.21.15.107:0/519978585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 0x557b063b8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff35db67700 1 --2- 172.21.15.107:0/519978585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 0x557b063b8f80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff348003dd0 tx=0x7ff3480045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2727b8e0a8ece6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff341ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:31:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff341ffb700 1 -- 172.21.15.107:0/519978585 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34802bd10 con 0x557b063ad580 2023-07-21T22:31:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff341ffb700 1 -- 172.21.15.107:0/519978585 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff348000e40 con 0x557b063ad580 2023-07-21T22:31:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff36c751b80 1 -- 172.21.15.107:0/519978585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b06257490 con 0x557b063ad580 2023-07-21T22:31:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff341ffb700 1 -- 172.21.15.107:0/519978585 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff34802be70 con 0x557b063ad580 2023-07-21T22:31:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff36c751b80 10 client.16167 fetch_fsmap learned FSMap version 34 2023-07-21T22:31:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.072+0000 7ff36c751b80 1 -- 172.21.15.107:0/519978585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557b0612a250 con 0x557b063ad580 2023-07-21T22:31:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.073+0000 7ff341ffb700 1 -- 172.21.15.107:0/519978585 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557b06257490 con 0x557b063ad580 2023-07-21T22:31:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.073+0000 7ff341ffb700 1 -- 172.21.15.107:0/519978585 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff348045050 con 0x557b063ad580 2023-07-21T22:31:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.073+0000 7ff36c751b80 10 client.16167 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:31:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.073+0000 7ff36c751b80 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.073+0000 7ff36c751b80 20 client.16167 populate_metadata read hostname 'smithi107' 2023-07-21T22:31:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.073+0000 7ff36c751b80 1 --2- 172.21.15.107:0/519978585 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557b064161e0 0x557b064185c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.073+0000 7ff36c751b80 4 client.16167 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:31:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.073+0000 7ff36c751b80 1 -- 172.21.15.107:0/519978585 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557b0614cf10 con 0x557b064161e0 2023-07-21T22:31:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.073+0000 7ff35e368700 1 --2- 172.21.15.107:0/519978585 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557b064161e0 0x557b064185c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.073+0000 7ff341ffb700 10 client.16167 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.073+0000 7ff35e368700 1 --2- 172.21.15.107:0/519978585 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557b064161e0 0x557b064185c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a9868582da9ebe server_cookie=9a6f32470ae1477a in_seq=0 out_seq=0 2023-07-21T22:31:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff341ffb700 1 -- 172.21.15.107:0/519978585 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557b0614cf10 con 0x557b064161e0 2023-07-21T22:31:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff341ffb700 10 client.16167 handle_command_reply: tid=0 2023-07-21T22:31:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff36c751b80 1 client.16167 shutdown 2023-07-21T22:31:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff36c751b80 1 -- 172.21.15.107:0/519978585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 msgr2=0x557b063b8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff36c751b80 1 --2- 172.21.15.107:0/519978585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 0x557b063b8f80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff348003dd0 tx=0x7ff3480045c0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff36c751b80 1 -- 172.21.15.107:0/519978585 shutdown_connections 2023-07-21T22:31:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff36c751b80 1 --2- 172.21.15.107:0/519978585 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557b064161e0 0x557b064185c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff36c751b80 1 --2- 172.21.15.107:0/519978585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557b063bbfa0 0x557b063bc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff341ffb700 0 client.16167 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff36c751b80 1 --2- 172.21.15.107:0/519978585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b062086b0 0x557b063b8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff36c751b80 1 --2- 172.21.15.107:0/519978585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557b063ad580 0x557b063b8f80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff36c751b80 1 -- 172.21.15.107:0/519978585 >> 172.21.15.107:0/519978585 conn(0x557b063a1ae0 msgr2=0x557b063b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff36c751b80 1 -- 172.21.15.107:0/519978585 shutdown_connections 2023-07-21T22:31:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff36c751b80 1 -- 172.21.15.107:0/519978585 wait complete. 2023-07-21T22:31:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.226+0000 7ff36c751b80 20 client.16167 trim_cache size 0 max 16384 2023-07-21T22:31:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:31:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:31:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:31:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:31:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.227+0000 7ff3605cc700 1 -- 172.21.15.107:0/3033290776 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff3440a1150 msgr2=0x7ff3440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.227+0000 7ff3605cc700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff3440a1150 0x7ff3440a3600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff35400b2e0 tx=0x7ff354005fb0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.227+0000 7ff3605cc700 1 -- 172.21.15.107:0/3033290776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff358101680 msgr2=0x7ff358198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.227+0000 7ff3605cc700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff358101680 0x7ff358198f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff35000efc0 tx=0x7ff35000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.227+0000 7ff3605cc700 1 -- 172.21.15.107:0/3033290776 shutdown_connections 2023-07-21T22:31:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.227+0000 7ff3605cc700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff3440a1150 0x7ff3440a3600 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.227+0000 7ff3605cc700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff358101680 0x7ff358198f30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.227+0000 7ff3605cc700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3580ffb00 0x7ff358197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.227+0000 7ff3605cc700 1 --2- 172.21.15.107:0/3033290776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff358100cf0 0x7ff3581980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.227+0000 7ff3605cc700 1 -- 172.21.15.107:0/3033290776 >> 172.21.15.107:0/3033290776 conn(0x7ff3580fb120 msgr2=0x7ff3581065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.227+0000 7ff3605cc700 1 -- 172.21.15.107:0/3033290776 shutdown_connections 2023-07-21T22:31:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:37.227+0000 7ff3605cc700 1 -- 172.21.15.107:0/3033290776 wait complete. 2023-07-21T22:31:37.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:37 smithi190 ceph-mon[98334]: pgmap v370: 97 pgs: 97 active+clean; 323 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 421 op/s 2023-07-21T22:31:37.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:37 smithi132 ceph-mon[97863]: pgmap v370: 97 pgs: 97 active+clean; 323 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 421 op/s 2023-07-21T22:31:37.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:37 smithi107 ceph-mon[96543]: pgmap v370: 97 pgs: 97 active+clean; 323 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 421 op/s 2023-07-21T22:31:39.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:39 smithi190 ceph-mon[98334]: pgmap v371: 97 pgs: 97 active+clean; 403 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 13 MiB/s wr, 689 op/s 2023-07-21T22:31:39.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:39 smithi132 ceph-mon[97863]: pgmap v371: 97 pgs: 97 active+clean; 403 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 13 MiB/s wr, 689 op/s 2023-07-21T22:31:39.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:39 smithi107 ceph-mon[96543]: pgmap v371: 97 pgs: 97 active+clean; 403 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 13 MiB/s wr, 689 op/s 2023-07-21T22:31:41.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:41 smithi132 ceph-mon[97863]: pgmap v372: 97 pgs: 97 active+clean; 404 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 588 op/s 2023-07-21T22:31:41.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:41 smithi107 ceph-mon[96543]: pgmap v372: 97 pgs: 97 active+clean; 404 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 588 op/s 2023-07-21T22:31:41.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:41 smithi190 ceph-mon[98334]: pgmap v372: 97 pgs: 97 active+clean; 404 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 588 op/s 2023-07-21T22:31:43.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:43 smithi132 ceph-mon[97863]: pgmap v373: 97 pgs: 97 active+clean; 409 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 681 op/s 2023-07-21T22:31:43.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:43 smithi107 ceph-mon[96543]: pgmap v373: 97 pgs: 97 active+clean; 409 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 681 op/s 2023-07-21T22:31:43.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:43 smithi190 ceph-mon[98334]: pgmap v373: 97 pgs: 97 active+clean; 409 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 681 op/s 2023-07-21T22:31:45.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:45 smithi132 ceph-mon[97863]: pgmap v374: 97 pgs: 97 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 632 op/s 2023-07-21T22:31:45.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:45 smithi107 ceph-mon[96543]: pgmap v374: 97 pgs: 97 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 632 op/s 2023-07-21T22:31:45.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:45 smithi190 ceph-mon[98334]: pgmap v374: 97 pgs: 97 active+clean; 409 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 632 op/s 2023-07-21T22:31:46.487 INFO:tasks.workunit.client.0.smithi107.stdout: 106 1069657 480 747190 544 290210 603 2023-07-21T22:31:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:47 smithi132 ceph-mon[97863]: pgmap v375: 97 pgs: 97 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 675 op/s 2023-07-21T22:31:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:47 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:31:47.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:47 smithi107 ceph-mon[96543]: pgmap v375: 97 pgs: 97 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 675 op/s 2023-07-21T22:31:47.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:47 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:31:47.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:47 smithi190 ceph-mon[98334]: pgmap v375: 97 pgs: 97 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 675 op/s 2023-07-21T22:31:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:47 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:31:49.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:49 smithi132 ceph-mon[97863]: pgmap v376: 97 pgs: 97 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.1 MiB/s wr, 509 op/s 2023-07-21T22:31:49.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:49 smithi107 ceph-mon[96543]: pgmap v376: 97 pgs: 97 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.1 MiB/s wr, 509 op/s 2023-07-21T22:31:49.504 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:31:49.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:49 smithi190 ceph-mon[98334]: pgmap v376: 97 pgs: 97 active+clean; 414 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.1 MiB/s wr, 509 op/s 2023-07-21T22:31:49.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.677+0000 7f50cb399700 1 -- 172.21.15.107:0/22613042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c41005d0 msgr2=0x7f50c4106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:49.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.677+0000 7f50cb399700 1 --2- 172.21.15.107:0/22613042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c41005d0 0x7f50c4106e90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00b0a0 tx=0x7f50bc00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.677+0000 7f50cb399700 1 -- 172.21.15.107:0/22613042 shutdown_connections 2023-07-21T22:31:49.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.677+0000 7f50cb399700 1 --2- 172.21.15.107:0/22613042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c41005d0 0x7f50c4106e90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.677+0000 7f50cb399700 1 --2- 172.21.15.107:0/22613042 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50c40fea50 0x7f50c40fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.677+0000 7f50cb399700 1 --2- 172.21.15.107:0/22613042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c40ffc40 0x7f50c4100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.677+0000 7f50cb399700 1 -- 172.21.15.107:0/22613042 >> 172.21.15.107:0/22613042 conn(0x7f50c40fa070 msgr2=0x7f50c40fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:49.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.677+0000 7f50cb399700 1 -- 172.21.15.107:0/22613042 shutdown_connections 2023-07-21T22:31:49.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.677+0000 7f50cb399700 1 -- 172.21.15.107:0/22613042 wait complete. 2023-07-21T22:31:49.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.678+0000 7f50cb399700 1 Processor -- start 2023-07-21T22:31:49.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.678+0000 7f50cb399700 1 -- start start 2023-07-21T22:31:49.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.678+0000 7f50cb399700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c40fea50 0x7f50c4194640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.678+0000 7f50cb399700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c40ffc40 0x7f50c4194b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.678+0000 7f50cb399700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50c41005d0 0x7f50c419bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.678+0000 7f50cb399700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c419c1c0 con 0x7f50c40ffc40 2023-07-21T22:31:49.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.678+0000 7f50cb399700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c419c300 con 0x7f50c41005d0 2023-07-21T22:31:49.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.678+0000 7f50cb399700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c419c490 con 0x7f50c40fea50 2023-07-21T22:31:49.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50c8934700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c40ffc40 0x7f50c4194b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50c9135700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c40fea50 0x7f50c4194640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50c8934700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c40ffc40 0x7f50c4194b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49846/0 (socket says 172.21.15.107:49846) 2023-07-21T22:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50c9135700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c40fea50 0x7f50c4194640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38402/0 (socket says 172.21.15.107:38402) 2023-07-21T22:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50c8934700 1 -- 172.21.15.107:0/184193807 learned_addr learned my addr 172.21.15.107:0/184193807 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50c9936700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50c41005d0 0x7f50c419bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50c9135700 1 -- 172.21.15.107:0/184193807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50c41005d0 msgr2=0x7f50c419bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50c9135700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50c41005d0 0x7f50c419bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50c9135700 1 -- 172.21.15.107:0/184193807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c40ffc40 msgr2=0x7f50c4194b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50c9135700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c40ffc40 0x7f50c4194b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50c9135700 1 -- 172.21.15.107:0/184193807 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c0009520 con 0x7f50c40fea50 2023-07-21T22:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50c8934700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c40ffc40 0x7f50c4194b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50c9135700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c40fea50 0x7f50c4194640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f50c000ead0 tx=0x7f50c000ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50ba7fc700 1 -- 172.21.15.107:0/184193807 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0013070 con 0x7f50c40fea50 2023-07-21T22:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50ba7fc700 1 -- 172.21.15.107:0/184193807 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50c000be20 con 0x7f50c40fea50 2023-07-21T22:31:49.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50cb399700 1 -- 172.21.15.107:0/184193807 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc009d00 con 0x7f50c40fea50 2023-07-21T22:31:49.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50ba7fc700 1 -- 172.21.15.107:0/184193807 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0017e90 con 0x7f50c40fea50 2023-07-21T22:31:49.731 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.679+0000 7f50cb399700 1 -- 172.21.15.107:0/184193807 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c4068c00 con 0x7f50c40fea50 2023-07-21T22:31:49.735 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.681+0000 7f50ba7fc700 1 -- 172.21.15.107:0/184193807 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f50c0017410 con 0x7f50c40fea50 2023-07-21T22:31:49.735 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.682+0000 7f50ba7fc700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f50b00a1150 0x7f50b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.735 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.682+0000 7f50ba7fc700 1 -- 172.21.15.107:0/184193807 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f50c00cdea0 con 0x7f50c40fea50 2023-07-21T22:31:49.735 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.682+0000 7f50c8934700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f50b00a1150 0x7f50b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50c8934700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f50b00a1150 0x7f50b00a3600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f50b4005950 tx=0x7f50b400b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50d751eb80 1 Processor -- start 2023-07-21T22:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50d751eb80 1 -- start start 2023-07-21T22:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50d751eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1ea80 0x55af60e1ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50d751eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1f3d0 0x55af60e1fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50d751eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1fff0 0x55af60e20460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50d751eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60d1fcb0 con 0x55af60e1fff0 2023-07-21T22:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50d751eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60d72a40 con 0x55af60e1ea80 2023-07-21T22:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50d751eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60d6bed0 con 0x55af60e1f3d0 2023-07-21T22:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50c9936700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1fff0 0x55af60e20460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50c9135700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1ea80 0x55af60e1ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50c8934700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1f3d0 0x55af60e1fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50c9936700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1fff0 0x55af60e20460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49856/0 (socket says 172.21.15.107:49856) 2023-07-21T22:31:49.737 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50c9936700 1 -- 172.21.15.107:0/584802203 learned_addr learned my addr 172.21.15.107:0/584802203 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:49.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50c9135700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1ea80 0x55af60e1ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40680/0 (socket says 172.21.15.107:40680) 2023-07-21T22:31:49.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50c9936700 1 -- 172.21.15.107:0/584802203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1f3d0 msgr2=0x55af60e1fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:49.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50c9936700 1 --2- 172.21.15.107:0/584802203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1f3d0 0x55af60e1fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50c9936700 1 -- 172.21.15.107:0/584802203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1ea80 msgr2=0x55af60e1ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:49.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.683+0000 7f50c9936700 1 --2- 172.21.15.107:0/584802203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1ea80 0x55af60e1ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.684+0000 7f50c9936700 1 -- 172.21.15.107:0/584802203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af60b99de0 con 0x55af60e1fff0 2023-07-21T22:31:49.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.684+0000 7f50c9135700 1 --2- 172.21.15.107:0/584802203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1ea80 0x55af60e1ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:49.738 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.684+0000 7f50c9936700 1 --2- 172.21.15.107:0/584802203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1fff0 0x55af60e20460 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00b0a0 tx=0x7f50bc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660b70d4da0d0bd7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:49.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.684+0000 7f50af7fe700 1 -- 172.21.15.107:0/584802203 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc00e040 con 0x55af60e1fff0 2023-07-21T22:31:49.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.684+0000 7f50af7fe700 1 -- 172.21.15.107:0/584802203 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50bc00bdb0 con 0x55af60e1fff0 2023-07-21T22:31:49.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.684+0000 7f50af7fe700 1 -- 172.21.15.107:0/584802203 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc01c800 con 0x55af60e1fff0 2023-07-21T22:31:49.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.684+0000 7f50d751eb80 1 -- 172.21.15.107:0/584802203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1fff0 msgr2=0x55af60e20460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:49.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.684+0000 7f50d751eb80 1 --2- 172.21.15.107:0/584802203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1fff0 0x55af60e20460 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00b0a0 tx=0x7f50bc009500 comp rx=0 tx=0).stop 2023-07-21T22:31:49.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 -- 172.21.15.107:0/584802203 shutdown_connections 2023-07-21T22:31:49.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 --2- 172.21.15.107:0/584802203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1fff0 0x55af60e20460 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.739 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 --2- 172.21.15.107:0/584802203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1ea80 0x55af60e1ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 --2- 172.21.15.107:0/584802203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1f3d0 0x55af60e1fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 -- 172.21.15.107:0/584802203 >> 172.21.15.107:0/584802203 conn(0x55af60e0fef0 msgr2=0x55af60e15360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 -- 172.21.15.107:0/584802203 shutdown_connections 2023-07-21T22:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 -- 172.21.15.107:0/584802203 wait complete. 2023-07-21T22:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 -- start start 2023-07-21T22:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 Processor -- start 2023-07-21T22:31:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 0x55af60e28430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 0x55af60e28970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1fff0 0x55af60e2f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60d0bac0 con 0x55af60e1f3d0 2023-07-21T22:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60d0b7c0 con 0x55af60e1fff0 2023-07-21T22:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60bef920 con 0x55af60e1ea80 2023-07-21T22:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50d751eb80 10 client.0 _ll_register_callbacks cb 0x55af60d09760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.685+0000 7f50c8934700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 0x55af60e28970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.741 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50c9135700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 0x55af60e28430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50c8934700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 0x55af60e28970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49860/0 (socket says 172.21.15.107:49860) 2023-07-21T22:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50c8934700 1 -- 172.21.15.107:0/2615060269 learned_addr learned my addr 172.21.15.107:0/2615060269 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50c9936700 1 --2- 172.21.15.107:0/2615060269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1fff0 0x55af60e2f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50c8934700 1 -- 172.21.15.107:0/2615060269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 msgr2=0x55af60e28430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50c8934700 1 --2- 172.21.15.107:0/2615060269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 0x55af60e28430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50c8934700 1 -- 172.21.15.107:0/2615060269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1fff0 msgr2=0x55af60e2f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:49.742 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50c8934700 1 --2- 172.21.15.107:0/2615060269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1fff0 0x55af60e2f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50c8934700 1 -- 172.21.15.107:0/2615060269 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c4068c00 con 0x55af60e1f3d0 2023-07-21T22:31:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50c8934700 1 --2- 172.21.15.107:0/2615060269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 0x55af60e28970 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f50b4011b70 tx=0x7f50b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d875efb4ec4fd9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50acff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:31:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50acff9700 1 -- 172.21.15.107:0/2615060269 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4028630 con 0x55af60e1f3d0 2023-07-21T22:31:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50acff9700 1 -- 172.21.15.107:0/2615060269 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f50b4017070 con 0x55af60e1f3d0 2023-07-21T22:31:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50d751eb80 1 -- 172.21.15.107:0/2615060269 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af60cc54f0 con 0x55af60e1f3d0 2023-07-21T22:31:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50acff9700 1 -- 172.21.15.107:0/2615060269 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50b4028790 con 0x55af60e1f3d0 2023-07-21T22:31:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50d751eb80 10 client.16179 fetch_fsmap learned FSMap version 34 2023-07-21T22:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50d751eb80 1 -- 172.21.15.107:0/2615060269 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f50c41077c0 con 0x55af60e1f3d0 2023-07-21T22:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.686+0000 7f50acff9700 1 -- 172.21.15.107:0/2615060269 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55af60cc54f0 con 0x55af60e1f3d0 2023-07-21T22:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.687+0000 7f50acff9700 1 -- 172.21.15.107:0/2615060269 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f50b4030020 con 0x55af60e1f3d0 2023-07-21T22:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.687+0000 7f50d751eb80 10 client.16179 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.687+0000 7f50d751eb80 10 client.16179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.687+0000 7f50d751eb80 20 client.16179 populate_metadata read hostname 'smithi107' 2023-07-21T22:31:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.687+0000 7f50d751eb80 1 --2- 172.21.15.107:0/2615060269 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55af60e342b0 0x55af60e366c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.687+0000 7f50d751eb80 4 client.16179 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.687+0000 7f50d751eb80 1 -- 172.21.15.107:0/2615060269 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af60d64eb0 con 0x55af60e342b0 2023-07-21T22:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.687+0000 7f50c9135700 1 --2- 172.21.15.107:0/2615060269 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55af60e342b0 0x55af60e366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.687+0000 7f50acff9700 10 client.16179 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.687+0000 7f50c9135700 1 --2- 172.21.15.107:0/2615060269 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55af60e342b0 0x55af60e366c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc2e95fedc3e47e server_cookie=b5fe305eae3d6055 in_seq=0 out_seq=0 2023-07-21T22:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50acff9700 1 -- 172.21.15.107:0/2615060269 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55af60d64eb0 con 0x55af60e342b0 2023-07-21T22:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50acff9700 10 client.16179 handle_command_reply: tid=0 2023-07-21T22:31:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50d751eb80 1 client.16179 shutdown 2023-07-21T22:31:49.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50d751eb80 1 -- 172.21.15.107:0/2615060269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 msgr2=0x55af60e28970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:49.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50d751eb80 1 --2- 172.21.15.107:0/2615060269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 0x55af60e28970 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f50b4011b70 tx=0x7f50b40047c0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50d751eb80 1 -- 172.21.15.107:0/2615060269 shutdown_connections 2023-07-21T22:31:49.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50d751eb80 1 --2- 172.21.15.107:0/2615060269 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55af60e342b0 0x55af60e366c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50d751eb80 1 --2- 172.21.15.107:0/2615060269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1fff0 0x55af60e2f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50acff9700 0 client.16179 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:49.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50d751eb80 1 --2- 172.21.15.107:0/2615060269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 0x55af60e28430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.747 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50d751eb80 1 --2- 172.21.15.107:0/2615060269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 0x55af60e28970 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.747 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50d751eb80 1 -- 172.21.15.107:0/2615060269 >> 172.21.15.107:0/2615060269 conn(0x55af60e0fef0 msgr2=0x55af60e14a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:49.747 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50d751eb80 1 -- 172.21.15.107:0/2615060269 shutdown_connections 2023-07-21T22:31:49.747 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50d751eb80 1 -- 172.21.15.107:0/2615060269 wait complete. 2023-07-21T22:31:49.747 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.688+0000 7f50d751eb80 20 client.16179 trim_cache size 0 max 16384 2023-07-21T22:31:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.707+0000 7f50d751eb80 1 Processor -- start 2023-07-21T22:31:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.707+0000 7f50d751eb80 1 -- start start 2023-07-21T22:31:49.754 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.707+0000 7f50d751eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 0x55af60e3afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.755 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.707+0000 7f50d751eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 0x55af60e2f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.755 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.707+0000 7f50d751eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1fff0 0x55af60e26300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.755 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.707+0000 7f50d751eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60e26120 con 0x55af60e1f3d0 2023-07-21T22:31:49.755 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.707+0000 7f50d751eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60e3b520 con 0x55af60e1fff0 2023-07-21T22:31:49.755 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.707+0000 7f50d751eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60e281e0 con 0x55af60e1ea80 2023-07-21T22:31:49.755 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.707+0000 7f50c8934700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 0x55af60e2f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.755 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.707+0000 7f50c8934700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 0x55af60e2f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49868/0 (socket says 172.21.15.107:49868) 2023-07-21T22:31:49.756 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.707+0000 7f50c8934700 1 -- 172.21.15.107:0/2711004169 learned_addr learned my addr 172.21.15.107:0/2711004169 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:49.756 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.707+0000 7f50c9135700 1 --2- 172.21.15.107:0/2711004169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 0x55af60e3afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.756 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.707+0000 7f50c9936700 1 --2- 172.21.15.107:0/2711004169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1fff0 0x55af60e26300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.756 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.708+0000 7f50c8934700 1 -- 172.21.15.107:0/2711004169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 msgr2=0x55af60e3afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:49.756 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.708+0000 7f50c8934700 1 --2- 172.21.15.107:0/2711004169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 0x55af60e3afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.759 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.708+0000 7f50c8934700 1 -- 172.21.15.107:0/2711004169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1fff0 msgr2=0x55af60e26300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:49.759 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.708+0000 7f50c8934700 1 --2- 172.21.15.107:0/2711004169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1fff0 0x55af60e26300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.759 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.708+0000 7f50c8934700 1 -- 172.21.15.107:0/2711004169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af60b82c60 con 0x55af60e1f3d0 2023-07-21T22:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.708+0000 7f50c9936700 1 --2- 172.21.15.107:0/2711004169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1fff0 0x55af60e26300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.708+0000 7f50c8934700 1 --2- 172.21.15.107:0/2711004169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 0x55af60e2f970 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f50b4010f40 tx=0x7f50b401fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbe208dc3a134ee server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.708+0000 7f50affff700 1 -- 172.21.15.107:0/2711004169 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b401f630 con 0x55af60e1f3d0 2023-07-21T22:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.708+0000 7f50affff700 1 -- 172.21.15.107:0/2711004169 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50b4023070 con 0x55af60e1f3d0 2023-07-21T22:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.708+0000 7f50affff700 1 -- 172.21.15.107:0/2711004169 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4032ea0 con 0x55af60e1f3d0 2023-07-21T22:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.709+0000 7f50d751eb80 1 -- 172.21.15.107:0/2711004169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 msgr2=0x55af60e2f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:49.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.709+0000 7f50d751eb80 1 --2- 172.21.15.107:0/2711004169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 0x55af60e2f970 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f50b4010f40 tx=0x7f50b401fd60 comp rx=0 tx=0).stop 2023-07-21T22:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.709+0000 7f50d751eb80 1 -- 172.21.15.107:0/2711004169 shutdown_connections 2023-07-21T22:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.709+0000 7f50d751eb80 1 --2- 172.21.15.107:0/2711004169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1fff0 0x55af60e26300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.709+0000 7f50d751eb80 1 --2- 172.21.15.107:0/2711004169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 0x55af60e3afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.709+0000 7f50d751eb80 1 --2- 172.21.15.107:0/2711004169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1f3d0 0x55af60e2f970 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.709+0000 7f50d751eb80 1 -- 172.21.15.107:0/2711004169 >> 172.21.15.107:0/2711004169 conn(0x55af60e0fef0 msgr2=0x55af60e271a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.709+0000 7f50d751eb80 1 -- 172.21.15.107:0/2711004169 shutdown_connections 2023-07-21T22:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.709+0000 7f50d751eb80 1 -- 172.21.15.107:0/2711004169 wait complete. 2023-07-21T22:31:49.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.709+0000 7f50d751eb80 1 -- start start 2023-07-21T22:31:49.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.709+0000 7f50d751eb80 1 Processor -- start 2023-07-21T22:31:49.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50d751eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 0x55af60e2f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50d751eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1f3d0 0x55af60e2bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50d751eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1fff0 0x55af60e2c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50d751eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60e2cb60 con 0x55af60e1fff0 2023-07-21T22:31:49.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50d751eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60e2fb20 con 0x55af60e1f3d0 2023-07-21T22:31:49.762 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50d751eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af60e2fcb0 con 0x55af60e1ea80 2023-07-21T22:31:49.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50d751eb80 10 client.0 _ll_register_callbacks cb 0x7f50b4011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:49.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50c8934700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1f3d0 0x55af60e2bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50c9936700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1fff0 0x55af60e2c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50c9135700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 0x55af60e2f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50c8934700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1f3d0 0x55af60e2bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40702/0 (socket says 172.21.15.107:40702) 2023-07-21T22:31:49.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50c9936700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1fff0 0x55af60e2c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49876/0 (socket says 172.21.15.107:49876) 2023-07-21T22:31:49.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50c8934700 1 -- 172.21.15.107:0/3316747131 learned_addr learned my addr 172.21.15.107:0/3316747131 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:49.763 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50c9135700 1 -- 172.21.15.107:0/3316747131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1f3d0 msgr2=0x55af60e2bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50c9135700 1 --2- 172.21.15.107:0/3316747131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1f3d0 0x55af60e2bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50c9135700 1 -- 172.21.15.107:0/3316747131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1fff0 msgr2=0x55af60e2c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50c9135700 1 --2- 172.21.15.107:0/3316747131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1fff0 0x55af60e2c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.710+0000 7f50c9135700 1 -- 172.21.15.107:0/3316747131 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af60b99de0 con 0x55af60e1ea80 2023-07-21T22:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50c9135700 1 --2- 172.21.15.107:0/3316747131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 0x55af60e2f4a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f50c009af90 tx=0x7f50c00cb850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19fbbb58d3a30f08 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50acff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:31:49.764 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50acff9700 1 -- 172.21.15.107:0/3316747131 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0095ea0 con 0x55af60e1ea80 2023-07-21T22:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50acff9700 1 -- 172.21.15.107:0/3316747131 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f50c00cbe40 con 0x55af60e1ea80 2023-07-21T22:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50d751eb80 1 -- 172.21.15.107:0/3316747131 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50bc00bdb0 con 0x55af60e1ea80 2023-07-21T22:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50acff9700 1 -- 172.21.15.107:0/3316747131 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50c009f2c0 con 0x55af60e1ea80 2023-07-21T22:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50d751eb80 10 client.25705 fetch_fsmap learned FSMap version 34 2023-07-21T22:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50d751eb80 1 -- 172.21.15.107:0/3316747131 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f50c41077c0 con 0x55af60e1ea80 2023-07-21T22:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50acff9700 1 -- 172.21.15.107:0/3316747131 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50bc00bdb0 con 0x55af60e1ea80 2023-07-21T22:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50acff9700 1 -- 172.21.15.107:0/3316747131 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f50c007d050 con 0x55af60e1ea80 2023-07-21T22:31:49.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50d751eb80 10 client.25705 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50d751eb80 10 client.25705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50d751eb80 20 client.25705 populate_metadata read hostname 'smithi107' 2023-07-21T22:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50d751eb80 1 --2- 172.21.15.107:0/3316747131 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55af60e89930 0x55af60e8bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50d751eb80 4 client.25705 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.711+0000 7f50d751eb80 1 -- 172.21.15.107:0/3316747131 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55af60d27050 con 0x55af60e89930 2023-07-21T22:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.712+0000 7f50c8934700 1 --2- 172.21.15.107:0/3316747131 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55af60e89930 0x55af60e8bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:49.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.712+0000 7f50acff9700 10 client.25705 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:49.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.712+0000 7f50c8934700 1 --2- 172.21.15.107:0/3316747131 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55af60e89930 0x55af60e8bd40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f781dbecff17a986 server_cookie=8d42535d45d70c52 in_seq=0 out_seq=0 2023-07-21T22:31:49.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.712+0000 7f50acff9700 1 -- 172.21.15.107:0/3316747131 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55af60d27050 con 0x55af60e89930 2023-07-21T22:31:49.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.712+0000 7f50acff9700 10 client.25705 handle_command_reply: tid=0 2023-07-21T22:31:49.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.712+0000 7f50d751eb80 1 client.25705 shutdown 2023-07-21T22:31:49.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.713+0000 7f50d751eb80 1 -- 172.21.15.107:0/3316747131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 msgr2=0x55af60e2f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:49.767 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.713+0000 7f50d751eb80 1 --2- 172.21.15.107:0/3316747131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 0x55af60e2f4a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f50c009af90 tx=0x7f50c00cb850 comp rx=0 tx=0).stop 2023-07-21T22:31:49.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.713+0000 7f50d751eb80 1 -- 172.21.15.107:0/3316747131 shutdown_connections 2023-07-21T22:31:49.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.713+0000 7f50d751eb80 1 --2- 172.21.15.107:0/3316747131 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55af60e89930 0x55af60e8bd40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.713+0000 7f50d751eb80 1 --2- 172.21.15.107:0/3316747131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af60e1fff0 0x55af60e2c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.713+0000 7f50acff9700 0 client.25705 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:49.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.713+0000 7f50d751eb80 1 --2- 172.21.15.107:0/3316747131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55af60e1ea80 0x55af60e2f4a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.713+0000 7f50d751eb80 1 --2- 172.21.15.107:0/3316747131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af60e1f3d0 0x55af60e2bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.713+0000 7f50d751eb80 1 -- 172.21.15.107:0/3316747131 >> 172.21.15.107:0/3316747131 conn(0x55af60e0fef0 msgr2=0x55af60e26300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:49.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.713+0000 7f50d751eb80 1 -- 172.21.15.107:0/3316747131 shutdown_connections 2023-07-21T22:31:49.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.713+0000 7f50d751eb80 1 -- 172.21.15.107:0/3316747131 wait complete. 2023-07-21T22:31:49.769 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.713+0000 7f50d751eb80 20 client.25705 trim_cache size 0 max 16384 2023-07-21T22:31:49.769 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:31:49.770 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:31:49.770 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:31:49.770 INFO:teuthology.orchestra.run.smithi107.stdout: "d037412e-e4e0-4d2c-8d6e-c7193371fe0f": { 2023-07-21T22:31:49.770 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:31:49.770 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d037412e-e4e0-4d2c-8d6e-c7193371fe0f", 2023-07-21T22:31:49.770 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:31:49.770 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:31:49.771 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:31:49.771 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:31:49.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.714+0000 7f50cb399700 1 -- 172.21.15.107:0/184193807 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f50b00a1150 msgr2=0x7f50b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:49.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.714+0000 7f50cb399700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f50b00a1150 0x7f50b00a3600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f50b4005950 tx=0x7f50b400b500 comp rx=0 tx=0).stop 2023-07-21T22:31:49.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.714+0000 7f50cb399700 1 -- 172.21.15.107:0/184193807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c40fea50 msgr2=0x7f50c4194640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:49.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.714+0000 7f50cb399700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c40fea50 0x7f50c4194640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f50c000ead0 tx=0x7f50c000ede0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.714+0000 7f50cb399700 1 -- 172.21.15.107:0/184193807 shutdown_connections 2023-07-21T22:31:49.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.714+0000 7f50cb399700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50c41005d0 0x7f50c419bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.714+0000 7f50cb399700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f50b00a1150 0x7f50b00a3600 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.714+0000 7f50cb399700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50c40fea50 0x7f50c4194640 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.714+0000 7f50cb399700 1 --2- 172.21.15.107:0/184193807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50c40ffc40 0x7f50c4194b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:49.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.714+0000 7f50cb399700 1 -- 172.21.15.107:0/184193807 >> 172.21.15.107:0/184193807 conn(0x7f50c40fa070 msgr2=0x7f50c41030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:49.773 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.714+0000 7f50cb399700 1 -- 172.21.15.107:0/184193807 shutdown_connections 2023-07-21T22:31:49.773 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:31:49.714+0000 7f50cb399700 1 -- 172.21.15.107:0/184193807 wait complete. 2023-07-21T22:31:49.775 INFO:tasks.cephfs.filesystem:scrub status for tag:d037412e-e4e0-4d2c-8d6e-c7193371fe0f - {'path': '/', 'tag': 'd037412e-e4e0-4d2c-8d6e-c7193371fe0f', 'options': 'recursive,force'} 2023-07-21T22:31:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.409+0000 7fa932250700 1 -- 172.21.15.107:0/444702773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92c0fe410 msgr2=0x7fa92c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.409+0000 7fa932250700 1 --2- 172.21.15.107:0/444702773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92c0fe410 0x7fa92c100830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa920009990 tx=0x7fa920009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.410+0000 7fa932250700 1 -- 172.21.15.107:0/444702773 shutdown_connections 2023-07-21T22:31:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.410+0000 7fa932250700 1 --2- 172.21.15.107:0/444702773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa92c103730 0x7fa92c103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.410+0000 7fa932250700 1 --2- 172.21.15.107:0/444702773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92c0fe410 0x7fa92c100830 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.410+0000 7fa932250700 1 --2- 172.21.15.107:0/444702773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c100d70 0x7fa92c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.410+0000 7fa932250700 1 -- 172.21.15.107:0/444702773 >> 172.21.15.107:0/444702773 conn(0x7fa92c0fa000 msgr2=0x7fa92c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.410+0000 7fa932250700 1 -- 172.21.15.107:0/444702773 shutdown_connections 2023-07-21T22:31:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.410+0000 7fa932250700 1 -- 172.21.15.107:0/444702773 wait complete. 2023-07-21T22:31:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.411+0000 7fa932250700 1 Processor -- start 2023-07-21T22:31:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.411+0000 7fa932250700 1 -- start start 2023-07-21T22:31:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.411+0000 7fa932250700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0fe410 0x7fa92c19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.411+0000 7fa932250700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa92c100d70 0x7fa92c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.411+0000 7fa932250700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92c103730 0x7fa92c19d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.411+0000 7fa932250700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19d980 con 0x7fa92c0fe410 2023-07-21T22:31:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.411+0000 7fa932250700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19daf0 con 0x7fa92c103730 2023-07-21T22:31:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.411+0000 7fa932250700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19dcb0 con 0x7fa92c100d70 2023-07-21T22:31:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.412+0000 7fa92affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa92c100d70 0x7fa92c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.412+0000 7fa92affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa92c100d70 0x7fa92c19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38446/0 (socket says 172.21.15.107:38446) 2023-07-21T22:31:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.412+0000 7fa92affd700 1 -- 172.21.15.107:0/1159765286 learned_addr learned my addr 172.21.15.107:0/1159765286 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.412+0000 7fa92b7fe700 1 --2- 172.21.15.107:0/1159765286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0fe410 0x7fa92c19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.412+0000 7fa92affd700 1 -- 172.21.15.107:0/1159765286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92c103730 msgr2=0x7fa92c19d320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T22:31:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.412+0000 7fa92affd700 1 --2- 172.21.15.107:0/1159765286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92c103730 0x7fa92c19d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.412+0000 7fa92affd700 1 -- 172.21.15.107:0/1159765286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0fe410 msgr2=0x7fa92c19bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.412+0000 7fa92affd700 1 --2- 172.21.15.107:0/1159765286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0fe410 0x7fa92c19bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.412+0000 7fa92affd700 1 -- 172.21.15.107:0/1159765286 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9200095f0 con 0x7fa92c100d70 2023-07-21T22:31:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.412+0000 7fa92affd700 1 --2- 172.21.15.107:0/1159765286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa92c100d70 0x7fa92c19c4a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00e9a0 tx=0x7fa91c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.413+0000 7fa928ff9700 1 -- 172.21.15.107:0/1159765286 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00cbd0 con 0x7fa92c100d70 2023-07-21T22:31:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.413+0000 7fa928ff9700 1 -- 172.21.15.107:0/1159765286 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa91c00cd30 con 0x7fa92c100d70 2023-07-21T22:31:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.413+0000 7fa928ff9700 1 -- 172.21.15.107:0/1159765286 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c018680 con 0x7fa92c100d70 2023-07-21T22:31:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.413+0000 7fa932250700 1 -- 172.21.15.107:0/1159765286 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92c19df90 con 0x7fa92c100d70 2023-07-21T22:31:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.413+0000 7fa932250700 1 -- 172.21.15.107:0/1159765286 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c104c10 con 0x7fa92c100d70 2023-07-21T22:31:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.417+0000 7fa93e3d5b80 1 Processor -- start 2023-07-21T22:31:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.418+0000 7fa93e3d5b80 1 -- start start 2023-07-21T22:31:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.418+0000 7fa93e3d5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257c9d6b0 0x563257e412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.418+0000 7fa93e3d5b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e41830 0x563257e42040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.418+0000 7fa93e3d5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e42580 0x563257e429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.418+0000 7fa93e3d5b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257d46c50 con 0x563257e41830 2023-07-21T22:31:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.418+0000 7fa93e3d5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257d999e0 con 0x563257c9d6b0 2023-07-21T22:31:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.419+0000 7fa93e3d5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257d92e70 con 0x563257e42580 2023-07-21T22:31:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.419+0000 7fa92b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e41830 0x563257e42040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.419+0000 7fa92b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e41830 0x563257e42040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49894/0 (socket says 172.21.15.107:49894) 2023-07-21T22:31:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.419+0000 7fa92b7fe700 1 -- 172.21.15.107:0/4127914643 learned_addr learned my addr 172.21.15.107:0/4127914643 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.420+0000 7fa92b7fe700 1 -- 172.21.15.107:0/4127914643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e42580 msgr2=0x563257e429f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:31:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.420+0000 7fa92b7fe700 1 --2- 172.21.15.107:0/4127914643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e42580 0x563257e429f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.420+0000 7fa92b7fe700 1 -- 172.21.15.107:0/4127914643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257c9d6b0 msgr2=0x563257e412f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:31:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.420+0000 7fa92b7fe700 1 --2- 172.21.15.107:0/4127914643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257c9d6b0 0x563257e412f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.420+0000 7fa92b7fe700 1 -- 172.21.15.107:0/4127914643 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563257bc0d80 con 0x563257e41830 2023-07-21T22:31:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.420+0000 7fa92b7fe700 1 --2- 172.21.15.107:0/4127914643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e41830 0x563257e42040 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0683c0 tx=0x7fa920005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9286a51b41fe61 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.420+0000 7fa928ff9700 1 -- 172.21.15.107:0/1159765286 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa91c010430 con 0x7fa92c100d70 2023-07-21T22:31:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.421+0000 7fa928ff9700 1 --2- 172.21.15.107:0/1159765286 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa9140a1250 0x7fa9140a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.421+0000 7fa928ff9700 1 -- 172.21.15.107:0/1159765286 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa91c014070 con 0x7fa92c100d70 2023-07-21T22:31:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.422+0000 7fa911ffb700 1 -- 172.21.15.107:0/4127914643 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92001d070 con 0x563257e41830 2023-07-21T22:31:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.422+0000 7fa911ffb700 1 -- 172.21.15.107:0/4127914643 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa92001e040 con 0x563257e41830 2023-07-21T22:31:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.422+0000 7fa911ffb700 1 -- 172.21.15.107:0/4127914643 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920004aa0 con 0x563257e41830 2023-07-21T22:31:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.422+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/4127914643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e41830 msgr2=0x563257e42040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.422+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/4127914643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e41830 0x563257e42040 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0683c0 tx=0x7fa920005bc0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.424+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/4127914643 shutdown_connections 2023-07-21T22:31:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.424+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/4127914643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e42580 0x563257e429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.424+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/4127914643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257c9d6b0 0x563257e412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.424+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/4127914643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e41830 0x563257e42040 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.424+0000 7fa92bfff700 1 --2- 172.21.15.107:0/1159765286 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa9140a1250 0x7fa9140a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.424+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/4127914643 >> 172.21.15.107:0/4127914643 conn(0x563257e36ae0 msgr2=0x563257e3bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.424+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/4127914643 shutdown_connections 2023-07-21T22:31:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.424+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/4127914643 wait complete. 2023-07-21T22:31:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.424+0000 7fa92bfff700 1 --2- 172.21.15.107:0/1159765286 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa9140a1250 0x7fa9140a3700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa9240060b0 tx=0x7fa924006040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.424+0000 7fa93e3d5b80 1 -- start start 2023-07-21T22:31:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa93e3d5b80 1 Processor -- start 2023-07-21T22:31:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa93e3d5b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257c9d6b0 0x563257e5b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa93e3d5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e41830 0x563257e5bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa93e3d5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257e42580 0x563257e62d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa93e3d5b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257d32a60 con 0x563257c9d6b0 2023-07-21T22:31:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa93e3d5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257d32760 con 0x563257e42580 2023-07-21T22:31:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa93e3d5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257c168c0 con 0x563257e41830 2023-07-21T22:31:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa93e3d5b80 10 client.0 _ll_register_callbacks cb 0x563257d30700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa92b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257c9d6b0 0x563257e5b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa92b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257c9d6b0 0x563257e5b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49898/0 (socket says 172.21.15.107:49898) 2023-07-21T22:31:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa92b7fe700 1 -- 172.21.15.107:0/1182822390 learned_addr learned my addr 172.21.15.107:0/1182822390 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa92bfff700 1 --2- 172.21.15.107:0/1182822390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257e42580 0x563257e62d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa92b7fe700 1 -- 172.21.15.107:0/1182822390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e41830 msgr2=0x563257e5bd50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:31:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa92b7fe700 1 --2- 172.21.15.107:0/1182822390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e41830 0x563257e5bd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa92b7fe700 1 -- 172.21.15.107:0/1182822390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257e42580 msgr2=0x563257e62d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa92b7fe700 1 --2- 172.21.15.107:0/1182822390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257e42580 0x563257e62d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.425+0000 7fa92b7fe700 1 -- 172.21.15.107:0/1182822390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c1046a0 con 0x563257c9d6b0 2023-07-21T22:31:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.426+0000 7fa92b7fe700 1 --2- 172.21.15.107:0/1182822390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257c9d6b0 0x563257e5b810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa920009350 tx=0x7fa92000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60a5fc10e02cc6b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.426+0000 7fa90f7f6700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:31:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.426+0000 7fa90f7f6700 1 -- 172.21.15.107:0/1182822390 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92001d040 con 0x563257c9d6b0 2023-07-21T22:31:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.426+0000 7fa90f7f6700 1 -- 172.21.15.107:0/1182822390 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa92000be50 con 0x563257c9d6b0 2023-07-21T22:31:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.426+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/1182822390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563257cec490 con 0x563257c9d6b0 2023-07-21T22:31:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.426+0000 7fa90f7f6700 1 -- 172.21.15.107:0/1182822390 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa920022600 con 0x563257c9d6b0 2023-07-21T22:31:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.426+0000 7fa93e3d5b80 10 client.16197 fetch_fsmap learned FSMap version 34 2023-07-21T22:31:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.426+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/1182822390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563257ba9c00 con 0x563257c9d6b0 2023-07-21T22:31:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.426+0000 7fa90f7f6700 1 -- 172.21.15.107:0/1182822390 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563257cec490 con 0x563257c9d6b0 2023-07-21T22:31:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.427+0000 7fa90f7f6700 1 -- 172.21.15.107:0/1182822390 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa920017760 con 0x563257c9d6b0 2023-07-21T22:31:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.427+0000 7fa93e3d5b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:31:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.427+0000 7fa93e3d5b80 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.427+0000 7fa93e3d5b80 20 client.16197 populate_metadata read hostname 'smithi107' 2023-07-21T22:31:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.427+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/1182822390 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563257e670f0 0x563257e694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.427+0000 7fa93e3d5b80 4 client.16197 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.427+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/1182822390 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563257d8be50 con 0x563257e670f0 2023-07-21T22:31:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.427+0000 7fa92affd700 1 --2- 172.21.15.107:0/1182822390 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563257e670f0 0x563257e694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.427+0000 7fa90f7f6700 10 client.16197 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.427+0000 7fa92affd700 1 --2- 172.21.15.107:0/1182822390 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563257e670f0 0x563257e694d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fffc74fe2ef58aa server_cookie=41f574e72bc74a07 in_seq=0 out_seq=0 2023-07-21T22:31:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.428+0000 7fa90f7f6700 1 -- 172.21.15.107:0/1182822390 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563257d8be50 con 0x563257e670f0 2023-07-21T22:31:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.428+0000 7fa90f7f6700 10 client.16197 handle_command_reply: tid=0 2023-07-21T22:31:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.428+0000 7fa93e3d5b80 1 client.16197 shutdown 2023-07-21T22:31:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.428+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/1182822390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257c9d6b0 msgr2=0x563257e5b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.428+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/1182822390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257c9d6b0 0x563257e5b810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa920009350 tx=0x7fa92000fab0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.428+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/1182822390 shutdown_connections 2023-07-21T22:31:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.428+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/1182822390 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563257e670f0 0x563257e694d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.428+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/1182822390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257e42580 0x563257e62d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.428+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/1182822390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257c9d6b0 0x563257e5b810 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.428+0000 7fa90f7f6700 0 client.16197 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.428+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/1182822390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e41830 0x563257e5bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.428+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/1182822390 >> 172.21.15.107:0/1182822390 conn(0x563257e36ae0 msgr2=0x563257e3b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.429+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/1182822390 shutdown_connections 2023-07-21T22:31:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.429+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/1182822390 wait complete. 2023-07-21T22:31:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.429+0000 7fa93e3d5b80 20 client.16197 trim_cache size 0 max 16384 2023-07-21T22:31:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.445+0000 7fa93e3d5b80 1 Processor -- start 2023-07-21T22:31:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.445+0000 7fa93e3d5b80 1 -- start start 2023-07-21T22:31:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa93e3d5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c9d6b0 0x563257e3bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa93e3d5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257e41830 0x563257e62d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa93e3d5b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e42580 0x563257e63500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa93e3d5b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257e5b3e0 con 0x563257e42580 2023-07-21T22:31:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa93e3d5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257e3d0a0 con 0x563257e41830 2023-07-21T22:31:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa93e3d5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257e41e70 con 0x563257c9d6b0 2023-07-21T22:31:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c9d6b0 0x563257e3bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e42580 0x563257e63500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257e41830 0x563257e62d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c9d6b0 0x563257e3bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38464/0 (socket says 172.21.15.107:38464) 2023-07-21T22:31:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e42580 0x563257e63500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49910/0 (socket says 172.21.15.107:49910) 2023-07-21T22:31:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257e41830 0x563257e62d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40736/0 (socket says 172.21.15.107:40736) 2023-07-21T22:31:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92b7fe700 1 -- 172.21.15.107:0/1731327084 learned_addr learned my addr 172.21.15.107:0/1731327084 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92b7fe700 1 -- 172.21.15.107:0/1731327084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257e41830 msgr2=0x563257e62d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92b7fe700 1 --2- 172.21.15.107:0/1731327084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257e41830 0x563257e62d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92b7fe700 1 -- 172.21.15.107:0/1731327084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e42580 msgr2=0x563257e63500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92b7fe700 1 --2- 172.21.15.107:0/1731327084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e42580 0x563257e63500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92b7fe700 1 -- 172.21.15.107:0/1731327084 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563257be71f0 con 0x563257c9d6b0 2023-07-21T22:31:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92bfff700 1 --2- 172.21.15.107:0/1731327084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e42580 0x563257e63500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.446+0000 7fa92b7fe700 1 --2- 172.21.15.107:0/1731327084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c9d6b0 0x563257e3bb10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa920017ac0 tx=0x7fa920005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5faa4e9c4d763777 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.447+0000 7fa9127fc700 1 -- 172.21.15.107:0/1731327084 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9200175f0 con 0x563257c9d6b0 2023-07-21T22:31:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.447+0000 7fa9127fc700 1 -- 172.21.15.107:0/1731327084 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa92000fbd0 con 0x563257c9d6b0 2023-07-21T22:31:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.447+0000 7fa9127fc700 1 -- 172.21.15.107:0/1731327084 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9200047d0 con 0x563257c9d6b0 2023-07-21T22:31:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.447+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/1731327084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c9d6b0 msgr2=0x563257e3bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.447+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/1731327084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c9d6b0 0x563257e3bb10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa920017ac0 tx=0x7fa920005bc0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.447+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/1731327084 shutdown_connections 2023-07-21T22:31:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.447+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/1731327084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e42580 0x563257e63500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.447+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/1731327084 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257c9d6b0 0x563257e3bb10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.447+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/1731327084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257e41830 0x563257e62d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.447+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/1731327084 >> 172.21.15.107:0/1731327084 conn(0x563257e36ae0 msgr2=0x563257e6de30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.447+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/1731327084 shutdown_connections 2023-07-21T22:31:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.447+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/1731327084 wait complete. 2023-07-21T22:31:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.448+0000 7fa93e3d5b80 1 -- start start 2023-07-21T22:31:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.448+0000 7fa93e3d5b80 1 Processor -- start 2023-07-21T22:31:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.448+0000 7fa93e3d5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257c9d6b0 0x563257e5e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.448+0000 7fa93e3d5b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e41830 0x563257e5ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.448+0000 7fa93e3d5b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e42580 0x563257e61be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.448+0000 7fa93e3d5b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257e62120 con 0x563257e42580 2023-07-21T22:31:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.448+0000 7fa93e3d5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257e62290 con 0x563257c9d6b0 2023-07-21T22:31:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.448+0000 7fa93e3d5b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257e62450 con 0x563257e41830 2023-07-21T22:31:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.448+0000 7fa93e3d5b80 10 client.0 _ll_register_callbacks cb 0x7fa920005ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.448+0000 7fa92b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257c9d6b0 0x563257e5e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.448+0000 7fa92b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257c9d6b0 0x563257e5e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40740/0 (socket says 172.21.15.107:40740) 2023-07-21T22:31:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.448+0000 7fa92b7fe700 1 -- 172.21.15.107:0/3333609407 learned_addr learned my addr 172.21.15.107:0/3333609407 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:31:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa92bfff700 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e42580 0x563257e61be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa92affd700 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e41830 0x563257e5ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa92b7fe700 1 -- 172.21.15.107:0/3333609407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e41830 msgr2=0x563257e5ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa92b7fe700 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e41830 0x563257e5ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa92b7fe700 1 -- 172.21.15.107:0/3333609407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e42580 msgr2=0x563257e61be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa92b7fe700 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e42580 0x563257e61be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa92b7fe700 1 -- 172.21.15.107:0/3333609407 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa924005e70 con 0x563257c9d6b0 2023-07-21T22:31:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa92b7fe700 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257c9d6b0 0x563257e5e680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa920019040 tx=0x7fa92000fab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6a9c57fcd26b19f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa90f7f6700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:31:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa92affd700 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e41830 0x563257e5ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa90f7f6700 1 -- 172.21.15.107:0/3333609407 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92001d030 con 0x563257c9d6b0 2023-07-21T22:31:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa90f7f6700 1 -- 172.21.15.107:0/3333609407 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa920000e40 con 0x563257c9d6b0 2023-07-21T22:31:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/3333609407 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563257d4dff0 con 0x563257c9d6b0 2023-07-21T22:31:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.449+0000 7fa90f7f6700 1 -- 172.21.15.107:0/3333609407 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa92000fbd0 con 0x563257c9d6b0 2023-07-21T22:31:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.450+0000 7fa93e3d5b80 10 client.25652 fetch_fsmap learned FSMap version 34 2023-07-21T22:31:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.450+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/3333609407 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563257ba9c00 con 0x563257c9d6b0 2023-07-21T22:31:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.450+0000 7fa90f7f6700 1 -- 172.21.15.107:0/3333609407 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563257d4dff0 con 0x563257c9d6b0 2023-07-21T22:31:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.450+0000 7fa90f7f6700 1 -- 172.21.15.107:0/3333609407 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa9200515a0 con 0x563257c9d6b0 2023-07-21T22:31:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.450+0000 7fa93e3d5b80 10 client.25652 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:31:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.450+0000 7fa93e3d5b80 10 client.25652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.450+0000 7fa93e3d5b80 20 client.25652 populate_metadata read hostname 'smithi107' 2023-07-21T22:31:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.450+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563257ebb620 0x563257ebda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.450+0000 7fa93e3d5b80 4 client.25652 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:31:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.450+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/3333609407 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563257be1f10 con 0x563257ebb620 2023-07-21T22:31:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.450+0000 7fa92affd700 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563257ebb620 0x563257ebda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.451+0000 7fa92affd700 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563257ebb620 0x563257ebda00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd85c7cc7decda54 server_cookie=3802632d6971b067 in_seq=0 out_seq=0 2023-07-21T22:31:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.451+0000 7fa90f7f6700 10 client.25652 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.963+0000 7fa90f7f6700 1 -- 172.21.15.107:0/3333609407 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563257be1f10 con 0x563257ebb620 2023-07-21T22:31:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.963+0000 7fa90f7f6700 10 client.25652 handle_command_reply: tid=0 2023-07-21T22:31:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.992+0000 7fa93e3d5b80 1 client.25652 shutdown 2023-07-21T22:31:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.992+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/3333609407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257c9d6b0 msgr2=0x563257e5e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.992+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257c9d6b0 0x563257e5e680 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa920019040 tx=0x7fa92000fab0 comp rx=0 tx=0).stop 2023-07-21T22:31:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.993+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/3333609407 shutdown_connections 2023-07-21T22:31:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.993+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563257ebb620 0x563257ebda00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.993+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563257e42580 0x563257e61be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.993+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563257c9d6b0 0x563257e5e680 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.993+0000 7fa93e3d5b80 1 --2- 172.21.15.107:0/3333609407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563257e41830 0x563257e5ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.993+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/3333609407 >> 172.21.15.107:0/3333609407 conn(0x563257e36ae0 msgr2=0x563257e63500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.993+0000 7fa90f7f6700 0 client.25652 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:31:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.997+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/3333609407 shutdown_connections 2023-07-21T22:31:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.997+0000 7fa93e3d5b80 1 -- 172.21.15.107:0/3333609407 wait complete. 2023-07-21T22:31:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:50.997+0000 7fa93e3d5b80 20 client.25652 trim_cache size 0 max 16384 2023-07-21T22:31:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:31:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:31:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:31:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:31:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:51.026+0000 7fa932250700 1 -- 172.21.15.107:0/1159765286 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa9140a1250 msgr2=0x7fa9140a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:51.026+0000 7fa932250700 1 --2- 172.21.15.107:0/1159765286 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa9140a1250 0x7fa9140a3700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa9240060b0 tx=0x7fa924006040 comp rx=0 tx=0).stop 2023-07-21T22:31:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:51.026+0000 7fa932250700 1 -- 172.21.15.107:0/1159765286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa92c100d70 msgr2=0x7fa92c19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:51.026+0000 7fa932250700 1 --2- 172.21.15.107:0/1159765286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa92c100d70 0x7fa92c19c4a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00e9a0 tx=0x7fa91c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:51.027+0000 7fa932250700 1 -- 172.21.15.107:0/1159765286 shutdown_connections 2023-07-21T22:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:51.027+0000 7fa932250700 1 --2- 172.21.15.107:0/1159765286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa92c103730 0x7fa92c19d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:51.027+0000 7fa932250700 1 --2- 172.21.15.107:0/1159765286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0fe410 0x7fa92c19bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:51.027+0000 7fa932250700 1 --2- 172.21.15.107:0/1159765286 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa9140a1250 0x7fa9140a3700 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:51.027+0000 7fa932250700 1 --2- 172.21.15.107:0/1159765286 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa92c100d70 0x7fa92c19c4a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:51.027+0000 7fa932250700 1 -- 172.21.15.107:0/1159765286 >> 172.21.15.107:0/1159765286 conn(0x7fa92c0fa000 msgr2=0x7fa92c101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:51.027+0000 7fa932250700 1 -- 172.21.15.107:0/1159765286 shutdown_connections 2023-07-21T22:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:31:51.027+0000 7fa932250700 1 -- 172.21.15.107:0/1159765286 wait complete. 2023-07-21T22:31:51.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:51 smithi132 ceph-mon[97863]: pgmap v377: 97 pgs: 97 active+clean; 417 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-07-21T22:31:51.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:51 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:31:51.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:51 smithi107 ceph-mon[96543]: pgmap v377: 97 pgs: 97 active+clean; 417 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-07-21T22:31:51.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:51 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:31:51.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:51 smithi190 ceph-mon[98334]: pgmap v377: 97 pgs: 97 active+clean; 417 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-07-21T22:31:51.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:51 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:31:52.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:52 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:31:52.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:52 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:31:52.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:52 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:31:52.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:52 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:31:52.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:52 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:31:52.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:52 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:31:52.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:52 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:31:52.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:52 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:31:52.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:52 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:31:52.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:52 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:31:52.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:52 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:31:52.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:52 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:31:52.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:52 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:31:52.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:52 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:31:52.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:52 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:31:53.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:53 smithi190 ceph-mon[98334]: pgmap v378: 97 pgs: 97 active+clean; 419 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 373 op/s 2023-07-21T22:31:53.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:53 smithi132 ceph-mon[97863]: pgmap v378: 97 pgs: 97 active+clean; 419 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 373 op/s 2023-07-21T22:31:53.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:53 smithi107 ceph-mon[96543]: pgmap v378: 97 pgs: 97 active+clean; 419 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 373 op/s 2023-07-21T22:31:55.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:55 smithi132 ceph-mon[97863]: pgmap v379: 97 pgs: 97 active+clean; 419 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 318 op/s 2023-07-21T22:31:55.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:55 smithi107 ceph-mon[96543]: pgmap v379: 97 pgs: 97 active+clean; 419 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 318 op/s 2023-07-21T22:31:55.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:55 smithi190 ceph-mon[98334]: pgmap v379: 97 pgs: 97 active+clean; 419 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 318 op/s 2023-07-21T22:31:56.487 INFO:tasks.workunit.client.0.smithi107.stdout: 120 1139972 775 786846 517 302014 717 2023-07-21T22:31:57.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:57 smithi190 ceph-mon[98334]: pgmap v380: 97 pgs: 97 active+clean; 422 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 437 op/s 2023-07-21T22:31:57.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:57 smithi132 ceph-mon[97863]: pgmap v380: 97 pgs: 97 active+clean; 422 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 437 op/s 2023-07-21T22:31:57.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:57 smithi107 ceph-mon[96543]: pgmap v380: 97 pgs: 97 active+clean; 422 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 437 op/s 2023-07-21T22:31:59.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:31:59 smithi132 ceph-mon[97863]: pgmap v381: 97 pgs: 97 active+clean; 425 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 427 op/s 2023-07-21T22:31:59.439 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:31:59 smithi107 ceph-mon[96543]: pgmap v381: 97 pgs: 97 active+clean; 425 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 427 op/s 2023-07-21T22:31:59.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:31:59 smithi190 ceph-mon[98334]: pgmap v381: 97 pgs: 97 active+clean; 425 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 427 op/s 2023-07-21T22:32:01.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:01 smithi190 ceph-mon[98334]: pgmap v382: 97 pgs: 97 active+clean; 428 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 422 op/s 2023-07-21T22:32:01.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:01 smithi132 ceph-mon[97863]: pgmap v382: 97 pgs: 97 active+clean; 428 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 422 op/s 2023-07-21T22:32:01.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:01 smithi107 ceph-mon[96543]: pgmap v382: 97 pgs: 97 active+clean; 428 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 422 op/s 2023-07-21T22:32:03.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:03 smithi190 ceph-mon[98334]: pgmap v383: 97 pgs: 97 active+clean; 516 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.4 MiB/s wr, 812 op/s 2023-07-21T22:32:03.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:03 smithi132 ceph-mon[97863]: pgmap v383: 97 pgs: 97 active+clean; 516 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.4 MiB/s wr, 812 op/s 2023-07-21T22:32:03.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:03 smithi107 ceph-mon[96543]: pgmap v383: 97 pgs: 97 active+clean; 516 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.4 MiB/s wr, 812 op/s 2023-07-21T22:32:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.451+0000 7f10f2bad700 1 -- 172.21.15.107:0/3394082482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec070f00 msgr2=0x7f10ec071310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.451+0000 7f10f2bad700 1 --2- 172.21.15.107:0/3394082482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec070f00 0x7f10ec071310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10dc01a9a0 tx=0x7f10dc01acb0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.453+0000 7f10f2bad700 1 -- 172.21.15.107:0/3394082482 shutdown_connections 2023-07-21T22:32:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.453+0000 7f10f2bad700 1 --2- 172.21.15.107:0/3394082482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10ec10bd60 0x7f10ec10e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.453+0000 7f10f2bad700 1 --2- 172.21.15.107:0/3394082482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec070f00 0x7f10ec071310 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.453+0000 7f10f2bad700 1 --2- 172.21.15.107:0/3394082482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10ec071850 0x7f10ec10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.453+0000 7f10f2bad700 1 -- 172.21.15.107:0/3394082482 >> 172.21.15.107:0/3394082482 conn(0x7f10ec06ca50 msgr2=0x7f10ec06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.453+0000 7f10f2bad700 1 -- 172.21.15.107:0/3394082482 shutdown_connections 2023-07-21T22:32:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.453+0000 7f10f2bad700 1 -- 172.21.15.107:0/3394082482 wait complete. 2023-07-21T22:32:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.454+0000 7f10f2bad700 1 Processor -- start 2023-07-21T22:32:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.454+0000 7f10f2bad700 1 -- start start 2023-07-21T22:32:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.455+0000 7f10f2bad700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec070f00 0x7f10ec19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.455+0000 7f10f2bad700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10ec071850 0x7f10ec1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.455+0000 7f10f2bad700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10ec10bd60 0x7f10ec1a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.455+0000 7f10f2bad700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec1a3b90 con 0x7f10ec10bd60 2023-07-21T22:32:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.455+0000 7f10f2bad700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec1a3d00 con 0x7f10ec071850 2023-07-21T22:32:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.455+0000 7f10f2bad700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec1a3ec0 con 0x7f10ec070f00 2023-07-21T22:32:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.456+0000 7f10f0949700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec070f00 0x7f10ec19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.456+0000 7f10f0949700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec070f00 0x7f10ec19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38484/0 (socket says 172.21.15.107:38484) 2023-07-21T22:32:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.456+0000 7f10f0949700 1 -- 172.21.15.107:0/3939550457 learned_addr learned my addr 172.21.15.107:0/3939550457 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.456+0000 7f10ebfff700 1 --2- 172.21.15.107:0/3939550457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10ec071850 0x7f10ec1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.456+0000 7f10f0949700 1 -- 172.21.15.107:0/3939550457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10ec071850 msgr2=0x7f10ec1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.456+0000 7f10f0949700 1 --2- 172.21.15.107:0/3939550457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10ec071850 0x7f10ec1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.456+0000 7f10f0949700 1 -- 172.21.15.107:0/3939550457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10ec10bd60 msgr2=0x7f10ec1a33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.456+0000 7f10f0949700 1 --2- 172.21.15.107:0/3939550457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10ec10bd60 0x7f10ec1a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.456+0000 7f10f0949700 1 -- 172.21.15.107:0/3939550457 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10dc01a600 con 0x7f10ec070f00 2023-07-21T22:32:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.457+0000 7f10f0949700 1 --2- 172.21.15.107:0/3939550457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec070f00 0x7f10ec19e790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f10dc01c5d0 tx=0x7f10dc003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.457+0000 7f10e9ffb700 1 -- 172.21.15.107:0/3939550457 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc02d070 con 0x7f10ec070f00 2023-07-21T22:32:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.457+0000 7f10e9ffb700 1 -- 172.21.15.107:0/3939550457 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10dc031410 con 0x7f10ec070f00 2023-07-21T22:32:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.457+0000 7f10e9ffb700 1 -- 172.21.15.107:0/3939550457 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0283f0 con 0x7f10ec070f00 2023-07-21T22:32:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.457+0000 7f10f2bad700 1 -- 172.21.15.107:0/3939550457 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ec1a4140 con 0x7f10ec070f00 2023-07-21T22:32:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.458+0000 7f10f2bad700 1 -- 172.21.15.107:0/3939550457 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ec1a4510 con 0x7f10ec070f00 2023-07-21T22:32:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.476+0000 7f10fed32b80 1 Processor -- start 2023-07-21T22:32:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.476+0000 7f10fed32b80 1 -- start start 2023-07-21T22:32:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.477+0000 7f10fed32b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfdea66b0 0x55bdfe04a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.477+0000 7f10fed32b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe04b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.477+0000 7f10fed32b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfe04b580 0x55bdfe04b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.477+0000 7f10fed32b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfdf4fc50 con 0x55bdfdea66b0 2023-07-21T22:32:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.477+0000 7f10fed32b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfdfa29e0 con 0x55bdfe04b580 2023-07-21T22:32:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.477+0000 7f10fed32b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfdf9be70 con 0x55bdfe04a830 2023-07-21T22:32:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.477+0000 7f10ebfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfe04b580 0x55bdfe04b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.477+0000 7f10ebfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfe04b580 0x55bdfe04b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40760/0 (socket says 172.21.15.107:40760) 2023-07-21T22:32:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.477+0000 7f10ebfff700 1 -- 172.21.15.107:0/2549701261 learned_addr learned my addr 172.21.15.107:0/2549701261 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.478+0000 7f10ebfff700 1 -- 172.21.15.107:0/2549701261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 msgr2=0x55bdfe04b040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:32:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.478+0000 7f10ebfff700 1 --2- 172.21.15.107:0/2549701261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe04b040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.478+0000 7f10ebfff700 1 -- 172.21.15.107:0/2549701261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfdea66b0 msgr2=0x55bdfe04a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.478+0000 7f10ebfff700 1 --2- 172.21.15.107:0/2549701261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfdea66b0 0x55bdfe04a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.478+0000 7f10ebfff700 1 -- 172.21.15.107:0/2549701261 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdfddc9d80 con 0x55bdfe04b580 2023-07-21T22:32:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.483+0000 7f10ebfff700 1 --2- 172.21.15.107:0/2549701261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfe04b580 0x55bdfe04b9f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f10e000bd40 tx=0x7f10e00106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9777f1cd296a74d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.489+0000 7f10e9ffb700 1 -- 172.21.15.107:0/3939550457 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f10dc028550 con 0x7f10ec070f00 2023-07-21T22:32:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.490+0000 7f10e9ffb700 1 --2- 172.21.15.107:0/3939550457 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10d40a1350 0x7f10d40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.538 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.491+0000 7f10e9ffb700 1 -- 172.21.15.107:0/3939550457 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f10dc0df080 con 0x7f10ec070f00 2023-07-21T22:32:04.544 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.497+0000 7f10f114a700 1 --2- 172.21.15.107:0/3939550457 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10d40a1350 0x7f10d40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.521+0000 7f10f114a700 1 --2- 172.21.15.107:0/3939550457 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10d40a1350 0x7f10d40a3800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009510 tx=0x7f10e4009050 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.522+0000 7f10d2ffd700 1 -- 172.21.15.107:0/2549701261 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e000be90 con 0x55bdfe04b580 2023-07-21T22:32:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.522+0000 7f10d2ffd700 1 -- 172.21.15.107:0/2549701261 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10e0015070 con 0x55bdfe04b580 2023-07-21T22:32:04.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.522+0000 7f10d2ffd700 1 -- 172.21.15.107:0/2549701261 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0004ea0 con 0x55bdfe04b580 2023-07-21T22:32:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.524+0000 7f10fed32b80 1 -- 172.21.15.107:0/2549701261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfe04b580 msgr2=0x55bdfe04b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.524+0000 7f10fed32b80 1 --2- 172.21.15.107:0/2549701261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfe04b580 0x55bdfe04b9f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f10e000bd40 tx=0x7f10e00106b0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.525+0000 7f10fed32b80 1 -- 172.21.15.107:0/2549701261 shutdown_connections 2023-07-21T22:32:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.525+0000 7f10fed32b80 1 --2- 172.21.15.107:0/2549701261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfe04b580 0x55bdfe04b9f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.525+0000 7f10fed32b80 1 --2- 172.21.15.107:0/2549701261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfdea66b0 0x55bdfe04a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.525+0000 7f10fed32b80 1 --2- 172.21.15.107:0/2549701261 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe04b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.525+0000 7f10fed32b80 1 -- 172.21.15.107:0/2549701261 >> 172.21.15.107:0/2549701261 conn(0x55bdfe03fae0 msgr2=0x55bdfe044c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.525+0000 7f10fed32b80 1 -- 172.21.15.107:0/2549701261 shutdown_connections 2023-07-21T22:32:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.525+0000 7f10fed32b80 1 -- 172.21.15.107:0/2549701261 wait complete. 2023-07-21T22:32:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.526+0000 7f10fed32b80 1 -- start start 2023-07-21T22:32:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.526+0000 7f10fed32b80 1 Processor -- start 2023-07-21T22:32:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.526+0000 7f10fed32b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfdea66b0 0x55bdfe0538c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.526+0000 7f10fed32b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe053e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.526+0000 7f10fed32b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfe04b580 0x55bdfe05ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.526+0000 7f10fed32b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfdf3ba60 con 0x55bdfe04b580 2023-07-21T22:32:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.526+0000 7f10fed32b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfdf3b760 con 0x55bdfdea66b0 2023-07-21T22:32:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.526+0000 7f10fed32b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfde1f8c0 con 0x55bdfe04a830 2023-07-21T22:32:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.526+0000 7f10fed32b80 10 client.0 _ll_register_callbacks cb 0x55bdfdf39700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.528+0000 7f10ebfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe053e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.528+0000 7f10ebfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe053e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38502/0 (socket says 172.21.15.107:38502) 2023-07-21T22:32:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.528+0000 7f10ebfff700 1 -- 172.21.15.107:0/3333263003 learned_addr learned my addr 172.21.15.107:0/3333263003 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.528+0000 7f10f0949700 1 --2- 172.21.15.107:0/3333263003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfdea66b0 0x55bdfe0538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.528+0000 7f10ebfff700 1 -- 172.21.15.107:0/3333263003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfdea66b0 msgr2=0x55bdfe0538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.528+0000 7f10ebfff700 1 --2- 172.21.15.107:0/3333263003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfdea66b0 0x55bdfe0538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.528+0000 7f10ebfff700 1 -- 172.21.15.107:0/3333263003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfe04b580 msgr2=0x55bdfe05ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.528+0000 7f10ebfff700 1 --2- 172.21.15.107:0/3333263003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfe04b580 0x55bdfe05ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.528+0000 7f10ebfff700 1 -- 172.21.15.107:0/3333263003 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e000b9a0 con 0x55bdfe04a830 2023-07-21T22:32:04.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.532+0000 7f10ebfff700 1 --2- 172.21.15.107:0/3333263003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe053e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f10e0004b30 tx=0x7f10e0010d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b13eb170515805 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.566+0000 7f10cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:32:04.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.566+0000 7f10cbfff700 1 -- 172.21.15.107:0/3333263003 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0020070 con 0x55bdfe04a830 2023-07-21T22:32:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.566+0000 7f10cbfff700 1 -- 172.21.15.107:0/3333263003 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f10e0017070 con 0x55bdfe04a830 2023-07-21T22:32:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.567+0000 7f10fed32b80 1 -- 172.21.15.107:0/3333263003 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdfdef5490 con 0x55bdfe04a830 2023-07-21T22:32:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.572+0000 7f10cbfff700 1 -- 172.21.15.107:0/3333263003 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10e0005010 con 0x55bdfe04a830 2023-07-21T22:32:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.572+0000 7f10fed32b80 10 client.25726 fetch_fsmap learned FSMap version 34 2023-07-21T22:32:04.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.572+0000 7f10fed32b80 1 -- 172.21.15.107:0/3333263003 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55bdfddb2c00 con 0x55bdfe04a830 2023-07-21T22:32:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.575+0000 7f10cbfff700 1 -- 172.21.15.107:0/3333263003 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bdfdef5490 con 0x55bdfe04a830 2023-07-21T22:32:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.575+0000 7f10cbfff700 1 -- 172.21.15.107:0/3333263003 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f10e002d070 con 0x55bdfe04a830 2023-07-21T22:32:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.576+0000 7f10fed32b80 10 client.25726 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:32:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.576+0000 7f10fed32b80 10 client.25726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.576+0000 7f10fed32b80 20 client.25726 populate_metadata read hostname 'smithi107' 2023-07-21T22:32:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.576+0000 7f10fed32b80 1 --2- 172.21.15.107:0/3333263003 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bdfe05f4e0 0x55bdfe0618f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.576+0000 7f10fed32b80 4 client.25726 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.576+0000 7f10fed32b80 1 -- 172.21.15.107:0/3333263003 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdfdf94e50 con 0x55bdfe05f4e0 2023-07-21T22:32:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.577+0000 7f10f0949700 1 --2- 172.21.15.107:0/3333263003 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bdfe05f4e0 0x55bdfe0618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.582+0000 7f10f0949700 1 --2- 172.21.15.107:0/3333263003 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bdfe05f4e0 0x55bdfe0618f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee8fafdc7c8a7214 server_cookie=9538f9d4ca529b2e in_seq=0 out_seq=0 2023-07-21T22:32:04.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.582+0000 7f10cbfff700 10 client.25726 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.589+0000 7f10cbfff700 1 -- 172.21.15.107:0/3333263003 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bdfdf94e50 con 0x55bdfe05f4e0 2023-07-21T22:32:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.589+0000 7f10cbfff700 10 client.25726 handle_command_reply: tid=0 2023-07-21T22:32:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.589+0000 7f10fed32b80 1 client.25726 shutdown 2023-07-21T22:32:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.590+0000 7f10fed32b80 1 -- 172.21.15.107:0/3333263003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 msgr2=0x55bdfe053e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.590+0000 7f10fed32b80 1 --2- 172.21.15.107:0/3333263003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe053e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f10e0004b30 tx=0x7f10e0010d70 comp rx=0 tx=0).stop 2023-07-21T22:32:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.591+0000 7f10fed32b80 1 -- 172.21.15.107:0/3333263003 shutdown_connections 2023-07-21T22:32:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.591+0000 7f10fed32b80 1 --2- 172.21.15.107:0/3333263003 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bdfe05f4e0 0x55bdfe0618f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.591+0000 7f10fed32b80 1 --2- 172.21.15.107:0/3333263003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfe04b580 0x55bdfe05ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.591+0000 7f10fed32b80 1 --2- 172.21.15.107:0/3333263003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfdea66b0 0x55bdfe0538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.591+0000 7f10fed32b80 1 --2- 172.21.15.107:0/3333263003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe053e00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.591+0000 7f10fed32b80 1 -- 172.21.15.107:0/3333263003 >> 172.21.15.107:0/3333263003 conn(0x55bdfe03fae0 msgr2=0x55bdfe0443e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.591+0000 7f10cbfff700 0 client.25726 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.591+0000 7f10fed32b80 1 -- 172.21.15.107:0/3333263003 shutdown_connections 2023-07-21T22:32:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.591+0000 7f10fed32b80 1 -- 172.21.15.107:0/3333263003 wait complete. 2023-07-21T22:32:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.591+0000 7f10fed32b80 20 client.25726 trim_cache size 0 max 16384 2023-07-21T22:32:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.630+0000 7f10fed32b80 1 Processor -- start 2023-07-21T22:32:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.630+0000 7f10fed32b80 1 -- start start 2023-07-21T22:32:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.631+0000 7f10fed32b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfdea66b0 0x55bdfe051980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.631+0000 7f10fed32b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfe04a830 0x55bdfe05ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.631+0000 7f10fed32b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04b580 0x55bdfe051ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.631+0000 7f10fed32b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe053430 con 0x55bdfdea66b0 2023-07-21T22:32:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.631+0000 7f10fed32b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe0466e0 con 0x55bdfe04a830 2023-07-21T22:32:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.631+0000 7f10fed32b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe054540 con 0x55bdfe04b580 2023-07-21T22:32:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.631+0000 7f10f0949700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfdea66b0 0x55bdfe051980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.631+0000 7f10f0949700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfdea66b0 0x55bdfe051980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49950/0 (socket says 172.21.15.107:49950) 2023-07-21T22:32:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.631+0000 7f10f0949700 1 -- 172.21.15.107:0/1312040900 learned_addr learned my addr 172.21.15.107:0/1312040900 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.631+0000 7f10ebfff700 1 --2- 172.21.15.107:0/1312040900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfe04a830 0x55bdfe05ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.631+0000 7f10f114a700 1 --2- 172.21.15.107:0/1312040900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04b580 0x55bdfe051ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.632+0000 7f10f0949700 1 -- 172.21.15.107:0/1312040900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04b580 msgr2=0x55bdfe051ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.632+0000 7f10f0949700 1 --2- 172.21.15.107:0/1312040900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04b580 0x55bdfe051ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.632+0000 7f10f0949700 1 -- 172.21.15.107:0/1312040900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfe04a830 msgr2=0x55bdfe05ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.632+0000 7f10f0949700 1 --2- 172.21.15.107:0/1312040900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfe04a830 0x55bdfe05ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.632+0000 7f10f0949700 1 -- 172.21.15.107:0/1312040900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdfddf01f0 con 0x55bdfdea66b0 2023-07-21T22:32:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.632+0000 7f10ebfff700 1 --2- 172.21.15.107:0/1312040900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfe04a830 0x55bdfe05ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.632+0000 7f10f0949700 1 --2- 172.21.15.107:0/1312040900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfdea66b0 0x55bdfe051980 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0ac1a0 tx=0x7f10dc02a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47901184e89d6f73 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.632+0000 7f10d37fe700 1 -- 172.21.15.107:0/1312040900 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0ac4a0 con 0x55bdfdea66b0 2023-07-21T22:32:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.632+0000 7f10d37fe700 1 -- 172.21.15.107:0/1312040900 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10dc0b04d0 con 0x55bdfdea66b0 2023-07-21T22:32:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.632+0000 7f10d37fe700 1 -- 172.21.15.107:0/1312040900 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0ab990 con 0x55bdfdea66b0 2023-07-21T22:32:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.633+0000 7f10fed32b80 1 -- 172.21.15.107:0/1312040900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfdea66b0 msgr2=0x55bdfe051980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.633+0000 7f10fed32b80 1 --2- 172.21.15.107:0/1312040900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfdea66b0 0x55bdfe051980 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0ac1a0 tx=0x7f10dc02a040 comp rx=0 tx=0).stop 2023-07-21T22:32:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.633+0000 7f10fed32b80 1 -- 172.21.15.107:0/1312040900 shutdown_connections 2023-07-21T22:32:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.633+0000 7f10fed32b80 1 --2- 172.21.15.107:0/1312040900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04b580 0x55bdfe051ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.633+0000 7f10fed32b80 1 --2- 172.21.15.107:0/1312040900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfdea66b0 0x55bdfe051980 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.633+0000 7f10fed32b80 1 --2- 172.21.15.107:0/1312040900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfe04a830 0x55bdfe05ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.633+0000 7f10fed32b80 1 -- 172.21.15.107:0/1312040900 >> 172.21.15.107:0/1312040900 conn(0x55bdfe03fae0 msgr2=0x55bdfe053880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.633+0000 7f10fed32b80 1 -- 172.21.15.107:0/1312040900 shutdown_connections 2023-07-21T22:32:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.633+0000 7f10fed32b80 1 -- 172.21.15.107:0/1312040900 wait complete. 2023-07-21T22:32:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.634+0000 7f10fed32b80 1 -- start start 2023-07-21T22:32:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.634+0000 7f10fed32b80 1 Processor -- start 2023-07-21T22:32:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.634+0000 7f10fed32b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfdea66b0 0x55bdfe057030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.634+0000 7f10fed32b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe057570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.634+0000 7f10fed32b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfe04b580 0x55bdfe05a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.634+0000 7f10fed32b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe05a900 con 0x55bdfe04b580 2023-07-21T22:32:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.634+0000 7f10fed32b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe05aa70 con 0x55bdfdea66b0 2023-07-21T22:32:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.634+0000 7f10fed32b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe05ac30 con 0x55bdfe04a830 2023-07-21T22:32:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.634+0000 7f10fed32b80 10 client.0 _ll_register_callbacks cb 0x7f10e000bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.634+0000 7f10f0949700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfdea66b0 0x55bdfe057030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.634+0000 7f10f0949700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfdea66b0 0x55bdfe057030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40782/0 (socket says 172.21.15.107:40782) 2023-07-21T22:32:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.634+0000 7f10f0949700 1 -- 172.21.15.107:0/3052665518 learned_addr learned my addr 172.21.15.107:0/3052665518 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.635+0000 7f10ebfff700 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe057570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.635+0000 7f10f114a700 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfe04b580 0x55bdfe05a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.635+0000 7f10ebfff700 1 -- 172.21.15.107:0/3052665518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfdea66b0 msgr2=0x55bdfe057030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.635+0000 7f10ebfff700 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfdea66b0 0x55bdfe057030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.635+0000 7f10ebfff700 1 -- 172.21.15.107:0/3052665518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfe04b580 msgr2=0x55bdfe05a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.635+0000 7f10ebfff700 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfe04b580 0x55bdfe05a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.635+0000 7f10ebfff700 1 -- 172.21.15.107:0/3052665518 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdfddb2c00 con 0x55bdfe04a830 2023-07-21T22:32:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.635+0000 7f10f114a700 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfe04b580 0x55bdfe05a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.635+0000 7f10ebfff700 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe057570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f10e0000c00 tx=0x7f10e0011a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81bf2f9f56a47197 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.635+0000 7f10cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:32:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10cbfff700 1 -- 172.21.15.107:0/3052665518 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0020030 con 0x55bdfe04a830 2023-07-21T22:32:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10cbfff700 1 -- 172.21.15.107:0/3052665518 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f10e0021070 con 0x55bdfe04a830 2023-07-21T22:32:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10fed32b80 1 -- 172.21.15.107:0/3052665518 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10e0015070 con 0x55bdfe04a830 2023-07-21T22:32:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10cbfff700 1 -- 172.21.15.107:0/3052665518 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10e000b410 con 0x55bdfe04a830 2023-07-21T22:32:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10fed32b80 10 client.25732 fetch_fsmap learned FSMap version 34 2023-07-21T22:32:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10fed32b80 1 -- 172.21.15.107:0/3052665518 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f10ec1a4510 con 0x55bdfe04a830 2023-07-21T22:32:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10cbfff700 1 -- 172.21.15.107:0/3052665518 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10e0015070 con 0x55bdfe04a830 2023-07-21T22:32:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10cbfff700 1 -- 172.21.15.107:0/3052665518 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f10e00107d0 con 0x55bdfe04a830 2023-07-21T22:32:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10fed32b80 10 client.25732 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:32:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10fed32b80 10 client.25732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10fed32b80 20 client.25732 populate_metadata read hostname 'smithi107' 2023-07-21T22:32:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10fed32b80 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bdfe0b42d0 0x55bdfe0b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10fed32b80 4 client.25732 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:32:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10fed32b80 1 -- 172.21.15.107:0/3052665518 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bdfdf56ff0 con 0x55bdfe0b42d0 2023-07-21T22:32:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.636+0000 7f10f0949700 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bdfe0b42d0 0x55bdfe0b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.637+0000 7f10cbfff700 10 client.25732 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.637+0000 7f10f0949700 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bdfe0b42d0 0x55bdfe0b66b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92794d2be9903482 server_cookie=bfad9ad8be6e9f88 in_seq=0 out_seq=0 2023-07-21T22:32:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.958+0000 7f10cbfff700 1 -- 172.21.15.107:0/3052665518 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bdfdf56ff0 con 0x55bdfe0b42d0 2023-07-21T22:32:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.958+0000 7f10cbfff700 10 client.25732 handle_command_reply: tid=0 2023-07-21T22:32:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.958+0000 7f10fed32b80 1 client.25732 shutdown 2023-07-21T22:32:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.958+0000 7f10fed32b80 1 -- 172.21.15.107:0/3052665518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 msgr2=0x55bdfe057570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.958+0000 7f10fed32b80 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe057570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f10e0000c00 tx=0x7f10e0011a70 comp rx=0 tx=0).stop 2023-07-21T22:32:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.958+0000 7f10fed32b80 1 -- 172.21.15.107:0/3052665518 shutdown_connections 2023-07-21T22:32:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.958+0000 7f10fed32b80 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bdfe0b42d0 0x55bdfe0b66b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.958+0000 7f10fed32b80 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdfe04b580 0x55bdfe05a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.958+0000 7f10fed32b80 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bdfdea66b0 0x55bdfe057030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.958+0000 7f10fed32b80 1 --2- 172.21.15.107:0/3052665518 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdfe04a830 0x55bdfe057570 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.958+0000 7f10fed32b80 1 -- 172.21.15.107:0/3052665518 >> 172.21.15.107:0/3052665518 conn(0x55bdfe03fae0 msgr2=0x55bdfe053860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.958+0000 7f10cbfff700 0 client.25732 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.959+0000 7f10fed32b80 1 -- 172.21.15.107:0/3052665518 shutdown_connections 2023-07-21T22:32:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.959+0000 7f10fed32b80 1 -- 172.21.15.107:0/3052665518 wait complete. 2023-07-21T22:32:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.959+0000 7f10fed32b80 20 client.25732 trim_cache size 0 max 16384 2023-07-21T22:32:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:32:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:32:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:32:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:32:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.960+0000 7f10f2bad700 1 -- 172.21.15.107:0/3939550457 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10d40a1350 msgr2=0x7f10d40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.960+0000 7f10f2bad700 1 --2- 172.21.15.107:0/3939550457 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10d40a1350 0x7f10d40a3800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009510 tx=0x7f10e4009050 comp rx=0 tx=0).stop 2023-07-21T22:32:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.960+0000 7f10f2bad700 1 -- 172.21.15.107:0/3939550457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec070f00 msgr2=0x7f10ec19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.960+0000 7f10f2bad700 1 --2- 172.21.15.107:0/3939550457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec070f00 0x7f10ec19e790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f10dc01c5d0 tx=0x7f10dc003b90 comp rx=0 tx=0).stop 2023-07-21T22:32:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.980+0000 7f10f2bad700 1 -- 172.21.15.107:0/3939550457 shutdown_connections 2023-07-21T22:32:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.980+0000 7f10f2bad700 1 --2- 172.21.15.107:0/3939550457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10ec10bd60 0x7f10ec1a33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.980+0000 7f10f2bad700 1 --2- 172.21.15.107:0/3939550457 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f10d40a1350 0x7f10d40a3800 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.980+0000 7f10f2bad700 1 --2- 172.21.15.107:0/3939550457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10ec070f00 0x7f10ec19e790 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.980+0000 7f10f2bad700 1 --2- 172.21.15.107:0/3939550457 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10ec071850 0x7f10ec1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.980+0000 7f10f2bad700 1 -- 172.21.15.107:0/3939550457 >> 172.21.15.107:0/3939550457 conn(0x7f10ec06ca50 msgr2=0x7f10ec06e870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.987+0000 7f10f2bad700 1 -- 172.21.15.107:0/3939550457 shutdown_connections 2023-07-21T22:32:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:04.988+0000 7f10f2bad700 1 -- 172.21.15.107:0/3939550457 wait complete. 2023-07-21T22:32:05.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:05 smithi190 ceph-mon[98334]: pgmap v384: 97 pgs: 97 active+clean; 533 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 717 op/s 2023-07-21T22:32:05.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:05 smithi132 ceph-mon[97863]: pgmap v384: 97 pgs: 97 active+clean; 533 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 717 op/s 2023-07-21T22:32:05.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:05 smithi107 ceph-mon[96543]: pgmap v384: 97 pgs: 97 active+clean; 533 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 717 op/s 2023-07-21T22:32:06.487 INFO:tasks.workunit.client.0.smithi107.stdout: 139 1317414 937 902983 1109 382579 1464 2023-07-21T22:32:07.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:07 smithi190 ceph-mon[98334]: pgmap v385: 97 pgs: 97 active+clean; 532 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 755 op/s 2023-07-21T22:32:07.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:07 smithi132 ceph-mon[97863]: pgmap v385: 97 pgs: 97 active+clean; 532 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 755 op/s 2023-07-21T22:32:07.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:07 smithi107 ceph-mon[96543]: pgmap v385: 97 pgs: 97 active+clean; 532 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 755 op/s 2023-07-21T22:32:09.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:09 smithi190 ceph-mon[98334]: pgmap v386: 97 pgs: 97 active+clean; 540 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 777 op/s 2023-07-21T22:32:09.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:09 smithi132 ceph-mon[97863]: pgmap v386: 97 pgs: 97 active+clean; 540 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 777 op/s 2023-07-21T22:32:09.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:09 smithi107 ceph-mon[96543]: pgmap v386: 97 pgs: 97 active+clean; 540 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 777 op/s 2023-07-21T22:32:11.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:11 smithi190 ceph-mon[98334]: pgmap v387: 97 pgs: 97 active+clean; 540 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 745 op/s 2023-07-21T22:32:11.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:11 smithi132 ceph-mon[97863]: pgmap v387: 97 pgs: 97 active+clean; 540 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 745 op/s 2023-07-21T22:32:11.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:11 smithi107 ceph-mon[96543]: pgmap v387: 97 pgs: 97 active+clean; 540 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 745 op/s 2023-07-21T22:32:13.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:13 smithi190 ceph-mon[98334]: pgmap v388: 97 pgs: 97 active+clean; 552 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 827 op/s 2023-07-21T22:32:13.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:13 smithi132 ceph-mon[97863]: pgmap v388: 97 pgs: 97 active+clean; 552 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 827 op/s 2023-07-21T22:32:13.695 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:13 smithi107 ceph-mon[96543]: pgmap v388: 97 pgs: 97 active+clean; 552 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 827 op/s 2023-07-21T22:32:15.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:15 smithi190 ceph-mon[98334]: pgmap v389: 97 pgs: 97 active+clean; 551 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 462 op/s 2023-07-21T22:32:15.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:15 smithi132 ceph-mon[97863]: pgmap v389: 97 pgs: 97 active+clean; 551 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 462 op/s 2023-07-21T22:32:15.702 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:15 smithi107 ceph-mon[96543]: pgmap v389: 97 pgs: 97 active+clean; 551 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.1 MiB/s wr, 462 op/s 2023-07-21T22:32:16.494 INFO:tasks.workunit.client.0.smithi107.stdout: 157 2394314 873 1636910 942 793121 1704 2023-07-21T22:32:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:17 smithi190 ceph-mon[98334]: pgmap v390: 97 pgs: 97 active+clean; 556 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 444 op/s 2023-07-21T22:32:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:17 smithi132 ceph-mon[97863]: pgmap v390: 97 pgs: 97 active+clean; 556 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 444 op/s 2023-07-21T22:32:17.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:17 smithi107 ceph-mon[96543]: pgmap v390: 97 pgs: 97 active+clean; 556 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 444 op/s 2023-07-21T22:32:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.934+0000 7f2895293700 1 -- 172.21.15.107:0/2767545751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890071850 msgr2=0x7f289010b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.934+0000 7f2895293700 1 --2- 172.21.15.107:0/2767545751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890071850 0x7f289010b7d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f288800b0a0 tx=0x7f288800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:32:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.935+0000 7f2895293700 1 -- 172.21.15.107:0/2767545751 shutdown_connections 2023-07-21T22:32:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.935+0000 7f2895293700 1 --2- 172.21.15.107:0/2767545751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f289010bd60 0x7f289010e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.935+0000 7f2895293700 1 --2- 172.21.15.107:0/2767545751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2890070f00 0x7f2890071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.935+0000 7f2895293700 1 --2- 172.21.15.107:0/2767545751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890071850 0x7f289010b7d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.935+0000 7f2895293700 1 -- 172.21.15.107:0/2767545751 >> 172.21.15.107:0/2767545751 conn(0x7f289006ca50 msgr2=0x7f289006eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.935+0000 7f2895293700 1 -- 172.21.15.107:0/2767545751 shutdown_connections 2023-07-21T22:32:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.935+0000 7f2895293700 1 -- 172.21.15.107:0/2767545751 wait complete. 2023-07-21T22:32:18.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.936+0000 7f2895293700 1 Processor -- start 2023-07-21T22:32:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.937+0000 7f2895293700 1 -- start start 2023-07-21T22:32:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.937+0000 7f2895293700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2890070f00 0x7f2890113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.937+0000 7f2895293700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890071850 0x7f2890114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.937+0000 7f2895293700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f289010bd60 0x7f2890114ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.937+0000 7f2895293700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2890115530 con 0x7f2890070f00 2023-07-21T22:32:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.937+0000 7f2895293700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28901156a0 con 0x7f2890071850 2023-07-21T22:32:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.937+0000 7f2895293700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2890115860 con 0x7f289010bd60 2023-07-21T22:32:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.938+0000 7f288effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2890070f00 0x7f2890113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.938+0000 7f288e7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890071850 0x7f2890114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.938+0000 7f288e7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890071850 0x7f2890114000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40802/0 (socket says 172.21.15.107:40802) 2023-07-21T22:32:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.938+0000 7f288e7fc700 1 -- 172.21.15.107:0/720416032 learned_addr learned my addr 172.21.15.107:0/720416032 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:18.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.938+0000 7f288f7fe700 1 --2- 172.21.15.107:0/720416032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f289010bd60 0x7f2890114ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.953+0000 7f288e7fc700 1 -- 172.21.15.107:0/720416032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f289010bd60 msgr2=0x7f2890114ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.953+0000 7f288e7fc700 1 --2- 172.21.15.107:0/720416032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f289010bd60 0x7f2890114ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.953+0000 7f288e7fc700 1 -- 172.21.15.107:0/720416032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2890070f00 msgr2=0x7f2890113ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.953+0000 7f288e7fc700 1 --2- 172.21.15.107:0/720416032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2890070f00 0x7f2890113ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.953+0000 7f288e7fc700 1 -- 172.21.15.107:0/720416032 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2888009d00 con 0x7f2890071850 2023-07-21T22:32:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:18.957+0000 7f288e7fc700 1 --2- 172.21.15.107:0/720416032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890071850 0x7f2890114000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2888007ba0 tx=0x7f2888009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.016+0000 7f2877fff700 1 -- 172.21.15.107:0/720416032 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288800e040 con 0x7f2890071850 2023-07-21T22:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.016+0000 7f2877fff700 1 -- 172.21.15.107:0/720416032 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f288801de10 con 0x7f2890071850 2023-07-21T22:32:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.016+0000 7f2877fff700 1 -- 172.21.15.107:0/720416032 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288801beb0 con 0x7f2890071850 2023-07-21T22:32:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.017+0000 7f2895293700 1 -- 172.21.15.107:0/720416032 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2890115ae0 con 0x7f2890071850 2023-07-21T22:32:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.017+0000 7f2895293700 1 -- 172.21.15.107:0/720416032 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2890116370 con 0x7f2890071850 2023-07-21T22:32:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.023+0000 7f28a1418b80 1 Processor -- start 2023-07-21T22:32:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.025+0000 7f2877fff700 1 -- 172.21.15.107:0/720416032 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f288801b550 con 0x7f2890071850 2023-07-21T22:32:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.026+0000 7f2877fff700 1 --2- 172.21.15.107:0/720416032 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f28780a1350 0x7f28780a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.026+0000 7f2877fff700 1 -- 172.21.15.107:0/720416032 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f28880ce550 con 0x7f2890071850 2023-07-21T22:32:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.027+0000 7f28a1418b80 1 -- start start 2023-07-21T22:32:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.027+0000 7f28a1418b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee69ef86b0 0x55ee6a09c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.027+0000 7f28a1418b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09c830 0x55ee6a09d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.027+0000 7f28a1418b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09d580 0x55ee6a09d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.027+0000 7f28a1418b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee69fa1c50 con 0x55ee6a09c830 2023-07-21T22:32:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.027+0000 7f28a1418b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee69ff49e0 con 0x55ee69ef86b0 2023-07-21T22:32:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.027+0000 7f28a1418b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee69fede70 con 0x55ee6a09d580 2023-07-21T22:32:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.037+0000 7f288f7fe700 1 --2- 172.21.15.107:0/720416032 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f28780a1350 0x7f28780a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.037+0000 7f288f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09d580 0x55ee6a09d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.038+0000 7f288f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09d580 0x55ee6a09d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38538/0 (socket says 172.21.15.107:38538) 2023-07-21T22:32:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.038+0000 7f288f7fe700 1 -- 172.21.15.107:0/1762578761 learned_addr learned my addr 172.21.15.107:0/1762578761 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.038+0000 7f288e7fc700 1 --2- 172.21.15.107:0/1762578761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09c830 0x55ee6a09d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.038+0000 7f288effd700 1 --2- 172.21.15.107:0/1762578761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee69ef86b0 0x55ee6a09c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.040+0000 7f288f7fe700 1 -- 172.21.15.107:0/1762578761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee69ef86b0 msgr2=0x55ee6a09c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.040+0000 7f288f7fe700 1 --2- 172.21.15.107:0/1762578761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee69ef86b0 0x55ee6a09c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.040+0000 7f288f7fe700 1 -- 172.21.15.107:0/1762578761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09c830 msgr2=0x55ee6a09d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.040+0000 7f288f7fe700 1 --2- 172.21.15.107:0/1762578761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09c830 0x55ee6a09d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.040+0000 7f288f7fe700 1 -- 172.21.15.107:0/1762578761 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee69e1bd80 con 0x55ee6a09d580 2023-07-21T22:32:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.041+0000 7f288f7fe700 1 --2- 172.21.15.107:0/1762578761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09d580 0x55ee6a09d9f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f288000a5b0 tx=0x7f288000a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d938df0f6498d849 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:19.105 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.048+0000 7f28757fa700 1 -- 172.21.15.107:0/1762578761 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288001d030 con 0x55ee6a09d580 2023-07-21T22:32:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.048+0000 7f28757fa700 1 -- 172.21.15.107:0/1762578761 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2880017ab0 con 0x55ee6a09d580 2023-07-21T22:32:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.048+0000 7f28757fa700 1 -- 172.21.15.107:0/1762578761 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880018ce0 con 0x55ee6a09d580 2023-07-21T22:32:19.106 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.060+0000 7f28a1418b80 1 -- 172.21.15.107:0/1762578761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09d580 msgr2=0x55ee6a09d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.060+0000 7f28a1418b80 1 --2- 172.21.15.107:0/1762578761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09d580 0x55ee6a09d9f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f288000a5b0 tx=0x7f288000a8c0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.077+0000 7f28a1418b80 1 -- 172.21.15.107:0/1762578761 shutdown_connections 2023-07-21T22:32:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.077+0000 7f28a1418b80 1 --2- 172.21.15.107:0/1762578761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09d580 0x55ee6a09d9f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.077+0000 7f28a1418b80 1 --2- 172.21.15.107:0/1762578761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee69ef86b0 0x55ee6a09c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.077+0000 7f28a1418b80 1 --2- 172.21.15.107:0/1762578761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09c830 0x55ee6a09d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.077+0000 7f28a1418b80 1 -- 172.21.15.107:0/1762578761 >> 172.21.15.107:0/1762578761 conn(0x55ee6a091ae0 msgr2=0x55ee6a096c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.077+0000 7f28a1418b80 1 -- 172.21.15.107:0/1762578761 shutdown_connections 2023-07-21T22:32:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.077+0000 7f28a1418b80 1 -- 172.21.15.107:0/1762578761 wait complete. 2023-07-21T22:32:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.078+0000 7f28a1418b80 1 -- start start 2023-07-21T22:32:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.078+0000 7f28a1418b80 1 Processor -- start 2023-07-21T22:32:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.078+0000 7f28a1418b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee69ef86b0 0x55ee6a0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.078+0000 7f28a1418b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09c830 0x55ee6a0a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.078+0000 7f28a1418b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09d580 0x55ee6a0acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.078+0000 7f28a1418b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee69f8da60 con 0x55ee6a09d580 2023-07-21T22:32:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.078+0000 7f28a1418b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee69f8d760 con 0x55ee69ef86b0 2023-07-21T22:32:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.078+0000 7f28a1418b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee69e718c0 con 0x55ee6a09c830 2023-07-21T22:32:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.078+0000 7f28a1418b80 10 client.0 _ll_register_callbacks cb 0x55ee69f8b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.079+0000 7f288f7fe700 1 --2- 172.21.15.107:0/720416032 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f28780a1350 0x7f28780a3800 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f288000a5b0 tx=0x7f288000a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.079+0000 7f288f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09d580 0x55ee6a0acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.079+0000 7f288f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09d580 0x55ee6a0acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49988/0 (socket says 172.21.15.107:49988) 2023-07-21T22:32:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.079+0000 7f288f7fe700 1 -- 172.21.15.107:0/2192694538 learned_addr learned my addr 172.21.15.107:0/2192694538 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.083+0000 7f288f7fe700 1 -- 172.21.15.107:0/2192694538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09c830 msgr2=0x55ee6a0a5d50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:32:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.083+0000 7f288f7fe700 1 --2- 172.21.15.107:0/2192694538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09c830 0x55ee6a0a5d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.083+0000 7f288f7fe700 1 -- 172.21.15.107:0/2192694538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee69ef86b0 msgr2=0x55ee6a0a5810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:32:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.083+0000 7f288f7fe700 1 --2- 172.21.15.107:0/2192694538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee69ef86b0 0x55ee6a0a5810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.083+0000 7f288f7fe700 1 -- 172.21.15.107:0/2192694538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2880013040 con 0x55ee6a09d580 2023-07-21T22:32:19.147 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.095+0000 7f288f7fe700 1 --2- 172.21.15.107:0/2192694538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09d580 0x55ee6a0acd90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f288000a580 tx=0x7f2880009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a70cfc13eed23be server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.172+0000 7f2872ff5700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:32:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.172+0000 7f2872ff5700 1 -- 172.21.15.107:0/2192694538 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880018ce0 con 0x55ee6a09d580 2023-07-21T22:32:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.172+0000 7f2872ff5700 1 -- 172.21.15.107:0/2192694538 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f288001e070 con 0x55ee6a09d580 2023-07-21T22:32:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.173+0000 7f28a1418b80 1 -- 172.21.15.107:0/2192694538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee69f47490 con 0x55ee6a09d580 2023-07-21T22:32:19.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.188+0000 7f2872ff5700 1 -- 172.21.15.107:0/2192694538 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2880018600 con 0x55ee6a09d580 2023-07-21T22:32:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.195+0000 7f28a1418b80 10 client.16221 fetch_fsmap learned FSMap version 34 2023-07-21T22:32:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.195+0000 7f28a1418b80 1 -- 172.21.15.107:0/2192694538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ee69e04c00 con 0x55ee6a09d580 2023-07-21T22:32:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.208+0000 7f2872ff5700 1 -- 172.21.15.107:0/2192694538 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee69f47490 con 0x55ee6a09d580 2023-07-21T22:32:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.208+0000 7f2872ff5700 1 -- 172.21.15.107:0/2192694538 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f288002f680 con 0x55ee6a09d580 2023-07-21T22:32:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.209+0000 7f28a1418b80 10 client.16221 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:32:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.209+0000 7f28a1418b80 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:19.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.209+0000 7f28a1418b80 20 client.16221 populate_metadata read hostname 'smithi107' 2023-07-21T22:32:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.209+0000 7f28a1418b80 1 --2- 172.21.15.107:0/2192694538 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ee6a0b15b0 0x55ee6a0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.209+0000 7f28a1418b80 4 client.16221 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:19.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.209+0000 7f28a1418b80 1 -- 172.21.15.107:0/2192694538 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee69fe6e50 con 0x55ee6a0b15b0 2023-07-21T22:32:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.216+0000 7f288effd700 1 --2- 172.21.15.107:0/2192694538 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ee6a0b15b0 0x55ee6a0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.225+0000 7f288effd700 1 --2- 172.21.15.107:0/2192694538 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ee6a0b15b0 0x55ee6a0b39c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4f8c8887e8a875e server_cookie=830502816fbd0e39 in_seq=0 out_seq=0 2023-07-21T22:32:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.230+0000 7f2872ff5700 10 client.16221 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.231+0000 7f2872ff5700 1 -- 172.21.15.107:0/2192694538 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ee69fe6e50 con 0x55ee6a0b15b0 2023-07-21T22:32:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.231+0000 7f2872ff5700 10 client.16221 handle_command_reply: tid=0 2023-07-21T22:32:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.234+0000 7f28a1418b80 1 client.16221 shutdown 2023-07-21T22:32:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.234+0000 7f28a1418b80 1 -- 172.21.15.107:0/2192694538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09d580 msgr2=0x55ee6a0acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.234+0000 7f28a1418b80 1 --2- 172.21.15.107:0/2192694538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09d580 0x55ee6a0acd90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f288000a580 tx=0x7f2880009cd0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.236+0000 7f28a1418b80 1 -- 172.21.15.107:0/2192694538 shutdown_connections 2023-07-21T22:32:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.236+0000 7f28a1418b80 1 --2- 172.21.15.107:0/2192694538 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ee6a0b15b0 0x55ee6a0b39c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.236+0000 7f28a1418b80 1 --2- 172.21.15.107:0/2192694538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09d580 0x55ee6a0acd90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.236+0000 7f28a1418b80 1 --2- 172.21.15.107:0/2192694538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee69ef86b0 0x55ee6a0a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.236+0000 7f28a1418b80 1 --2- 172.21.15.107:0/2192694538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09c830 0x55ee6a0a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.236+0000 7f28a1418b80 1 -- 172.21.15.107:0/2192694538 >> 172.21.15.107:0/2192694538 conn(0x55ee6a091ae0 msgr2=0x55ee6a0963e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.237+0000 7f2872ff5700 0 client.16221 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.237+0000 7f28a1418b80 1 -- 172.21.15.107:0/2192694538 shutdown_connections 2023-07-21T22:32:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.237+0000 7f28a1418b80 1 -- 172.21.15.107:0/2192694538 wait complete. 2023-07-21T22:32:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.237+0000 7f28a1418b80 20 client.16221 trim_cache size 0 max 16384 2023-07-21T22:32:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.277+0000 7f28a1418b80 1 Processor -- start 2023-07-21T22:32:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.277+0000 7f28a1418b80 1 -- start start 2023-07-21T22:32:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f28a1418b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee69ef86b0 0x55ee6a0a4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f28a1418b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09c830 0x55ee6a0acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f28a1418b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09d580 0x55ee6a096b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f28a1418b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6a0a53e0 con 0x55ee6a09c830 2023-07-21T22:32:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f28a1418b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6a09ced0 con 0x55ee6a09d580 2023-07-21T22:32:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f28a1418b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6a0a4d10 con 0x55ee69ef86b0 2023-07-21T22:32:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f288f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09d580 0x55ee6a096b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f288f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09d580 0x55ee6a096b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40820/0 (socket says 172.21.15.107:40820) 2023-07-21T22:32:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f288f7fe700 1 -- 172.21.15.107:0/2753729061 learned_addr learned my addr 172.21.15.107:0/2753729061 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f288effd700 1 --2- 172.21.15.107:0/2753729061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee69ef86b0 0x55ee6a0a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f288f7fe700 1 -- 172.21.15.107:0/2753729061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee69ef86b0 msgr2=0x55ee6a0a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f288f7fe700 1 --2- 172.21.15.107:0/2753729061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee69ef86b0 0x55ee6a0a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f288f7fe700 1 -- 172.21.15.107:0/2753729061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09c830 msgr2=0x55ee6a0acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f288f7fe700 1 --2- 172.21.15.107:0/2753729061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09c830 0x55ee6a0acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.278+0000 7f288f7fe700 1 -- 172.21.15.107:0/2753729061 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee69e421f0 con 0x55ee6a09d580 2023-07-21T22:32:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.289+0000 7f288f7fe700 1 --2- 172.21.15.107:0/2753729061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09d580 0x55ee6a096b10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f288000a4d0 tx=0x7f2880030f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbed08491dc5d64c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.289+0000 7f2875ffb700 1 -- 172.21.15.107:0/2753729061 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288003d070 con 0x55ee6a09d580 2023-07-21T22:32:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.289+0000 7f2875ffb700 1 -- 172.21.15.107:0/2753729061 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2880007000 con 0x55ee6a09d580 2023-07-21T22:32:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.289+0000 7f2875ffb700 1 -- 172.21.15.107:0/2753729061 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288001d030 con 0x55ee6a09d580 2023-07-21T22:32:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.290+0000 7f28a1418b80 1 -- 172.21.15.107:0/2753729061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09d580 msgr2=0x55ee6a096b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.290+0000 7f28a1418b80 1 --2- 172.21.15.107:0/2753729061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09d580 0x55ee6a096b10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f288000a4d0 tx=0x7f2880030f70 comp rx=0 tx=0).stop 2023-07-21T22:32:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.291+0000 7f28a1418b80 1 -- 172.21.15.107:0/2753729061 shutdown_connections 2023-07-21T22:32:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.291+0000 7f28a1418b80 1 --2- 172.21.15.107:0/2753729061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09d580 0x55ee6a096b10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.291+0000 7f28a1418b80 1 --2- 172.21.15.107:0/2753729061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee69ef86b0 0x55ee6a0a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.291+0000 7f28a1418b80 1 --2- 172.21.15.107:0/2753729061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee6a09c830 0x55ee6a0acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.291+0000 7f28a1418b80 1 -- 172.21.15.107:0/2753729061 >> 172.21.15.107:0/2753729061 conn(0x55ee6a091ae0 msgr2=0x55ee6a0b8300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.340+0000 7f28a1418b80 1 -- 172.21.15.107:0/2753729061 shutdown_connections 2023-07-21T22:32:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.349+0000 7f28a1418b80 1 -- 172.21.15.107:0/2753729061 wait complete. 2023-07-21T22:32:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.349+0000 7f28a1418b80 1 -- start start 2023-07-21T22:32:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.349+0000 7f28a1418b80 1 Processor -- start 2023-07-21T22:32:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.351+0000 7f28a1418b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee69ef86b0 0x55ee6a0a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.351+0000 7f28a1418b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09c830 0x55ee6a0a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.351+0000 7f28a1418b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09d580 0x55ee6a0abb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.351+0000 7f28a1418b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6a0ac0b0 con 0x55ee69ef86b0 2023-07-21T22:32:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.351+0000 7f28a1418b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6a0ac220 con 0x55ee6a09c830 2023-07-21T22:32:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.351+0000 7f28a1418b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee6a0ac3e0 con 0x55ee6a09d580 2023-07-21T22:32:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.351+0000 7f28a1418b80 10 client.0 _ll_register_callbacks cb 0x7f2880009330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.351+0000 7f288e7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09c830 0x55ee6a0a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.351+0000 7f288e7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09c830 0x55ee6a0a8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40828/0 (socket says 172.21.15.107:40828) 2023-07-21T22:32:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.351+0000 7f288e7fc700 1 -- 172.21.15.107:0/3953730810 learned_addr learned my addr 172.21.15.107:0/3953730810 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.352+0000 7f288f7fe700 1 --2- 172.21.15.107:0/3953730810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09d580 0x55ee6a0abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.356+0000 7f288e7fc700 1 -- 172.21.15.107:0/3953730810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09d580 msgr2=0x55ee6a0abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.356+0000 7f288e7fc700 1 --2- 172.21.15.107:0/3953730810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09d580 0x55ee6a0abb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.356+0000 7f288e7fc700 1 -- 172.21.15.107:0/3953730810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee69ef86b0 msgr2=0x55ee6a0a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.356+0000 7f288e7fc700 1 --2- 172.21.15.107:0/3953730810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee69ef86b0 0x55ee6a0a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.356+0000 7f288e7fc700 1 -- 172.21.15.107:0/3953730810 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee69e04c00 con 0x55ee6a09c830 2023-07-21T22:32:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.360+0000 7f288e7fc700 1 --2- 172.21.15.107:0/3953730810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09c830 0x55ee6a0a8b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f28880cbfd0 tx=0x7f28880099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f8706ddb5be3b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.363+0000 7f2872ff5700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:32:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.363+0000 7f2872ff5700 1 -- 172.21.15.107:0/3953730810 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28880ce310 con 0x55ee6a09c830 2023-07-21T22:32:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.363+0000 7f2872ff5700 1 -- 172.21.15.107:0/3953730810 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f288801a3b0 con 0x55ee6a09c830 2023-07-21T22:32:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.363+0000 7f28a1418b80 1 -- 172.21.15.107:0/3953730810 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2880017ab0 con 0x55ee6a09c830 2023-07-21T22:32:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.366+0000 7f2872ff5700 1 -- 172.21.15.107:0/3953730810 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f288809b970 con 0x55ee6a09c830 2023-07-21T22:32:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.366+0000 7f28a1418b80 10 client.25685 fetch_fsmap learned FSMap version 34 2023-07-21T22:32:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.366+0000 7f28a1418b80 1 -- 172.21.15.107:0/3953730810 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55ee69e1a250 con 0x55ee6a09c830 2023-07-21T22:32:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.377+0000 7f2872ff5700 1 -- 172.21.15.107:0/3953730810 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2880017ab0 con 0x55ee6a09c830 2023-07-21T22:32:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.377+0000 7f2872ff5700 1 -- 172.21.15.107:0/3953730810 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f28880a38a0 con 0x55ee6a09c830 2023-07-21T22:32:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.378+0000 7f28a1418b80 10 client.25685 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:32:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.378+0000 7f28a1418b80 10 client.25685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.378+0000 7f28a1418b80 20 client.25685 populate_metadata read hostname 'smithi107' 2023-07-21T22:32:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.378+0000 7f28a1418b80 1 --2- 172.21.15.107:0/3953730810 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ee6a105c90 0x55ee6a108070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.379+0000 7f28a1418b80 4 client.25685 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:32:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.379+0000 7f28a1418b80 1 -- 172.21.15.107:0/3953730810 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ee69fa8ff0 con 0x55ee6a105c90 2023-07-21T22:32:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.379+0000 7f288effd700 1 --2- 172.21.15.107:0/3953730810 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ee6a105c90 0x55ee6a108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.381+0000 7f288effd700 1 --2- 172.21.15.107:0/3953730810 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ee6a105c90 0x55ee6a108070 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b735f1319346fc server_cookie=cca76c4454cfbbb7 in_seq=0 out_seq=0 2023-07-21T22:32:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.385+0000 7f2872ff5700 10 client.25685 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:19.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:19 smithi190 ceph-mon[98334]: pgmap v391: 97 pgs: 97 active+clean; 561 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 411 op/s 2023-07-21T22:32:19.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:19 smithi132 ceph-mon[97863]: pgmap v391: 97 pgs: 97 active+clean; 561 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 411 op/s 2023-07-21T22:32:19.704 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:19 smithi107 ceph-mon[96543]: pgmap v391: 97 pgs: 97 active+clean; 561 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 411 op/s 2023-07-21T22:32:19.775 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:32:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:32:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:32:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:32:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:32:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.834+0000 7f2872ff5700 1 -- 172.21.15.107:0/3953730810 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ee69fa8ff0 con 0x55ee6a105c90 2023-07-21T22:32:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.834+0000 7f2872ff5700 10 client.25685 handle_command_reply: tid=0 2023-07-21T22:32:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.834+0000 7f28a1418b80 1 client.25685 shutdown 2023-07-21T22:32:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.834+0000 7f28a1418b80 1 -- 172.21.15.107:0/3953730810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09c830 msgr2=0x55ee6a0a8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.834+0000 7f28a1418b80 1 --2- 172.21.15.107:0/3953730810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09c830 0x55ee6a0a8b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f28880cbfd0 tx=0x7f28880099e0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.834+0000 7f28a1418b80 1 -- 172.21.15.107:0/3953730810 shutdown_connections 2023-07-21T22:32:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.834+0000 7f28a1418b80 1 --2- 172.21.15.107:0/3953730810 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ee6a105c90 0x55ee6a108070 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.834+0000 7f28a1418b80 1 --2- 172.21.15.107:0/3953730810 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ee6a09d580 0x55ee6a0abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.834+0000 7f28a1418b80 1 --2- 172.21.15.107:0/3953730810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee69ef86b0 0x55ee6a0a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.834+0000 7f28a1418b80 1 --2- 172.21.15.107:0/3953730810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ee6a09c830 0x55ee6a0a8b50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.834+0000 7f28a1418b80 1 -- 172.21.15.107:0/3953730810 >> 172.21.15.107:0/3953730810 conn(0x55ee6a091ae0 msgr2=0x55ee6a0a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.834+0000 7f2872ff5700 0 client.25685 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.835+0000 7f28a1418b80 1 -- 172.21.15.107:0/3953730810 shutdown_connections 2023-07-21T22:32:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.835+0000 7f28a1418b80 1 -- 172.21.15.107:0/3953730810 wait complete. 2023-07-21T22:32:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.846+0000 7f28a1418b80 20 client.25685 trim_cache size 0 max 16384 2023-07-21T22:32:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.890+0000 7f2895293700 1 -- 172.21.15.107:0/720416032 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f28780a1350 msgr2=0x7f28780a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.890+0000 7f2895293700 1 --2- 172.21.15.107:0/720416032 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f28780a1350 0x7f28780a3800 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f288000a5b0 tx=0x7f288000a8c0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.890+0000 7f2895293700 1 -- 172.21.15.107:0/720416032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890071850 msgr2=0x7f2890114000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.890+0000 7f2895293700 1 --2- 172.21.15.107:0/720416032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890071850 0x7f2890114000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2888007ba0 tx=0x7f2888009300 comp rx=0 tx=0).stop 2023-07-21T22:32:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.890+0000 7f2895293700 1 -- 172.21.15.107:0/720416032 shutdown_connections 2023-07-21T22:32:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.890+0000 7f2895293700 1 --2- 172.21.15.107:0/720416032 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f28780a1350 0x7f28780a3800 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.890+0000 7f2895293700 1 --2- 172.21.15.107:0/720416032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f289010bd60 0x7f2890114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.890+0000 7f2895293700 1 --2- 172.21.15.107:0/720416032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2890070f00 0x7f2890113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.890+0000 7f2895293700 1 --2- 172.21.15.107:0/720416032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890071850 0x7f2890114000 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.890+0000 7f2895293700 1 -- 172.21.15.107:0/720416032 >> 172.21.15.107:0/720416032 conn(0x7f289006ca50 msgr2=0x7f289006dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.891+0000 7f2895293700 1 -- 172.21.15.107:0/720416032 shutdown_connections 2023-07-21T22:32:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:19.891+0000 7f2895293700 1 -- 172.21.15.107:0/720416032 wait complete. 2023-07-21T22:32:20.554 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.507+0000 7f1e409ea700 1 -- 172.21.15.107:0/978917779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e3c0fec00 msgr2=0x7f1e3c0ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:20.555 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.507+0000 7f1e409ea700 1 --2- 172.21.15.107:0/978917779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e3c0fec00 0x7f1e3c0ff070 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009990 tx=0x7f1e30009ca0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.555 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.507+0000 7f1e409ea700 1 -- 172.21.15.107:0/978917779 shutdown_connections 2023-07-21T22:32:20.555 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.507+0000 7f1e409ea700 1 --2- 172.21.15.107:0/978917779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c0ff5b0 0x7f1e3c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.555 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.507+0000 7f1e409ea700 1 --2- 172.21.15.107:0/978917779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e3c0fffb0 0x7f1e3c1003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.555 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.507+0000 7f1e409ea700 1 --2- 172.21.15.107:0/978917779 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e3c0fec00 0x7f1e3c0ff070 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.556 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.507+0000 7f1e409ea700 1 -- 172.21.15.107:0/978917779 >> 172.21.15.107:0/978917779 conn(0x7f1e3c0f97a0 msgr2=0x7f1e3c0fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:20.556 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.508+0000 7f1e409ea700 1 -- 172.21.15.107:0/978917779 shutdown_connections 2023-07-21T22:32:20.556 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.508+0000 7f1e409ea700 1 -- 172.21.15.107:0/978917779 wait complete. 2023-07-21T22:32:20.557 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.508+0000 7f1e409ea700 1 Processor -- start 2023-07-21T22:32:20.558 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.508+0000 7f1e409ea700 1 -- start start 2023-07-21T22:32:20.558 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.508+0000 7f1e409ea700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e3c0fec00 0x7f1e3c1935f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.558 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.508+0000 7f1e409ea700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c0ff5b0 0x7f1e3c193b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.558 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.508+0000 7f1e409ea700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e3c0fffb0 0x7f1e3c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.558 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e409ea700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c19b170 con 0x7f1e3c0ff5b0 2023-07-21T22:32:20.558 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e409ea700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c19b2b0 con 0x7f1e3c0fec00 2023-07-21T22:32:20.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e409ea700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c19b440 con 0x7f1e3c0fffb0 2023-07-21T22:32:20.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e3a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e3c0fec00 0x7f1e3c1935f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e39d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c0ff5b0 0x7f1e3c193b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e3a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e3c0fec00 0x7f1e3c1935f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40840/0 (socket says 172.21.15.107:40840) 2023-07-21T22:32:20.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e39d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c0ff5b0 0x7f1e3c193b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50014/0 (socket says 172.21.15.107:50014) 2023-07-21T22:32:20.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e3a59c700 1 -- 172.21.15.107:0/2775688516 learned_addr learned my addr 172.21.15.107:0/2775688516 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:20.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e3ad9d700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e3c0fffb0 0x7f1e3c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e3a59c700 1 -- 172.21.15.107:0/2775688516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e3c0fffb0 msgr2=0x7f1e3c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:20.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e3c0fffb0 0x7f1e3c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e3a59c700 1 -- 172.21.15.107:0/2775688516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c0ff5b0 msgr2=0x7f1e3c193b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:20.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c0ff5b0 0x7f1e3c193b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e3a59c700 1 -- 172.21.15.107:0/2775688516 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e300095f0 con 0x7f1e3c0fec00 2023-07-21T22:32:20.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e3ad9d700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e3c0fffb0 0x7f1e3c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:20.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e39d9b700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c0ff5b0 0x7f1e3c193b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:20.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.509+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e3c0fec00 0x7f1e3c1935f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00e9a0 tx=0x7f1e2c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:20.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.510+0000 7f1e2b7fe700 1 -- 172.21.15.107:0/2775688516 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c009790 con 0x7f1e3c0fec00 2023-07-21T22:32:20.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.510+0000 7f1e409ea700 1 -- 172.21.15.107:0/2775688516 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3c19b720 con 0x7f1e3c0fec00 2023-07-21T22:32:20.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.510+0000 7f1e2b7fe700 1 -- 172.21.15.107:0/2775688516 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e2c010430 con 0x7f1e3c0fec00 2023-07-21T22:32:20.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.510+0000 7f1e2b7fe700 1 -- 172.21.15.107:0/2775688516 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c018680 con 0x7f1e3c0fec00 2023-07-21T22:32:20.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.510+0000 7f1e409ea700 1 -- 172.21.15.107:0/2775688516 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3c19bc80 con 0x7f1e3c0fec00 2023-07-21T22:32:20.563 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.513+0000 7f1e2b7fe700 1 -- 172.21.15.107:0/2775688516 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1e2c0105a0 con 0x7f1e3c0fec00 2023-07-21T22:32:20.563 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.513+0000 7f1e2b7fe700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1e240a1150 0x7f1e240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.563 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.513+0000 7f1e4cb6fb80 1 Processor -- start 2023-07-21T22:32:20.564 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.514+0000 7f1e4cb6fb80 1 -- start start 2023-07-21T22:32:20.564 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.514+0000 7f1e39d9b700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1e240a1150 0x7f1e240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.564 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.514+0000 7f1e2b7fe700 1 -- 172.21.15.107:0/2775688516 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1e2c014070 con 0x7f1e3c0fec00 2023-07-21T22:32:20.564 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.514+0000 7f1e4cb6fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e060 0x55f02379e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.564 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.514+0000 7f1e4cb6fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e9f0 0x55f02379cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.564 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.514+0000 7f1e4cb6fb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f02379d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.564 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.514+0000 7f1e4cb6fb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f023699cb0 con 0x55f02379d190 2023-07-21T22:32:20.565 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.514+0000 7f1e4cb6fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0236eca40 con 0x55f02379e9f0 2023-07-21T22:32:20.565 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.514+0000 7f1e4cb6fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0236e5ed0 con 0x55f02379e060 2023-07-21T22:32:20.565 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.514+0000 7f1e3a59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e060 0x55f02379e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.565 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.514+0000 7f1e3a59c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e060 0x55f02379e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38578/0 (socket says 172.21.15.107:38578) 2023-07-21T22:32:20.565 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.514+0000 7f1e3a59c700 1 -- 172.21.15.107:0/2442202485 learned_addr learned my addr 172.21.15.107:0/2442202485 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:20.565 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.514+0000 7f1e3ad9d700 1 --2- 172.21.15.107:0/2442202485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f02379d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.565 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.515+0000 7f1e39d9b700 1 --2- 172.21.15.107:0/2442202485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e9f0 0x55f02379cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.565 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.515+0000 7f1e3a59c700 1 -- 172.21.15.107:0/2442202485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e9f0 msgr2=0x55f02379cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.566 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.515+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/2442202485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e9f0 0x55f02379cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.566 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.515+0000 7f1e3a59c700 1 -- 172.21.15.107:0/2442202485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 msgr2=0x55f02379d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.566 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.515+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/2442202485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f02379d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.566 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.515+0000 7f1e3a59c700 1 -- 172.21.15.107:0/2442202485 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f023513de0 con 0x55f02379e060 2023-07-21T22:32:20.566 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.515+0000 7f1e39d9b700 1 --2- 172.21.15.107:0/2442202485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e9f0 0x55f02379cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:32:20.566 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.515+0000 7f1e39d9b700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1e240a1150 0x7f1e240a3600 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000b2e0 tx=0x7f1e30004600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:20.566 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.515+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/2442202485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e060 0x55f02379e470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0963e0 tx=0x7f1e2c099910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a4ef6bd2317beec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:20.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.515+0000 7f1e28ff9700 1 -- 172.21.15.107:0/2442202485 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0a2330 con 0x55f02379e060 2023-07-21T22:32:20.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.515+0000 7f1e28ff9700 1 -- 172.21.15.107:0/2442202485 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e2c098280 con 0x55f02379e060 2023-07-21T22:32:20.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.515+0000 7f1e28ff9700 1 -- 172.21.15.107:0/2442202485 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0a2100 con 0x55f02379e060 2023-07-21T22:32:20.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.516+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/2442202485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e060 msgr2=0x55f02379e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:20.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.516+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/2442202485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e060 0x55f02379e470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0963e0 tx=0x7f1e2c099910 comp rx=0 tx=0).stop 2023-07-21T22:32:20.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.516+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/2442202485 shutdown_connections 2023-07-21T22:32:20.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.516+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/2442202485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f02379d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.516+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/2442202485 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e060 0x55f02379e470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.516+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/2442202485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e9f0 0x55f02379cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.516+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/2442202485 >> 172.21.15.107:0/2442202485 conn(0x55f023789ef0 msgr2=0x55f02378f360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:20.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.516+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/2442202485 shutdown_connections 2023-07-21T22:32:20.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.516+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/2442202485 wait complete. 2023-07-21T22:32:20.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.516+0000 7f1e4cb6fb80 1 -- start start 2023-07-21T22:32:20.571 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.517+0000 7f1e4cb6fb80 1 Processor -- start 2023-07-21T22:32:20.571 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.517+0000 7f1e4cb6fb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.571 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.517+0000 7f1e4cb6fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 0x55f0237a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.571 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.517+0000 7f1e4cb6fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.571 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.517+0000 7f1e4cb6fb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f023685ac0 con 0x55f02379d190 2023-07-21T22:32:20.571 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.517+0000 7f1e4cb6fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0236857c0 con 0x55f02379e060 2023-07-21T22:32:20.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.517+0000 7f1e4cb6fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f023569920 con 0x55f02379e9f0 2023-07-21T22:32:20.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.517+0000 7f1e4cb6fb80 10 client.0 _ll_register_callbacks cb 0x55f023683760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:20.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.517+0000 7f1e3a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.517+0000 7f1e3a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237a68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50026/0 (socket says 172.21.15.107:50026) 2023-07-21T22:32:20.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.517+0000 7f1e3ad9d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.517+0000 7f1e3a59c700 1 -- 172.21.15.107:0/498811626 learned_addr learned my addr 172.21.15.107:0/498811626 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:20.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.517+0000 7f1e39d9b700 1 --2- 172.21.15.107:0/498811626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 0x55f0237a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e3ad9d700 1 -- 172.21.15.107:0/498811626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 msgr2=0x55f0237a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e3ad9d700 1 --2- 172.21.15.107:0/498811626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 0x55f0237a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e3ad9d700 1 -- 172.21.15.107:0/498811626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 msgr2=0x55f0237a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e3ad9d700 1 --2- 172.21.15.107:0/498811626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e3ad9d700 1 -- 172.21.15.107:0/498811626 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3c19bc80 con 0x55f02379e9f0 2023-07-21T22:32:20.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/498811626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:20.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e3ad9d700 1 --2- 172.21.15.107:0/498811626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237ade70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400edb0 tx=0x7f1e340086d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a354c73c18780012 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:20.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:32:20.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e21ffb700 1 -- 172.21.15.107:0/498811626 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3401d070 con 0x55f02379e9f0 2023-07-21T22:32:20.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/498811626 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f02363f4f0 con 0x55f02379e9f0 2023-07-21T22:32:20.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e21ffb700 1 -- 172.21.15.107:0/498811626 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1e34004bb0 con 0x55f02379e9f0 2023-07-21T22:32:20.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e21ffb700 1 -- 172.21.15.107:0/498811626 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e34014070 con 0x55f02379e9f0 2023-07-21T22:32:20.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e4cb6fb80 10 client.25759 fetch_fsmap learned FSMap version 34 2023-07-21T22:32:20.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.518+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/498811626 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f0234fcc60 con 0x55f02379e9f0 2023-07-21T22:32:20.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.519+0000 7f1e21ffb700 1 -- 172.21.15.107:0/498811626 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f02363f4f0 con 0x55f02379e9f0 2023-07-21T22:32:20.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.519+0000 7f1e21ffb700 1 -- 172.21.15.107:0/498811626 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1e34020020 con 0x55f02379e9f0 2023-07-21T22:32:20.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.519+0000 7f1e4cb6fb80 10 client.25759 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:32:20.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.519+0000 7f1e4cb6fb80 10 client.25759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:20.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.519+0000 7f1e4cb6fb80 20 client.25759 populate_metadata read hostname 'smithi107' 2023-07-21T22:32:20.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.519+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/498811626 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f0237b26a0 0x55f0237b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.519+0000 7f1e4cb6fb80 4 client.25759 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:20.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.519+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/498811626 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0236deeb0 con 0x55f0237b26a0 2023-07-21T22:32:20.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.519+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/498811626 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f0237b26a0 0x55f0237b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.520+0000 7f1e21ffb700 10 client.25759 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:20.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.520+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/498811626 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f0237b26a0 0x55f0237b4ab0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=577e54d96ff133b7 server_cookie=49d9e93bdb5871b2 in_seq=0 out_seq=0 2023-07-21T22:32:20.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.520+0000 7f1e21ffb700 1 -- 172.21.15.107:0/498811626 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f0236deeb0 con 0x55f0237b26a0 2023-07-21T22:32:20.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.520+0000 7f1e21ffb700 10 client.25759 handle_command_reply: tid=0 2023-07-21T22:32:20.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.520+0000 7f1e4cb6fb80 1 client.25759 shutdown 2023-07-21T22:32:20.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.520+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/498811626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 msgr2=0x55f0237ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:20.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.520+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/498811626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237ade70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400edb0 tx=0x7f1e340086d0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.521+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/498811626 shutdown_connections 2023-07-21T22:32:20.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.521+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/498811626 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f0237b26a0 0x55f0237b4ab0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.521+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/498811626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237ade70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.521+0000 7f1e21ffb700 0 client.25759 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:20.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.521+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/498811626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.521+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/498811626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 0x55f0237a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.521+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/498811626 >> 172.21.15.107:0/498811626 conn(0x55f023789ef0 msgr2=0x55f02378ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:20.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.521+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/498811626 shutdown_connections 2023-07-21T22:32:20.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.521+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/498811626 wait complete. 2023-07-21T22:32:20.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.521+0000 7f1e4cb6fb80 20 client.25759 trim_cache size 0 max 16384 2023-07-21T22:32:20.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.548+0000 7f1e4cb6fb80 1 Processor -- start 2023-07-21T22:32:20.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e4cb6fb80 1 -- start start 2023-07-21T22:32:20.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e4cb6fb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e4cb6fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 0x55f0237ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e4cb6fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e4cb6fb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0237a4600 con 0x55f02379d190 2023-07-21T22:32:20.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e4cb6fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f02378e5e0 con 0x55f02379e060 2023-07-21T22:32:20.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e4cb6fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f023790bd0 con 0x55f02379e9f0 2023-07-21T22:32:20.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e3a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e3a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50034/0 (socket says 172.21.15.107:50034) 2023-07-21T22:32:20.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e3a59c700 1 -- 172.21.15.107:0/1192904040 learned_addr learned my addr 172.21.15.107:0/1192904040 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:20.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e39d9b700 1 --2- 172.21.15.107:0/1192904040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 0x55f0237ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e3ad9d700 1 --2- 172.21.15.107:0/1192904040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e3a59c700 1 -- 172.21.15.107:0/1192904040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 msgr2=0x55f0237a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/1192904040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e3a59c700 1 -- 172.21.15.107:0/1192904040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 msgr2=0x55f0237ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/1192904040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 0x55f0237ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e3a59c700 1 -- 172.21.15.107:0/1192904040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f02353a250 con 0x55f02379d190 2023-07-21T22:32:20.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.549+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/1192904040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237a4810 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c096950 tx=0x7f1e2c010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813883b345bcfad5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:20.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.550+0000 7f1e297fa700 1 -- 172.21.15.107:0/1192904040 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0b8310 con 0x55f02379d190 2023-07-21T22:32:20.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.550+0000 7f1e297fa700 1 -- 172.21.15.107:0/1192904040 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e2c0993d0 con 0x55f02379d190 2023-07-21T22:32:20.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.550+0000 7f1e297fa700 1 -- 172.21.15.107:0/1192904040 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0a2100 con 0x55f02379d190 2023-07-21T22:32:20.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.569+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/1192904040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 msgr2=0x55f0237a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:20.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.569+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/1192904040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237a4810 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c096950 tx=0x7f1e2c010c80 comp rx=0 tx=0).stop 2023-07-21T22:32:20.894 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.841+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/1192904040 shutdown_connections 2023-07-21T22:32:20.894 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.841+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/1192904040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.894 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.841+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/1192904040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237a4810 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.841+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/1192904040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 0x55f0237ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.841+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/1192904040 >> 172.21.15.107:0/1192904040 conn(0x55f023789ef0 msgr2=0x55f0237a55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:20.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.856+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/1192904040 shutdown_connections 2023-07-21T22:32:20.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.856+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/1192904040 wait complete. 2023-07-21T22:32:20.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.856+0000 7f1e4cb6fb80 1 -- start start 2023-07-21T22:32:20.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.857+0000 7f1e4cb6fb80 1 Processor -- start 2023-07-21T22:32:20.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.857+0000 7f1e4cb6fb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237ad610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.857+0000 7f1e4cb6fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 0x55f0237adb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.857+0000 7f1e4cb6fb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237aa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.857+0000 7f1e4cb6fb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0237aa5f0 con 0x55f02379d190 2023-07-21T22:32:20.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.857+0000 7f1e4cb6fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0237aa760 con 0x55f02379e060 2023-07-21T22:32:20.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.857+0000 7f1e4cb6fb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0237aa920 con 0x55f02379e9f0 2023-07-21T22:32:20.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.857+0000 7f1e4cb6fb80 10 client.0 _ll_register_callbacks cb 0x7f1e340093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:20.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.858+0000 7f1e39d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 0x55f0237adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.858+0000 7f1e39d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 0x55f0237adb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40870/0 (socket says 172.21.15.107:40870) 2023-07-21T22:32:20.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.858+0000 7f1e39d9b700 1 -- 172.21.15.107:0/1961688302 learned_addr learned my addr 172.21.15.107:0/1961688302 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:20.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.858+0000 7f1e3ad9d700 1 --2- 172.21.15.107:0/1961688302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237aa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.858+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/1961688302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.861+0000 7f1e3ad9d700 1 -- 172.21.15.107:0/1961688302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 msgr2=0x55f0237adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.861+0000 7f1e3ad9d700 1 --2- 172.21.15.107:0/1961688302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 0x55f0237adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.861+0000 7f1e3ad9d700 1 -- 172.21.15.107:0/1961688302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 msgr2=0x55f0237ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:20.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.861+0000 7f1e3ad9d700 1 --2- 172.21.15.107:0/1961688302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237ad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.861+0000 7f1e3ad9d700 1 -- 172.21.15.107:0/1961688302 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f023513de0 con 0x55f02379e9f0 2023-07-21T22:32:20.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.865+0000 7f1e3ad9d700 1 --2- 172.21.15.107:0/1961688302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237aa0b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e34004690 tx=0x7f1e34004ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eec79788a8406e05 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:20.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.875+0000 7f1e21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:32:20.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.875+0000 7f1e21ffb700 1 -- 172.21.15.107:0/1961688302 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3401d040 con 0x55f02379e9f0 2023-07-21T22:32:20.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.875+0000 7f1e21ffb700 1 -- 172.21.15.107:0/1961688302 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1e340224d0 con 0x55f02379e9f0 2023-07-21T22:32:20.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.876+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/1961688302 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e2c098280 con 0x55f02379e9f0 2023-07-21T22:32:20.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.879+0000 7f1e21ffb700 1 -- 172.21.15.107:0/1961688302 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e34014040 con 0x55f02379e9f0 2023-07-21T22:32:20.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.880+0000 7f1e4cb6fb80 10 client.25765 fetch_fsmap learned FSMap version 34 2023-07-21T22:32:20.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.880+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/1961688302 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f0235122b0 con 0x55f02379e9f0 2023-07-21T22:32:20.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.885+0000 7f1e21ffb700 1 -- 172.21.15.107:0/1961688302 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e34038030 con 0x55f02379e9f0 2023-07-21T22:32:20.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.885+0000 7f1e21ffb700 1 -- 172.21.15.107:0/1961688302 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1e34032060 con 0x55f02379e9f0 2023-07-21T22:32:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.886+0000 7f1e4cb6fb80 10 client.25765 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:32:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.886+0000 7f1e4cb6fb80 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.886+0000 7f1e4cb6fb80 20 client.25765 populate_metadata read hostname 'smithi107' 2023-07-21T22:32:20.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.886+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/1961688302 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f023818980 0x55f02381ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:20.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.886+0000 7f1e4cb6fb80 4 client.25765 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:32:20.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.886+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/1961688302 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f0236a1050 con 0x55f023818980 2023-07-21T22:32:20.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.887+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/1961688302 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f023818980 0x55f02381ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:20.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.905+0000 7f1e3a59c700 1 --2- 172.21.15.107:0/1961688302 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f023818980 0x55f02381ad90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7f93c5b17fc88a server_cookie=358bc5c10099a703 in_seq=0 out_seq=0 2023-07-21T22:32:20.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.905+0000 7f1e21ffb700 10 client.25765 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:20.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.905+0000 7f1e21ffb700 1 -- 172.21.15.107:0/1961688302 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f0236a1050 con 0x55f023818980 2023-07-21T22:32:20.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.905+0000 7f1e21ffb700 10 client.25765 handle_command_reply: tid=0 2023-07-21T22:32:20.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.905+0000 7f1e4cb6fb80 1 client.25765 shutdown 2023-07-21T22:32:20.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.944+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/1961688302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 msgr2=0x55f0237aa0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:20.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.945+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/1961688302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237aa0b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e34004690 tx=0x7f1e34004ec0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.945+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/1961688302 shutdown_connections 2023-07-21T22:32:20.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.945+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/1961688302 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55f023818980 0x55f02381ad90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.945+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/1961688302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f02379e9f0 0x55f0237aa0b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.945+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/1961688302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f02379d190 0x55f0237ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.945+0000 7f1e4cb6fb80 1 --2- 172.21.15.107:0/1961688302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f02379e060 0x55f0237adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:20.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.945+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/1961688302 >> 172.21.15.107:0/1961688302 conn(0x55f023789ef0 msgr2=0x55f0237a55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:20.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.945+0000 7f1e21ffb700 0 client.25765 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:20.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.945+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/1961688302 shutdown_connections 2023-07-21T22:32:20.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.945+0000 7f1e4cb6fb80 1 -- 172.21.15.107:0/1961688302 wait complete. 2023-07-21T22:32:20.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.945+0000 7f1e4cb6fb80 20 client.25765 trim_cache size 0 max 16384 2023-07-21T22:32:20.994 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:32:20.994 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:32:20.994 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:32:20.995 INFO:teuthology.orchestra.run.smithi107.stdout: "d037412e-e4e0-4d2c-8d6e-c7193371fe0f": { 2023-07-21T22:32:20.995 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:32:20.995 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d037412e-e4e0-4d2c-8d6e-c7193371fe0f", 2023-07-21T22:32:20.995 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:32:20.995 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:32:20.995 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:32:20.995 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:32:21.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.974+0000 7f1e409ea700 1 -- 172.21.15.107:0/2775688516 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1e240a1150 msgr2=0x7f1e240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:21.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.974+0000 7f1e409ea700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1e240a1150 0x7f1e240a3600 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000b2e0 tx=0x7f1e30004600 comp rx=0 tx=0).stop 2023-07-21T22:32:21.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.974+0000 7f1e409ea700 1 -- 172.21.15.107:0/2775688516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e3c0fec00 msgr2=0x7f1e3c1935f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:21.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.974+0000 7f1e409ea700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e3c0fec00 0x7f1e3c1935f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00e9a0 tx=0x7f1e2c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.975+0000 7f1e409ea700 1 -- 172.21.15.107:0/2775688516 shutdown_connections 2023-07-21T22:32:21.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.975+0000 7f1e409ea700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f1e240a1150 0x7f1e240a3600 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.975+0000 7f1e409ea700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e3c0fffb0 0x7f1e3c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.975+0000 7f1e409ea700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e3c0fec00 0x7f1e3c1935f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.975+0000 7f1e409ea700 1 --2- 172.21.15.107:0/2775688516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c0ff5b0 0x7f1e3c193b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:21.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.975+0000 7f1e409ea700 1 -- 172.21.15.107:0/2775688516 >> 172.21.15.107:0/2775688516 conn(0x7f1e3c0f97a0 msgr2=0x7f1e3c0fb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:21.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.975+0000 7f1e409ea700 1 -- 172.21.15.107:0/2775688516 shutdown_connections 2023-07-21T22:32:21.024 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:20.975+0000 7f1e409ea700 1 -- 172.21.15.107:0/2775688516 wait complete. 2023-07-21T22:32:21.133 INFO:tasks.cephfs.filesystem:scrub status for tag:d037412e-e4e0-4d2c-8d6e-c7193371fe0f - {'path': '/', 'tag': 'd037412e-e4e0-4d2c-8d6e-c7193371fe0f', 'options': 'recursive,force'} 2023-07-21T22:32:21.419 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:21 smithi107 ceph-mon[96543]: pgmap v392: 97 pgs: 97 active+clean; 562 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 266 op/s 2023-07-21T22:32:21.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:21 smithi132 ceph-mon[97863]: pgmap v392: 97 pgs: 97 active+clean; 562 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 266 op/s 2023-07-21T22:32:21.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:21 smithi190 ceph-mon[98334]: pgmap v392: 97 pgs: 97 active+clean; 562 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 266 op/s 2023-07-21T22:32:22.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:22 smithi190 ceph-mon[98334]: pgmap v393: 97 pgs: 97 active+clean; 567 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 414 op/s 2023-07-21T22:32:22.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:22 smithi132 ceph-mon[97863]: pgmap v393: 97 pgs: 97 active+clean; 567 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 414 op/s 2023-07-21T22:32:22.949 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:22 smithi107 ceph-mon[96543]: pgmap v393: 97 pgs: 97 active+clean; 567 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 414 op/s 2023-07-21T22:32:25.395 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:25 smithi190 ceph-mon[98334]: pgmap v394: 97 pgs: 97 active+clean; 564 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 357 op/s 2023-07-21T22:32:25.434 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:25 smithi132 ceph-mon[97863]: pgmap v394: 97 pgs: 97 active+clean; 564 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 357 op/s 2023-07-21T22:32:25.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:24 smithi107 ceph-mon[96543]: pgmap v394: 97 pgs: 97 active+clean; 564 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 357 op/s 2023-07-21T22:32:26.452 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:26 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:26.495 INFO:tasks.workunit.client.0.smithi107.stdout: 170 2307075 729 1596188 641 809692 794 2023-07-21T22:32:26.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:26 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:26.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:26 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:27.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:27 smithi190 ceph-mon[98334]: pgmap v395: 97 pgs: 97 active+clean; 594 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 574 op/s 2023-07-21T22:32:27.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:27 smithi132 ceph-mon[97863]: pgmap v395: 97 pgs: 97 active+clean; 594 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 574 op/s 2023-07-21T22:32:27.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:27 smithi107 ceph-mon[96543]: pgmap v395: 97 pgs: 97 active+clean; 594 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.4 MiB/s wr, 574 op/s 2023-07-21T22:32:29.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:29 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:32:29.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:29 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:29 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:32:29.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:29 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:29 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:32:29.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:29 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:29 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:32:29.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:29 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.642 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:29 smithi190 ceph-mon[98334]: pgmap v396: 97 pgs: 97 active+clean; 685 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-07-21T22:32:29.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:29 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:32:29.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:29 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:29 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:32:29.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:29 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:29 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:32:29.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:29 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:29 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:32:29.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:29 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:29 smithi132 ceph-mon[97863]: pgmap v396: 97 pgs: 97 active+clean; 685 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-07-21T22:32:29.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:29 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:32:29.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:29 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:29 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:32:29.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:29 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:29 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:32:29.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:29 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:29 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:32:29.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:29 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:29 smithi107 ceph-mon[96543]: pgmap v396: 97 pgs: 97 active+clean; 685 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-07-21T22:32:31.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:31 smithi190 ceph-mon[98334]: pgmap v397: 97 pgs: 97 active+clean; 687 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 985 op/s 2023-07-21T22:32:31.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:31 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:31.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:31 smithi132 ceph-mon[97863]: pgmap v397: 97 pgs: 97 active+clean; 687 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 985 op/s 2023-07-21T22:32:31.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:31 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:31.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:31 smithi107 ceph-mon[96543]: pgmap v397: 97 pgs: 97 active+clean; 687 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 985 op/s 2023-07-21T22:32:31.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:31 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:32:32.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:32 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:32:32.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:32 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:32.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:32 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:32:32.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:32 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:32.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:32 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:32:32.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:32 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.297+0000 7fa97a203700 1 -- 172.21.15.107:0/4065283487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9740ff4e0 msgr2=0x7fa974103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.297+0000 7fa97a203700 1 --2- 172.21.15.107:0/4065283487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9740ff4e0 0x7fa974103d90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa96c01c0d0 tx=0x7fa96c01c3e0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.297+0000 7fa97a203700 1 -- 172.21.15.107:0/4065283487 shutdown_connections 2023-07-21T22:32:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.297+0000 7fa97a203700 1 --2- 172.21.15.107:0/4065283487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9740ff4e0 0x7fa974103d90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.297+0000 7fa97a203700 1 --2- 172.21.15.107:0/4065283487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9740fddd0 0x7fa9740fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.297+0000 7fa97a203700 1 --2- 172.21.15.107:0/4065283487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9740feb30 0x7fa9740fefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.297+0000 7fa97a203700 1 -- 172.21.15.107:0/4065283487 >> 172.21.15.107:0/4065283487 conn(0x7fa9740f97b0 msgr2=0x7fa9740fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.545+0000 7fa97a203700 1 -- 172.21.15.107:0/4065283487 shutdown_connections 2023-07-21T22:32:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.552+0000 7fa97a203700 1 -- 172.21.15.107:0/4065283487 wait complete. 2023-07-21T22:32:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.553+0000 7fa97a203700 1 Processor -- start 2023-07-21T22:32:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.558+0000 7fa97a203700 1 -- start start 2023-07-21T22:32:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.558+0000 7fa97a203700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9740fddd0 0x7fa97410ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.558+0000 7fa97a203700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9740ff4e0 0x7fa97410f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.558+0000 7fa97a203700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9741b6c90 0x7fa9741b7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.558+0000 7fa97a203700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9741b7730 con 0x7fa9740fddd0 2023-07-21T22:32:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.558+0000 7fa97a203700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9741b78a0 con 0x7fa9740ff4e0 2023-07-21T22:32:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.558+0000 7fa97a203700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9741b7a60 con 0x7fa9741b6c90 2023-07-21T22:32:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.559+0000 7fa972ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9740ff4e0 0x7fa97410f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.559+0000 7fa972ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9740ff4e0 0x7fa97410f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40880/0 (socket says 172.21.15.107:40880) 2023-07-21T22:32:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.559+0000 7fa972ffd700 1 -- 172.21.15.107:0/3296883899 learned_addr learned my addr 172.21.15.107:0/3296883899 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.559+0000 7fa973fff700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9741b6c90 0x7fa9741b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.560+0000 7fa9737fe700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9740fddd0 0x7fa97410ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.560+0000 7fa973fff700 1 -- 172.21.15.107:0/3296883899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9740ff4e0 msgr2=0x7fa97410f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.560+0000 7fa973fff700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9740ff4e0 0x7fa97410f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.561+0000 7fa973fff700 1 -- 172.21.15.107:0/3296883899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9740fddd0 msgr2=0x7fa97410ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.561+0000 7fa973fff700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9740fddd0 0x7fa97410ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.561+0000 7fa973fff700 1 -- 172.21.15.107:0/3296883899 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c01ad10 con 0x7fa9741b6c90 2023-07-21T22:32:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.564+0000 7fa973fff700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9741b6c90 0x7fa9741b7130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa96c01c0d0 tx=0x7fa96c003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:33.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:33 smithi190 ceph-mon[98334]: pgmap v398: 97 pgs: 97 active+clean; 715 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-07-21T22:32:33.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:33 smithi132 ceph-mon[97863]: pgmap v398: 97 pgs: 97 active+clean; 715 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-07-21T22:32:33.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:33 smithi107 ceph-mon[96543]: pgmap v398: 97 pgs: 97 active+clean; 715 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-07-21T22:32:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.708+0000 7fa970ff9700 1 -- 172.21.15.107:0/3296883899 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c01f050 con 0x7fa9741b6c90 2023-07-21T22:32:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.708+0000 7fa970ff9700 1 -- 172.21.15.107:0/3296883899 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa96c01de60 con 0x7fa9741b6c90 2023-07-21T22:32:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.708+0000 7fa970ff9700 1 -- 172.21.15.107:0/3296883899 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c007a00 con 0x7fa9741b6c90 2023-07-21T22:32:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.714+0000 7fa97a203700 1 -- 172.21.15.107:0/3296883899 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa974069f30 con 0x7fa9741b6c90 2023-07-21T22:32:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.714+0000 7fa97a203700 1 -- 172.21.15.107:0/3296883899 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97406a470 con 0x7fa9741b6c90 2023-07-21T22:32:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.724+0000 7fa986388b80 1 Processor -- start 2023-07-21T22:32:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.725+0000 7fa986388b80 1 -- start start 2023-07-21T22:32:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.725+0000 7fa986388b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9c226b0 0x5610a9dc62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.725+0000 7fa986388b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 0x5610a9dc7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.725+0000 7fa986388b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dc7580 0x5610a9dc79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.725+0000 7fa986388b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610a9ccbc50 con 0x5610a9c226b0 2023-07-21T22:32:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.725+0000 7fa986388b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610a9d1e9e0 con 0x5610a9dc7580 2023-07-21T22:32:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.725+0000 7fa986388b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610a9d17e70 con 0x5610a9dc6830 2023-07-21T22:32:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.725+0000 7fa9737fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 0x5610a9dc7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.725+0000 7fa9737fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 0x5610a9dc7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38620/0 (socket says 172.21.15.107:38620) 2023-07-21T22:32:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.725+0000 7fa9737fe700 1 -- 172.21.15.107:0/3999584397 learned_addr learned my addr 172.21.15.107:0/3999584397 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.725+0000 7fa972ffd700 1 --2- 172.21.15.107:0/3999584397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dc7580 0x5610a9dc79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.730+0000 7fa9737fe700 1 -- 172.21.15.107:0/3999584397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dc7580 msgr2=0x5610a9dc79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.730+0000 7fa9737fe700 1 --2- 172.21.15.107:0/3999584397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dc7580 0x5610a9dc79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.730+0000 7fa9737fe700 1 -- 172.21.15.107:0/3999584397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9c226b0 msgr2=0x5610a9dc62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.730+0000 7fa9737fe700 1 --2- 172.21.15.107:0/3999584397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9c226b0 0x5610a9dc62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.730+0000 7fa9737fe700 1 -- 172.21.15.107:0/3999584397 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610a9b45d80 con 0x5610a9dc6830 2023-07-21T22:32:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.735+0000 7fa9737fe700 1 --2- 172.21.15.107:0/3999584397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 0x5610a9dc7040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa964009d20 tx=0x7fa964011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63475c5436e15a5c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.740+0000 7fa959ffb700 1 -- 172.21.15.107:0/3999584397 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964004a30 con 0x5610a9dc6830 2023-07-21T22:32:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.740+0000 7fa959ffb700 1 -- 172.21.15.107:0/3999584397 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9640054e0 con 0x5610a9dc6830 2023-07-21T22:32:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.740+0000 7fa959ffb700 1 -- 172.21.15.107:0/3999584397 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964019830 con 0x5610a9dc6830 2023-07-21T22:32:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.741+0000 7fa986388b80 1 -- 172.21.15.107:0/3999584397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 msgr2=0x5610a9dc7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.741+0000 7fa986388b80 1 --2- 172.21.15.107:0/3999584397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 0x5610a9dc7040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa964009d20 tx=0x7fa964011720 comp rx=0 tx=0).stop 2023-07-21T22:32:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.742+0000 7fa986388b80 1 -- 172.21.15.107:0/3999584397 shutdown_connections 2023-07-21T22:32:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.742+0000 7fa986388b80 1 --2- 172.21.15.107:0/3999584397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dc7580 0x5610a9dc79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.742+0000 7fa986388b80 1 --2- 172.21.15.107:0/3999584397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9c226b0 0x5610a9dc62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.742+0000 7fa986388b80 1 --2- 172.21.15.107:0/3999584397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 0x5610a9dc7040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.742+0000 7fa986388b80 1 -- 172.21.15.107:0/3999584397 >> 172.21.15.107:0/3999584397 conn(0x5610a9dbbae0 msgr2=0x5610a9dc0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.742+0000 7fa986388b80 1 -- 172.21.15.107:0/3999584397 shutdown_connections 2023-07-21T22:32:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.742+0000 7fa986388b80 1 -- 172.21.15.107:0/3999584397 wait complete. 2023-07-21T22:32:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.742+0000 7fa970ff9700 1 -- 172.21.15.107:0/3296883899 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa96c0052c0 con 0x7fa9741b6c90 2023-07-21T22:32:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.773+0000 7fa986388b80 1 -- start start 2023-07-21T22:32:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.773+0000 7fa986388b80 1 Processor -- start 2023-07-21T22:32:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.773+0000 7fa986388b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9c226b0 0x5610a9dcf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.773+0000 7fa986388b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dc6830 0x5610a9dcfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.773+0000 7fa986388b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc7580 0x5610a9dd6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.773+0000 7fa986388b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610a9cb7a60 con 0x5610a9c226b0 2023-07-21T22:32:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.773+0000 7fa986388b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610a9cb7760 con 0x5610a9dc6830 2023-07-21T22:32:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.773+0000 7fa986388b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610a9b9b8c0 con 0x5610a9dc7580 2023-07-21T22:32:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.774+0000 7fa986388b80 10 client.0 _ll_register_callbacks cb 0x5610a9cb5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.774+0000 7fa972ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc7580 0x5610a9dd6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.774+0000 7fa972ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc7580 0x5610a9dd6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38624/0 (socket says 172.21.15.107:38624) 2023-07-21T22:32:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.774+0000 7fa972ffd700 1 -- 172.21.15.107:0/4121002661 learned_addr learned my addr 172.21.15.107:0/4121002661 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.774+0000 7fa9737fe700 1 --2- 172.21.15.107:0/4121002661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dc6830 0x5610a9dcfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.775+0000 7fa972ffd700 1 -- 172.21.15.107:0/4121002661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dc6830 msgr2=0x5610a9dcfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.775+0000 7fa972ffd700 1 --2- 172.21.15.107:0/4121002661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dc6830 0x5610a9dcfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.775+0000 7fa972ffd700 1 -- 172.21.15.107:0/4121002661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9c226b0 msgr2=0x5610a9dcf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.775+0000 7fa972ffd700 1 --2- 172.21.15.107:0/4121002661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9c226b0 0x5610a9dcf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.775+0000 7fa972ffd700 1 -- 172.21.15.107:0/4121002661 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa964009980 con 0x5610a9dc7580 2023-07-21T22:32:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.775+0000 7fa972ffd700 1 --2- 172.21.15.107:0/4121002661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc7580 0x5610a9dd6de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa968009e80 tx=0x7fa968013840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2da7e29e1ba68bb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.793+0000 7fa970ff9700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa95c0a1230 0x7fa95c0a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.793+0000 7fa970ff9700 1 -- 172.21.15.107:0/3296883899 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa96c0cd1c0 con 0x7fa9741b6c90 2023-07-21T22:32:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.794+0000 7fa9737fe700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa95c0a1230 0x7fa95c0a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.832+0000 7fa9737fe700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa95c0a1230 0x7fa95c0a36e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa964005920 tx=0x7fa964005fa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.839+0000 7fa9577f6700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:32:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.839+0000 7fa9577f6700 1 -- 172.21.15.107:0/4121002661 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968038470 con 0x5610a9dc7580 2023-07-21T22:32:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.839+0000 7fa9577f6700 1 -- 172.21.15.107:0/4121002661 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa9680385d0 con 0x5610a9dc7580 2023-07-21T22:32:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.840+0000 7fa986388b80 1 -- 172.21.15.107:0/4121002661 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610a9c71490 con 0x5610a9dc7580 2023-07-21T22:32:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.847+0000 7fa9577f6700 1 -- 172.21.15.107:0/4121002661 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa968038790 con 0x5610a9dc7580 2023-07-21T22:32:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.847+0000 7fa986388b80 10 client.25774 fetch_fsmap learned FSMap version 36 2023-07-21T22:32:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.847+0000 7fa986388b80 1 -- 172.21.15.107:0/4121002661 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5610a9b2ec00 con 0x5610a9dc7580 2023-07-21T22:32:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.876+0000 7fa9577f6700 1 -- 172.21.15.107:0/4121002661 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5610a9c71490 con 0x5610a9dc7580 2023-07-21T22:32:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.876+0000 7fa9577f6700 1 -- 172.21.15.107:0/4121002661 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fa9680379f0 con 0x5610a9dc7580 2023-07-21T22:32:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.879+0000 7fa986388b80 10 client.25774 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:32:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.879+0000 7fa986388b80 10 client.25774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.879+0000 7fa986388b80 20 client.25774 populate_metadata read hostname 'smithi107' 2023-07-21T22:32:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.879+0000 7fa986388b80 1 --2- 172.21.15.107:0/4121002661 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5610a9ddb670 0x5610a9ddda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.880+0000 7fa986388b80 4 client.25774 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.880+0000 7fa986388b80 1 -- 172.21.15.107:0/4121002661 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610a9d10e50 con 0x5610a9ddb670 2023-07-21T22:32:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.884+0000 7fa972ffd700 1 --2- 172.21.15.107:0/4121002661 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5610a9ddb670 0x5610a9ddda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.908+0000 7fa972ffd700 1 --2- 172.21.15.107:0/4121002661 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5610a9ddb670 0x5610a9ddda80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=369f6dc05f3ff679 server_cookie=127ca3e69fe6a8be in_seq=0 out_seq=0 2023-07-21T22:32:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.911+0000 7fa9577f6700 10 client.25774 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.912+0000 7fa9577f6700 1 -- 172.21.15.107:0/4121002661 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5610a9d10e50 con 0x5610a9ddb670 2023-07-21T22:32:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.912+0000 7fa9577f6700 10 client.25774 handle_command_reply: tid=0 2023-07-21T22:32:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.912+0000 7fa986388b80 1 client.25774 shutdown 2023-07-21T22:32:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.912+0000 7fa986388b80 1 -- 172.21.15.107:0/4121002661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc7580 msgr2=0x5610a9dd6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.912+0000 7fa986388b80 1 --2- 172.21.15.107:0/4121002661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc7580 0x5610a9dd6de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa968009e80 tx=0x7fa968013840 comp rx=0 tx=0).stop 2023-07-21T22:32:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.912+0000 7fa986388b80 1 -- 172.21.15.107:0/4121002661 shutdown_connections 2023-07-21T22:32:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.912+0000 7fa986388b80 1 --2- 172.21.15.107:0/4121002661 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5610a9ddb670 0x5610a9ddda80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.912+0000 7fa986388b80 1 --2- 172.21.15.107:0/4121002661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc7580 0x5610a9dd6de0 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa968009e80 tx=0x7fa968013840 comp rx=0 tx=0).stop 2023-07-21T22:32:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.912+0000 7fa986388b80 1 --2- 172.21.15.107:0/4121002661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9c226b0 0x5610a9dcf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.912+0000 7fa986388b80 1 --2- 172.21.15.107:0/4121002661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dc6830 0x5610a9dcfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.912+0000 7fa986388b80 1 -- 172.21.15.107:0/4121002661 >> 172.21.15.107:0/4121002661 conn(0x5610a9dbbae0 msgr2=0x5610a9dc03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.913+0000 7fa9577f6700 0 client.25774 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.913+0000 7fa986388b80 1 -- 172.21.15.107:0/4121002661 shutdown_connections 2023-07-21T22:32:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.913+0000 7fa986388b80 1 -- 172.21.15.107:0/4121002661 wait complete. 2023-07-21T22:32:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.913+0000 7fa986388b80 20 client.25774 trim_cache size 0 max 16384 2023-07-21T22:32:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.996+0000 7fa986388b80 1 Processor -- start 2023-07-21T22:32:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.998+0000 7fa986388b80 1 -- start start 2023-07-21T22:32:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.999+0000 7fa986388b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9c226b0 0x5610a9dc62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.999+0000 7fa986388b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 0x5610a9dd6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.999+0000 7fa986388b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dca430 0x5610a9dca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.999+0000 7fa986388b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610a9db8a40 con 0x5610a9c226b0 2023-07-21T22:32:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.999+0000 7fa986388b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610a9dcadf0 con 0x5610a9dca430 2023-07-21T22:32:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.999+0000 7fa986388b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610a9c71490 con 0x5610a9dc6830 2023-07-21T22:32:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.999+0000 7fa972ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 0x5610a9dd6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.999+0000 7fa972ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 0x5610a9dd6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38634/0 (socket says 172.21.15.107:38634) 2023-07-21T22:32:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.999+0000 7fa972ffd700 1 -- 172.21.15.107:0/734029541 learned_addr learned my addr 172.21.15.107:0/734029541 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:33.999+0000 7fa973fff700 1 --2- 172.21.15.107:0/734029541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dca430 0x5610a9dca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.010+0000 7fa9737fe700 1 --2- 172.21.15.107:0/734029541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9c226b0 0x5610a9dc62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.019+0000 7fa972ffd700 1 -- 172.21.15.107:0/734029541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dca430 msgr2=0x5610a9dca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.019+0000 7fa972ffd700 1 --2- 172.21.15.107:0/734029541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dca430 0x5610a9dca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.019+0000 7fa972ffd700 1 -- 172.21.15.107:0/734029541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9c226b0 msgr2=0x5610a9dc62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.019+0000 7fa972ffd700 1 --2- 172.21.15.107:0/734029541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9c226b0 0x5610a9dc62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.019+0000 7fa972ffd700 1 -- 172.21.15.107:0/734029541 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610a9b6c1f0 con 0x5610a9dc6830 2023-07-21T22:32:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.037+0000 7fa972ffd700 1 --2- 172.21.15.107:0/734029541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 0x5610a9dd6d00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa968037d80 tx=0x7fa9680376d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a12f3f5345127984 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.037+0000 7fa95a7fc700 1 -- 172.21.15.107:0/734029541 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968009280 con 0x5610a9dc6830 2023-07-21T22:32:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.037+0000 7fa95a7fc700 1 -- 172.21.15.107:0/734029541 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa968037ed0 con 0x5610a9dc6830 2023-07-21T22:32:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.037+0000 7fa95a7fc700 1 -- 172.21.15.107:0/734029541 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968009460 con 0x5610a9dc6830 2023-07-21T22:32:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.038+0000 7fa986388b80 1 -- 172.21.15.107:0/734029541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 msgr2=0x5610a9dd6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.038+0000 7fa986388b80 1 --2- 172.21.15.107:0/734029541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 0x5610a9dd6d00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa968037d80 tx=0x7fa9680376d0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.045+0000 7fa986388b80 1 -- 172.21.15.107:0/734029541 shutdown_connections 2023-07-21T22:32:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.045+0000 7fa986388b80 1 --2- 172.21.15.107:0/734029541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dca430 0x5610a9dca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.045+0000 7fa986388b80 1 --2- 172.21.15.107:0/734029541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9c226b0 0x5610a9dc62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.045+0000 7fa986388b80 1 --2- 172.21.15.107:0/734029541 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9dc6830 0x5610a9dd6d00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.045+0000 7fa986388b80 1 -- 172.21.15.107:0/734029541 >> 172.21.15.107:0/734029541 conn(0x5610a9dbbae0 msgr2=0x5610a9de23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.046+0000 7fa986388b80 1 -- 172.21.15.107:0/734029541 shutdown_connections 2023-07-21T22:32:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.046+0000 7fa986388b80 1 -- 172.21.15.107:0/734029541 wait complete. 2023-07-21T22:32:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.046+0000 7fa986388b80 1 -- start start 2023-07-21T22:32:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.046+0000 7fa986388b80 1 Processor -- start 2023-07-21T22:32:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.047+0000 7fa986388b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9c226b0 0x5610a9dd2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.047+0000 7fa986388b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9dc6830 0x5610a9dd3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.047+0000 7fa986388b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dca430 0x5610a9dd61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.047+0000 7fa986388b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610a9dd66e0 con 0x5610a9dc6830 2023-07-21T22:32:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.047+0000 7fa986388b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610a9dd6850 con 0x5610a9dca430 2023-07-21T22:32:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.047+0000 7fa986388b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610a9dd6a10 con 0x5610a9c226b0 2023-07-21T22:32:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.047+0000 7fa986388b80 10 client.0 _ll_register_callbacks cb 0x7fa9680099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.047+0000 7fa973fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dca430 0x5610a9dd61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.047+0000 7fa973fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dca430 0x5610a9dd61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40908/0 (socket says 172.21.15.107:40908) 2023-07-21T22:32:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.047+0000 7fa973fff700 1 -- 172.21.15.107:0/2158453356 learned_addr learned my addr 172.21.15.107:0/2158453356 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.047+0000 7fa9737fe700 1 --2- 172.21.15.107:0/2158453356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9c226b0 0x5610a9dd2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.048+0000 7fa9737fe700 1 -- 172.21.15.107:0/2158453356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dca430 msgr2=0x5610a9dd61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.048+0000 7fa9737fe700 1 --2- 172.21.15.107:0/2158453356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dca430 0x5610a9dd61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.048+0000 7fa9737fe700 1 -- 172.21.15.107:0/2158453356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9dc6830 msgr2=0x5610a9dd3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.048+0000 7fa9737fe700 1 --2- 172.21.15.107:0/2158453356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9dc6830 0x5610a9dd3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.048+0000 7fa9737fe700 1 -- 172.21.15.107:0/2158453356 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610a9b2ec00 con 0x5610a9c226b0 2023-07-21T22:32:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.048+0000 7fa9737fe700 1 --2- 172.21.15.107:0/2158453356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9c226b0 0x5610a9dd2c40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa964006010 tx=0x7fa964009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6fc36f7d50b4284 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.048+0000 7fa9577f6700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:32:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.048+0000 7fa9577f6700 1 -- 172.21.15.107:0/2158453356 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964003950 con 0x5610a9c226b0 2023-07-21T22:32:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.048+0000 7fa9577f6700 1 -- 172.21.15.107:0/2158453356 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa964003ab0 con 0x5610a9c226b0 2023-07-21T22:32:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.049+0000 7fa986388b80 1 -- 172.21.15.107:0/2158453356 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9640054e0 con 0x5610a9c226b0 2023-07-21T22:32:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.061+0000 7fa9577f6700 1 -- 172.21.15.107:0/2158453356 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa964003c70 con 0x5610a9c226b0 2023-07-21T22:32:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.061+0000 7fa986388b80 10 client.25780 fetch_fsmap learned FSMap version 36 2023-07-21T22:32:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.061+0000 7fa986388b80 1 -- 172.21.15.107:0/2158453356 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5610a9b44250 con 0x5610a9c226b0 2023-07-21T22:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.063+0000 7fa9577f6700 1 -- 172.21.15.107:0/2158453356 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9640054e0 con 0x5610a9c226b0 2023-07-21T22:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.063+0000 7fa9577f6700 1 -- 172.21.15.107:0/2158453356 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fa964019430 con 0x5610a9c226b0 2023-07-21T22:32:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.063+0000 7fa986388b80 10 client.25780 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:32:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.063+0000 7fa986388b80 10 client.25780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.063+0000 7fa986388b80 20 client.25780 populate_metadata read hostname 'smithi107' 2023-07-21T22:32:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.063+0000 7fa986388b80 1 --2- 172.21.15.107:0/2158453356 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5610a9e30590 0x5610a9e32a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.063+0000 7fa986388b80 4 client.25780 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:32:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.063+0000 7fa986388b80 1 -- 172.21.15.107:0/2158453356 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5610a9cd2ff0 con 0x5610a9e30590 2023-07-21T22:32:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.063+0000 7fa972ffd700 1 --2- 172.21.15.107:0/2158453356 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5610a9e30590 0x5610a9e32a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.066+0000 7fa972ffd700 1 --2- 172.21.15.107:0/2158453356 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5610a9e30590 0x5610a9e32a20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c096df705ff63c69 server_cookie=81df24e13a621ccc in_seq=0 out_seq=0 2023-07-21T22:32:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.066+0000 7fa9577f6700 10 client.25780 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.843+0000 7fa9577f6700 1 -- 172.21.15.107:0/2158453356 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5610a9cd2ff0 con 0x5610a9e30590 2023-07-21T22:32:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.843+0000 7fa9577f6700 10 client.25780 handle_command_reply: tid=0 2023-07-21T22:32:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.843+0000 7fa986388b80 1 client.25780 shutdown 2023-07-21T22:32:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.843+0000 7fa986388b80 1 -- 172.21.15.107:0/2158453356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9c226b0 msgr2=0x5610a9dd2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.843+0000 7fa986388b80 1 --2- 172.21.15.107:0/2158453356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9c226b0 0x5610a9dd2c40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa964006010 tx=0x7fa964009260 comp rx=0 tx=0).stop 2023-07-21T22:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.844+0000 7fa986388b80 1 -- 172.21.15.107:0/2158453356 shutdown_connections 2023-07-21T22:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.844+0000 7fa986388b80 1 --2- 172.21.15.107:0/2158453356 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5610a9e30590 0x5610a9e32a20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.844+0000 7fa986388b80 1 --2- 172.21.15.107:0/2158453356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5610a9dca430 0x5610a9dd61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.844+0000 7fa986388b80 1 --2- 172.21.15.107:0/2158453356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5610a9c226b0 0x5610a9dd2c40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.844+0000 7fa986388b80 1 --2- 172.21.15.107:0/2158453356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610a9dc6830 0x5610a9dd3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.844+0000 7fa986388b80 1 -- 172.21.15.107:0/2158453356 >> 172.21.15.107:0/2158453356 conn(0x5610a9dbbae0 msgr2=0x5610a9dcf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.844+0000 7fa9577f6700 0 client.25780 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.844+0000 7fa986388b80 1 -- 172.21.15.107:0/2158453356 shutdown_connections 2023-07-21T22:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.844+0000 7fa986388b80 1 -- 172.21.15.107:0/2158453356 wait complete. 2023-07-21T22:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.844+0000 7fa986388b80 20 client.25780 trim_cache size 0 max 16384 2023-07-21T22:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:32:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.950+0000 7fa97a203700 1 -- 172.21.15.107:0/3296883899 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa95c0a1230 msgr2=0x7fa95c0a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.950+0000 7fa97a203700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa95c0a1230 0x7fa95c0a36e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa964005920 tx=0x7fa964005fa0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.950+0000 7fa97a203700 1 -- 172.21.15.107:0/3296883899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9741b6c90 msgr2=0x7fa9741b7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.950+0000 7fa97a203700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9741b6c90 0x7fa9741b7130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa96c01c0d0 tx=0x7fa96c003c30 comp rx=0 tx=0).stop 2023-07-21T22:32:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.950+0000 7fa97a203700 1 -- 172.21.15.107:0/3296883899 shutdown_connections 2023-07-21T22:32:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.950+0000 7fa97a203700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa95c0a1230 0x7fa95c0a36e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.950+0000 7fa97a203700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa9741b6c90 0x7fa9741b7130 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.950+0000 7fa97a203700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9740fddd0 0x7fa97410ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.950+0000 7fa97a203700 1 --2- 172.21.15.107:0/3296883899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9740ff4e0 0x7fa97410f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.950+0000 7fa97a203700 1 -- 172.21.15.107:0/3296883899 >> 172.21.15.107:0/3296883899 conn(0x7fa9740f97b0 msgr2=0x7fa9740fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.951+0000 7fa97a203700 1 -- 172.21.15.107:0/3296883899 shutdown_connections 2023-07-21T22:32:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:34.952+0000 7fa97a203700 1 -- 172.21.15.107:0/3296883899 wait complete. 2023-07-21T22:32:35.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:35 smithi132 ceph-mon[97863]: pgmap v399: 97 pgs: 97 active+clean; 715 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-07-21T22:32:35.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:35 smithi107 ceph-mon[96543]: pgmap v399: 97 pgs: 97 active+clean; 715 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-07-21T22:32:35.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:35 smithi190 ceph-mon[98334]: pgmap v399: 97 pgs: 97 active+clean; 715 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-07-21T22:32:36.502 INFO:tasks.workunit.client.0.smithi107.stdout: 188 2217654 1035 1535317 981 750172 327 2023-07-21T22:32:37.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:37 smithi190 ceph-mon[98334]: pgmap v400: 97 pgs: 97 active+clean; 717 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 14 MiB/s wr, 1.13k op/s 2023-07-21T22:32:37.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:37 smithi132 ceph-mon[97863]: pgmap v400: 97 pgs: 97 active+clean; 717 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 14 MiB/s wr, 1.13k op/s 2023-07-21T22:32:37.948 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:37 smithi107 ceph-mon[96543]: pgmap v400: 97 pgs: 97 active+clean; 717 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 14 MiB/s wr, 1.13k op/s 2023-07-21T22:32:39.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:39 smithi190 ceph-mon[98334]: pgmap v401: 97 pgs: 97 active+clean; 718 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-07-21T22:32:39.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:39 smithi132 ceph-mon[97863]: pgmap v401: 97 pgs: 97 active+clean; 718 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-07-21T22:32:39.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:39 smithi107 ceph-mon[96543]: pgmap v401: 97 pgs: 97 active+clean; 718 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-07-21T22:32:41.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:41 smithi190 ceph-mon[98334]: pgmap v402: 97 pgs: 97 active+clean; 720 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 633 op/s 2023-07-21T22:32:41.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:41 smithi132 ceph-mon[97863]: pgmap v402: 97 pgs: 97 active+clean; 720 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 633 op/s 2023-07-21T22:32:41.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:41 smithi107 ceph-mon[96543]: pgmap v402: 97 pgs: 97 active+clean; 720 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 633 op/s 2023-07-21T22:32:43.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:43 smithi190 ceph-mon[98334]: pgmap v403: 97 pgs: 97 active+clean; 726 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 716 op/s 2023-07-21T22:32:43.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:43 smithi132 ceph-mon[97863]: pgmap v403: 97 pgs: 97 active+clean; 726 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 716 op/s 2023-07-21T22:32:43.948 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:43 smithi107 ceph-mon[96543]: pgmap v403: 97 pgs: 97 active+clean; 726 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.7 MiB/s wr, 716 op/s 2023-07-21T22:32:44.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:44 smithi190 ceph-mon[98334]: pgmap v404: 97 pgs: 97 active+clean; 726 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 539 op/s 2023-07-21T22:32:44.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:44 smithi132 ceph-mon[97863]: pgmap v404: 97 pgs: 97 active+clean; 726 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 539 op/s 2023-07-21T22:32:44.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:44 smithi107 ceph-mon[96543]: pgmap v404: 97 pgs: 97 active+clean; 726 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 539 op/s 2023-07-21T22:32:46.505 INFO:tasks.workunit.client.0.smithi107.stdout: 217 1925459 1336 1343213 1521 598377 353 2023-07-21T22:32:47.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:47 smithi132 ceph-mon[97863]: pgmap v405: 97 pgs: 97 active+clean; 733 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 548 op/s 2023-07-21T22:32:47.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:47 smithi107 ceph-mon[96543]: pgmap v405: 97 pgs: 97 active+clean; 733 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 548 op/s 2023-07-21T22:32:47.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:47 smithi190 ceph-mon[98334]: pgmap v405: 97 pgs: 97 active+clean; 733 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.7 MiB/s wr, 548 op/s 2023-07-21T22:32:49.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:49 smithi190 ceph-mon[98334]: pgmap v406: 97 pgs: 97 active+clean; 739 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 437 op/s 2023-07-21T22:32:49.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:49 smithi132 ceph-mon[97863]: pgmap v406: 97 pgs: 97 active+clean; 739 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 437 op/s 2023-07-21T22:32:49.692 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:49 smithi107 ceph-mon[96543]: pgmap v406: 97 pgs: 97 active+clean; 739 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 437 op/s 2023-07-21T22:32:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.898+0000 7fe89878d700 1 -- 172.21.15.107:0/1651323731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe89010bd60 msgr2=0x7fe89010e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.898+0000 7fe89878d700 1 --2- 172.21.15.107:0/1651323731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe89010bd60 0x7fe89010e210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe888009e00 tx=0x7fe88800b330 comp rx=0 tx=0).stop 2023-07-21T22:32:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.898+0000 7fe89878d700 1 -- 172.21.15.107:0/1651323731 shutdown_connections 2023-07-21T22:32:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.898+0000 7fe89878d700 1 --2- 172.21.15.107:0/1651323731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe89010bd60 0x7fe89010e210 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.898+0000 7fe89878d700 1 --2- 172.21.15.107:0/1651323731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe890070f00 0x7fe890071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.898+0000 7fe89878d700 1 --2- 172.21.15.107:0/1651323731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe890071850 0x7fe89010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.898+0000 7fe89878d700 1 -- 172.21.15.107:0/1651323731 >> 172.21.15.107:0/1651323731 conn(0x7fe89006ca50 msgr2=0x7fe89006eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.898+0000 7fe89878d700 1 -- 172.21.15.107:0/1651323731 shutdown_connections 2023-07-21T22:32:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.898+0000 7fe89878d700 1 -- 172.21.15.107:0/1651323731 wait complete. 2023-07-21T22:32:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.899+0000 7fe89878d700 1 Processor -- start 2023-07-21T22:32:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.899+0000 7fe89878d700 1 -- start start 2023-07-21T22:32:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.900+0000 7fe89878d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe890070f00 0x7fe890074750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.900+0000 7fe89878d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe890071850 0x7fe890076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.900+0000 7fe89878d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89010bd60 0x7fe890079380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.900+0000 7fe89878d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe890079b50 con 0x7fe89010bd60 2023-07-21T22:32:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.900+0000 7fe89878d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe890079cc0 con 0x7fe890070f00 2023-07-21T22:32:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.900+0000 7fe89878d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe890079e80 con 0x7fe890071850 2023-07-21T22:32:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.900+0000 7fe895d28700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe890071850 0x7fe890076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.900+0000 7fe896d2a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89010bd60 0x7fe890079380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.900+0000 7fe896d2a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89010bd60 0x7fe890079380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50096/0 (socket says 172.21.15.107:50096) 2023-07-21T22:32:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.900+0000 7fe896d2a700 1 -- 172.21.15.107:0/1167680096 learned_addr learned my addr 172.21.15.107:0/1167680096 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:49.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.910+0000 7fe896529700 1 --2- 172.21.15.107:0/1167680096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe890070f00 0x7fe890074750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:49.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.910+0000 7fe895d28700 1 -- 172.21.15.107:0/1167680096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe890070f00 msgr2=0x7fe890074750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:49.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.910+0000 7fe895d28700 1 --2- 172.21.15.107:0/1167680096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe890070f00 0x7fe890074750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:49.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.910+0000 7fe895d28700 1 -- 172.21.15.107:0/1167680096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89010bd60 msgr2=0x7fe890079380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:49.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.910+0000 7fe895d28700 1 --2- 172.21.15.107:0/1167680096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89010bd60 0x7fe890079380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:49.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.910+0000 7fe895d28700 1 -- 172.21.15.107:0/1167680096 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe888005e70 con 0x7fe890071850 2023-07-21T22:32:49.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.911+0000 7fe895d28700 1 --2- 172.21.15.107:0/1167680096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe890071850 0x7fe890076ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe88c005fd0 tx=0x7fe88c01fdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:49.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.911+0000 7fe8877fe700 1 -- 172.21.15.107:0/1167680096 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c01daa0 con 0x7fe890071850 2023-07-21T22:32:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.911+0000 7fe89878d700 1 -- 172.21.15.107:0/1167680096 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89007a160 con 0x7fe890071850 2023-07-21T22:32:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.911+0000 7fe89878d700 1 -- 172.21.15.107:0/1167680096 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89007a6c0 con 0x7fe890071850 2023-07-21T22:32:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.919+0000 7fe8a4912b80 1 Processor -- start 2023-07-21T22:32:49.973 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.920+0000 7fe8a4912b80 1 -- start start 2023-07-21T22:32:49.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.921+0000 7fe8a4912b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1c716b0 0x5567e1e152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:49.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.921+0000 7fe8a4912b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e15830 0x5567e1e16040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.921+0000 7fe8a4912b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1e16580 0x5567e1e169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.921+0000 7fe8a4912b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e1d1ac50 con 0x5567e1e16580 2023-07-21T22:32:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.921+0000 7fe8a4912b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e1d6d9e0 con 0x5567e1e15830 2023-07-21T22:32:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.921+0000 7fe8a4912b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e1d66e70 con 0x5567e1c716b0 2023-07-21T22:32:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.922+0000 7fe896d2a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1c716b0 0x5567e1e152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.922+0000 7fe896d2a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1c716b0 0x5567e1e152f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38660/0 (socket says 172.21.15.107:38660) 2023-07-21T22:32:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.922+0000 7fe896d2a700 1 -- 172.21.15.107:0/1399957248 learned_addr learned my addr 172.21.15.107:0/1399957248 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.922+0000 7fe896529700 1 --2- 172.21.15.107:0/1399957248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e15830 0x5567e1e16040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.923+0000 7fe896529700 1 -- 172.21.15.107:0/1399957248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1c716b0 msgr2=0x5567e1e152f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.923+0000 7fe896529700 1 --2- 172.21.15.107:0/1399957248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1c716b0 0x5567e1e152f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.923+0000 7fe896529700 1 -- 172.21.15.107:0/1399957248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1e16580 msgr2=0x5567e1e169f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.923+0000 7fe896529700 1 --2- 172.21.15.107:0/1399957248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1e16580 0x5567e1e169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.923+0000 7fe896529700 1 -- 172.21.15.107:0/1399957248 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567e1b94d80 con 0x5567e1e15830 2023-07-21T22:32:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.926+0000 7fe896529700 1 --2- 172.21.15.107:0/1399957248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e15830 0x5567e1e16040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe880007750 tx=0x7fe880007a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=623596d4cfae03e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.927+0000 7fe8877fe700 1 -- 172.21.15.107:0/1167680096 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe88c0044e0 con 0x7fe890071850 2023-07-21T22:32:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.927+0000 7fe8877fe700 1 -- 172.21.15.107:0/1167680096 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c003ce0 con 0x7fe890071850 2023-07-21T22:32:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.927+0000 7fe8877fe700 1 -- 172.21.15.107:0/1167680096 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe88c032440 con 0x7fe890071850 2023-07-21T22:32:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.928+0000 7fe8877fe700 1 --2- 172.21.15.107:0/1167680096 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe87c0a1320 0x7fe87c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:49.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.928+0000 7fe8877fe700 1 -- 172.21.15.107:0/1167680096 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe88c025070 con 0x7fe890071850 2023-07-21T22:32:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.935+0000 7fe896d2a700 1 --2- 172.21.15.107:0/1167680096 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe87c0a1320 0x7fe87c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:50.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:49.960+0000 7fe896d2a700 1 --2- 172.21.15.107:0/1167680096 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe87c0a1320 0x7fe87c0a37d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe888009fd0 tx=0x7fe888009300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.049+0000 7fe884ff9700 1 -- 172.21.15.107:0/1399957248 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880011590 con 0x5567e1e15830 2023-07-21T22:32:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.049+0000 7fe884ff9700 1 -- 172.21.15.107:0/1399957248 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe880011bb0 con 0x5567e1e15830 2023-07-21T22:32:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.049+0000 7fe884ff9700 1 -- 172.21.15.107:0/1399957248 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880012c60 con 0x5567e1e15830 2023-07-21T22:32:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.050+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1399957248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e15830 msgr2=0x5567e1e16040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.050+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1399957248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e15830 0x5567e1e16040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe880007750 tx=0x7fe880007a60 comp rx=0 tx=0).stop 2023-07-21T22:32:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.060+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1399957248 shutdown_connections 2023-07-21T22:32:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.060+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1399957248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1e16580 0x5567e1e169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.060+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1399957248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1c716b0 0x5567e1e152f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.060+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1399957248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e15830 0x5567e1e16040 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.060+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1399957248 >> 172.21.15.107:0/1399957248 conn(0x5567e1e0aae0 msgr2=0x5567e1e0fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.066+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1399957248 shutdown_connections 2023-07-21T22:32:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.066+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1399957248 wait complete. 2023-07-21T22:32:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.066+0000 7fe8a4912b80 1 -- start start 2023-07-21T22:32:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.066+0000 7fe8a4912b80 1 Processor -- start 2023-07-21T22:32:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.067+0000 7fe8a4912b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1c716b0 0x5567e1e1e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.067+0000 7fe8a4912b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1e15830 0x5567e1e1ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.067+0000 7fe8a4912b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e16580 0x5567e1e25ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.067+0000 7fe8a4912b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e1d06a60 con 0x5567e1c716b0 2023-07-21T22:32:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.067+0000 7fe8a4912b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e1d06760 con 0x5567e1e16580 2023-07-21T22:32:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.067+0000 7fe8a4912b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e1bea8c0 con 0x5567e1e15830 2023-07-21T22:32:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.067+0000 7fe8a4912b80 10 client.0 _ll_register_callbacks cb 0x5567e1d04700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.067+0000 7fe895d28700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1e15830 0x5567e1e1ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.067+0000 7fe895d28700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1e15830 0x5567e1e1ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38666/0 (socket says 172.21.15.107:38666) 2023-07-21T22:32:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.067+0000 7fe895d28700 1 -- 172.21.15.107:0/1934195803 learned_addr learned my addr 172.21.15.107:0/1934195803 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.068+0000 7fe896d2a700 1 --2- 172.21.15.107:0/1934195803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e16580 0x5567e1e25ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.068+0000 7fe895d28700 1 -- 172.21.15.107:0/1934195803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e16580 msgr2=0x5567e1e25ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.068+0000 7fe895d28700 1 --2- 172.21.15.107:0/1934195803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e16580 0x5567e1e25ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.068+0000 7fe895d28700 1 -- 172.21.15.107:0/1934195803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1c716b0 msgr2=0x5567e1e1e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.068+0000 7fe895d28700 1 --2- 172.21.15.107:0/1934195803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1c716b0 0x5567e1e1e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.068+0000 7fe895d28700 1 -- 172.21.15.107:0/1934195803 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88000e040 con 0x5567e1e15830 2023-07-21T22:32:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.071+0000 7fe895d28700 1 --2- 172.21.15.107:0/1934195803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1e15830 0x5567e1e1ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0a8440 tx=0x7fe88c0ab9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c779ed940dd4c5e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.157+0000 7fe879ffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:32:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.157+0000 7fe879ffb700 1 -- 172.21.15.107:0/1934195803 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c01de00 con 0x5567e1e15830 2023-07-21T22:32:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.157+0000 7fe879ffb700 1 -- 172.21.15.107:0/1934195803 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe88c0aff70 con 0x5567e1e15830 2023-07-21T22:32:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.157+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1934195803 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567e1cc0490 con 0x5567e1e15830 2023-07-21T22:32:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.162+0000 7fe879ffb700 1 -- 172.21.15.107:0/1934195803 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe88c0b4390 con 0x5567e1e15830 2023-07-21T22:32:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.162+0000 7fe8a4912b80 10 client.25792 fetch_fsmap learned FSMap version 36 2023-07-21T22:32:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.162+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1934195803 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5567e1b7dc00 con 0x5567e1e15830 2023-07-21T22:32:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.170+0000 7fe879ffb700 1 -- 172.21.15.107:0/1934195803 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5567e1cc0490 con 0x5567e1e15830 2023-07-21T22:32:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.170+0000 7fe879ffb700 1 -- 172.21.15.107:0/1934195803 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fe88c0ac1a0 con 0x5567e1e15830 2023-07-21T22:32:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.171+0000 7fe8a4912b80 10 client.25792 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:32:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.171+0000 7fe8a4912b80 10 client.25792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.171+0000 7fe8a4912b80 20 client.25792 populate_metadata read hostname 'smithi107' 2023-07-21T22:32:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.171+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1934195803 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5567e1e2a780 0x5567e1e2cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.171+0000 7fe8a4912b80 4 client.25792 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.171+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1934195803 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567e1d5fe50 con 0x5567e1e2a780 2023-07-21T22:32:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.171+0000 7fe896529700 1 --2- 172.21.15.107:0/1934195803 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5567e1e2a780 0x5567e1e2cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.184+0000 7fe896529700 1 --2- 172.21.15.107:0/1934195803 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5567e1e2a780 0x5567e1e2cb90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feab6e9557284f14 server_cookie=346903c1e3481523 in_seq=0 out_seq=0 2023-07-21T22:32:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.185+0000 7fe879ffb700 10 client.25792 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.196+0000 7fe879ffb700 1 -- 172.21.15.107:0/1934195803 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5567e1d5fe50 con 0x5567e1e2a780 2023-07-21T22:32:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.196+0000 7fe879ffb700 10 client.25792 handle_command_reply: tid=0 2023-07-21T22:32:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.196+0000 7fe8a4912b80 1 client.25792 shutdown 2023-07-21T22:32:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.200+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1934195803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1e15830 msgr2=0x5567e1e1ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.200+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1934195803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1e15830 0x5567e1e1ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0a8440 tx=0x7fe88c0ab9d0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.200+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1934195803 shutdown_connections 2023-07-21T22:32:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.200+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1934195803 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5567e1e2a780 0x5567e1e2cb90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.200+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1934195803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e16580 0x5567e1e25ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.200+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1934195803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1c716b0 0x5567e1e1e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.200+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1934195803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1e15830 0x5567e1e1ee80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.200+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1934195803 >> 172.21.15.107:0/1934195803 conn(0x5567e1e0aae0 msgr2=0x5567e1e0f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.201+0000 7fe879ffb700 0 client.25792 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.201+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1934195803 shutdown_connections 2023-07-21T22:32:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.201+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1934195803 wait complete. 2023-07-21T22:32:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.207+0000 7fe8a4912b80 20 client.25792 trim_cache size 0 max 16384 2023-07-21T22:32:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.476+0000 7fe8a4912b80 1 Processor -- start 2023-07-21T22:32:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.498+0000 7fe8a4912b80 1 -- start start 2023-07-21T22:32:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.498+0000 7fe8a4912b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1c716b0 0x5567e1e1e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.498+0000 7fe8a4912b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1e15830 0x5567e1e1ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.498+0000 7fe8a4912b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1e16580 0x5567e1e25de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.498+0000 7fe8a4912b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e1e27bb0 con 0x5567e1e15830 2023-07-21T22:32:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.498+0000 7fe8a4912b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e1e15c00 con 0x5567e1c716b0 2023-07-21T22:32:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.498+0000 7fe8a4912b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e1e1ddf0 con 0x5567e1e16580 2023-07-21T22:32:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.499+0000 7fe896529700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1c716b0 0x5567e1e1e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.499+0000 7fe896529700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1c716b0 0x5567e1e1e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40944/0 (socket says 172.21.15.107:40944) 2023-07-21T22:32:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.499+0000 7fe896529700 1 -- 172.21.15.107:0/1293724623 learned_addr learned my addr 172.21.15.107:0/1293724623 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.499+0000 7fe896d2a700 1 --2- 172.21.15.107:0/1293724623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1e16580 0x5567e1e25de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:50.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.505+0000 7fe895d28700 1 --2- 172.21.15.107:0/1293724623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1e15830 0x5567e1e1ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.506+0000 7fe896d2a700 1 -- 172.21.15.107:0/1293724623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1c716b0 msgr2=0x5567e1e1e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.506+0000 7fe896d2a700 1 --2- 172.21.15.107:0/1293724623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1c716b0 0x5567e1e1e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.506+0000 7fe896d2a700 1 -- 172.21.15.107:0/1293724623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1e15830 msgr2=0x5567e1e1ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.506+0000 7fe896d2a700 1 --2- 172.21.15.107:0/1293724623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1e15830 0x5567e1e1ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.506+0000 7fe896d2a700 1 -- 172.21.15.107:0/1293724623 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567e1bbb1f0 con 0x5567e1e16580 2023-07-21T22:32:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.517+0000 7fe896d2a700 1 --2- 172.21.15.107:0/1293724623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1e16580 0x5567e1e25de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe88800b700 tx=0x7fe88801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f62d218d413aea0e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.570+0000 7fe8857fa700 1 -- 172.21.15.107:0/1293724623 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88800e040 con 0x5567e1e16580 2023-07-21T22:32:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.570+0000 7fe8857fa700 1 -- 172.21.15.107:0/1293724623 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe888020070 con 0x5567e1e16580 2023-07-21T22:32:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.570+0000 7fe8857fa700 1 -- 172.21.15.107:0/1293724623 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8880286b0 con 0x5567e1e16580 2023-07-21T22:32:50.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.571+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1293724623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1e16580 msgr2=0x5567e1e25de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:50.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.571+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1293724623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1e16580 0x5567e1e25de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe88800b700 tx=0x7fe88801e000 comp rx=0 tx=0).stop 2023-07-21T22:32:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.572+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1293724623 shutdown_connections 2023-07-21T22:32:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.573+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1293724623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1e16580 0x5567e1e25de0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.573+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1293724623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1c716b0 0x5567e1e1e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.573+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/1293724623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1e15830 0x5567e1e1ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.573+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1293724623 >> 172.21.15.107:0/1293724623 conn(0x5567e1e0aae0 msgr2=0x5567e1e1d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.573+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1293724623 shutdown_connections 2023-07-21T22:32:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.573+0000 7fe8a4912b80 1 -- 172.21.15.107:0/1293724623 wait complete. 2023-07-21T22:32:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.574+0000 7fe8a4912b80 1 -- start start 2023-07-21T22:32:50.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.574+0000 7fe8a4912b80 1 Processor -- start 2023-07-21T22:32:50.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.574+0000 7fe8a4912b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1c716b0 0x5567e1e21a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:50.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.574+0000 7fe8a4912b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1e15830 0x5567e1e21fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:50.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.574+0000 7fe8a4912b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e16580 0x5567e1e24fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:50.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.574+0000 7fe8a4912b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e1e25500 con 0x5567e1e15830 2023-07-21T22:32:50.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.574+0000 7fe8a4912b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e1e25670 con 0x5567e1e16580 2023-07-21T22:32:50.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.574+0000 7fe8a4912b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e1e25830 con 0x5567e1c716b0 2023-07-21T22:32:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.574+0000 7fe8a4912b80 10 client.0 _ll_register_callbacks cb 0x7fe88c0abd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.575+0000 7fe896529700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1c716b0 0x5567e1e21a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.575+0000 7fe896529700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1c716b0 0x5567e1e21a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38680/0 (socket says 172.21.15.107:38680) 2023-07-21T22:32:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.575+0000 7fe896529700 1 -- 172.21.15.107:0/3898834493 learned_addr learned my addr 172.21.15.107:0/3898834493 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.575+0000 7fe896d2a700 1 --2- 172.21.15.107:0/3898834493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e16580 0x5567e1e24fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.583+0000 7fe896d2a700 1 -- 172.21.15.107:0/3898834493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1c716b0 msgr2=0x5567e1e21a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.583+0000 7fe896d2a700 1 --2- 172.21.15.107:0/3898834493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1c716b0 0x5567e1e21a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.583+0000 7fe896d2a700 1 -- 172.21.15.107:0/3898834493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1e15830 msgr2=0x5567e1e21fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.583+0000 7fe896d2a700 1 --2- 172.21.15.107:0/3898834493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1e15830 0x5567e1e21fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:50.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.583+0000 7fe896d2a700 1 -- 172.21.15.107:0/3898834493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe890117640 con 0x5567e1e16580 2023-07-21T22:32:50.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.592+0000 7fe896d2a700 1 --2- 172.21.15.107:0/3898834493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e16580 0x5567e1e24fc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe888007900 tx=0x7fe888007e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15327712f3857be3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.618+0000 7fe879ffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:32:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.618+0000 7fe879ffb700 1 -- 172.21.15.107:0/3898834493 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888036070 con 0x5567e1e16580 2023-07-21T22:32:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.618+0000 7fe879ffb700 1 -- 172.21.15.107:0/3898834493 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe8880294f0 con 0x5567e1e16580 2023-07-21T22:32:50.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.618+0000 7fe8a4912b80 1 -- 172.21.15.107:0/3898834493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe880011bb0 con 0x5567e1e16580 2023-07-21T22:32:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.619+0000 7fe879ffb700 1 -- 172.21.15.107:0/3898834493 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe88800e040 con 0x5567e1e16580 2023-07-21T22:32:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.619+0000 7fe8a4912b80 10 client.25736 fetch_fsmap learned FSMap version 36 2023-07-21T22:32:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.619+0000 7fe8a4912b80 1 -- 172.21.15.107:0/3898834493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5567e1b93250 con 0x5567e1e16580 2023-07-21T22:32:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.630+0000 7fe879ffb700 1 -- 172.21.15.107:0/3898834493 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe880011bb0 con 0x5567e1e16580 2023-07-21T22:32:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.630+0000 7fe879ffb700 1 -- 172.21.15.107:0/3898834493 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 36) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fe88801c410 con 0x5567e1e16580 2023-07-21T22:32:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.630+0000 7fe8a4912b80 10 client.25736 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:32:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.630+0000 7fe8a4912b80 10 client.25736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.630+0000 7fe8a4912b80 20 client.25736 populate_metadata read hostname 'smithi107' 2023-07-21T22:32:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.630+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/3898834493 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5567e1e7f050 0x5567e1e81430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.630+0000 7fe8a4912b80 4 client.25736 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:32:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.630+0000 7fe8a4912b80 1 -- 172.21.15.107:0/3898834493 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5567e1d21ff0 con 0x5567e1e7f050 2023-07-21T22:32:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.631+0000 7fe896529700 1 --2- 172.21.15.107:0/3898834493 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5567e1e7f050 0x5567e1e81430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.639+0000 7fe896529700 1 --2- 172.21.15.107:0/3898834493 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5567e1e7f050 0x5567e1e81430 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d72496c5e9e1b7f server_cookie=5ff7894968cfb4c8 in_seq=0 out_seq=0 2023-07-21T22:32:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:50.641+0000 7fe879ffb700 10 client.25736 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:51.134 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:32:51.311 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.262+0000 7fe879ffb700 1 -- 172.21.15.107:0/3898834493 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5567e1d21ff0 con 0x5567e1e7f050 2023-07-21T22:32:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.262+0000 7fe879ffb700 10 client.25736 handle_command_reply: tid=0 2023-07-21T22:32:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.270+0000 7fe8a4912b80 1 client.25736 shutdown 2023-07-21T22:32:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.384+0000 7fe8a4912b80 1 -- 172.21.15.107:0/3898834493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e16580 msgr2=0x5567e1e24fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.384+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/3898834493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e16580 0x5567e1e24fc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe888007900 tx=0x7fe888007e90 comp rx=0 tx=0).stop 2023-07-21T22:32:51.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.384+0000 7fe8a4912b80 1 -- 172.21.15.107:0/3898834493 shutdown_connections 2023-07-21T22:32:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.384+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/3898834493 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5567e1e7f050 0x5567e1e81430 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.384+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/3898834493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5567e1e16580 0x5567e1e24fc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.384+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/3898834493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5567e1c716b0 0x5567e1e21a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.384+0000 7fe8a4912b80 1 --2- 172.21.15.107:0/3898834493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5567e1e15830 0x5567e1e21fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.384+0000 7fe8a4912b80 1 -- 172.21.15.107:0/3898834493 >> 172.21.15.107:0/3898834493 conn(0x5567e1e0aae0 msgr2=0x5567e1e1e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:51.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.384+0000 7fe879ffb700 0 client.25736 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.386+0000 7fe8a4912b80 1 -- 172.21.15.107:0/3898834493 shutdown_connections 2023-07-21T22:32:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.386+0000 7fe8a4912b80 1 -- 172.21.15.107:0/3898834493 wait complete. 2023-07-21T22:32:51.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.386+0000 7fe8a4912b80 20 client.25736 trim_cache size 0 max 16384 2023-07-21T22:32:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:32:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:32:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:32:51.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:32:51.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:51 smithi190 ceph-mon[98334]: pgmap v407: 97 pgs: 97 active+clean; 741 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 200 op/s 2023-07-21T22:32:51.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:51 smithi132 ceph-mon[97863]: pgmap v407: 97 pgs: 97 active+clean; 741 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 200 op/s 2023-07-21T22:32:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.653+0000 7fe89878d700 1 -- 172.21.15.107:0/1167680096 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe87c0a1320 msgr2=0x7fe87c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.653+0000 7fe89878d700 1 --2- 172.21.15.107:0/1167680096 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe87c0a1320 0x7fe87c0a37d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe888009fd0 tx=0x7fe888009300 comp rx=0 tx=0).stop 2023-07-21T22:32:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.653+0000 7fe89878d700 1 -- 172.21.15.107:0/1167680096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe890071850 msgr2=0x7fe890076ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.653+0000 7fe89878d700 1 --2- 172.21.15.107:0/1167680096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe890071850 0x7fe890076ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe88c005fd0 tx=0x7fe88c01fdd0 comp rx=0 tx=0).stop 2023-07-21T22:32:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.653+0000 7fe89878d700 1 -- 172.21.15.107:0/1167680096 shutdown_connections 2023-07-21T22:32:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.654+0000 7fe89878d700 1 --2- 172.21.15.107:0/1167680096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe89010bd60 0x7fe890079380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.654+0000 7fe89878d700 1 --2- 172.21.15.107:0/1167680096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe890070f00 0x7fe890074750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.654+0000 7fe89878d700 1 --2- 172.21.15.107:0/1167680096 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe87c0a1320 0x7fe87c0a37d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.654+0000 7fe89878d700 1 --2- 172.21.15.107:0/1167680096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe890071850 0x7fe890076ca0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.654+0000 7fe89878d700 1 -- 172.21.15.107:0/1167680096 >> 172.21.15.107:0/1167680096 conn(0x7fe89006ca50 msgr2=0x7fe89006dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:51.702 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:51 smithi107 ceph-mon[96543]: pgmap v407: 97 pgs: 97 active+clean; 741 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 200 op/s 2023-07-21T22:32:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.654+0000 7fe89878d700 1 -- 172.21.15.107:0/1167680096 shutdown_connections 2023-07-21T22:32:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:32:51.654+0000 7fe89878d700 1 -- 172.21.15.107:0/1167680096 wait complete. 2023-07-21T22:32:52.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:52 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:32:52.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:52 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:32:52.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:52 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:32:53.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.484+0000 7f6988d43700 1 -- 172.21.15.107:0/1357779653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69840711d0 msgr2=0x7f69840715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:53.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.484+0000 7f6988d43700 1 --2- 172.21.15.107:0/1357779653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69840711d0 0x7f69840715e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f69740098f0 tx=0x7f6974009c00 comp rx=0 tx=0).stop 2023-07-21T22:32:53.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.485+0000 7f6988d43700 1 -- 172.21.15.107:0/1357779653 shutdown_connections 2023-07-21T22:32:53.532 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.485+0000 7f6988d43700 1 --2- 172.21.15.107:0/1357779653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f698410bba0 0x7f698410e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.485+0000 7f6988d43700 1 --2- 172.21.15.107:0/1357779653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69840711d0 0x7f69840715e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.485+0000 7f6988d43700 1 --2- 172.21.15.107:0/1357779653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6984071b20 0x7f698410b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.485+0000 7f6988d43700 1 -- 172.21.15.107:0/1357779653 >> 172.21.15.107:0/1357779653 conn(0x7f698406c8b0 msgr2=0x7f698406ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:53.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.485+0000 7f6988d43700 1 -- 172.21.15.107:0/1357779653 shutdown_connections 2023-07-21T22:32:53.533 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.485+0000 7f6988d43700 1 -- 172.21.15.107:0/1357779653 wait complete. 2023-07-21T22:32:53.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.486+0000 7f6988d43700 1 Processor -- start 2023-07-21T22:32:53.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.487+0000 7f6988d43700 1 -- start start 2023-07-21T22:32:53.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.487+0000 7f6988d43700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69840711d0 0x7f69841a4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.534 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.487+0000 7f6988d43700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6984071b20 0x7f69841a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.487+0000 7f6988d43700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410bba0 0x7f69841aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.487+0000 7f6988d43700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69841ac000 con 0x7f698410bba0 2023-07-21T22:32:53.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.487+0000 7f6988d43700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698410ec40 con 0x7f6984071b20 2023-07-21T22:32:53.535 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.487+0000 7f6988d43700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698410edd0 con 0x7f69840711d0 2023-07-21T22:32:53.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.494+0000 7f69837fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69840711d0 0x7f69841a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.494+0000 7f69837fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69840711d0 0x7f69841a4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38694/0 (socket says 172.21.15.107:38694) 2023-07-21T22:32:53.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.494+0000 7f69837fe700 1 -- 172.21.15.107:0/3321427748 learned_addr learned my addr 172.21.15.107:0/3321427748 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:53.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.494+0000 7f6982ffd700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6984071b20 0x7f69841a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.494+0000 7f6983fff700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410bba0 0x7f69841aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.542 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.494+0000 7f69837fe700 1 -- 172.21.15.107:0/3321427748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6984071b20 msgr2=0x7f69841a49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:53.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.494+0000 7f69837fe700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6984071b20 0x7f69841a49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.494+0000 7f69837fe700 1 -- 172.21.15.107:0/3321427748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410bba0 msgr2=0x7f69841aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:53.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.494+0000 7f69837fe700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410bba0 0x7f69841aba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.494+0000 7f69837fe700 1 -- 172.21.15.107:0/3321427748 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6974009520 con 0x7f69840711d0 2023-07-21T22:32:53.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.494+0000 7f6983fff700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410bba0 0x7f69841aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:32:53.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.501+0000 7f69837fe700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69840711d0 0x7f69841a4480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f697400fbd0 tx=0x7f697400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:53.549 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.501+0000 7f6980ff9700 1 -- 172.21.15.107:0/3321427748 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697401c070 con 0x7f69840711d0 2023-07-21T22:32:53.549 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.501+0000 7f6980ff9700 1 -- 172.21.15.107:0/3321427748 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6974005530 con 0x7f69840711d0 2023-07-21T22:32:53.549 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.501+0000 7f6980ff9700 1 -- 172.21.15.107:0/3321427748 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974003e90 con 0x7f69840711d0 2023-07-21T22:32:53.550 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.501+0000 7f6988d43700 1 -- 172.21.15.107:0/3321427748 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698410f050 con 0x7f69840711d0 2023-07-21T22:32:53.550 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.501+0000 7f6988d43700 1 -- 172.21.15.107:0/3321427748 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698410f4b0 con 0x7f69840711d0 2023-07-21T22:32:53.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.517+0000 7f699612ab80 1 Processor -- start 2023-07-21T22:32:53.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.518+0000 7f6980ff9700 1 -- 172.21.15.107:0/3321427748 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6974003680 con 0x7f69840711d0 2023-07-21T22:32:53.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.519+0000 7f6980ff9700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f696c0a13e0 0x7f696c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.519+0000 7f6980ff9700 1 -- 172.21.15.107:0/3321427748 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f69740cde00 con 0x7f69840711d0 2023-07-21T22:32:53.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.529+0000 7f699612ab80 1 -- start start 2023-07-21T22:32:53.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.530+0000 7f699612ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e745a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.530+0000 7f699612ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e745f70 0x55ce3e746830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.530+0000 7f699612ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e746dc0 0x55ce3e747230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.530+0000 7f699612ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3e64acb0 con 0x55ce3e745620 2023-07-21T22:32:53.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.530+0000 7f699612ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3e69da40 con 0x55ce3e746dc0 2023-07-21T22:32:53.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.530+0000 7f699612ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3e696ed0 con 0x55ce3e745f70 2023-07-21T22:32:53.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.530+0000 7f6983fff700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f696c0a13e0 0x7f696c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.530+0000 7f69837fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e745a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.530+0000 7f69837fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e745a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50144/0 (socket says 172.21.15.107:50144) 2023-07-21T22:32:53.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.530+0000 7f69837fe700 1 -- 172.21.15.107:0/3870378345 learned_addr learned my addr 172.21.15.107:0/3870378345 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:53.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.530+0000 7f6982ffd700 1 --2- 172.21.15.107:0/3870378345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e745f70 0x55ce3e746830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.579 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.530+0000 7f6983fff700 1 --2- 172.21.15.107:0/3870378345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e746dc0 0x55ce3e747230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.531+0000 7f6983fff700 1 -- 172.21.15.107:0/3870378345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e745f70 msgr2=0x55ce3e746830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:53.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.531+0000 7f6983fff700 1 --2- 172.21.15.107:0/3870378345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e745f70 0x55ce3e746830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.531+0000 7f6983fff700 1 -- 172.21.15.107:0/3870378345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 msgr2=0x55ce3e745a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:53.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.531+0000 7f6983fff700 1 --2- 172.21.15.107:0/3870378345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e745a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.531+0000 7f6983fff700 1 -- 172.21.15.107:0/3870378345 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce3e4c4de0 con 0x55ce3e746dc0 2023-07-21T22:32:53.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.545+0000 7f6983fff700 1 --2- 172.21.15.107:0/3870378345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e746dc0 0x55ce3e747230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f697c007770 tx=0x7f697c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fc5407cbf050dc9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:53.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.554+0000 7f6983fff700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f696c0a13e0 0x7f696c0a3890 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f697c007fd0 tx=0x7f697c01a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:53.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.568+0000 7f6969ffb700 1 -- 172.21.15.107:0/3870378345 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c009150 con 0x55ce3e746dc0 2023-07-21T22:32:53.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.568+0000 7f6969ffb700 1 -- 172.21.15.107:0/3870378345 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f697c019e20 con 0x55ce3e746dc0 2023-07-21T22:32:53.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.568+0000 7f6969ffb700 1 -- 172.21.15.107:0/3870378345 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c018940 con 0x55ce3e746dc0 2023-07-21T22:32:53.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.569+0000 7f699612ab80 1 -- 172.21.15.107:0/3870378345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e746dc0 msgr2=0x55ce3e747230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:53.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.569+0000 7f699612ab80 1 --2- 172.21.15.107:0/3870378345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e746dc0 0x55ce3e747230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f697c007770 tx=0x7f697c009f90 comp rx=0 tx=0).stop 2023-07-21T22:32:53.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.570+0000 7f699612ab80 1 -- 172.21.15.107:0/3870378345 shutdown_connections 2023-07-21T22:32:53.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.570+0000 7f699612ab80 1 --2- 172.21.15.107:0/3870378345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e746dc0 0x55ce3e747230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.570+0000 7f699612ab80 1 --2- 172.21.15.107:0/3870378345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e745a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.570+0000 7f699612ab80 1 --2- 172.21.15.107:0/3870378345 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e745f70 0x55ce3e746830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.570+0000 7f699612ab80 1 -- 172.21.15.107:0/3870378345 >> 172.21.15.107:0/3870378345 conn(0x55ce3e73aef0 msgr2=0x55ce3e740390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:53.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.570+0000 7f699612ab80 1 -- 172.21.15.107:0/3870378345 shutdown_connections 2023-07-21T22:32:53.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.570+0000 7f699612ab80 1 -- 172.21.15.107:0/3870378345 wait complete. 2023-07-21T22:32:53.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.570+0000 7f699612ab80 1 -- start start 2023-07-21T22:32:53.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.570+0000 7f699612ab80 1 Processor -- start 2023-07-21T22:32:53.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.571+0000 7f699612ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e74f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.571+0000 7f699612ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745f70 0x55ce3e74f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.571+0000 7f699612ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e746dc0 0x55ce3e756880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.571+0000 7f699612ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3e636ac0 con 0x55ce3e745620 2023-07-21T22:32:53.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.571+0000 7f699612ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3e6367c0 con 0x55ce3e745f70 2023-07-21T22:32:53.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.571+0000 7f699612ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3e51a920 con 0x55ce3e746dc0 2023-07-21T22:32:53.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.571+0000 7f699612ab80 10 client.0 _ll_register_callbacks cb 0x55ce3e634760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:53.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.571+0000 7f6982ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745f70 0x55ce3e74f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.571+0000 7f6982ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745f70 0x55ce3e74f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40980/0 (socket says 172.21.15.107:40980) 2023-07-21T22:32:53.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.571+0000 7f6982ffd700 1 -- 172.21.15.107:0/2316589240 learned_addr learned my addr 172.21.15.107:0/2316589240 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:53.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.571+0000 7f6983fff700 1 --2- 172.21.15.107:0/2316589240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e746dc0 0x55ce3e756880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.578+0000 7f69837fe700 1 --2- 172.21.15.107:0/2316589240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e74f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.580+0000 7f6982ffd700 1 -- 172.21.15.107:0/2316589240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e746dc0 msgr2=0x55ce3e756880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:53.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.580+0000 7f6982ffd700 1 --2- 172.21.15.107:0/2316589240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e746dc0 0x55ce3e756880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.580+0000 7f6982ffd700 1 -- 172.21.15.107:0/2316589240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 msgr2=0x55ce3e74f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:53.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.580+0000 7f6982ffd700 1 --2- 172.21.15.107:0/2316589240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e74f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.580+0000 7f6982ffd700 1 -- 172.21.15.107:0/2316589240 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c005e70 con 0x55ce3e745f70 2023-07-21T22:32:53.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.591+0000 7f6982ffd700 1 --2- 172.21.15.107:0/2316589240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745f70 0x55ce3e74f840 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6978009f20 tx=0x7f69780116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d10a837ace555978 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:53.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.591+0000 7f69677f6700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:32:53.638 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.591+0000 7f69677f6700 1 -- 172.21.15.107:0/2316589240 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978012650 con 0x55ce3e745f70 2023-07-21T22:32:53.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.591+0000 7f69677f6700 1 -- 172.21.15.107:0/2316589240 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f69780127b0 con 0x55ce3e745f70 2023-07-21T22:32:53.639 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.591+0000 7f699612ab80 1 -- 172.21.15.107:0/2316589240 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce3e5f04f0 con 0x55ce3e745f70 2023-07-21T22:32:53.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:53 smithi190 ceph-mon[98334]: pgmap v408: 97 pgs: 97 active+clean; 746 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2023-07-21T22:32:53.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:53.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:53.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:53.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:53 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:53.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.602+0000 7f69677f6700 1 -- 172.21.15.107:0/2316589240 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6978011e90 con 0x55ce3e745f70 2023-07-21T22:32:53.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.602+0000 7f699612ab80 10 client.25745 fetch_fsmap learned FSMap version 36 2023-07-21T22:32:53.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.602+0000 7f699612ab80 1 -- 172.21.15.107:0/2316589240 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ce3e4adc60 con 0x55ce3e745f70 2023-07-21T22:32:53.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.625+0000 7f69677f6700 1 -- 172.21.15.107:0/2316589240 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce3e5f04f0 con 0x55ce3e745f70 2023-07-21T22:32:53.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.625+0000 7f69677f6700 1 -- 172.21.15.107:0/2316589240 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 36) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f697800c070 con 0x55ce3e745f70 2023-07-21T22:32:53.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.626+0000 7f699612ab80 10 client.25745 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:32:53.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.626+0000 7f699612ab80 10 client.25745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:53.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.626+0000 7f699612ab80 20 client.25745 populate_metadata read hostname 'smithi107' 2023-07-21T22:32:53.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.626+0000 7f699612ab80 1 --2- 172.21.15.107:0/2316589240 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ce3e75b140 0x55ce3e75d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.626+0000 7f699612ab80 4 client.25745 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:53.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.626+0000 7f699612ab80 1 -- 172.21.15.107:0/2316589240 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce3e68feb0 con 0x55ce3e75b140 2023-07-21T22:32:53.674 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.626+0000 7f69837fe700 1 --2- 172.21.15.107:0/2316589240 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ce3e75b140 0x55ce3e75d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.627+0000 7f69837fe700 1 --2- 172.21.15.107:0/2316589240 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ce3e75b140 0x55ce3e75d550 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=670374440b801991 server_cookie=f76cf817c877cedf in_seq=0 out_seq=0 2023-07-21T22:32:53.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.627+0000 7f69677f6700 10 client.25745 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:53.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.628+0000 7f69677f6700 1 -- 172.21.15.107:0/2316589240 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ce3e68feb0 con 0x55ce3e75b140 2023-07-21T22:32:53.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.628+0000 7f69677f6700 10 client.25745 handle_command_reply: tid=0 2023-07-21T22:32:53.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.628+0000 7f699612ab80 1 client.25745 shutdown 2023-07-21T22:32:53.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.628+0000 7f699612ab80 1 -- 172.21.15.107:0/2316589240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745f70 msgr2=0x55ce3e74f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:53.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.628+0000 7f699612ab80 1 --2- 172.21.15.107:0/2316589240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745f70 0x55ce3e74f840 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6978009f20 tx=0x7f69780116c0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.629+0000 7f699612ab80 1 -- 172.21.15.107:0/2316589240 shutdown_connections 2023-07-21T22:32:53.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.629+0000 7f699612ab80 1 --2- 172.21.15.107:0/2316589240 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ce3e75b140 0x55ce3e75d550 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.629+0000 7f699612ab80 1 --2- 172.21.15.107:0/2316589240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e746dc0 0x55ce3e756880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.629+0000 7f699612ab80 1 --2- 172.21.15.107:0/2316589240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e74f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.629+0000 7f699612ab80 1 --2- 172.21.15.107:0/2316589240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745f70 0x55ce3e74f840 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.629+0000 7f699612ab80 1 -- 172.21.15.107:0/2316589240 >> 172.21.15.107:0/2316589240 conn(0x55ce3e73aef0 msgr2=0x55ce3e73fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:53.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.629+0000 7f69677f6700 0 client.25745 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:53.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.634+0000 7f699612ab80 1 -- 172.21.15.107:0/2316589240 shutdown_connections 2023-07-21T22:32:53.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.634+0000 7f699612ab80 1 -- 172.21.15.107:0/2316589240 wait complete. 2023-07-21T22:32:53.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.634+0000 7f699612ab80 20 client.25745 trim_cache size 0 max 16384 2023-07-21T22:32:53.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:53 smithi132 ceph-mon[97863]: pgmap v408: 97 pgs: 97 active+clean; 746 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2023-07-21T22:32:53.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:53.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:53.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:53.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:53 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:53.693 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:53 smithi107 ceph-mon[96543]: pgmap v408: 97 pgs: 97 active+clean; 746 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2023-07-21T22:32:53.693 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:53.693 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:53.693 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:53.693 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:53 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:53.839 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.792+0000 7f699612ab80 1 Processor -- start 2023-07-21T22:32:53.840 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.793+0000 7f699612ab80 1 -- start start 2023-07-21T22:32:53.840 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.793+0000 7f699612ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e75d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.851 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.793+0000 7f699612ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745f70 0x55ce3e7567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.852 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.793+0000 7f699612ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e746dc0 0x55ce3e74d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.852 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.793+0000 7f699612ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3e73a8f0 con 0x55ce3e745620 2023-07-21T22:32:53.852 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.794+0000 7f699612ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3e741620 con 0x55ce3e745f70 2023-07-21T22:32:53.852 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.794+0000 7f699612ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3e74e7b0 con 0x55ce3e746dc0 2023-07-21T22:32:53.853 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.805+0000 7f6983fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e746dc0 0x55ce3e74d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.853 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.805+0000 7f6983fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e746dc0 0x55ce3e74d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38720/0 (socket says 172.21.15.107:38720) 2023-07-21T22:32:53.853 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.805+0000 7f6983fff700 1 -- 172.21.15.107:0/1706197033 learned_addr learned my addr 172.21.15.107:0/1706197033 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:53.853 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.805+0000 7f6982ffd700 1 --2- 172.21.15.107:0/1706197033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745f70 0x55ce3e7567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.860 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.813+0000 7f69837fe700 1 --2- 172.21.15.107:0/1706197033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e75d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.821+0000 7f69837fe700 1 -- 172.21.15.107:0/1706197033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e746dc0 msgr2=0x55ce3e74d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:53.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.821+0000 7f69837fe700 1 --2- 172.21.15.107:0/1706197033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e746dc0 0x55ce3e74d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.821+0000 7f69837fe700 1 -- 172.21.15.107:0/1706197033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745f70 msgr2=0x55ce3e7567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:53.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.821+0000 7f69837fe700 1 --2- 172.21.15.107:0/1706197033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745f70 0x55ce3e7567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.821+0000 7f69837fe700 1 -- 172.21.15.107:0/1706197033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce3e4eb250 con 0x55ce3e745620 2023-07-21T22:32:53.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.831+0000 7f69837fe700 1 --2- 172.21.15.107:0/1706197033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e75d520 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f69740a30b0 tx=0x7f697400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82fecb263fd62a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:53.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.842+0000 7f696a7fc700 1 -- 172.21.15.107:0/1706197033 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69740a1110 con 0x55ce3e745620 2023-07-21T22:32:53.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.842+0000 7f696a7fc700 1 -- 172.21.15.107:0/1706197033 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69740aee90 con 0x55ce3e745620 2023-07-21T22:32:53.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.842+0000 7f696a7fc700 1 -- 172.21.15.107:0/1706197033 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974097930 con 0x55ce3e745620 2023-07-21T22:32:53.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.862+0000 7f699612ab80 1 -- 172.21.15.107:0/1706197033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 msgr2=0x55ce3e75d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:53.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.862+0000 7f699612ab80 1 --2- 172.21.15.107:0/1706197033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e75d520 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f69740a30b0 tx=0x7f697400bda0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.938 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.891+0000 7f699612ab80 1 -- 172.21.15.107:0/1706197033 shutdown_connections 2023-07-21T22:32:53.938 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.891+0000 7f699612ab80 1 --2- 172.21.15.107:0/1706197033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e746dc0 0x55ce3e74d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.938 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.891+0000 7f699612ab80 1 --2- 172.21.15.107:0/1706197033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e745620 0x55ce3e75d520 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.938 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.891+0000 7f699612ab80 1 --2- 172.21.15.107:0/1706197033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745f70 0x55ce3e7567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.939 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.891+0000 7f699612ab80 1 -- 172.21.15.107:0/1706197033 >> 172.21.15.107:0/1706197033 conn(0x55ce3e73aef0 msgr2=0x55ce3e761eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:53.940 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.893+0000 7f699612ab80 1 -- 172.21.15.107:0/1706197033 shutdown_connections 2023-07-21T22:32:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.893+0000 7f699612ab80 1 -- 172.21.15.107:0/1706197033 wait complete. 2023-07-21T22:32:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.894+0000 7f699612ab80 1 -- start start 2023-07-21T22:32:53.941 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.894+0000 7f699612ab80 1 Processor -- start 2023-07-21T22:32:53.942 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.894+0000 7f699612ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745620 0x55ce3e752710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.894+0000 7f699612ab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e745f70 0x55ce3e752c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.894+0000 7f699612ab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e746dc0 0x55ce3e755c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:53.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.894+0000 7f699612ab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3e7561b0 con 0x55ce3e746dc0 2023-07-21T22:32:53.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.894+0000 7f699612ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3e756320 con 0x55ce3e745620 2023-07-21T22:32:53.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.894+0000 7f699612ab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3e7564e0 con 0x55ce3e745f70 2023-07-21T22:32:53.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.894+0000 7f699612ab80 10 client.0 _ll_register_callbacks cb 0x7f69780099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:53.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.895+0000 7f69837fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745620 0x55ce3e752710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.943 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.895+0000 7f69837fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745620 0x55ce3e752710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:40992/0 (socket says 172.21.15.107:40992) 2023-07-21T22:32:53.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.895+0000 7f69837fe700 1 -- 172.21.15.107:0/1633679213 learned_addr learned my addr 172.21.15.107:0/1633679213 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:32:53.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.896+0000 7f6982ffd700 1 --2- 172.21.15.107:0/1633679213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e745f70 0x55ce3e752c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:53.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.897+0000 7f6982ffd700 1 -- 172.21.15.107:0/1633679213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745620 msgr2=0x55ce3e752710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:53.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.897+0000 7f6982ffd700 1 --2- 172.21.15.107:0/1633679213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745620 0x55ce3e752710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.897+0000 7f6982ffd700 1 -- 172.21.15.107:0/1633679213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e746dc0 msgr2=0x55ce3e755c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:53.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.897+0000 7f6982ffd700 1 --2- 172.21.15.107:0/1633679213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e746dc0 0x55ce3e755c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:53.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.897+0000 7f6982ffd700 1 -- 172.21.15.107:0/1633679213 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698410f4b0 con 0x55ce3e745f70 2023-07-21T22:32:53.955 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.908+0000 7f6982ffd700 1 --2- 172.21.15.107:0/1633679213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e745f70 0x55ce3e752c50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f697801aba0 tx=0x7f69780094c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=397d015995d3eea3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:53.989 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.941+0000 7f69677f6700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:32:53.990 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.941+0000 7f69677f6700 1 -- 172.21.15.107:0/1633679213 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978011e90 con 0x55ce3e745f70 2023-07-21T22:32:53.990 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.942+0000 7f69677f6700 1 -- 172.21.15.107:0/1633679213 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6978005250 con 0x55ce3e745f70 2023-07-21T22:32:53.990 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.942+0000 7f699612ab80 1 -- 172.21.15.107:0/1633679213 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f697c019e20 con 0x55ce3e745f70 2023-07-21T22:32:54.006 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.952+0000 7f69677f6700 1 -- 172.21.15.107:0/1633679213 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6978005430 con 0x55ce3e745f70 2023-07-21T22:32:54.007 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.954+0000 7f699612ab80 10 client.25816 fetch_fsmap learned FSMap version 36 2023-07-21T22:32:54.007 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.954+0000 7f699612ab80 1 -- 172.21.15.107:0/1633679213 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ce3e4c32b0 con 0x55ce3e745f70 2023-07-21T22:32:54.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.960+0000 7f69677f6700 1 -- 172.21.15.107:0/1633679213 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f697c019e20 con 0x55ce3e745f70 2023-07-21T22:32:54.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.960+0000 7f69677f6700 1 -- 172.21.15.107:0/1633679213 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f697802b070 con 0x55ce3e745f70 2023-07-21T22:32:54.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.961+0000 7f699612ab80 10 client.25816 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:32:54.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.961+0000 7f699612ab80 10 client.25816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:54.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.961+0000 7f699612ab80 20 client.25816 populate_metadata read hostname 'smithi107' 2023-07-21T22:32:54.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.961+0000 7f699612ab80 1 --2- 172.21.15.107:0/1633679213 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ce3e7b00e0 0x55ce3e7b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:54.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.961+0000 7f699612ab80 4 client.25816 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:32:54.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.961+0000 7f699612ab80 1 -- 172.21.15.107:0/1633679213 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ce3e652050 con 0x55ce3e7b00e0 2023-07-21T22:32:54.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.961+0000 7f69837fe700 1 --2- 172.21.15.107:0/1633679213 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ce3e7b00e0 0x55ce3e7b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:54.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.969+0000 7f69677f6700 10 client.25816 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:54.016 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.969+0000 7f69837fe700 1 --2- 172.21.15.107:0/1633679213 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ce3e7b00e0 0x55ce3e7b24f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d2ad3cab8d627d server_cookie=222a93baa9e7c9c1 in_seq=0 out_seq=0 2023-07-21T22:32:54.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.969+0000 7f69677f6700 1 -- 172.21.15.107:0/1633679213 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ce3e652050 con 0x55ce3e7b00e0 2023-07-21T22:32:54.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.969+0000 7f69677f6700 10 client.25816 handle_command_reply: tid=0 2023-07-21T22:32:54.017 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.969+0000 7f699612ab80 1 client.25816 shutdown 2023-07-21T22:32:54.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.970+0000 7f699612ab80 1 -- 172.21.15.107:0/1633679213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e745f70 msgr2=0x55ce3e752c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:54.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.970+0000 7f699612ab80 1 --2- 172.21.15.107:0/1633679213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e745f70 0x55ce3e752c50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f697801aba0 tx=0x7f69780094c0 comp rx=0 tx=0).stop 2023-07-21T22:32:54.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.970+0000 7f699612ab80 1 -- 172.21.15.107:0/1633679213 shutdown_connections 2023-07-21T22:32:54.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.970+0000 7f699612ab80 1 --2- 172.21.15.107:0/1633679213 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ce3e7b00e0 0x55ce3e7b24f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:54.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.970+0000 7f699612ab80 1 --2- 172.21.15.107:0/1633679213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce3e746dc0 0x55ce3e755c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:54.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.970+0000 7f699612ab80 1 --2- 172.21.15.107:0/1633679213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce3e745620 0x55ce3e752710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:54.018 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.970+0000 7f699612ab80 1 --2- 172.21.15.107:0/1633679213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ce3e745f70 0x55ce3e752c50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:54.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.970+0000 7f69677f6700 0 client.25816 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:32:54.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.970+0000 7f699612ab80 1 -- 172.21.15.107:0/1633679213 >> 172.21.15.107:0/1633679213 conn(0x55ce3e73aef0 msgr2=0x55ce3e74d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:54.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.971+0000 7f699612ab80 1 -- 172.21.15.107:0/1633679213 shutdown_connections 2023-07-21T22:32:54.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.971+0000 7f699612ab80 1 -- 172.21.15.107:0/1633679213 wait complete. 2023-07-21T22:32:54.019 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.971+0000 7f699612ab80 20 client.25816 trim_cache size 0 max 16384 2023-07-21T22:32:54.020 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:32:54.020 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:32:54.020 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:32:54.020 INFO:teuthology.orchestra.run.smithi107.stdout: "d037412e-e4e0-4d2c-8d6e-c7193371fe0f": { 2023-07-21T22:32:54.020 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:32:54.021 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d037412e-e4e0-4d2c-8d6e-c7193371fe0f", 2023-07-21T22:32:54.021 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:32:54.021 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:32:54.021 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:32:54.021 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:32:54.021 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.972+0000 7f6988d43700 1 -- 172.21.15.107:0/3321427748 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f696c0a13e0 msgr2=0x7f696c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.972+0000 7f6988d43700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f696c0a13e0 0x7f696c0a3890 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f697c007fd0 tx=0x7f697c01a000 comp rx=0 tx=0).stop 2023-07-21T22:32:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.972+0000 7f6988d43700 1 -- 172.21.15.107:0/3321427748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69840711d0 msgr2=0x7f69841a4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.972+0000 7f6988d43700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69840711d0 0x7f69841a4480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f697400fbd0 tx=0x7f697400b8f0 comp rx=0 tx=0).stop 2023-07-21T22:32:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.972+0000 7f6988d43700 1 -- 172.21.15.107:0/3321427748 shutdown_connections 2023-07-21T22:32:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.972+0000 7f6988d43700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410bba0 0x7f69841aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.972+0000 7f6988d43700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f696c0a13e0 0x7f696c0a3890 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:54.022 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.972+0000 7f6988d43700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69840711d0 0x7f69841a4480 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:54.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.972+0000 7f6988d43700 1 --2- 172.21.15.107:0/3321427748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6984071b20 0x7f69841a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:54.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.972+0000 7f6988d43700 1 -- 172.21.15.107:0/3321427748 >> 172.21.15.107:0/3321427748 conn(0x7f698406c8b0 msgr2=0x7f698406dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:54.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.976+0000 7f6988d43700 1 -- 172.21.15.107:0/3321427748 shutdown_connections 2023-07-21T22:32:54.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:32:53.976+0000 7f6988d43700 1 -- 172.21.15.107:0/3321427748 wait complete. 2023-07-21T22:32:54.059 INFO:tasks.cephfs.filesystem:scrub status for tag:d037412e-e4e0-4d2c-8d6e-c7193371fe0f - {'path': '/', 'tag': 'd037412e-e4e0-4d2c-8d6e-c7193371fe0f', 'options': 'recursive,force'} 2023-07-21T22:32:55.646 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:55 smithi190 ceph-mon[98334]: pgmap v409: 97 pgs: 97 active+clean; 746 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 261 op/s 2023-07-21T22:32:55.683 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:55 smithi132 ceph-mon[97863]: pgmap v409: 97 pgs: 97 active+clean; 746 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 261 op/s 2023-07-21T22:32:55.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:55 smithi107 ceph-mon[96543]: pgmap v409: 97 pgs: 97 active+clean; 746 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 261 op/s 2023-07-21T22:32:56.509 INFO:tasks.workunit.client.0.smithi107.stdout: 232 2338192 919 1629838 789 710272 1039 2023-07-21T22:32:56.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:32:56.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:32:56.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:56 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:56.691 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:32:56.691 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:32:56.692 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:56 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:56.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:32:56.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:32:56.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:56 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:32:57.692 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:57 smithi107 ceph-mon[96543]: pgmap v410: 97 pgs: 97 active+clean; 755 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2023-07-21T22:32:57.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:57 smithi190 ceph-mon[98334]: pgmap v410: 97 pgs: 97 active+clean; 755 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2023-07-21T22:32:57.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:57 smithi132 ceph-mon[97863]: pgmap v410: 97 pgs: 97 active+clean; 755 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2023-07-21T22:32:59.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:32:59 smithi190 ceph-mon[98334]: pgmap v411: 97 pgs: 97 active+clean; 860 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 754 op/s 2023-07-21T22:32:59.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:32:59 smithi132 ceph-mon[97863]: pgmap v411: 97 pgs: 97 active+clean; 860 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 754 op/s 2023-07-21T22:32:59.943 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:32:59 smithi107 ceph-mon[96543]: pgmap v411: 97 pgs: 97 active+clean; 860 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 754 op/s 2023-07-21T22:33:01.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:01 smithi190 ceph-mon[98334]: pgmap v412: 97 pgs: 97 active+clean; 865 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 736 op/s 2023-07-21T22:33:01.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:01 smithi132 ceph-mon[97863]: pgmap v412: 97 pgs: 97 active+clean; 865 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 736 op/s 2023-07-21T22:33:01.946 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:01 smithi107 ceph-mon[96543]: pgmap v412: 97 pgs: 97 active+clean; 865 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 736 op/s 2023-07-21T22:33:03.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:03 smithi190 ceph-mon[98334]: pgmap v413: 97 pgs: 97 active+clean; 922 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 16 MiB/s wr, 990 op/s 2023-07-21T22:33:03.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:03 smithi132 ceph-mon[97863]: pgmap v413: 97 pgs: 97 active+clean; 922 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 16 MiB/s wr, 990 op/s 2023-07-21T22:33:03.949 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:03 smithi107 ceph-mon[96543]: pgmap v413: 97 pgs: 97 active+clean; 922 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 16 MiB/s wr, 990 op/s 2023-07-21T22:33:04.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:04 smithi190 ceph-mon[98334]: pgmap v414: 97 pgs: 97 active+clean; 941 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 17 MiB/s wr, 954 op/s 2023-07-21T22:33:04.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:04 smithi132 ceph-mon[97863]: pgmap v414: 97 pgs: 97 active+clean; 941 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 17 MiB/s wr, 954 op/s 2023-07-21T22:33:04.945 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:04 smithi107 ceph-mon[96543]: pgmap v414: 97 pgs: 97 active+clean; 941 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 17 MiB/s wr, 954 op/s 2023-07-21T22:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.228+0000 7f077f44a700 1 -- 172.21.15.107:0/3883877932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07780ff370 msgr2=0x7f07780ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.228+0000 7f077f44a700 1 --2- 172.21.15.107:0/3883877932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07780ff370 0x7f07780ff7c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0768009990 tx=0x7f0768009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.228+0000 7f077f44a700 1 -- 172.21.15.107:0/3883877932 shutdown_connections 2023-07-21T22:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.228+0000 7f077f44a700 1 --2- 172.21.15.107:0/3883877932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07780ffd00 0x7f07781065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.228+0000 7f077f44a700 1 --2- 172.21.15.107:0/3883877932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07780fe180 0x7f07780fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.228+0000 7f077f44a700 1 --2- 172.21.15.107:0/3883877932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07780ff370 0x7f07780ff7c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.228+0000 7f077f44a700 1 -- 172.21.15.107:0/3883877932 >> 172.21.15.107:0/3883877932 conn(0x7f07780f9780 msgr2=0x7f07780fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.228+0000 7f077f44a700 1 -- 172.21.15.107:0/3883877932 shutdown_connections 2023-07-21T22:33:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.228+0000 7f077f44a700 1 -- 172.21.15.107:0/3883877932 wait complete. 2023-07-21T22:33:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077f44a700 1 Processor -- start 2023-07-21T22:33:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077f44a700 1 -- start start 2023-07-21T22:33:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077f44a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07780fe180 0x7f07781936d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077f44a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07780ff370 0x7f0778193c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077f44a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07780ffd00 0x7f0778194a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077f44a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07781950f0 con 0x7f07780ff370 2023-07-21T22:33:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077f44a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0778195260 con 0x7f07780ffd00 2023-07-21T22:33:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077f44a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0778195420 con 0x7f07780fe180 2023-07-21T22:33:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077c9e5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07780ff370 0x7f0778193c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077c9e5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07780ff370 0x7f0778193c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50180/0 (socket says 172.21.15.107:50180) 2023-07-21T22:33:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077c9e5700 1 -- 172.21.15.107:0/4046980135 learned_addr learned my addr 172.21.15.107:0/4046980135 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077d1e6700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07780fe180 0x7f07781936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077d9e7700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07780ffd00 0x7f0778194a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077c9e5700 1 -- 172.21.15.107:0/4046980135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07780fe180 msgr2=0x7f07781936d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077c9e5700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07780fe180 0x7f07781936d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077c9e5700 1 -- 172.21.15.107:0/4046980135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07780ffd00 msgr2=0x7f0778194a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077c9e5700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07780ffd00 0x7f0778194a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077c9e5700 1 -- 172.21.15.107:0/4046980135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0774009520 con 0x7f07780ff370 2023-07-21T22:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.229+0000 7f077d1e6700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07780fe180 0x7f07781936d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.230+0000 7f077c9e5700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07780ff370 0x7f0778193c10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0768000c00 tx=0x7f07680057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.230+0000 7f076e7fc700 1 -- 172.21.15.107:0/4046980135 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076801d070 con 0x7f07780ff370 2023-07-21T22:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.230+0000 7f077f44a700 1 -- 172.21.15.107:0/4046980135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07680095f0 con 0x7f07780ff370 2023-07-21T22:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.230+0000 7f076e7fc700 1 -- 172.21.15.107:0/4046980135 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f076800fb90 con 0x7f07780ff370 2023-07-21T22:33:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.230+0000 7f076e7fc700 1 -- 172.21.15.107:0/4046980135 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680173f0 con 0x7f07780ff370 2023-07-21T22:33:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.230+0000 7f077f44a700 1 -- 172.21.15.107:0/4046980135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077819bce0 con 0x7f07780ff370 2023-07-21T22:33:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.233+0000 7f076e7fc700 1 -- 172.21.15.107:0/4046980135 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0768021410 con 0x7f07780ff370 2023-07-21T22:33:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.233+0000 7f076e7fc700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f07640a11d0 0x7f07640a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.234+0000 7f076e7fc700 1 -- 172.21.15.107:0/4046980135 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f07680cdd10 con 0x7f07780ff370 2023-07-21T22:33:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.234+0000 7f077d1e6700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f07640a11d0 0x7f07640a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.234+0000 7f078b5cfb80 1 Processor -- start 2023-07-21T22:33:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.234+0000 7f078b5cfb80 1 -- start start 2023-07-21T22:33:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.234+0000 7f077d1e6700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f07640a11d0 0x7f07640a3680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0774009fd0 tx=0x7f0774009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.234+0000 7f078b5cfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d4f946b0 0x5653d51382f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.234+0000 7f078b5cfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d5138830 0x5653d5139040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.234+0000 7f078b5cfb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5139580 0x5653d51399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.234+0000 7f078b5cfb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d503dc50 con 0x5653d5139580 2023-07-21T22:33:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.234+0000 7f078b5cfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d50909e0 con 0x5653d5138830 2023-07-21T22:33:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.234+0000 7f078b5cfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d5089e70 con 0x5653d4f946b0 2023-07-21T22:33:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f077d9e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5139580 0x5653d51399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f077c9e5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d5138830 0x5653d5139040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f077d9e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5139580 0x5653d51399f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50188/0 (socket says 172.21.15.107:50188) 2023-07-21T22:33:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f077d9e7700 1 -- 172.21.15.107:0/915427820 learned_addr learned my addr 172.21.15.107:0/915427820 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f077d1e6700 1 --2- 172.21.15.107:0/915427820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d4f946b0 0x5653d51382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f077d9e7700 1 -- 172.21.15.107:0/915427820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d4f946b0 msgr2=0x5653d51382f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f077d9e7700 1 --2- 172.21.15.107:0/915427820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d4f946b0 0x5653d51382f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f077d9e7700 1 -- 172.21.15.107:0/915427820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d5138830 msgr2=0x5653d5139040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f077d9e7700 1 --2- 172.21.15.107:0/915427820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d5138830 0x5653d5139040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f077d9e7700 1 -- 172.21.15.107:0/915427820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d4eb7d80 con 0x5653d5139580 2023-07-21T22:33:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f077c9e5700 1 --2- 172.21.15.107:0/915427820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d5138830 0x5653d5139040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f077d9e7700 1 --2- 172.21.15.107:0/915427820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5139580 0x5653d51399f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f077000e8a0 tx=0x7f0770009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a22868078f38a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f07637fe700 1 -- 172.21.15.107:0/915427820 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770014040 con 0x5653d5139580 2023-07-21T22:33:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f07637fe700 1 -- 172.21.15.107:0/915427820 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07700092e0 con 0x5653d5139580 2023-07-21T22:33:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.235+0000 7f07637fe700 1 -- 172.21.15.107:0/915427820 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770007500 con 0x5653d5139580 2023-07-21T22:33:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.236+0000 7f078b5cfb80 1 -- 172.21.15.107:0/915427820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5139580 msgr2=0x5653d51399f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.236+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/915427820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5139580 0x5653d51399f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f077000e8a0 tx=0x7f0770009920 comp rx=0 tx=0).stop 2023-07-21T22:33:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.236+0000 7f078b5cfb80 1 -- 172.21.15.107:0/915427820 shutdown_connections 2023-07-21T22:33:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.236+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/915427820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5139580 0x5653d51399f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.236+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/915427820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d4f946b0 0x5653d51382f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.236+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/915427820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d5138830 0x5653d5139040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.236+0000 7f078b5cfb80 1 -- 172.21.15.107:0/915427820 >> 172.21.15.107:0/915427820 conn(0x5653d512dae0 msgr2=0x5653d5132c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.236+0000 7f078b5cfb80 1 -- 172.21.15.107:0/915427820 shutdown_connections 2023-07-21T22:33:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.236+0000 7f078b5cfb80 1 -- 172.21.15.107:0/915427820 wait complete. 2023-07-21T22:33:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.237+0000 7f078b5cfb80 1 -- start start 2023-07-21T22:33:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.237+0000 7f078b5cfb80 1 Processor -- start 2023-07-21T22:33:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.237+0000 7f078b5cfb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d4f946b0 0x5653d51417a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.237+0000 7f078b5cfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5138830 0x5653d5141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.237+0000 7f078b5cfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d5139580 0x5653d5148d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.237+0000 7f078b5cfb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d5029a60 con 0x5653d4f946b0 2023-07-21T22:33:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.237+0000 7f078b5cfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d5029760 con 0x5653d5139580 2023-07-21T22:33:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.237+0000 7f078b5cfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d4f0d8c0 con 0x5653d5138830 2023-07-21T22:33:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.237+0000 7f078b5cfb80 10 client.0 _ll_register_callbacks cb 0x5653d5027700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f077d1e6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d4f946b0 0x5653d51417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f077c9e5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5138830 0x5653d5141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f077d9e7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d5139580 0x5653d5148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f077c9e5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5138830 0x5653d5141ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38752/0 (socket says 172.21.15.107:38752) 2023-07-21T22:33:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f077c9e5700 1 -- 172.21.15.107:0/117097336 learned_addr learned my addr 172.21.15.107:0/117097336 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f077c9e5700 1 -- 172.21.15.107:0/117097336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d5139580 msgr2=0x5653d5148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f077c9e5700 1 --2- 172.21.15.107:0/117097336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d5139580 0x5653d5148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f077c9e5700 1 -- 172.21.15.107:0/117097336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d4f946b0 msgr2=0x5653d51417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f077c9e5700 1 --2- 172.21.15.107:0/117097336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d4f946b0 0x5653d51417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f077c9e5700 1 -- 172.21.15.107:0/117097336 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077819bce0 con 0x5653d5138830 2023-07-21T22:33:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f077c9e5700 1 --2- 172.21.15.107:0/117097336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5138830 0x5653d5141ce0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f076809b0c0 tx=0x7f07680219a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9f81aa4575cff10 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f0760ff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:33:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f0760ff9700 1 -- 172.21.15.107:0/117097336 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680b2070 con 0x5653d5138830 2023-07-21T22:33:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f0760ff9700 1 -- 172.21.15.107:0/117097336 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f076809f160 con 0x5653d5138830 2023-07-21T22:33:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f078b5cfb80 1 -- 172.21.15.107:0/117097336 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653d4fe3490 con 0x5653d5138830 2023-07-21T22:33:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f0760ff9700 1 -- 172.21.15.107:0/117097336 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07680a34f0 con 0x5653d5138830 2023-07-21T22:33:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f078b5cfb80 10 client.25828 fetch_fsmap learned FSMap version 36 2023-07-21T22:33:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.238+0000 7f078b5cfb80 1 -- 172.21.15.107:0/117097336 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f077000e550 con 0x5653d5138830 2023-07-21T22:33:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.239+0000 7f0760ff9700 1 -- 172.21.15.107:0/117097336 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5653d4fe3490 con 0x5653d5138830 2023-07-21T22:33:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.239+0000 7f0760ff9700 1 -- 172.21.15.107:0/117097336 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f076809a090 con 0x5653d5138830 2023-07-21T22:33:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.239+0000 7f078b5cfb80 10 client.25828 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:33:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.239+0000 7f078b5cfb80 10 client.25828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.239+0000 7f078b5cfb80 20 client.25828 populate_metadata read hostname 'smithi107' 2023-07-21T22:33:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.239+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/117097336 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5653d514d200 0x5653d514f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.239+0000 7f078b5cfb80 4 client.25828 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.239+0000 7f078b5cfb80 1 -- 172.21.15.107:0/117097336 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653d5082e50 con 0x5653d514d200 2023-07-21T22:33:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.239+0000 7f077d1e6700 1 --2- 172.21.15.107:0/117097336 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5653d514d200 0x5653d514f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.239+0000 7f0760ff9700 10 client.25828 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.239+0000 7f077d1e6700 1 --2- 172.21.15.107:0/117097336 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5653d514d200 0x5653d514f610 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51bd9202a284d6aa server_cookie=1632a9bdd781a358 in_seq=0 out_seq=0 2023-07-21T22:33:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.240+0000 7f0760ff9700 1 -- 172.21.15.107:0/117097336 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5653d5082e50 con 0x5653d514d200 2023-07-21T22:33:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.240+0000 7f0760ff9700 10 client.25828 handle_command_reply: tid=0 2023-07-21T22:33:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.240+0000 7f078b5cfb80 1 client.25828 shutdown 2023-07-21T22:33:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.240+0000 7f078b5cfb80 1 -- 172.21.15.107:0/117097336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5138830 msgr2=0x5653d5141ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.240+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/117097336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5138830 0x5653d5141ce0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f076809b0c0 tx=0x7f07680219a0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.240+0000 7f078b5cfb80 1 -- 172.21.15.107:0/117097336 shutdown_connections 2023-07-21T22:33:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.240+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/117097336 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5653d514d200 0x5653d514f610 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.240+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/117097336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d5139580 0x5653d5148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.240+0000 7f0760ff9700 0 client.25828 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.240+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/117097336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d4f946b0 0x5653d51417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.240+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/117097336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5138830 0x5653d5141ce0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.240+0000 7f078b5cfb80 1 -- 172.21.15.107:0/117097336 >> 172.21.15.107:0/117097336 conn(0x5653d512dae0 msgr2=0x5653d51323e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.240+0000 7f078b5cfb80 1 -- 172.21.15.107:0/117097336 shutdown_connections 2023-07-21T22:33:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.241+0000 7f078b5cfb80 1 -- 172.21.15.107:0/117097336 wait complete. 2023-07-21T22:33:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.241+0000 7f078b5cfb80 20 client.25828 trim_cache size 0 max 16384 2023-07-21T22:33:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.259+0000 7f078b5cfb80 1 Processor -- start 2023-07-21T22:33:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.264+0000 7f078b5cfb80 1 -- start start 2023-07-21T22:33:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.264+0000 7f078b5cfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d4f946b0 0x5653d5132be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.264+0000 7f078b5cfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5138830 0x5653d5148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.264+0000 7f078b5cfb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5139580 0x5653d5153f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.264+0000 7f078b5cfb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d5141370 con 0x5653d5139580 2023-07-21T22:33:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.264+0000 7f078b5cfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d5138c00 con 0x5653d4f946b0 2023-07-21T22:33:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.264+0000 7f078b5cfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d5140c50 con 0x5653d5138830 2023-07-21T22:33:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f077d1e6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d4f946b0 0x5653d5132be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f077d9e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5139580 0x5653d5153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f077d1e6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d4f946b0 0x5653d5132be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41028/0 (socket says 172.21.15.107:41028) 2023-07-21T22:33:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f077d1e6700 1 -- 172.21.15.107:0/3785644594 learned_addr learned my addr 172.21.15.107:0/3785644594 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f077c9e5700 1 --2- 172.21.15.107:0/3785644594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5138830 0x5653d5148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f077c9e5700 1 -- 172.21.15.107:0/3785644594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d4f946b0 msgr2=0x5653d5132be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f077c9e5700 1 --2- 172.21.15.107:0/3785644594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d4f946b0 0x5653d5132be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f077c9e5700 1 -- 172.21.15.107:0/3785644594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5139580 msgr2=0x5653d5153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f077c9e5700 1 --2- 172.21.15.107:0/3785644594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5139580 0x5653d5153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f077c9e5700 1 -- 172.21.15.107:0/3785644594 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d4ea0c00 con 0x5653d5138830 2023-07-21T22:33:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f077c9e5700 1 --2- 172.21.15.107:0/3785644594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5138830 0x5653d5148ce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07680a2fc0 tx=0x7f07680996e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58b7d4b1b6fbf30e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f0763fff700 1 -- 172.21.15.107:0/3785644594 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680b2030 con 0x5653d5138830 2023-07-21T22:33:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f0763fff700 1 -- 172.21.15.107:0/3785644594 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07680a34f0 con 0x5653d5138830 2023-07-21T22:33:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.265+0000 7f0763fff700 1 -- 172.21.15.107:0/3785644594 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680b90d0 con 0x5653d5138830 2023-07-21T22:33:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.266+0000 7f078b5cfb80 1 -- 172.21.15.107:0/3785644594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5138830 msgr2=0x5653d5148ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.266+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/3785644594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5138830 0x5653d5148ce0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07680a2fc0 tx=0x7f07680996e0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.266+0000 7f078b5cfb80 1 -- 172.21.15.107:0/3785644594 shutdown_connections 2023-07-21T22:33:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.266+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/3785644594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5139580 0x5653d5153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.266+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/3785644594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d4f946b0 0x5653d5132be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.266+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/3785644594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5138830 0x5653d5148ce0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.266+0000 7f078b5cfb80 1 -- 172.21.15.107:0/3785644594 >> 172.21.15.107:0/3785644594 conn(0x5653d512dae0 msgr2=0x5653d5140500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.266+0000 7f078b5cfb80 1 -- 172.21.15.107:0/3785644594 shutdown_connections 2023-07-21T22:33:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.266+0000 7f078b5cfb80 1 -- 172.21.15.107:0/3785644594 wait complete. 2023-07-21T22:33:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.267+0000 7f078b5cfb80 1 -- start start 2023-07-21T22:33:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.267+0000 7f078b5cfb80 1 Processor -- start 2023-07-21T22:33:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.267+0000 7f078b5cfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d4f946b0 0x5653d5146e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.267+0000 7f078b5cfb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5138830 0x5653d51473c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.267+0000 7f078b5cfb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5139580 0x5653d51479a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.267+0000 7f078b5cfb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d5148100 con 0x5653d5138830 2023-07-21T22:33:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.267+0000 7f078b5cfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d5148270 con 0x5653d4f946b0 2023-07-21T22:33:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.267+0000 7f078b5cfb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d5148430 con 0x5653d5139580 2023-07-21T22:33:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.267+0000 7f078b5cfb80 10 client.0 _ll_register_callbacks cb 0x5653d5027700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.267+0000 7f077d1e6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d4f946b0 0x5653d5146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.320 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.267+0000 7f077d1e6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d4f946b0 0x5653d5146e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41034/0 (socket says 172.21.15.107:41034) 2023-07-21T22:33:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.267+0000 7f077d1e6700 1 -- 172.21.15.107:0/2357919625 learned_addr learned my addr 172.21.15.107:0/2357919625 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.268+0000 7f077d9e7700 1 --2- 172.21.15.107:0/2357919625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5139580 0x5653d51479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.268+0000 7f077c9e5700 1 --2- 172.21.15.107:0/2357919625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5138830 0x5653d51473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.268+0000 7f077d1e6700 1 -- 172.21.15.107:0/2357919625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5139580 msgr2=0x5653d51479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.268+0000 7f077d1e6700 1 --2- 172.21.15.107:0/2357919625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5139580 0x5653d51479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.268+0000 7f077d1e6700 1 -- 172.21.15.107:0/2357919625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5138830 msgr2=0x5653d51473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.268+0000 7f077d1e6700 1 --2- 172.21.15.107:0/2357919625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5138830 0x5653d51473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.268+0000 7f077d1e6700 1 -- 172.21.15.107:0/2357919625 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077000e550 con 0x5653d4f946b0 2023-07-21T22:33:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.268+0000 7f077d1e6700 1 --2- 172.21.15.107:0/2357919625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d4f946b0 0x5653d5146e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0774013010 tx=0x7f07740047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=873834d6594b1e1a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.268+0000 7f0760ff9700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:33:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.268+0000 7f0760ff9700 1 -- 172.21.15.107:0/2357919625 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774024040 con 0x5653d4f946b0 2023-07-21T22:33:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.268+0000 7f0760ff9700 1 -- 172.21.15.107:0/2357919625 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0774028c80 con 0x5653d4f946b0 2023-07-21T22:33:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.268+0000 7f078b5cfb80 1 -- 172.21.15.107:0/2357919625 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07700092e0 con 0x5653d4f946b0 2023-07-21T22:33:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.269+0000 7f0760ff9700 1 -- 172.21.15.107:0/2357919625 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0774028780 con 0x5653d4f946b0 2023-07-21T22:33:05.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.269+0000 7f078b5cfb80 10 client.25769 fetch_fsmap learned FSMap version 36 2023-07-21T22:33:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.269+0000 7f078b5cfb80 1 -- 172.21.15.107:0/2357919625 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5653d4ede1f0 con 0x5653d4f946b0 2023-07-21T22:33:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.269+0000 7f0760ff9700 1 -- 172.21.15.107:0/2357919625 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07700092e0 con 0x5653d4f946b0 2023-07-21T22:33:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.269+0000 7f0760ff9700 1 -- 172.21.15.107:0/2357919625 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 36) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0774039020 con 0x5653d4f946b0 2023-07-21T22:33:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.269+0000 7f078b5cfb80 10 client.25769 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:33:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.269+0000 7f078b5cfb80 10 client.25769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.269+0000 7f078b5cfb80 20 client.25769 populate_metadata read hostname 'smithi107' 2023-07-21T22:33:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.269+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/2357919625 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5653d51aa4c0 0x5653d51ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.269+0000 7f078b5cfb80 4 client.25769 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:33:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.269+0000 7f078b5cfb80 1 -- 172.21.15.107:0/2357919625 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5653d5044ff0 con 0x5653d51aa4c0 2023-07-21T22:33:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.269+0000 7f077c9e5700 1 --2- 172.21.15.107:0/2357919625 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5653d51aa4c0 0x5653d51ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.270+0000 7f0760ff9700 10 client.25769 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.270+0000 7f077c9e5700 1 --2- 172.21.15.107:0/2357919625 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5653d51aa4c0 0x5653d51ac8d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da646b6283fde27a server_cookie=f2f13a2aaa825b1e in_seq=0 out_seq=0 2023-07-21T22:33:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.572+0000 7f0760ff9700 1 -- 172.21.15.107:0/2357919625 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5653d5044ff0 con 0x5653d51aa4c0 2023-07-21T22:33:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.572+0000 7f0760ff9700 10 client.25769 handle_command_reply: tid=0 2023-07-21T22:33:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.572+0000 7f078b5cfb80 1 client.25769 shutdown 2023-07-21T22:33:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.572+0000 7f078b5cfb80 1 -- 172.21.15.107:0/2357919625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d4f946b0 msgr2=0x5653d5146e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.572+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/2357919625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d4f946b0 0x5653d5146e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0774013010 tx=0x7f07740047c0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.573+0000 7f078b5cfb80 1 -- 172.21.15.107:0/2357919625 shutdown_connections 2023-07-21T22:33:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.573+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/2357919625 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5653d51aa4c0 0x5653d51ac8d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.573+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/2357919625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5653d5139580 0x5653d51479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.573+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/2357919625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5653d4f946b0 0x5653d5146e80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.573+0000 7f0760ff9700 0 client.25769 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.573+0000 7f078b5cfb80 1 --2- 172.21.15.107:0/2357919625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d5138830 0x5653d51473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.573+0000 7f078b5cfb80 1 -- 172.21.15.107:0/2357919625 >> 172.21.15.107:0/2357919625 conn(0x5653d512dae0 msgr2=0x5653d513c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.573+0000 7f078b5cfb80 1 -- 172.21.15.107:0/2357919625 shutdown_connections 2023-07-21T22:33:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.573+0000 7f078b5cfb80 1 -- 172.21.15.107:0/2357919625 wait complete. 2023-07-21T22:33:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.573+0000 7f078b5cfb80 20 client.25769 trim_cache size 0 max 16384 2023-07-21T22:33:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:33:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:33:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:33:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:33:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.574+0000 7f077f44a700 1 -- 172.21.15.107:0/4046980135 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f07640a11d0 msgr2=0x7f07640a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.574+0000 7f077f44a700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f07640a11d0 0x7f07640a3680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0774009fd0 tx=0x7f0774009260 comp rx=0 tx=0).stop 2023-07-21T22:33:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.574+0000 7f077f44a700 1 -- 172.21.15.107:0/4046980135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07780ff370 msgr2=0x7f0778193c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.574+0000 7f077f44a700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07780ff370 0x7f0778193c10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0768000c00 tx=0x7f07680057a0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.574+0000 7f077f44a700 1 -- 172.21.15.107:0/4046980135 shutdown_connections 2023-07-21T22:33:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.574+0000 7f077f44a700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07780ffd00 0x7f0778194a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.574+0000 7f077f44a700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f07640a11d0 0x7f07640a3680 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.574+0000 7f077f44a700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07780fe180 0x7f07781936d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.574+0000 7f077f44a700 1 --2- 172.21.15.107:0/4046980135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07780ff370 0x7f0778193c10 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.574+0000 7f077f44a700 1 -- 172.21.15.107:0/4046980135 >> 172.21.15.107:0/4046980135 conn(0x7f07780f9780 msgr2=0x7f07780fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.574+0000 7f077f44a700 1 -- 172.21.15.107:0/4046980135 shutdown_connections 2023-07-21T22:33:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:05.574+0000 7f077f44a700 1 -- 172.21.15.107:0/4046980135 wait complete. 2023-07-21T22:33:06.518 INFO:tasks.workunit.client.0.smithi107.stdout: 241 2480751 619 1737904 555 781516 1210 2023-07-21T22:33:07.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:07 smithi190 ceph-mon[98334]: pgmap v415: 97 pgs: 97 active+clean; 941 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 17 MiB/s wr, 940 op/s 2023-07-21T22:33:07.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:07 smithi132 ceph-mon[97863]: pgmap v415: 97 pgs: 97 active+clean; 941 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 17 MiB/s wr, 940 op/s 2023-07-21T22:33:07.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:07 smithi107 ceph-mon[96543]: pgmap v415: 97 pgs: 97 active+clean; 941 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 17 MiB/s wr, 940 op/s 2023-07-21T22:33:09.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:09 smithi190 ceph-mon[98334]: pgmap v416: 97 pgs: 97 active+clean; 945 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 17 MiB/s wr, 1.01k op/s 2023-07-21T22:33:09.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:09 smithi132 ceph-mon[97863]: pgmap v416: 97 pgs: 97 active+clean; 945 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 17 MiB/s wr, 1.01k op/s 2023-07-21T22:33:09.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:09 smithi107 ceph-mon[96543]: pgmap v416: 97 pgs: 97 active+clean; 945 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 17 MiB/s wr, 1.01k op/s 2023-07-21T22:33:11.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:11 smithi190 ceph-mon[98334]: pgmap v417: 97 pgs: 97 active+clean; 947 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.5 MiB/s wr, 586 op/s 2023-07-21T22:33:11.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:11 smithi132 ceph-mon[97863]: pgmap v417: 97 pgs: 97 active+clean; 947 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.5 MiB/s wr, 586 op/s 2023-07-21T22:33:11.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:11 smithi107 ceph-mon[96543]: pgmap v417: 97 pgs: 97 active+clean; 947 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.5 MiB/s wr, 586 op/s 2023-07-21T22:33:13.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:13 smithi190 ceph-mon[98334]: pgmap v418: 97 pgs: 97 active+clean; 952 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.6 MiB/s wr, 636 op/s 2023-07-21T22:33:13.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:13 smithi132 ceph-mon[97863]: pgmap v418: 97 pgs: 97 active+clean; 952 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.6 MiB/s wr, 636 op/s 2023-07-21T22:33:13.691 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:13 smithi107 ceph-mon[96543]: pgmap v418: 97 pgs: 97 active+clean; 952 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.6 MiB/s wr, 636 op/s 2023-07-21T22:33:15.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:15 smithi190 ceph-mon[98334]: pgmap v419: 97 pgs: 97 active+clean; 951 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 396 op/s 2023-07-21T22:33:15.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:15 smithi132 ceph-mon[97863]: pgmap v419: 97 pgs: 97 active+clean; 951 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 396 op/s 2023-07-21T22:33:15.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:15 smithi107 ceph-mon[96543]: pgmap v419: 97 pgs: 97 active+clean; 951 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 396 op/s 2023-07-21T22:33:16.518 INFO:tasks.workunit.client.0.smithi107.stdout: 251 2712744 639 1906388 688 921329 1379 2023-07-21T22:33:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:17 smithi190 ceph-mon[98334]: pgmap v420: 97 pgs: 97 active+clean; 956 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 314 op/s 2023-07-21T22:33:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:17 smithi132 ceph-mon[97863]: pgmap v420: 97 pgs: 97 active+clean; 956 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 314 op/s 2023-07-21T22:33:17.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:17 smithi107 ceph-mon[96543]: pgmap v420: 97 pgs: 97 active+clean; 956 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 314 op/s 2023-07-21T22:33:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.828+0000 7f6f57a37700 1 -- 172.21.15.107:0/2563654913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f500fe180 msgr2=0x7f6f500fe590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.828+0000 7f6f57a37700 1 --2- 172.21.15.107:0/2563654913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f500fe180 0x7f6f500fe590 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00aaa0 tx=0x7f6f4c00adb0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.828+0000 7f6f57a37700 1 -- 172.21.15.107:0/2563654913 shutdown_connections 2023-07-21T22:33:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.828+0000 7f6f57a37700 1 --2- 172.21.15.107:0/2563654913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f500ffd00 0x7f6f501065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.828+0000 7f6f57a37700 1 --2- 172.21.15.107:0/2563654913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f500fe180 0x7f6f500fe590 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.828+0000 7f6f57a37700 1 --2- 172.21.15.107:0/2563654913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f500ff370 0x7f6f500ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.828+0000 7f6f57a37700 1 -- 172.21.15.107:0/2563654913 >> 172.21.15.107:0/2563654913 conn(0x7f6f500f9780 msgr2=0x7f6f500fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.828+0000 7f6f57a37700 1 -- 172.21.15.107:0/2563654913 shutdown_connections 2023-07-21T22:33:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.828+0000 7f6f57a37700 1 -- 172.21.15.107:0/2563654913 wait complete. 2023-07-21T22:33:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.829+0000 7f6f57a37700 1 Processor -- start 2023-07-21T22:33:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.829+0000 7f6f57a37700 1 -- start start 2023-07-21T22:33:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f57a37700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f500fe180 0x7f6f50100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f57a37700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f500ff370 0x7f6f50100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f57a37700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f500ffd00 0x7f6f501011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f57a37700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5019f8a0 con 0x7f6f500ff370 2023-07-21T22:33:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f57a37700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5019f9e0 con 0x7f6f500fe180 2023-07-21T22:33:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f57a37700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5019fb20 con 0x7f6f500ffd00 2023-07-21T22:33:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f557d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f500fe180 0x7f6f50100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f55fd4700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f500ffd00 0x7f6f501011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f557d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f500fe180 0x7f6f50100750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41050/0 (socket says 172.21.15.107:41050) 2023-07-21T22:33:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f557d3700 1 -- 172.21.15.107:0/574299214 learned_addr learned my addr 172.21.15.107:0/574299214 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f557d3700 1 -- 172.21.15.107:0/574299214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f500ffd00 msgr2=0x7f6f501011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f557d3700 1 --2- 172.21.15.107:0/574299214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f500ffd00 0x7f6f501011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f557d3700 1 -- 172.21.15.107:0/574299214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f500ff370 msgr2=0x7f6f50100c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:33:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f557d3700 1 --2- 172.21.15.107:0/574299214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f500ff370 0x7f6f50100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f557d3700 1 -- 172.21.15.107:0/574299214 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f40009520 con 0x7f6f500fe180 2023-07-21T22:33:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.830+0000 7f6f55fd4700 1 --2- 172.21.15.107:0/574299214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f500ffd00 0x7f6f501011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.831+0000 7f6f557d3700 1 --2- 172.21.15.107:0/574299214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f500fe180 0x7f6f50100750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c003b00 tx=0x7f6f4c004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.831+0000 7f6f467fc700 1 -- 172.21.15.107:0/574299214 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c0051d0 con 0x7f6f500fe180 2023-07-21T22:33:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.831+0000 7f6f57a37700 1 -- 172.21.15.107:0/574299214 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f48009c40 con 0x7f6f500fe180 2023-07-21T22:33:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.831+0000 7f6f467fc700 1 -- 172.21.15.107:0/574299214 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f4c00e450 con 0x7f6f500fe180 2023-07-21T22:33:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.831+0000 7f6f467fc700 1 -- 172.21.15.107:0/574299214 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c016680 con 0x7f6f500fe180 2023-07-21T22:33:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.831+0000 7f6f57a37700 1 -- 172.21.15.107:0/574299214 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4c00a700 con 0x7f6f500fe180 2023-07-21T22:33:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.837+0000 7f6f63bbcb80 1 Processor -- start 2023-07-21T22:33:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.839+0000 7f6f63bbcb80 1 -- start start 2023-07-21T22:33:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.839+0000 7f6f63bbcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 0x55cf997dab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.839+0000 7f6f63bbcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997db040 0x55cf997db850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.839+0000 7f6f63bbcb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 0x55cf997dc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.839+0000 7f6f63bbcb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf996d7c50 con 0x55cf997dbd90 2023-07-21T22:33:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.839+0000 7f6f63bbcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9972a9e0 con 0x55cf9962e6b0 2023-07-21T22:33:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.839+0000 7f6f63bbcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf99723e70 con 0x55cf997db040 2023-07-21T22:33:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.839+0000 7f6f55fd4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 0x55cf997dab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.840+0000 7f6f55fd4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 0x55cf997dab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41056/0 (socket says 172.21.15.107:41056) 2023-07-21T22:33:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.840+0000 7f6f55fd4700 1 -- 172.21.15.107:0/2345738493 learned_addr learned my addr 172.21.15.107:0/2345738493 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.840+0000 7f6f467fc700 1 -- 172.21.15.107:0/574299214 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6f4c00e5c0 con 0x7f6f500fe180 2023-07-21T22:33:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.841+0000 7f6f467fc700 1 --2- 172.21.15.107:0/574299214 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f3c0a11d0 0x7f6f3c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.842+0000 7f6f467fc700 1 -- 172.21.15.107:0/574299214 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6f4c012070 con 0x7f6f500fe180 2023-07-21T22:33:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.845+0000 7f6f55fd4700 1 -- 172.21.15.107:0/2345738493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997db040 msgr2=0x55cf997db850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.845+0000 7f6f55fd4700 1 --2- 172.21.15.107:0/2345738493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997db040 0x55cf997db850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.845+0000 7f6f55fd4700 1 -- 172.21.15.107:0/2345738493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 msgr2=0x55cf997dc200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:33:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.845+0000 7f6f55fd4700 1 --2- 172.21.15.107:0/2345738493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 0x55cf997dc200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.845+0000 7f6f55fd4700 1 -- 172.21.15.107:0/2345738493 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf99551d80 con 0x55cf9962e6b0 2023-07-21T22:33:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.845+0000 7f6f55fd4700 1 --2- 172.21.15.107:0/2345738493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 0x55cf997dab00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6f48007610 tx=0x7f6f48009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1566111fea44aada server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.846+0000 7f6f55fd4700 1 --2- 172.21.15.107:0/574299214 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f3c0a11d0 0x7f6f3c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.847+0000 7f6f55fd4700 1 --2- 172.21.15.107:0/574299214 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f3c0a11d0 0x7f6f3c0a3680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6f48017fd0 tx=0x7f6f48015700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.849+0000 7f6f3b7fe700 1 -- 172.21.15.107:0/2345738493 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4800e900 con 0x55cf9962e6b0 2023-07-21T22:33:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.849+0000 7f6f3b7fe700 1 -- 172.21.15.107:0/2345738493 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f4800f040 con 0x55cf9962e6b0 2023-07-21T22:33:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.855+0000 7f6f3b7fe700 1 -- 172.21.15.107:0/2345738493 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48015410 con 0x55cf9962e6b0 2023-07-21T22:33:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.856+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/2345738493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 msgr2=0x55cf997dab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.856+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/2345738493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 0x55cf997dab00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6f48007610 tx=0x7f6f48009920 comp rx=0 tx=0).stop 2023-07-21T22:33:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.856+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/2345738493 shutdown_connections 2023-07-21T22:33:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.856+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/2345738493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 0x55cf997dc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.856+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/2345738493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 0x55cf997dab00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.856+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/2345738493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997db040 0x55cf997db850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.856+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/2345738493 >> 172.21.15.107:0/2345738493 conn(0x55cf997c7ae0 msgr2=0x55cf997ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.857+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/2345738493 shutdown_connections 2023-07-21T22:33:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.861+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/2345738493 wait complete. 2023-07-21T22:33:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.861+0000 7f6f63bbcb80 1 -- start start 2023-07-21T22:33:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.861+0000 7f6f63bbcb80 1 Processor -- start 2023-07-21T22:33:18.910 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.863+0000 7f6f63bbcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf9962e6b0 0x55cf997e4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.863+0000 7f6f63bbcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf997db040 0x55cf997e4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.863+0000 7f6f63bbcb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 0x55cf997eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.863+0000 7f6f63bbcb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf996c3a60 con 0x55cf997dbd90 2023-07-21T22:33:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.863+0000 7f6f63bbcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf996c3760 con 0x55cf997db040 2023-07-21T22:33:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.863+0000 7f6f63bbcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf995a78c0 con 0x55cf9962e6b0 2023-07-21T22:33:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.863+0000 7f6f63bbcb80 10 client.0 _ll_register_callbacks cb 0x55cf996c1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.864+0000 7f6f55fd4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 0x55cf997eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.864+0000 7f6f55fd4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 0x55cf997eb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50238/0 (socket says 172.21.15.107:50238) 2023-07-21T22:33:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.864+0000 7f6f55fd4700 1 -- 172.21.15.107:0/621915205 learned_addr learned my addr 172.21.15.107:0/621915205 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.866+0000 7f6f54fd2700 1 --2- 172.21.15.107:0/621915205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf997db040 0x55cf997e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.866+0000 7f6f55fd4700 1 -- 172.21.15.107:0/621915205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf9962e6b0 msgr2=0x55cf997e4040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:33:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.866+0000 7f6f55fd4700 1 --2- 172.21.15.107:0/621915205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf9962e6b0 0x55cf997e4040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.866+0000 7f6f55fd4700 1 -- 172.21.15.107:0/621915205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf997db040 msgr2=0x55cf997e4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.866+0000 7f6f55fd4700 1 --2- 172.21.15.107:0/621915205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf997db040 0x55cf997e4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.866+0000 7f6f55fd4700 1 -- 172.21.15.107:0/621915205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f48009260 con 0x55cf997dbd90 2023-07-21T22:33:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.866+0000 7f6f54fd2700 1 --2- 172.21.15.107:0/621915205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf997db040 0x55cf997e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:33:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.866+0000 7f6f55fd4700 1 --2- 172.21.15.107:0/621915205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 0x55cf997eb5c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6f48017b20 tx=0x7f6f48015630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc5877e1b6f4435 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.880+0000 7f6f38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:33:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.880+0000 7f6f38ff9700 1 -- 172.21.15.107:0/621915205 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48025070 con 0x55cf997dbd90 2023-07-21T22:33:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.880+0000 7f6f38ff9700 1 -- 172.21.15.107:0/621915205 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6f48015b50 con 0x55cf997dbd90 2023-07-21T22:33:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.888+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/621915205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf9967d490 con 0x55cf997dbd90 2023-07-21T22:33:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.890+0000 7f6f38ff9700 1 -- 172.21.15.107:0/621915205 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f48015410 con 0x55cf997dbd90 2023-07-21T22:33:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.932+0000 7f6f63bbcb80 10 client.16290 fetch_fsmap learned FSMap version 36 2023-07-21T22:33:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.932+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/621915205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55cf9953ac00 con 0x55cf997dbd90 2023-07-21T22:33:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.943+0000 7f6f38ff9700 1 -- 172.21.15.107:0/621915205 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cf9967d490 con 0x55cf997dbd90 2023-07-21T22:33:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.943+0000 7f6f38ff9700 1 -- 172.21.15.107:0/621915205 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f6f4800e700 con 0x55cf997dbd90 2023-07-21T22:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.944+0000 7f6f63bbcb80 10 client.16290 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.944+0000 7f6f63bbcb80 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.944+0000 7f6f63bbcb80 20 client.16290 populate_metadata read hostname 'smithi107' 2023-07-21T22:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.944+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/621915205 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cf99800d40 0x55cf99803150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.944+0000 7f6f63bbcb80 4 client.16290 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.944+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/621915205 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf9971ce50 con 0x55cf99800d40 2023-07-21T22:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.944+0000 7f6f557d3700 1 --2- 172.21.15.107:0/621915205 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cf99800d40 0x55cf99803150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.978+0000 7f6f557d3700 1 --2- 172.21.15.107:0/621915205 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cf99800d40 0x55cf99803150 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa132a8cf879a6b server_cookie=91741b41f58507d2 in_seq=0 out_seq=0 2023-07-21T22:33:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:18.978+0000 7f6f38ff9700 10 client.16290 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.000+0000 7f6f38ff9700 1 -- 172.21.15.107:0/621915205 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cf9971ce50 con 0x55cf99800d40 2023-07-21T22:33:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.000+0000 7f6f38ff9700 10 client.16290 handle_command_reply: tid=0 2023-07-21T22:33:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.000+0000 7f6f63bbcb80 1 client.16290 shutdown 2023-07-21T22:33:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.000+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/621915205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 msgr2=0x55cf997eb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.000+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/621915205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 0x55cf997eb5c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6f48017b20 tx=0x7f6f48015630 comp rx=0 tx=0).stop 2023-07-21T22:33:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.001+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/621915205 shutdown_connections 2023-07-21T22:33:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.001+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/621915205 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cf99800d40 0x55cf99803150 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.001+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/621915205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 0x55cf997eb5c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.001+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/621915205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf9962e6b0 0x55cf997e4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.001+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/621915205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf997db040 0x55cf997e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.001+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/621915205 >> 172.21.15.107:0/621915205 conn(0x55cf997c7ae0 msgr2=0x55cf997cc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.001+0000 7f6f38ff9700 0 client.16290 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.001+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/621915205 shutdown_connections 2023-07-21T22:33:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.001+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/621915205 wait complete. 2023-07-21T22:33:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.002+0000 7f6f63bbcb80 20 client.16290 trim_cache size 0 max 16384 2023-07-21T22:33:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.151+0000 7f6f63bbcb80 1 Processor -- start 2023-07-21T22:33:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.157+0000 7f6f63bbcb80 1 -- start start 2023-07-21T22:33:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.157+0000 7f6f63bbcb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf9962e6b0 0x55cf997dec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.157+0000 7f6f63bbcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf997db040 0x55cf997eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.157+0000 7f6f63bbcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997dbd90 0x55cf997df140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.157+0000 7f6f63bbcb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf997e3c10 con 0x55cf9962e6b0 2023-07-21T22:33:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.157+0000 7f6f63bbcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf9967d4c0 con 0x55cf997db040 2023-07-21T22:33:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.157+0000 7f6f63bbcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf997ce440 con 0x55cf997dbd90 2023-07-21T22:33:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.158+0000 7f6f54fd2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf997db040 0x55cf997eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.158+0000 7f6f54fd2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf997db040 0x55cf997eb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41072/0 (socket says 172.21.15.107:41072) 2023-07-21T22:33:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.158+0000 7f6f54fd2700 1 -- 172.21.15.107:0/2783057040 learned_addr learned my addr 172.21.15.107:0/2783057040 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.158+0000 7f6f55fd4700 1 --2- 172.21.15.107:0/2783057040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997dbd90 0x55cf997df140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.159+0000 7f6f55fd4700 1 -- 172.21.15.107:0/2783057040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf997db040 msgr2=0x55cf997eb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.159+0000 7f6f55fd4700 1 --2- 172.21.15.107:0/2783057040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf997db040 0x55cf997eb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.159+0000 7f6f55fd4700 1 -- 172.21.15.107:0/2783057040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf9962e6b0 msgr2=0x55cf997dec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.159+0000 7f6f55fd4700 1 --2- 172.21.15.107:0/2783057040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf9962e6b0 0x55cf997dec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.159+0000 7f6f55fd4700 1 -- 172.21.15.107:0/2783057040 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf995781f0 con 0x55cf997dbd90 2023-07-21T22:33:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.159+0000 7f6f55fd4700 1 --2- 172.21.15.107:0/2783057040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997dbd90 0x55cf997df140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6f48002230 tx=0x7f6f480177f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec2b394462daa1a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:19.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.173+0000 7f6f3bfff700 1 -- 172.21.15.107:0/2783057040 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48044070 con 0x55cf997dbd90 2023-07-21T22:33:19.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.173+0000 7f6f3bfff700 1 -- 172.21.15.107:0/2783057040 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f48030070 con 0x55cf997dbd90 2023-07-21T22:33:19.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.173+0000 7f6f3bfff700 1 -- 172.21.15.107:0/2783057040 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48015c60 con 0x55cf997dbd90 2023-07-21T22:33:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.198+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/2783057040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997dbd90 msgr2=0x55cf997df140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.198+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/2783057040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997dbd90 0x55cf997df140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6f48002230 tx=0x7f6f480177f0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.214+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/2783057040 shutdown_connections 2023-07-21T22:33:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.214+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/2783057040 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997dbd90 0x55cf997df140 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.214+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/2783057040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf9962e6b0 0x55cf997dec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.214+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/2783057040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf997db040 0x55cf997eb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.214+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/2783057040 >> 172.21.15.107:0/2783057040 conn(0x55cf997c7ae0 msgr2=0x55cf99807ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.215+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/2783057040 shutdown_connections 2023-07-21T22:33:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.215+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/2783057040 wait complete. 2023-07-21T22:33:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.215+0000 7f6f63bbcb80 1 -- start start 2023-07-21T22:33:19.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.215+0000 7f6f63bbcb80 1 Processor -- start 2023-07-21T22:33:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.216+0000 7f6f63bbcb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 0x55cf997e7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.216+0000 7f6f63bbcb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997db040 0x55cf997e9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.216+0000 7f6f63bbcb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 0x55cf997ea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.216+0000 7f6f63bbcb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf997ea960 con 0x55cf997dbd90 2023-07-21T22:33:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.216+0000 7f6f63bbcb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf997eaad0 con 0x55cf9962e6b0 2023-07-21T22:33:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.216+0000 7f6f63bbcb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf997eac90 con 0x55cf997db040 2023-07-21T22:33:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.216+0000 7f6f63bbcb80 10 client.0 _ll_register_callbacks cb 0x55cf996c1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.220+0000 7f6f54fd2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997db040 0x55cf997e9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.220+0000 7f6f557d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 0x55cf997e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.220+0000 7f6f557d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 0x55cf997e7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41076/0 (socket says 172.21.15.107:41076) 2023-07-21T22:33:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.220+0000 7f6f557d3700 1 -- 172.21.15.107:0/133735934 learned_addr learned my addr 172.21.15.107:0/133735934 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.221+0000 7f6f557d3700 1 -- 172.21.15.107:0/133735934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997db040 msgr2=0x55cf997e9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.221+0000 7f6f557d3700 1 --2- 172.21.15.107:0/133735934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997db040 0x55cf997e9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.221+0000 7f6f557d3700 1 -- 172.21.15.107:0/133735934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 msgr2=0x55cf997ea290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.221+0000 7f6f557d3700 1 --2- 172.21.15.107:0/133735934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 0x55cf997ea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.221+0000 7f6f557d3700 1 -- 172.21.15.107:0/133735934 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf9953ac00 con 0x55cf9962e6b0 2023-07-21T22:33:19.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.221+0000 7f6f557d3700 1 --2- 172.21.15.107:0/133735934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 0x55cf997e7710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c097e00 tx=0x7f6f4c09b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67de20d4b104711 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.227+0000 7f6f38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:33:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.227+0000 7f6f38ff9700 1 -- 172.21.15.107:0/133735934 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c0b9ea0 con 0x55cf9962e6b0 2023-07-21T22:33:19.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.227+0000 7f6f38ff9700 1 -- 172.21.15.107:0/133735934 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6f4c09b900 con 0x55cf9962e6b0 2023-07-21T22:33:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.229+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/133735934 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f4800f040 con 0x55cf9962e6b0 2023-07-21T22:33:19.285 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.238+0000 7f6f38ff9700 1 -- 172.21.15.107:0/133735934 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f4c09f930 con 0x55cf9962e6b0 2023-07-21T22:33:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.244+0000 7f6f63bbcb80 10 client.25784 fetch_fsmap learned FSMap version 36 2023-07-21T22:33:19.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.244+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/133735934 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55cf99550250 con 0x55cf9962e6b0 2023-07-21T22:33:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.246+0000 7f6f38ff9700 1 -- 172.21.15.107:0/133735934 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f4800f040 con 0x55cf9962e6b0 2023-07-21T22:33:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.247+0000 7f6f38ff9700 1 -- 172.21.15.107:0/133735934 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 36) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f6f4c09bb60 con 0x55cf9962e6b0 2023-07-21T22:33:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.247+0000 7f6f63bbcb80 10 client.25784 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:33:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.247+0000 7f6f63bbcb80 10 client.25784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.247+0000 7f6f63bbcb80 20 client.25784 populate_metadata read hostname 'smithi107' 2023-07-21T22:33:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.248+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/133735934 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cf997d18f0 0x55cf997d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.248+0000 7f6f63bbcb80 4 client.25784 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:33:19.295 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.248+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/133735934 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cf996deff0 con 0x55cf997d18f0 2023-07-21T22:33:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.249+0000 7f6f54fd2700 1 --2- 172.21.15.107:0/133735934 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cf997d18f0 0x55cf997d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.249+0000 7f6f54fd2700 1 --2- 172.21.15.107:0/133735934 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cf997d18f0 0x55cf997d3d00 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9894d8584942d9 server_cookie=5840353fddd0859f in_seq=0 out_seq=0 2023-07-21T22:33:19.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.249+0000 7f6f38ff9700 10 client.25784 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:19.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:19 smithi132 ceph-mon[97863]: pgmap v421: 97 pgs: 97 active+clean; 960 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 331 op/s 2023-07-21T22:33:19.742 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:19 smithi107 ceph-mon[96543]: pgmap v421: 97 pgs: 97 active+clean; 960 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 331 op/s 2023-07-21T22:33:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.748+0000 7f6f38ff9700 1 -- 172.21.15.107:0/133735934 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cf996deff0 con 0x55cf997d18f0 2023-07-21T22:33:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.748+0000 7f6f38ff9700 10 client.25784 handle_command_reply: tid=0 2023-07-21T22:33:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.748+0000 7f6f63bbcb80 1 client.25784 shutdown 2023-07-21T22:33:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.751+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/133735934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 msgr2=0x55cf997e7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.751+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/133735934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 0x55cf997e7710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c097e00 tx=0x7f6f4c09b260 comp rx=0 tx=0).stop 2023-07-21T22:33:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.751+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/133735934 shutdown_connections 2023-07-21T22:33:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.751+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/133735934 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55cf997d18f0 0x55cf997d3d00 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.751+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/133735934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cf997dbd90 0x55cf997ea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.751+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/133735934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cf9962e6b0 0x55cf997e7710 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.751+0000 7f6f63bbcb80 1 --2- 172.21.15.107:0/133735934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cf997db040 0x55cf997e9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.751+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/133735934 >> 172.21.15.107:0/133735934 conn(0x55cf997c7ae0 msgr2=0x55cf997dec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.751+0000 7f6f38ff9700 0 client.25784 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.751+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/133735934 shutdown_connections 2023-07-21T22:33:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.751+0000 7f6f63bbcb80 1 -- 172.21.15.107:0/133735934 wait complete. 2023-07-21T22:33:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.752+0000 7f6f63bbcb80 20 client.25784 trim_cache size 0 max 16384 2023-07-21T22:33:19.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:33:19.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:33:19.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:33:19.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:33:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.796+0000 7f6f57a37700 1 -- 172.21.15.107:0/574299214 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f3c0a11d0 msgr2=0x7f6f3c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.796+0000 7f6f57a37700 1 --2- 172.21.15.107:0/574299214 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f3c0a11d0 0x7f6f3c0a3680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6f48017fd0 tx=0x7f6f48015700 comp rx=0 tx=0).stop 2023-07-21T22:33:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.796+0000 7f6f57a37700 1 -- 172.21.15.107:0/574299214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f500fe180 msgr2=0x7f6f50100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.796+0000 7f6f57a37700 1 --2- 172.21.15.107:0/574299214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f500fe180 0x7f6f50100750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c003b00 tx=0x7f6f4c004ce0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.797+0000 7f6f57a37700 1 -- 172.21.15.107:0/574299214 shutdown_connections 2023-07-21T22:33:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.797+0000 7f6f57a37700 1 --2- 172.21.15.107:0/574299214 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6f3c0a11d0 0x7f6f3c0a3680 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.797+0000 7f6f57a37700 1 --2- 172.21.15.107:0/574299214 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f500ffd00 0x7f6f501011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.797+0000 7f6f57a37700 1 --2- 172.21.15.107:0/574299214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6f500fe180 0x7f6f50100750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.797+0000 7f6f57a37700 1 --2- 172.21.15.107:0/574299214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f500ff370 0x7f6f50100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.797+0000 7f6f57a37700 1 -- 172.21.15.107:0/574299214 >> 172.21.15.107:0/574299214 conn(0x7f6f500f9780 msgr2=0x7f6f500fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.797+0000 7f6f57a37700 1 -- 172.21.15.107:0/574299214 shutdown_connections 2023-07-21T22:33:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:19.797+0000 7f6f57a37700 1 -- 172.21.15.107:0/574299214 wait complete. 2023-07-21T22:33:19.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:19 smithi190 ceph-mon[98334]: pgmap v421: 97 pgs: 97 active+clean; 960 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 331 op/s 2023-07-21T22:33:21.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:21 smithi190 ceph-mon[98334]: pgmap v422: 97 pgs: 97 active+clean; 960 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2023-07-21T22:33:21.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:21 smithi132 ceph-mon[97863]: pgmap v422: 97 pgs: 97 active+clean; 960 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2023-07-21T22:33:21.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:21 smithi107 ceph-mon[96543]: pgmap v422: 97 pgs: 97 active+clean; 960 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2023-07-21T22:33:23.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:23 smithi190 ceph-mon[98334]: pgmap v423: 97 pgs: 97 active+clean; 962 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 286 op/s 2023-07-21T22:33:23.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:23 smithi132 ceph-mon[97863]: pgmap v423: 97 pgs: 97 active+clean; 962 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 286 op/s 2023-07-21T22:33:23.943 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:23 smithi107 ceph-mon[96543]: pgmap v423: 97 pgs: 97 active+clean; 962 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 286 op/s 2023-07-21T22:33:24.061 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:33:25.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:25 smithi190 ceph-mon[98334]: pgmap v424: 97 pgs: 97 active+clean; 966 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 275 op/s 2023-07-21T22:33:25.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:25 smithi132 ceph-mon[97863]: pgmap v424: 97 pgs: 97 active+clean; 966 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 275 op/s 2023-07-21T22:33:25.949 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:25 smithi107 ceph-mon[96543]: pgmap v424: 97 pgs: 97 active+clean; 966 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 275 op/s 2023-07-21T22:33:26.527 INFO:tasks.workunit.client.0.smithi107.stdout: 260 2640139 682 1870371 529 963859 1433 2023-07-21T22:33:27.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.185+0000 7f27e7fff700 1 -- 172.21.15.107:0/2965869523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27e80711d0 msgr2=0x7f27e80715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:27.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.185+0000 7f27e7fff700 1 --2- 172.21.15.107:0/2965869523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27e80711d0 0x7f27e80715e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009f90 tx=0x7f27e000b330 comp rx=0 tx=0).stop 2023-07-21T22:33:27.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.199+0000 7f27e7fff700 1 -- 172.21.15.107:0/2965869523 shutdown_connections 2023-07-21T22:33:27.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.199+0000 7f27e7fff700 1 --2- 172.21.15.107:0/2965869523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e810bba0 0x7f27e810e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.199+0000 7f27e7fff700 1 --2- 172.21.15.107:0/2965869523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27e80711d0 0x7f27e80715e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.199+0000 7f27e7fff700 1 --2- 172.21.15.107:0/2965869523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27e8071b20 0x7f27e810b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.199+0000 7f27e7fff700 1 -- 172.21.15.107:0/2965869523 >> 172.21.15.107:0/2965869523 conn(0x7f27e806c8b0 msgr2=0x7f27e806ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:27.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.208+0000 7f27e7fff700 1 -- 172.21.15.107:0/2965869523 shutdown_connections 2023-07-21T22:33:27.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.210+0000 7f27e7fff700 1 -- 172.21.15.107:0/2965869523 wait complete. 2023-07-21T22:33:27.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.210+0000 7f27e7fff700 1 Processor -- start 2023-07-21T22:33:27.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.211+0000 7f27e7fff700 1 -- start start 2023-07-21T22:33:27.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.211+0000 7f27e7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27e80711d0 0x7f27e8113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.275 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.211+0000 7f27e7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e8071b20 0x7f27e8113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.211+0000 7f27e7fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27e810bba0 0x7f27e81a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.211+0000 7f27e7fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e8117050 con 0x7f27e8071b20 2023-07-21T22:33:27.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.211+0000 7f27e7fff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e81171c0 con 0x7f27e80711d0 2023-07-21T22:33:27.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.211+0000 7f27e7fff700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e81a3b90 con 0x7f27e810bba0 2023-07-21T22:33:27.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.213+0000 7f27e77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27e810bba0 0x7f27e81a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.221+0000 7f27e6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27e80711d0 0x7f27e8113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.221+0000 7f27e6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27e80711d0 0x7f27e8113a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41094/0 (socket says 172.21.15.107:41094) 2023-07-21T22:33:27.276 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.221+0000 7f27e6ffd700 1 -- 172.21.15.107:0/961172116 learned_addr learned my addr 172.21.15.107:0/961172116 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:27.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.222+0000 7f27e6ffd700 1 -- 172.21.15.107:0/961172116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27e810bba0 msgr2=0x7f27e81a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:27.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.222+0000 7f27e6ffd700 1 --2- 172.21.15.107:0/961172116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27e810bba0 0x7f27e81a3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.222+0000 7f27e6ffd700 1 -- 172.21.15.107:0/961172116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e8071b20 msgr2=0x7f27e8113f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:27.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.222+0000 7f27e6ffd700 1 --2- 172.21.15.107:0/961172116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e8071b20 0x7f27e8113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.222+0000 7f27e6ffd700 1 -- 172.21.15.107:0/961172116 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e0009c40 con 0x7f27e80711d0 2023-07-21T22:33:27.277 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.222+0000 7f27e6ffd700 1 --2- 172.21.15.107:0/961172116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27e80711d0 0x7f27e8113a30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f27e000b6c0 tx=0x7f27e0004d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:27.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.227+0000 7f27ec89d700 1 -- 172.21.15.107:0/961172116 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e000e040 con 0x7f27e80711d0 2023-07-21T22:33:27.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.227+0000 7f27ec89d700 1 -- 172.21.15.107:0/961172116 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27e000be00 con 0x7f27e80711d0 2023-07-21T22:33:27.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.227+0000 7f27ec89d700 1 -- 172.21.15.107:0/961172116 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e001b860 con 0x7f27e80711d0 2023-07-21T22:33:27.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.227+0000 7f27e7fff700 1 -- 172.21.15.107:0/961172116 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e81a3db0 con 0x7f27e80711d0 2023-07-21T22:33:27.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.227+0000 7f27e7fff700 1 -- 172.21.15.107:0/961172116 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e81a4210 con 0x7f27e80711d0 2023-07-21T22:33:27.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.233+0000 7f27f9a24b80 1 Processor -- start 2023-07-21T22:33:27.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.234+0000 7f27f9a24b80 1 -- start start 2023-07-21T22:33:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.234+0000 7f27f9a24b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec3620 0x557530ec3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.234+0000 7f27f9a24b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3f70 0x557530ec4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.234+0000 7f27f9a24b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec4dc0 0x557530ec5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.234+0000 7f27f9a24b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557530dc8cb0 con 0x557530ec3620 2023-07-21T22:33:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.234+0000 7f27f9a24b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557530e1ba40 con 0x557530ec4dc0 2023-07-21T22:33:27.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.234+0000 7f27f9a24b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557530e14ed0 con 0x557530ec3f70 2023-07-21T22:33:27.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.243+0000 7f27e6ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3f70 0x557530ec4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.246+0000 7f27e6ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3f70 0x557530ec4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38830/0 (socket says 172.21.15.107:38830) 2023-07-21T22:33:27.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.246+0000 7f27e6ffd700 1 -- 172.21.15.107:0/2585034973 learned_addr learned my addr 172.21.15.107:0/2585034973 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:27.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.247+0000 7f27ec89d700 1 -- 172.21.15.107:0/961172116 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f27e001d470 con 0x7f27e80711d0 2023-07-21T22:33:27.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.248+0000 7f27ec89d700 1 --2- 172.21.15.107:0/961172116 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f27d00a1430 0x7f27d00a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.248+0000 7f27ec89d700 1 -- 172.21.15.107:0/961172116 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f27e00cdcd0 con 0x7f27e80711d0 2023-07-21T22:33:27.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.249+0000 7f27e77fe700 1 --2- 172.21.15.107:0/2585034973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec3620 0x557530ec3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.249+0000 7f27e67fc700 1 --2- 172.21.15.107:0/2585034973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec4dc0 0x557530ec5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.252+0000 7f27e6ffd700 1 -- 172.21.15.107:0/2585034973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec4dc0 msgr2=0x557530ec5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:27.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.252+0000 7f27e6ffd700 1 --2- 172.21.15.107:0/2585034973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec4dc0 0x557530ec5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.252+0000 7f27e6ffd700 1 -- 172.21.15.107:0/2585034973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec3620 msgr2=0x557530ec3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:27.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.252+0000 7f27e6ffd700 1 --2- 172.21.15.107:0/2585034973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec3620 0x557530ec3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.252+0000 7f27e6ffd700 1 -- 172.21.15.107:0/2585034973 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557530c42de0 con 0x557530ec3f70 2023-07-21T22:33:27.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.252+0000 7f27e6ffd700 1 --2- 172.21.15.107:0/2585034973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3f70 0x557530ec4830 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f27e009b080 tx=0x7f27e001ddb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ba77774108a37bb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:27.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.254+0000 7f27cdffb700 1 -- 172.21.15.107:0/2585034973 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e001ae90 con 0x557530ec3f70 2023-07-21T22:33:27.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.254+0000 7f27cdffb700 1 -- 172.21.15.107:0/2585034973 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27e009b0f0 con 0x557530ec3f70 2023-07-21T22:33:27.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.254+0000 7f27cdffb700 1 -- 172.21.15.107:0/2585034973 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e009b5b0 con 0x557530ec3f70 2023-07-21T22:33:27.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.255+0000 7f27f9a24b80 1 -- 172.21.15.107:0/2585034973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3f70 msgr2=0x557530ec4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:27.303 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.255+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/2585034973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3f70 0x557530ec4830 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f27e009b080 tx=0x7f27e001ddb0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.280+0000 7f27e77fe700 1 --2- 172.21.15.107:0/961172116 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f27d00a1430 0x7f27d00a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.323+0000 7f27f9a24b80 1 -- 172.21.15.107:0/2585034973 shutdown_connections 2023-07-21T22:33:27.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.323+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/2585034973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec4dc0 0x557530ec5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.323+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/2585034973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec3620 0x557530ec3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.323+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/2585034973 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3f70 0x557530ec4830 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.323+0000 7f27f9a24b80 1 -- 172.21.15.107:0/2585034973 >> 172.21.15.107:0/2585034973 conn(0x557530eb8ef0 msgr2=0x557530ebe390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:27.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.323+0000 7f27f9a24b80 1 -- 172.21.15.107:0/2585034973 shutdown_connections 2023-07-21T22:33:27.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.323+0000 7f27f9a24b80 1 -- 172.21.15.107:0/2585034973 wait complete. 2023-07-21T22:33:27.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.324+0000 7f27f9a24b80 1 -- start start 2023-07-21T22:33:27.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.324+0000 7f27f9a24b80 1 Processor -- start 2023-07-21T22:33:27.372 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.324+0000 7f27f9a24b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3620 0x557530ecd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.372 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.324+0000 7f27f9a24b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3f70 0x557530ecd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.372 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.324+0000 7f27f9a24b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec4dc0 0x557530ed4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.372 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.324+0000 7f27f9a24b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557530db4ac0 con 0x557530ec4dc0 2023-07-21T22:33:27.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.324+0000 7f27f9a24b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557530db47c0 con 0x557530ec3f70 2023-07-21T22:33:27.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.324+0000 7f27f9a24b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557530c98920 con 0x557530ec3620 2023-07-21T22:33:27.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.324+0000 7f27f9a24b80 10 client.0 _ll_register_callbacks cb 0x557530db2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:27.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.325+0000 7f27e67fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3f70 0x557530ecd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.342+0000 7f27e77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec4dc0 0x557530ed4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.342+0000 7f27e77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec4dc0 0x557530ed4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50278/0 (socket says 172.21.15.107:50278) 2023-07-21T22:33:27.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.342+0000 7f27e77fe700 1 -- 172.21.15.107:0/842410705 learned_addr learned my addr 172.21.15.107:0/842410705 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:27.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.342+0000 7f27e6ffd700 1 --2- 172.21.15.107:0/842410705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3620 0x557530ecd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.395 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.348+0000 7f27e67fc700 1 -- 172.21.15.107:0/842410705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3620 msgr2=0x557530ecd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:27.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.348+0000 7f27e67fc700 1 --2- 172.21.15.107:0/842410705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3620 0x557530ecd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.348+0000 7f27e67fc700 1 -- 172.21.15.107:0/842410705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec4dc0 msgr2=0x557530ed4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:27.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.348+0000 7f27e67fc700 1 --2- 172.21.15.107:0/842410705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec4dc0 0x557530ed4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.348+0000 7f27e67fc700 1 -- 172.21.15.107:0/842410705 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e81a4210 con 0x557530ec3f70 2023-07-21T22:33:27.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.353+0000 7f27e77fe700 1 --2- 172.21.15.107:0/961172116 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f27d00a1430 0x7f27d00a38e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f27d8005fd0 tx=0x7f27d8004960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:27.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.353+0000 7f27e67fc700 1 --2- 172.21.15.107:0/842410705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3f70 0x557530ecd600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f27dc011660 tx=0x7f27dc011970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=455e263bbac8cd73 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:27.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.353+0000 7f27cb7f6700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:33:27.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.356+0000 7f27cb7f6700 1 -- 172.21.15.107:0/842410705 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc009700 con 0x557530ec3f70 2023-07-21T22:33:27.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.356+0000 7f27cb7f6700 1 -- 172.21.15.107:0/842410705 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f27dc012860 con 0x557530ec3f70 2023-07-21T22:33:27.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.356+0000 7f27f9a24b80 1 -- 172.21.15.107:0/842410705 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557530d6e4f0 con 0x557530ec3f70 2023-07-21T22:33:27.406 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.359+0000 7f27cb7f6700 1 -- 172.21.15.107:0/842410705 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27dc012a20 con 0x557530ec3f70 2023-07-21T22:33:27.407 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.359+0000 7f27f9a24b80 10 client.25796 fetch_fsmap learned FSMap version 36 2023-07-21T22:33:27.407 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.359+0000 7f27f9a24b80 1 -- 172.21.15.107:0/842410705 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x557530c2bc60 con 0x557530ec3f70 2023-07-21T22:33:27.408 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.360+0000 7f27cb7f6700 1 -- 172.21.15.107:0/842410705 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557530d6e4f0 con 0x557530ec3f70 2023-07-21T22:33:27.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.360+0000 7f27cb7f6700 1 -- 172.21.15.107:0/842410705 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 36) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f27dc01a3f0 con 0x557530ec3f70 2023-07-21T22:33:27.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.361+0000 7f27f9a24b80 10 client.25796 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:33:27.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.361+0000 7f27f9a24b80 10 client.25796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:27.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.361+0000 7f27f9a24b80 20 client.25796 populate_metadata read hostname 'smithi107' 2023-07-21T22:33:27.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.361+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/842410705 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557530ed8f80 0x557530edb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.361+0000 7f27f9a24b80 4 client.25796 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:27.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.361+0000 7f27f9a24b80 1 -- 172.21.15.107:0/842410705 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557530e0deb0 con 0x557530ed8f80 2023-07-21T22:33:27.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.365+0000 7f27e6ffd700 1 --2- 172.21.15.107:0/842410705 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557530ed8f80 0x557530edb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.407+0000 7f27e6ffd700 1 --2- 172.21.15.107:0/842410705 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557530ed8f80 0x557530edb390 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=674590d7f33c189d server_cookie=471247d9660e60df in_seq=0 out_seq=0 2023-07-21T22:33:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.407+0000 7f27cb7f6700 10 client.25796 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:27.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.418+0000 7f27cb7f6700 1 -- 172.21.15.107:0/842410705 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557530e0deb0 con 0x557530ed8f80 2023-07-21T22:33:27.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.418+0000 7f27cb7f6700 10 client.25796 handle_command_reply: tid=0 2023-07-21T22:33:27.465 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.418+0000 7f27f9a24b80 1 client.25796 shutdown 2023-07-21T22:33:27.467 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.421+0000 7f27f9a24b80 1 -- 172.21.15.107:0/842410705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3f70 msgr2=0x557530ecd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:27.468 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.421+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/842410705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3f70 0x557530ecd600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f27dc011660 tx=0x7f27dc011970 comp rx=0 tx=0).stop 2023-07-21T22:33:27.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.458+0000 7f27f9a24b80 1 -- 172.21.15.107:0/842410705 shutdown_connections 2023-07-21T22:33:27.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.458+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/842410705 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557530ed8f80 0x557530edb390 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.458+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/842410705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec4dc0 0x557530ed4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.458+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/842410705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3620 0x557530ecd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.458+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/842410705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3f70 0x557530ecd600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.458+0000 7f27f9a24b80 1 -- 172.21.15.107:0/842410705 >> 172.21.15.107:0/842410705 conn(0x557530eb8ef0 msgr2=0x557530ebda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:27.506 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.458+0000 7f27cb7f6700 0 client.25796 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:27.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.459+0000 7f27f9a24b80 1 -- 172.21.15.107:0/842410705 shutdown_connections 2023-07-21T22:33:27.507 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.459+0000 7f27f9a24b80 1 -- 172.21.15.107:0/842410705 wait complete. 2023-07-21T22:33:27.513 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.465+0000 7f27f9a24b80 20 client.25796 trim_cache size 0 max 16384 2023-07-21T22:33:27.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:27 smithi190 ceph-mon[98334]: pgmap v425: 97 pgs: 97 active+clean; 967 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 267 op/s 2023-07-21T22:33:27.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.594+0000 7f27f9a24b80 1 Processor -- start 2023-07-21T22:33:27.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.595+0000 7f27f9a24b80 1 -- start start 2023-07-21T22:33:27.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.595+0000 7f27f9a24b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3620 0x557530ecb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.595+0000 7f27f9a24b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3f70 0x557530ed4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.595+0000 7f27f9a24b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec4dc0 0x557530ecd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.595+0000 7f27f9a24b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557530d6e4f0 con 0x557530ec4dc0 2023-07-21T22:33:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.595+0000 7f27f9a24b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557530ece2b0 con 0x557530ec3620 2023-07-21T22:33:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.595+0000 7f27f9a24b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557530eb88f0 con 0x557530ec3f70 2023-07-21T22:33:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.608+0000 7f27e77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec4dc0 0x557530ecd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.608+0000 7f27e77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec4dc0 0x557530ecd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50284/0 (socket says 172.21.15.107:50284) 2023-07-21T22:33:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.608+0000 7f27e77fe700 1 -- 172.21.15.107:0/1419184852 learned_addr learned my addr 172.21.15.107:0/1419184852 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:27.664 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.608+0000 7f27e67fc700 1 --2- 172.21.15.107:0/1419184852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3f70 0x557530ed4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.614+0000 7f27e6ffd700 1 --2- 172.21.15.107:0/1419184852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3620 0x557530ecb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.614+0000 7f27e67fc700 1 -- 172.21.15.107:0/1419184852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3620 msgr2=0x557530ecb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.614+0000 7f27e67fc700 1 --2- 172.21.15.107:0/1419184852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3620 0x557530ecb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.614+0000 7f27e67fc700 1 -- 172.21.15.107:0/1419184852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec4dc0 msgr2=0x557530ecd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.614+0000 7f27e67fc700 1 --2- 172.21.15.107:0/1419184852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec4dc0 0x557530ecd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.614+0000 7f27e67fc700 1 -- 172.21.15.107:0/1419184852 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557530ec5460 con 0x557530ec3f70 2023-07-21T22:33:27.665 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.617+0000 7f27e67fc700 1 --2- 172.21.15.107:0/1419184852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3f70 0x557530ed4600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f27dc01a750 tx=0x7f27dc011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca6e0ffcd61800c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.620+0000 7f27ce7fc700 1 -- 172.21.15.107:0/1419184852 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc01a3f0 con 0x557530ec3f70 2023-07-21T22:33:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.620+0000 7f27ce7fc700 1 -- 172.21.15.107:0/1419184852 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27dc01a550 con 0x557530ec3f70 2023-07-21T22:33:27.667 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.620+0000 7f27ce7fc700 1 -- 172.21.15.107:0/1419184852 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc004c90 con 0x557530ec3f70 2023-07-21T22:33:27.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.621+0000 7f27f9a24b80 1 -- 172.21.15.107:0/1419184852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3f70 msgr2=0x557530ed4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:27.668 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.621+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/1419184852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3f70 0x557530ed4600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f27dc01a750 tx=0x7f27dc011f90 comp rx=0 tx=0).stop 2023-07-21T22:33:27.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.628+0000 7f27f9a24b80 1 -- 172.21.15.107:0/1419184852 shutdown_connections 2023-07-21T22:33:27.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.628+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/1419184852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec4dc0 0x557530ecd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.628+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/1419184852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3620 0x557530ecb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.628+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/1419184852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec3f70 0x557530ed4600 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.628+0000 7f27f9a24b80 1 -- 172.21.15.107:0/1419184852 >> 172.21.15.107:0/1419184852 conn(0x557530eb8ef0 msgr2=0x557530edaf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:27.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.628+0000 7f27f9a24b80 1 -- 172.21.15.107:0/1419184852 shutdown_connections 2023-07-21T22:33:27.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.628+0000 7f27f9a24b80 1 -- 172.21.15.107:0/1419184852 wait complete. 2023-07-21T22:33:27.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.628+0000 7f27f9a24b80 1 -- start start 2023-07-21T22:33:27.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.629+0000 7f27f9a24b80 1 Processor -- start 2023-07-21T22:33:27.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.629+0000 7f27f9a24b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3620 0x557530ed07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.629+0000 7f27f9a24b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec3f70 0x557530ed0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.629+0000 7f27f9a24b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec4dc0 0x557530ed3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.629+0000 7f27f9a24b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557530ed4290 con 0x557530ec3f70 2023-07-21T22:33:27.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.629+0000 7f27f9a24b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557530ed4400 con 0x557530ec3620 2023-07-21T22:33:27.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.629+0000 7f27f9a24b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557530ed45c0 con 0x557530ec4dc0 2023-07-21T22:33:27.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.629+0000 7f27f9a24b80 10 client.0 _ll_register_callbacks cb 0x7f27dc009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:27.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.629+0000 7f27e77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec4dc0 0x557530ed3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.629+0000 7f27e77fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec4dc0 0x557530ed3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38848/0 (socket says 172.21.15.107:38848) 2023-07-21T22:33:27.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.629+0000 7f27e77fe700 1 -- 172.21.15.107:0/178143603 learned_addr learned my addr 172.21.15.107:0/178143603 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:27.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.629+0000 7f27e6ffd700 1 --2- 172.21.15.107:0/178143603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3620 0x557530ed07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.630+0000 7f27e77fe700 1 -- 172.21.15.107:0/178143603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3620 msgr2=0x557530ed07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:27.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.630+0000 7f27e77fe700 1 --2- 172.21.15.107:0/178143603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3620 0x557530ed07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.630+0000 7f27e77fe700 1 -- 172.21.15.107:0/178143603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec3f70 msgr2=0x557530ed0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:27.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.630+0000 7f27e77fe700 1 --2- 172.21.15.107:0/178143603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec3f70 0x557530ed0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.630+0000 7f27e77fe700 1 -- 172.21.15.107:0/178143603 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557530c2bc60 con 0x557530ec4dc0 2023-07-21T22:33:27.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.630+0000 7f27e77fe700 1 --2- 172.21.15.107:0/178143603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec4dc0 0x557530ed3d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009f30 tx=0x7f27d8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b43f694d8d39defb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:27.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:27 smithi132 ceph-mon[97863]: pgmap v425: 97 pgs: 97 active+clean; 967 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 267 op/s 2023-07-21T22:33:27.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.635+0000 7f27cb7f6700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:33:27.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.635+0000 7f27cb7f6700 1 -- 172.21.15.107:0/178143603 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8024040 con 0x557530ec4dc0 2023-07-21T22:33:27.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.635+0000 7f27cb7f6700 1 -- 172.21.15.107:0/178143603 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f27d8016030 con 0x557530ec4dc0 2023-07-21T22:33:27.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.635+0000 7f27f9a24b80 1 -- 172.21.15.107:0/178143603 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27e009b0f0 con 0x557530ec4dc0 2023-07-21T22:33:27.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:27 smithi107 ceph-mon[96543]: pgmap v425: 97 pgs: 97 active+clean; 967 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 267 op/s 2023-07-21T22:33:27.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.645+0000 7f27cb7f6700 1 -- 172.21.15.107:0/178143603 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27d801d2e0 con 0x557530ec4dc0 2023-07-21T22:33:27.735 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.650+0000 7f27f9a24b80 10 client.25861 fetch_fsmap learned FSMap version 36 2023-07-21T22:33:27.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.650+0000 7f27f9a24b80 1 -- 172.21.15.107:0/178143603 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x557530c69250 con 0x557530ec4dc0 2023-07-21T22:33:27.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.718+0000 7f27cb7f6700 1 -- 172.21.15.107:0/178143603 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27e009b0f0 con 0x557530ec4dc0 2023-07-21T22:33:27.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.718+0000 7f27cb7f6700 1 -- 172.21.15.107:0/178143603 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 36) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f27d8029430 con 0x557530ec4dc0 2023-07-21T22:33:27.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.719+0000 7f27f9a24b80 10 client.25861 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:33:27.765 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.719+0000 7f27f9a24b80 10 client.25861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:27.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.719+0000 7f27f9a24b80 20 client.25861 populate_metadata read hostname 'smithi107' 2023-07-21T22:33:27.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.719+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/178143603 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557530f2e250 0x557530f30660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:27.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.719+0000 7f27f9a24b80 4 client.25861 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:33:27.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.719+0000 7f27f9a24b80 1 -- 172.21.15.107:0/178143603 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557530dd0050 con 0x557530f2e250 2023-07-21T22:33:27.766 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.719+0000 7f27e6ffd700 1 --2- 172.21.15.107:0/178143603 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557530f2e250 0x557530f30660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:27.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.881+0000 7f27e6ffd700 1 --2- 172.21.15.107:0/178143603 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557530f2e250 0x557530f30660 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd6efb242c9b679e server_cookie=aa62950b32c67224 in_seq=0 out_seq=0 2023-07-21T22:33:27.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.881+0000 7f27cb7f6700 10 client.25861 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:27.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.881+0000 7f27cb7f6700 1 -- 172.21.15.107:0/178143603 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557530dd0050 con 0x557530f2e250 2023-07-21T22:33:27.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.881+0000 7f27cb7f6700 10 client.25861 handle_command_reply: tid=0 2023-07-21T22:33:27.928 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.881+0000 7f27f9a24b80 1 client.25861 shutdown 2023-07-21T22:33:27.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.885+0000 7f27f9a24b80 1 -- 172.21.15.107:0/178143603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec4dc0 msgr2=0x557530ed3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:27.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.885+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/178143603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec4dc0 0x557530ed3d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009f30 tx=0x7f27d8013000 comp rx=0 tx=0).stop 2023-07-21T22:33:27.968 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.896+0000 7f27f9a24b80 1 -- 172.21.15.107:0/178143603 shutdown_connections 2023-07-21T22:33:27.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.896+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/178143603 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x557530f2e250 0x557530f30660 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.896+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/178143603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557530ec4dc0 0x557530ed3d50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.896+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/178143603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557530ec3620 0x557530ed07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.896+0000 7f27f9a24b80 1 --2- 172.21.15.107:0/178143603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557530ec3f70 0x557530ed0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:27.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.896+0000 7f27f9a24b80 1 -- 172.21.15.107:0/178143603 >> 172.21.15.107:0/178143603 conn(0x557530eb8ef0 msgr2=0x557530ecbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:27.969 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.922+0000 7f27cb7f6700 0 client.25861 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:27.978 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.922+0000 7f27f9a24b80 1 -- 172.21.15.107:0/178143603 shutdown_connections 2023-07-21T22:33:27.978 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.932+0000 7f27f9a24b80 1 -- 172.21.15.107:0/178143603 wait complete. 2023-07-21T22:33:27.984 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.932+0000 7f27f9a24b80 20 client.25861 trim_cache size 0 max 16384 2023-07-21T22:33:27.986 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:33:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:33:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:33:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "d037412e-e4e0-4d2c-8d6e-c7193371fe0f": { 2023-07-21T22:33:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:33:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d037412e-e4e0-4d2c-8d6e-c7193371fe0f", 2023-07-21T22:33:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:33:27.987 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:33:27.988 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:33:27.988 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:33:27.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.939+0000 7f27e7fff700 1 -- 172.21.15.107:0/961172116 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f27d00a1430 msgr2=0x7f27d00a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.939+0000 7f27e7fff700 1 --2- 172.21.15.107:0/961172116 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f27d00a1430 0x7f27d00a38e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f27d8005fd0 tx=0x7f27d8004960 comp rx=0 tx=0).stop 2023-07-21T22:33:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.939+0000 7f27e7fff700 1 -- 172.21.15.107:0/961172116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27e80711d0 msgr2=0x7f27e8113a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.939+0000 7f27e7fff700 1 --2- 172.21.15.107:0/961172116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27e80711d0 0x7f27e8113a30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f27e000b6c0 tx=0x7f27e0004d30 comp rx=0 tx=0).stop 2023-07-21T22:33:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.939+0000 7f27e7fff700 1 -- 172.21.15.107:0/961172116 shutdown_connections 2023-07-21T22:33:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.939+0000 7f27e7fff700 1 --2- 172.21.15.107:0/961172116 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f27d00a1430 0x7f27d00a38e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:28.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.939+0000 7f27e7fff700 1 --2- 172.21.15.107:0/961172116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27e810bba0 0x7f27e81a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.939+0000 7f27e7fff700 1 --2- 172.21.15.107:0/961172116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27e80711d0 0x7f27e8113a30 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f27e000b6c0 tx=0x7f27e0004d30 comp rx=0 tx=0).stop 2023-07-21T22:33:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.939+0000 7f27e7fff700 1 --2- 172.21.15.107:0/961172116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e8071b20 0x7f27e8113f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.939+0000 7f27e7fff700 1 -- 172.21.15.107:0/961172116 >> 172.21.15.107:0/961172116 conn(0x7f27e806c8b0 msgr2=0x7f27e806e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:28.001 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.955+0000 7f27e7fff700 1 -- 172.21.15.107:0/961172116 shutdown_connections 2023-07-21T22:33:28.002 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:33:27.956+0000 7f27e7fff700 1 -- 172.21.15.107:0/961172116 wait complete. 2023-07-21T22:33:28.051 INFO:tasks.cephfs.filesystem:scrub status for tag:d037412e-e4e0-4d2c-8d6e-c7193371fe0f - {'path': '/', 'tag': 'd037412e-e4e0-4d2c-8d6e-c7193371fe0f', 'options': 'recursive,force'} 2023-07-21T22:33:29.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:29 smithi190 ceph-mon[98334]: pgmap v426: 97 pgs: 97 active+clean; 1000 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 417 op/s 2023-07-21T22:33:29.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:29 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:29.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:29 smithi132 ceph-mon[97863]: pgmap v426: 97 pgs: 97 active+clean; 1000 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 417 op/s 2023-07-21T22:33:29.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:29 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:29.702 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:29 smithi107 ceph-mon[96543]: pgmap v426: 97 pgs: 97 active+clean; 1000 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 417 op/s 2023-07-21T22:33:29.702 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:29 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:31.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:31 smithi132 ceph-mon[97863]: pgmap v427: 97 pgs: 97 active+clean; 1019 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 483 op/s 2023-07-21T22:33:31.692 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:31 smithi107 ceph-mon[96543]: pgmap v427: 97 pgs: 97 active+clean; 1019 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 483 op/s 2023-07-21T22:33:31.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:31 smithi190 ceph-mon[98334]: pgmap v427: 97 pgs: 97 active+clean; 1019 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 483 op/s 2023-07-21T22:33:33.691 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:33 smithi107 ceph-mon[96543]: pgmap v428: 97 pgs: 97 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 777 op/s 2023-07-21T22:33:33.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:33 smithi190 ceph-mon[98334]: pgmap v428: 97 pgs: 97 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 777 op/s 2023-07-21T22:33:33.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:33 smithi132 ceph-mon[97863]: pgmap v428: 97 pgs: 97 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 777 op/s 2023-07-21T22:33:34.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:34 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:34.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:34 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:34.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:34 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.950+0000 7f7081513700 1 -- 172.21.15.107:0/791293621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707c10bd60 msgr2=0x7f707c10e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.950+0000 7f7081513700 1 --2- 172.21.15.107:0/791293621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707c10bd60 0x7f707c10e210 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7074009ec0 tx=0x7f707400b330 comp rx=0 tx=0).stop 2023-07-21T22:33:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.976+0000 7f7081513700 1 -- 172.21.15.107:0/791293621 shutdown_connections 2023-07-21T22:33:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.976+0000 7f7081513700 1 --2- 172.21.15.107:0/791293621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707c10bd60 0x7f707c10e210 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.976+0000 7f7081513700 1 --2- 172.21.15.107:0/791293621 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f707c070f00 0x7f707c071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.976+0000 7f7081513700 1 --2- 172.21.15.107:0/791293621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f707c071850 0x7f707c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.976+0000 7f7081513700 1 -- 172.21.15.107:0/791293621 >> 172.21.15.107:0/791293621 conn(0x7f707c06ca50 msgr2=0x7f707c06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.976+0000 7f7081513700 1 -- 172.21.15.107:0/791293621 shutdown_connections 2023-07-21T22:33:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.976+0000 7f7081513700 1 -- 172.21.15.107:0/791293621 wait complete. 2023-07-21T22:33:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.977+0000 7f7081513700 1 Processor -- start 2023-07-21T22:33:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.978+0000 7f7081513700 1 -- start start 2023-07-21T22:33:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.978+0000 7f7081513700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f707c070f00 0x7f707c113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.978+0000 7f7081513700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707c071850 0x7f707c114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.978+0000 7f7081513700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f707c10bd60 0x7f707c114f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.978+0000 7f7081513700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c1155c0 con 0x7f707c071850 2023-07-21T22:33:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.978+0000 7f7081513700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c115730 con 0x7f707c10bd60 2023-07-21T22:33:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.978+0000 7f7081513700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c1158f0 con 0x7f707c070f00 2023-07-21T22:33:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.978+0000 7f707affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f707c070f00 0x7f707c113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.978+0000 7f707affd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f707c070f00 0x7f707c113b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38858/0 (socket says 172.21.15.107:38858) 2023-07-21T22:33:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.978+0000 7f707affd700 1 -- 172.21.15.107:0/4156173007 learned_addr learned my addr 172.21.15.107:0/4156173007 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.979+0000 7f707b7fe700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f707c10bd60 0x7f707c114f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.981+0000 7f707a7fc700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707c071850 0x7f707c114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.981+0000 7f707b7fe700 1 -- 172.21.15.107:0/4156173007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f707c070f00 msgr2=0x7f707c113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.981+0000 7f707b7fe700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f707c070f00 0x7f707c113b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.981+0000 7f707b7fe700 1 -- 172.21.15.107:0/4156173007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707c071850 msgr2=0x7f707c114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.981+0000 7f707b7fe700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707c071850 0x7f707c114090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.981+0000 7f707b7fe700 1 -- 172.21.15.107:0/4156173007 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7074005e70 con 0x7f707c10bd60 2023-07-21T22:33:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:34.985+0000 7f707b7fe700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f707c10bd60 0x7f707c114f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7074009ec0 tx=0x7f707400bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.022+0000 7f706bfff700 1 -- 172.21.15.107:0/4156173007 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707400e040 con 0x7f707c10bd60 2023-07-21T22:33:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.022+0000 7f706bfff700 1 -- 172.21.15.107:0/4156173007 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f707401bd30 con 0x7f707c10bd60 2023-07-21T22:33:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.022+0000 7f706bfff700 1 -- 172.21.15.107:0/4156173007 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707401db70 con 0x7f707c10bd60 2023-07-21T22:33:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.031+0000 7f7081513700 1 -- 172.21.15.107:0/4156173007 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707c115b70 con 0x7f707c10bd60 2023-07-21T22:33:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.031+0000 7f7081513700 1 -- 172.21.15.107:0/4156173007 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c116400 con 0x7f707c10bd60 2023-07-21T22:33:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.039+0000 7f706bfff700 1 -- 172.21.15.107:0/4156173007 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f707401b870 con 0x7f707c10bd60 2023-07-21T22:33:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.040+0000 7f706bfff700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f70640a1350 0x7f70640a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.040+0000 7f706bfff700 1 -- 172.21.15.107:0/4156173007 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f70740cdd70 con 0x7f707c10bd60 2023-07-21T22:33:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.041+0000 7f708d698b80 1 Processor -- start 2023-07-21T22:33:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.041+0000 7f708d698b80 1 -- start start 2023-07-21T22:33:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.042+0000 7f708d698b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb49022f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.042+0000 7f708d698b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4902830 0x563cb4903040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.042+0000 7f708d698b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4903580 0x563cb49039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.042+0000 7f708d698b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb4807c50 con 0x563cb4902830 2023-07-21T22:33:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.042+0000 7f708d698b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb485a9e0 con 0x563cb475e6b0 2023-07-21T22:33:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.042+0000 7f708d698b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb4853e70 con 0x563cb4903580 2023-07-21T22:33:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.043+0000 7f707a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4902830 0x563cb4903040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.043+0000 7f707a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4902830 0x563cb4903040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50308/0 (socket says 172.21.15.107:50308) 2023-07-21T22:33:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.043+0000 7f707a7fc700 1 -- 172.21.15.107:0/3559424796 learned_addr learned my addr 172.21.15.107:0/3559424796 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.043+0000 7f707affd700 1 --2- 172.21.15.107:0/3559424796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb49022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.047+0000 7f707affd700 1 -- 172.21.15.107:0/3559424796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4903580 msgr2=0x563cb49039f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:33:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.047+0000 7f707affd700 1 --2- 172.21.15.107:0/3559424796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4903580 0x563cb49039f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.047+0000 7f707affd700 1 -- 172.21.15.107:0/3559424796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4902830 msgr2=0x563cb4903040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.047+0000 7f707affd700 1 --2- 172.21.15.107:0/3559424796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4902830 0x563cb4903040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.047+0000 7f707affd700 1 -- 172.21.15.107:0/3559424796 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cb4681d80 con 0x563cb475e6b0 2023-07-21T22:33:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.047+0000 7f707affd700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f70640a1350 0x7f70640a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.054+0000 7f707affd700 1 --2- 172.21.15.107:0/3559424796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb49022f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f706c009bc0 tx=0x7f706c009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df194f85f1060411 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.063+0000 7f707affd700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f70640a1350 0x7f70640a3800 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f706c020000 tx=0x7f706c0146c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.080+0000 7f70697fa700 1 -- 172.21.15.107:0/3559424796 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c010e70 con 0x563cb475e6b0 2023-07-21T22:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.080+0000 7f70697fa700 1 -- 172.21.15.107:0/3559424796 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f706c0169a0 con 0x563cb475e6b0 2023-07-21T22:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.080+0000 7f70697fa700 1 -- 172.21.15.107:0/3559424796 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c021b70 con 0x563cb475e6b0 2023-07-21T22:33:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.089+0000 7f708d698b80 1 -- 172.21.15.107:0/3559424796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 msgr2=0x563cb49022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.089+0000 7f708d698b80 1 --2- 172.21.15.107:0/3559424796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb49022f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f706c009bc0 tx=0x7f706c009ed0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.089+0000 7f708d698b80 1 -- 172.21.15.107:0/3559424796 shutdown_connections 2023-07-21T22:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.089+0000 7f708d698b80 1 --2- 172.21.15.107:0/3559424796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4903580 0x563cb49039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.089+0000 7f708d698b80 1 --2- 172.21.15.107:0/3559424796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb49022f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.089+0000 7f708d698b80 1 --2- 172.21.15.107:0/3559424796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4902830 0x563cb4903040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.089+0000 7f708d698b80 1 -- 172.21.15.107:0/3559424796 >> 172.21.15.107:0/3559424796 conn(0x563cb48f7ae0 msgr2=0x563cb48fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.089+0000 7f708d698b80 1 -- 172.21.15.107:0/3559424796 shutdown_connections 2023-07-21T22:33:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.089+0000 7f708d698b80 1 -- 172.21.15.107:0/3559424796 wait complete. 2023-07-21T22:33:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.090+0000 7f708d698b80 1 -- start start 2023-07-21T22:33:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.090+0000 7f708d698b80 1 Processor -- start 2023-07-21T22:33:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.090+0000 7f708d698b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb490b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.090+0000 7f708d698b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4902830 0x563cb490bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.090+0000 7f708d698b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4903580 0x563cb4912df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.090+0000 7f708d698b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb47f3a60 con 0x563cb4902830 2023-07-21T22:33:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.090+0000 7f708d698b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb47f3760 con 0x563cb475e6b0 2023-07-21T22:33:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.090+0000 7f708d698b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb46d78c0 con 0x563cb4903580 2023-07-21T22:33:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.090+0000 7f708d698b80 10 client.0 _ll_register_callbacks cb 0x563cb47f1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.091+0000 7f707b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4903580 0x563cb4912df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.091+0000 7f707b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4903580 0x563cb4912df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38874/0 (socket says 172.21.15.107:38874) 2023-07-21T22:33:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.091+0000 7f707b7fe700 1 -- 172.21.15.107:0/3315187440 learned_addr learned my addr 172.21.15.107:0/3315187440 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.091+0000 7f707affd700 1 --2- 172.21.15.107:0/3315187440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb490b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.097+0000 7f707b7fe700 1 -- 172.21.15.107:0/3315187440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 msgr2=0x563cb490b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.097+0000 7f707b7fe700 1 --2- 172.21.15.107:0/3315187440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb490b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.097+0000 7f707b7fe700 1 -- 172.21.15.107:0/3315187440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4902830 msgr2=0x563cb490bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.097+0000 7f707b7fe700 1 --2- 172.21.15.107:0/3315187440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4902830 0x563cb490bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.097+0000 7f707b7fe700 1 -- 172.21.15.107:0/3315187440 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706c009820 con 0x563cb4903580 2023-07-21T22:33:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.102+0000 7f707b7fe700 1 --2- 172.21.15.107:0/3315187440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4903580 0x563cb4912df0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7074097690 tx=0x7f707400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e76949f7c55fc19 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.103+0000 7f705e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:33:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.103+0000 7f705e7fc700 1 -- 172.21.15.107:0/3315187440 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074099530 con 0x563cb4903580 2023-07-21T22:33:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.103+0000 7f705e7fc700 1 -- 172.21.15.107:0/3315187440 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f70740cb900 con 0x563cb4903580 2023-07-21T22:33:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.103+0000 7f708d698b80 1 -- 172.21.15.107:0/3315187440 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cb47ad490 con 0x563cb4903580 2023-07-21T22:33:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.108+0000 7f705e7fc700 1 -- 172.21.15.107:0/3315187440 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f707409a680 con 0x563cb4903580 2023-07-21T22:33:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.108+0000 7f708d698b80 10 client.25867 fetch_fsmap learned FSMap version 38 2023-07-21T22:33:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.108+0000 7f708d698b80 1 -- 172.21.15.107:0/3315187440 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x563cb466ac00 con 0x563cb4903580 2023-07-21T22:33:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.110+0000 7f705e7fc700 1 -- 172.21.15.107:0/3315187440 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563cb47ad490 con 0x563cb4903580 2023-07-21T22:33:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.110+0000 7f705e7fc700 1 -- 172.21.15.107:0/3315187440 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 38) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f707409a0f0 con 0x563cb4903580 2023-07-21T22:33:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.111+0000 7f708d698b80 10 client.25867 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:33:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.111+0000 7f708d698b80 10 client.25867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.111+0000 7f708d698b80 20 client.25867 populate_metadata read hostname 'smithi107' 2023-07-21T22:33:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.111+0000 7f708d698b80 1 --2- 172.21.15.107:0/3315187440 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563cb4917370 0x563cb4919750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.111+0000 7f708d698b80 4 client.25867 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.111+0000 7f708d698b80 1 -- 172.21.15.107:0/3315187440 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cb484ce50 con 0x563cb4917370 2023-07-21T22:33:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.111+0000 7f707affd700 1 --2- 172.21.15.107:0/3315187440 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563cb4917370 0x563cb4919750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.122+0000 7f707affd700 1 --2- 172.21.15.107:0/3315187440 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563cb4917370 0x563cb4919750 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42d5efee4b9f6ff3 server_cookie=982ddc84f51c3516 in_seq=0 out_seq=0 2023-07-21T22:33:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.123+0000 7f705e7fc700 10 client.25867 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.129+0000 7f705e7fc700 1 -- 172.21.15.107:0/3315187440 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563cb484ce50 con 0x563cb4917370 2023-07-21T22:33:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.129+0000 7f705e7fc700 10 client.25867 handle_command_reply: tid=0 2023-07-21T22:33:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.129+0000 7f708d698b80 1 client.25867 shutdown 2023-07-21T22:33:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.130+0000 7f708d698b80 1 -- 172.21.15.107:0/3315187440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4903580 msgr2=0x563cb4912df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.130+0000 7f708d698b80 1 --2- 172.21.15.107:0/3315187440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4903580 0x563cb4912df0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7074097690 tx=0x7f707400bf10 comp rx=0 tx=0).stop 2023-07-21T22:33:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.131+0000 7f708d698b80 1 -- 172.21.15.107:0/3315187440 shutdown_connections 2023-07-21T22:33:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.131+0000 7f708d698b80 1 --2- 172.21.15.107:0/3315187440 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563cb4917370 0x563cb4919750 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.131+0000 7f708d698b80 1 --2- 172.21.15.107:0/3315187440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4903580 0x563cb4912df0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.131+0000 7f708d698b80 1 --2- 172.21.15.107:0/3315187440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb490b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.131+0000 7f708d698b80 1 --2- 172.21.15.107:0/3315187440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4902830 0x563cb490bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.131+0000 7f708d698b80 1 -- 172.21.15.107:0/3315187440 >> 172.21.15.107:0/3315187440 conn(0x563cb48f7ae0 msgr2=0x563cb48fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.131+0000 7f705e7fc700 0 client.25867 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.131+0000 7f708d698b80 1 -- 172.21.15.107:0/3315187440 shutdown_connections 2023-07-21T22:33:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.131+0000 7f708d698b80 1 -- 172.21.15.107:0/3315187440 wait complete. 2023-07-21T22:33:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.132+0000 7f708d698b80 20 client.25867 trim_cache size 0 max 16384 2023-07-21T22:33:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.151+0000 7f708d698b80 1 Processor -- start 2023-07-21T22:33:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.151+0000 7f708d698b80 1 -- start start 2023-07-21T22:33:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.151+0000 7f708d698b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb48fcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.151+0000 7f708d698b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4902830 0x563cb4912db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.151+0000 7f708d698b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4903580 0x563cb4909950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.151+0000 7f708d698b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb48fdaf0 con 0x563cb4903580 2023-07-21T22:33:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.151+0000 7f708d698b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb490ad20 con 0x563cb475e6b0 2023-07-21T22:33:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.151+0000 7f708d698b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb4917190 con 0x563cb4902830 2023-07-21T22:33:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707a7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4902830 0x563cb4912db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb48fcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb48fcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41150/0 (socket says 172.21.15.107:41150) 2023-07-21T22:33:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707a7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4902830 0x563cb4912db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38882/0 (socket says 172.21.15.107:38882) 2023-07-21T22:33:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707affd700 1 -- 172.21.15.107:0/2821836020 learned_addr learned my addr 172.21.15.107:0/2821836020 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707b7fe700 1 --2- 172.21.15.107:0/2821836020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4903580 0x563cb4909950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707affd700 1 -- 172.21.15.107:0/2821836020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4902830 msgr2=0x563cb4912db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707affd700 1 --2- 172.21.15.107:0/2821836020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4902830 0x563cb4912db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707affd700 1 -- 172.21.15.107:0/2821836020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4903580 msgr2=0x563cb4909950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707affd700 1 --2- 172.21.15.107:0/2821836020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4903580 0x563cb4909950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707affd700 1 -- 172.21.15.107:0/2821836020 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cb46a81f0 con 0x563cb475e6b0 2023-07-21T22:33:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707a7fc700 1 --2- 172.21.15.107:0/2821836020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4902830 0x563cb4912db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707b7fe700 1 --2- 172.21.15.107:0/2821836020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4903580 0x563cb4909950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f707affd700 1 --2- 172.21.15.107:0/2821836020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb48fcb10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f706c0214c0 tx=0x7f706c027000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84c2d5481377d289 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.152+0000 7f7069ffb700 1 -- 172.21.15.107:0/2821836020 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c006da0 con 0x563cb475e6b0 2023-07-21T22:33:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.153+0000 7f7069ffb700 1 -- 172.21.15.107:0/2821836020 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f706c010e10 con 0x563cb475e6b0 2023-07-21T22:33:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.153+0000 7f7069ffb700 1 -- 172.21.15.107:0/2821836020 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c0166c0 con 0x563cb475e6b0 2023-07-21T22:33:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.154+0000 7f708d698b80 1 -- 172.21.15.107:0/2821836020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 msgr2=0x563cb48fcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.154+0000 7f708d698b80 1 --2- 172.21.15.107:0/2821836020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb48fcb10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f706c0214c0 tx=0x7f706c027000 comp rx=0 tx=0).stop 2023-07-21T22:33:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.154+0000 7f708d698b80 1 -- 172.21.15.107:0/2821836020 shutdown_connections 2023-07-21T22:33:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.154+0000 7f708d698b80 1 --2- 172.21.15.107:0/2821836020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb4903580 0x563cb4909950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.154+0000 7f708d698b80 1 --2- 172.21.15.107:0/2821836020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb475e6b0 0x563cb48fcb10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.154+0000 7f708d698b80 1 --2- 172.21.15.107:0/2821836020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4902830 0x563cb4912db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.154+0000 7f708d698b80 1 -- 172.21.15.107:0/2821836020 >> 172.21.15.107:0/2821836020 conn(0x563cb48f7ae0 msgr2=0x563cb490a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.154+0000 7f708d698b80 1 -- 172.21.15.107:0/2821836020 shutdown_connections 2023-07-21T22:33:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.154+0000 7f708d698b80 1 -- 172.21.15.107:0/2821836020 wait complete. 2023-07-21T22:33:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.154+0000 7f708d698b80 1 -- start start 2023-07-21T22:33:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.154+0000 7f708d698b80 1 Processor -- start 2023-07-21T22:33:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.155+0000 7f708d698b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb475e6b0 0x563cb4912ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.155+0000 7f708d698b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4902830 0x563cb490f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.155+0000 7f708d698b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb4903580 0x563cb490fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.155+0000 7f708d698b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb49100a0 con 0x563cb475e6b0 2023-07-21T22:33:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.155+0000 7f708d698b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb4910210 con 0x563cb4903580 2023-07-21T22:33:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.155+0000 7f708d698b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb496b1f0 con 0x563cb4902830 2023-07-21T22:33:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.155+0000 7f708d698b80 10 client.0 _ll_register_callbacks cb 0x7f707409adc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.155+0000 7f707affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb475e6b0 0x563cb4912ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.155+0000 7f707affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb475e6b0 0x563cb4912ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50328/0 (socket says 172.21.15.107:50328) 2023-07-21T22:33:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.155+0000 7f707affd700 1 -- 172.21.15.107:0/3831997687 learned_addr learned my addr 172.21.15.107:0/3831997687 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.155+0000 7f707b7fe700 1 --2- 172.21.15.107:0/3831997687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb4903580 0x563cb490fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f707b7fe700 1 -- 172.21.15.107:0/3831997687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4902830 msgr2=0x563cb490f620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:33:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f707b7fe700 1 --2- 172.21.15.107:0/3831997687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4902830 0x563cb490f620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f707b7fe700 1 -- 172.21.15.107:0/3831997687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb475e6b0 msgr2=0x563cb4912ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f707b7fe700 1 --2- 172.21.15.107:0/3831997687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb475e6b0 0x563cb4912ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f707b7fe700 1 -- 172.21.15.107:0/3831997687 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707000e040 con 0x563cb4903580 2023-07-21T22:33:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f707b7fe700 1 --2- 172.21.15.107:0/3831997687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb4903580 0x563cb490fb60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f707409ae90 tx=0x7f70740991c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94ca309bf9544542 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f705e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:33:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f705e7fc700 1 -- 172.21.15.107:0/3831997687 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707409af60 con 0x563cb4903580 2023-07-21T22:33:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f705e7fc700 1 -- 172.21.15.107:0/3831997687 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f70740b92b0 con 0x563cb4903580 2023-07-21T22:33:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f708d698b80 1 -- 172.21.15.107:0/3831997687 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f706c0169a0 con 0x563cb4903580 2023-07-21T22:33:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f705e7fc700 1 -- 172.21.15.107:0/3831997687 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70740974d0 con 0x563cb4903580 2023-07-21T22:33:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f708d698b80 10 client.25817 fetch_fsmap learned FSMap version 38 2023-07-21T22:33:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f708d698b80 1 -- 172.21.15.107:0/3831997687 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x563cb466ac00 con 0x563cb4903580 2023-07-21T22:33:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.156+0000 7f705e7fc700 1 -- 172.21.15.107:0/3831997687 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70740cb030 con 0x563cb4903580 2023-07-21T22:33:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.157+0000 7f705e7fc700 1 -- 172.21.15.107:0/3831997687 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 38) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f7074096f80 con 0x563cb4903580 2023-07-21T22:33:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.157+0000 7f708d698b80 10 client.25817 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:33:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.157+0000 7f708d698b80 10 client.25817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.157+0000 7f708d698b80 20 client.25817 populate_metadata read hostname 'smithi107' 2023-07-21T22:33:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.157+0000 7f708d698b80 1 --2- 172.21.15.107:0/3831997687 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563cb496c770 0x563cb496eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.157+0000 7f708d698b80 4 client.25817 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:33:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.157+0000 7f708d698b80 1 -- 172.21.15.107:0/3831997687 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563cb480eff0 con 0x563cb496c770 2023-07-21T22:33:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.157+0000 7f707affd700 1 --2- 172.21.15.107:0/3831997687 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563cb496c770 0x563cb496eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.158+0000 7f705e7fc700 10 client.25817 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.158+0000 7f707affd700 1 --2- 172.21.15.107:0/3831997687 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563cb496c770 0x563cb496eb80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16793ea68a1ba855 server_cookie=979bea281be68e9e in_seq=0 out_seq=0 2023-07-21T22:33:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f705e7fc700 1 -- 172.21.15.107:0/3831997687 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563cb480eff0 con 0x563cb496c770 2023-07-21T22:33:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f705e7fc700 10 client.25817 handle_command_reply: tid=0 2023-07-21T22:33:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f708d698b80 1 client.25817 shutdown 2023-07-21T22:33:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f708d698b80 1 -- 172.21.15.107:0/3831997687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb4903580 msgr2=0x563cb490fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f708d698b80 1 --2- 172.21.15.107:0/3831997687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb4903580 0x563cb490fb60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f707409ae90 tx=0x7f70740991c0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f708d698b80 1 -- 172.21.15.107:0/3831997687 shutdown_connections 2023-07-21T22:33:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f708d698b80 1 --2- 172.21.15.107:0/3831997687 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563cb496c770 0x563cb496eb80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f708d698b80 1 --2- 172.21.15.107:0/3831997687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563cb4903580 0x563cb490fb60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f708d698b80 1 --2- 172.21.15.107:0/3831997687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cb475e6b0 0x563cb4912ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f708d698b80 1 --2- 172.21.15.107:0/3831997687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563cb4902830 0x563cb490f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f708d698b80 1 -- 172.21.15.107:0/3831997687 >> 172.21.15.107:0/3831997687 conn(0x563cb48f7ae0 msgr2=0x563cb4909950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f705e7fc700 0 client.25817 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f708d698b80 1 -- 172.21.15.107:0/3831997687 shutdown_connections 2023-07-21T22:33:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f708d698b80 1 -- 172.21.15.107:0/3831997687 wait complete. 2023-07-21T22:33:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.414+0000 7f708d698b80 20 client.25817 trim_cache size 0 max 16384 2023-07-21T22:33:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:33:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:33:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:33:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:33:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.415+0000 7f7081513700 1 -- 172.21.15.107:0/4156173007 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f70640a1350 msgr2=0x7f70640a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.415+0000 7f7081513700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f70640a1350 0x7f70640a3800 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f706c020000 tx=0x7f706c0146c0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.415+0000 7f7081513700 1 -- 172.21.15.107:0/4156173007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f707c10bd60 msgr2=0x7f707c114f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.415+0000 7f7081513700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f707c10bd60 0x7f707c114f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7074009ec0 tx=0x7f707400bcb0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.415+0000 7f7081513700 1 -- 172.21.15.107:0/4156173007 shutdown_connections 2023-07-21T22:33:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.415+0000 7f7081513700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f707c10bd60 0x7f707c114f60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.415+0000 7f7081513700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f70640a1350 0x7f70640a3800 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.415+0000 7f7081513700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f707c070f00 0x7f707c113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.415+0000 7f7081513700 1 --2- 172.21.15.107:0/4156173007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f707c071850 0x7f707c114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.415+0000 7f7081513700 1 -- 172.21.15.107:0/4156173007 >> 172.21.15.107:0/4156173007 conn(0x7f707c06ca50 msgr2=0x7f707c06dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.417+0000 7f7081513700 1 -- 172.21.15.107:0/4156173007 shutdown_connections 2023-07-21T22:33:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:35.417+0000 7f7081513700 1 -- 172.21.15.107:0/4156173007 wait complete. 2023-07-21T22:33:35.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:35 smithi190 ceph-mon[98334]: pgmap v429: 97 pgs: 97 active+clean; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 813 op/s 2023-07-21T22:33:35.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:35 smithi132 ceph-mon[97863]: pgmap v429: 97 pgs: 97 active+clean; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 813 op/s 2023-07-21T22:33:35.943 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:35 smithi107 ceph-mon[96543]: pgmap v429: 97 pgs: 97 active+clean; 1.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 813 op/s 2023-07-21T22:33:36.527 INFO:tasks.workunit.client.0.smithi107.stdout: 271 2195381 671 1540687 602 818596 991 2023-07-21T22:33:37.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:37 smithi190 ceph-mon[98334]: pgmap v430: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 MiB/s wr, 835 op/s 2023-07-21T22:33:37.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:37 smithi132 ceph-mon[97863]: pgmap v430: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 MiB/s wr, 835 op/s 2023-07-21T22:33:37.958 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:37 smithi107 ceph-mon[96543]: pgmap v430: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 MiB/s wr, 835 op/s 2023-07-21T22:33:39.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:39 smithi190 ceph-mon[98334]: pgmap v431: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 934 op/s 2023-07-21T22:33:39.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:39 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:39.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:39 smithi132 ceph-mon[97863]: pgmap v431: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 934 op/s 2023-07-21T22:33:39.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:39 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:39.942 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:39 smithi107 ceph-mon[96543]: pgmap v431: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 934 op/s 2023-07-21T22:33:39.942 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:39 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:33:41.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:41 smithi190 ceph-mon[98334]: pgmap v432: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 804 op/s 2023-07-21T22:33:41.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:41 smithi132 ceph-mon[97863]: pgmap v432: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 804 op/s 2023-07-21T22:33:41.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:41 smithi107 ceph-mon[96543]: pgmap v432: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 804 op/s 2023-07-21T22:33:43.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:43 smithi190 ceph-mon[98334]: pgmap v433: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.2 MiB/s wr, 753 op/s 2023-07-21T22:33:43.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:43 smithi132 ceph-mon[97863]: pgmap v433: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.2 MiB/s wr, 753 op/s 2023-07-21T22:33:43.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:43 smithi107 ceph-mon[96543]: pgmap v433: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.2 MiB/s wr, 753 op/s 2023-07-21T22:33:45.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:45 smithi190 ceph-mon[98334]: pgmap v434: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 477 op/s 2023-07-21T22:33:45.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:45 smithi132 ceph-mon[97863]: pgmap v434: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 477 op/s 2023-07-21T22:33:45.949 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:45 smithi107 ceph-mon[96543]: pgmap v434: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 477 op/s 2023-07-21T22:33:46.537 INFO:tasks.workunit.client.0.smithi107.stdout: 286 2331410 871 1635687 863 893162 1255 2023-07-21T22:33:47.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:47 smithi132 ceph-mon[97863]: pgmap v435: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 347 op/s 2023-07-21T22:33:47.942 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:47 smithi107 ceph-mon[96543]: pgmap v435: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 347 op/s 2023-07-21T22:33:48.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:47 smithi190 ceph-mon[98334]: pgmap v435: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 347 op/s 2023-07-21T22:33:48.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:48 smithi107 ceph-mon[96543]: pgmap v436: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 315 op/s 2023-07-21T22:33:49.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:48 smithi190 ceph-mon[98334]: pgmap v436: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 315 op/s 2023-07-21T22:33:49.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:48 smithi132 ceph-mon[97863]: pgmap v436: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 315 op/s 2023-07-21T22:33:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.816+0000 7f5654c29700 1 -- 172.21.15.107:0/3484562004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5650071480 msgr2=0x7f565010b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.816+0000 7f5654c29700 1 --2- 172.21.15.107:0/3484562004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5650071480 0x7f565010b7d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f564800b0a0 tx=0x7f564800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:33:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.818+0000 7f5654c29700 1 -- 172.21.15.107:0/3484562004 shutdown_connections 2023-07-21T22:33:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.818+0000 7f5654c29700 1 --2- 172.21.15.107:0/3484562004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565010bd10 0x7f565010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.818+0000 7f5654c29700 1 --2- 172.21.15.107:0/3484562004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5650070b30 0x7f5650070f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.818+0000 7f5654c29700 1 --2- 172.21.15.107:0/3484562004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5650071480 0x7f565010b7d0 secure :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f564800b0a0 tx=0x7f564800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:33:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.818+0000 7f5654c29700 1 -- 172.21.15.107:0/3484562004 >> 172.21.15.107:0/3484562004 conn(0x7f565006c910 msgr2=0x7f565006ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.821+0000 7f5654c29700 1 -- 172.21.15.107:0/3484562004 shutdown_connections 2023-07-21T22:33:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.822+0000 7f5654c29700 1 -- 172.21.15.107:0/3484562004 wait complete. 2023-07-21T22:33:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.823+0000 7f5654c29700 1 Processor -- start 2023-07-21T22:33:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.823+0000 7f5654c29700 1 -- start start 2023-07-21T22:33:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.823+0000 7f5654c29700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5650070b30 0x7f5650114e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.823+0000 7f5654c29700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565010bd10 0x7f56501153d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.823+0000 7f5654c29700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5650116410 0x7f5650116880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.823+0000 7f5654c29700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5650116dc0 con 0x7f565010bd10 2023-07-21T22:33:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.823+0000 7f5654c29700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5650116f30 con 0x7f5650070b30 2023-07-21T22:33:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.823+0000 7f5654c29700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56501170f0 con 0x7f5650116410 2023-07-21T22:33:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.824+0000 7f564ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5650116410 0x7f5650116880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.824+0000 7f564ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5650116410 0x7f5650116880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38902/0 (socket says 172.21.15.107:38902) 2023-07-21T22:33:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.824+0000 7f564ffff700 1 -- 172.21.15.107:0/2496484436 learned_addr learned my addr 172.21.15.107:0/2496484436 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.824+0000 7f564f7fe700 1 --2- 172.21.15.107:0/2496484436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5650070b30 0x7f5650114e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.826+0000 7f564ffff700 1 -- 172.21.15.107:0/2496484436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5650070b30 msgr2=0x7f5650114e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.826+0000 7f564ffff700 1 --2- 172.21.15.107:0/2496484436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5650070b30 0x7f5650114e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.826+0000 7f564ffff700 1 -- 172.21.15.107:0/2496484436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565010bd10 msgr2=0x7f56501153d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.826+0000 7f564ffff700 1 --2- 172.21.15.107:0/2496484436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565010bd10 0x7f56501153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.826+0000 7f564ffff700 1 -- 172.21.15.107:0/2496484436 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5648009d00 con 0x7f5650116410 2023-07-21T22:33:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.834+0000 7f564ffff700 1 --2- 172.21.15.107:0/2496484436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5650116410 0x7f5650116880 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f564400b4a0 tx=0x7f564400b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.852+0000 7f564cff9700 1 -- 172.21.15.107:0/2496484436 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644011830 con 0x7f5650116410 2023-07-21T22:33:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.852+0000 7f564cff9700 1 -- 172.21.15.107:0/2496484436 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5644011e50 con 0x7f5650116410 2023-07-21T22:33:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.852+0000 7f564cff9700 1 -- 172.21.15.107:0/2496484436 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644010e40 con 0x7f5650116410 2023-07-21T22:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.855+0000 7f5654c29700 1 -- 172.21.15.107:0/2496484436 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56501b5680 con 0x7f5650116410 2023-07-21T22:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.855+0000 7f5654c29700 1 -- 172.21.15.107:0/2496484436 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56501b5b30 con 0x7f5650116410 2023-07-21T22:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.861+0000 7f5662010b80 1 Processor -- start 2023-07-21T22:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.863+0000 7f5662010b80 1 -- start start 2023-07-21T22:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.863+0000 7f5662010b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 0x5628b15f1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.863+0000 7f5662010b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b15f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.863+0000 7f5662010b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b15f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.863+0000 7f5662010b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b14f5c50 con 0x5628b144c6b0 2023-07-21T22:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.863+0000 7f5662010b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b15489e0 con 0x5628b15f0630 2023-07-21T22:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.863+0000 7f5662010b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b1541e70 con 0x5628b15f1a50 2023-07-21T22:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.864+0000 7f564f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b15f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.864+0000 7f564f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b15f00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38912/0 (socket says 172.21.15.107:38912) 2023-07-21T22:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.864+0000 7f564f7fe700 1 -- 172.21.15.107:0/154172361 learned_addr learned my addr 172.21.15.107:0/154172361 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.864+0000 7f564effd700 1 --2- 172.21.15.107:0/154172361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b15f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.864+0000 7f564f7fe700 1 -- 172.21.15.107:0/154172361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 msgr2=0x5628b15f0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.864+0000 7f564f7fe700 1 --2- 172.21.15.107:0/154172361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b15f0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.864+0000 7f564f7fe700 1 -- 172.21.15.107:0/154172361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 msgr2=0x5628b15f1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.864+0000 7f564f7fe700 1 --2- 172.21.15.107:0/154172361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 0x5628b15f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.864+0000 7f564f7fe700 1 -- 172.21.15.107:0/154172361 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b136fd80 con 0x5628b15f1a50 2023-07-21T22:33:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.865+0000 7f564cff9700 1 -- 172.21.15.107:0/2496484436 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f564401a400 con 0x7f5650116410 2023-07-21T22:33:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.866+0000 7f564cff9700 1 --2- 172.21.15.107:0/2496484436 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56380a1350 0x7f56380a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.866+0000 7f564cff9700 1 -- 172.21.15.107:0/2496484436 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f56440cd230 con 0x7f5650116410 2023-07-21T22:33:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.867+0000 7f564f7fe700 1 --2- 172.21.15.107:0/154172361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b15f00f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5640009aa0 tx=0x7f5640009db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=add5b40f1aec78d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.868+0000 7f564effd700 1 --2- 172.21.15.107:0/2496484436 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56380a1350 0x7f56380a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.884+0000 7f5635ffb700 1 -- 172.21.15.107:0/154172361 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564000e990 con 0x5628b15f1a50 2023-07-21T22:33:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.884+0000 7f5635ffb700 1 -- 172.21.15.107:0/154172361 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5640013470 con 0x5628b15f1a50 2023-07-21T22:33:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.884+0000 7f5635ffb700 1 -- 172.21.15.107:0/154172361 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640012630 con 0x5628b15f1a50 2023-07-21T22:33:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.885+0000 7f5662010b80 1 -- 172.21.15.107:0/154172361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 msgr2=0x5628b15f00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.885+0000 7f5662010b80 1 --2- 172.21.15.107:0/154172361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b15f00f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5640009aa0 tx=0x7f5640009db0 comp rx=0 tx=0).stop 2023-07-21T22:33:49.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.908+0000 7f564effd700 1 --2- 172.21.15.107:0/2496484436 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56380a1350 0x7f56380a3800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5648007bd0 tx=0x7f5648006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.944+0000 7f5662010b80 1 -- 172.21.15.107:0/154172361 shutdown_connections 2023-07-21T22:33:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.944+0000 7f5662010b80 1 --2- 172.21.15.107:0/154172361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b15f0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.944+0000 7f5662010b80 1 --2- 172.21.15.107:0/154172361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 0x5628b15f1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.944+0000 7f5662010b80 1 --2- 172.21.15.107:0/154172361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b15f00f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.944+0000 7f5662010b80 1 -- 172.21.15.107:0/154172361 >> 172.21.15.107:0/154172361 conn(0x5628b15e5c40 msgr2=0x5628b15eae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.946+0000 7f5662010b80 1 -- 172.21.15.107:0/154172361 shutdown_connections 2023-07-21T22:33:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.946+0000 7f5662010b80 1 -- 172.21.15.107:0/154172361 wait complete. 2023-07-21T22:33:49.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.946+0000 7f5662010b80 1 -- start start 2023-07-21T22:33:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.946+0000 7f5662010b80 1 Processor -- start 2023-07-21T22:33:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.947+0000 7f5662010b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 0x5628b15f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.947+0000 7f5662010b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b15f9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.947+0000 7f5662010b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b1600f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.947+0000 7f5662010b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b14e1a60 con 0x5628b144c6b0 2023-07-21T22:33:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.947+0000 7f5662010b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b14e1760 con 0x5628b15f0630 2023-07-21T22:33:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.947+0000 7f5662010b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b13c58c0 con 0x5628b15f1a50 2023-07-21T22:33:49.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.947+0000 7f5662010b80 10 client.0 _ll_register_callbacks cb 0x5628b14df700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.947+0000 7f564effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b15f9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.947+0000 7f564ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b1600f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.947+0000 7f564ffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b1600f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38920/0 (socket says 172.21.15.107:38920) 2023-07-21T22:33:49.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.947+0000 7f564ffff700 1 -- 172.21.15.107:0/3712886195 learned_addr learned my addr 172.21.15.107:0/3712886195 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.956+0000 7f564effd700 1 -- 172.21.15.107:0/3712886195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 msgr2=0x5628b1600f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.956+0000 7f564effd700 1 --2- 172.21.15.107:0/3712886195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b1600f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.956+0000 7f564effd700 1 -- 172.21.15.107:0/3712886195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 msgr2=0x5628b15f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.956+0000 7f564effd700 1 --2- 172.21.15.107:0/3712886195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 0x5628b15f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.956+0000 7f564effd700 1 -- 172.21.15.107:0/3712886195 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56400096d0 con 0x5628b15f0630 2023-07-21T22:33:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:49.958+0000 7f564effd700 1 --2- 172.21.15.107:0/3712886195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b15f9f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5648008000 tx=0x7f5648000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3750b80b4b83d1dd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:50.111 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.063+0000 7f56337f6700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:33:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.063+0000 7f56337f6700 1 -- 172.21.15.107:0/3712886195 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564802c070 con 0x5628b15f0630 2023-07-21T22:33:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.063+0000 7f56337f6700 1 -- 172.21.15.107:0/3712886195 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5648023070 con 0x5628b15f0630 2023-07-21T22:33:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.064+0000 7f5662010b80 1 -- 172.21.15.107:0/3712886195 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628b149b490 con 0x5628b15f0630 2023-07-21T22:33:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.086+0000 7f56337f6700 1 -- 172.21.15.107:0/3712886195 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5648007870 con 0x5628b15f0630 2023-07-21T22:33:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.086+0000 7f5662010b80 10 client.25823 fetch_fsmap learned FSMap version 39 2023-07-21T22:33:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.087+0000 7f5662010b80 1 -- 172.21.15.107:0/3712886195 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5628b1358c00 con 0x5628b15f0630 2023-07-21T22:33:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.095+0000 7f56337f6700 1 -- 172.21.15.107:0/3712886195 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5628b149b490 con 0x5628b15f0630 2023-07-21T22:33:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.095+0000 7f56337f6700 1 -- 172.21.15.107:0/3712886195 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f564802f030 con 0x5628b15f0630 2023-07-21T22:33:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.096+0000 7f5662010b80 10 client.25823 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:33:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.096+0000 7f5662010b80 10 client.25823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.096+0000 7f5662010b80 20 client.25823 populate_metadata read hostname 'smithi107' 2023-07-21T22:33:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.096+0000 7f5662010b80 1 --2- 172.21.15.107:0/3712886195 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5628b1605790 0x5628b1607ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.096+0000 7f5662010b80 4 client.25823 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.096+0000 7f5662010b80 1 -- 172.21.15.107:0/3712886195 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628b153ae50 con 0x5628b1605790 2023-07-21T22:33:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.096+0000 7f564f7fe700 1 --2- 172.21.15.107:0/3712886195 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5628b1605790 0x5628b1607ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.107+0000 7f564f7fe700 1 --2- 172.21.15.107:0/3712886195 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5628b1605790 0x5628b1607ba0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c11216b410fc525 server_cookie=6c25b6210f1b3fb7 in_seq=0 out_seq=0 2023-07-21T22:33:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.107+0000 7f56337f6700 10 client.25823 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.110+0000 7f56337f6700 1 -- 172.21.15.107:0/3712886195 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5628b153ae50 con 0x5628b1605790 2023-07-21T22:33:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.110+0000 7f56337f6700 10 client.25823 handle_command_reply: tid=0 2023-07-21T22:33:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.110+0000 7f5662010b80 1 client.25823 shutdown 2023-07-21T22:33:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.112+0000 7f5662010b80 1 -- 172.21.15.107:0/3712886195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 msgr2=0x5628b15f9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.112+0000 7f5662010b80 1 --2- 172.21.15.107:0/3712886195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b15f9f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5648008000 tx=0x7f5648000f40 comp rx=0 tx=0).stop 2023-07-21T22:33:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.113+0000 7f5662010b80 1 -- 172.21.15.107:0/3712886195 shutdown_connections 2023-07-21T22:33:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.113+0000 7f5662010b80 1 --2- 172.21.15.107:0/3712886195 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5628b1605790 0x5628b1607ba0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.113+0000 7f5662010b80 1 --2- 172.21.15.107:0/3712886195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b1600f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.113+0000 7f5662010b80 1 --2- 172.21.15.107:0/3712886195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 0x5628b15f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.113+0000 7f5662010b80 1 --2- 172.21.15.107:0/3712886195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b15f9f00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.113+0000 7f5662010b80 1 -- 172.21.15.107:0/3712886195 >> 172.21.15.107:0/3712886195 conn(0x5628b15e5c40 msgr2=0x5628b15ea540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.113+0000 7f56337f6700 0 client.25823 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.113+0000 7f5662010b80 1 -- 172.21.15.107:0/3712886195 shutdown_connections 2023-07-21T22:33:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.113+0000 7f5662010b80 1 -- 172.21.15.107:0/3712886195 wait complete. 2023-07-21T22:33:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.113+0000 7f5662010b80 20 client.25823 trim_cache size 0 max 16384 2023-07-21T22:33:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.162+0000 7f5662010b80 1 Processor -- start 2023-07-21T22:33:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.162+0000 7f5662010b80 1 -- start start 2023-07-21T22:33:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.162+0000 7f5662010b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 0x5628b15f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.162+0000 7f5662010b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b1600f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.162+0000 7f5662010b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b15f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.162+0000 7f5662010b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b15f1f60 con 0x5628b144c6b0 2023-07-21T22:33:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.163+0000 7f5662010b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b15e5640 con 0x5628b15f0630 2023-07-21T22:33:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.163+0000 7f5662010b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b15f1070 con 0x5628b15f1a50 2023-07-21T22:33:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.164+0000 7f564f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 0x5628b15f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.164+0000 7f564f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 0x5628b15f00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50366/0 (socket says 172.21.15.107:50366) 2023-07-21T22:33:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.164+0000 7f564f7fe700 1 -- 172.21.15.107:0/1340125743 learned_addr learned my addr 172.21.15.107:0/1340125743 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.165+0000 7f564f7fe700 1 -- 172.21.15.107:0/1340125743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 msgr2=0x5628b15f7b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:33:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.165+0000 7f564f7fe700 1 --2- 172.21.15.107:0/1340125743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b15f7b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.165+0000 7f564f7fe700 1 -- 172.21.15.107:0/1340125743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 msgr2=0x5628b1600f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:33:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.165+0000 7f564f7fe700 1 --2- 172.21.15.107:0/1340125743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b1600f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.165+0000 7f564f7fe700 1 -- 172.21.15.107:0/1340125743 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b13961f0 con 0x5628b144c6b0 2023-07-21T22:33:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.165+0000 7f564f7fe700 1 --2- 172.21.15.107:0/1340125743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 0x5628b15f00a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f564000e4b0 tx=0x7f5640015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11c76d3e30bea35 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.166+0000 7f56367fc700 1 -- 172.21.15.107:0/1340125743 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640013d40 con 0x5628b144c6b0 2023-07-21T22:33:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.166+0000 7f56367fc700 1 -- 172.21.15.107:0/1340125743 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5640023250 con 0x5628b144c6b0 2023-07-21T22:33:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.166+0000 7f56367fc700 1 -- 172.21.15.107:0/1340125743 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640004280 con 0x5628b144c6b0 2023-07-21T22:33:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.167+0000 7f5662010b80 1 -- 172.21.15.107:0/1340125743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 msgr2=0x5628b15f00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.167+0000 7f5662010b80 1 --2- 172.21.15.107:0/1340125743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 0x5628b15f00a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f564000e4b0 tx=0x7f5640015040 comp rx=0 tx=0).stop 2023-07-21T22:33:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.168+0000 7f5662010b80 1 -- 172.21.15.107:0/1340125743 shutdown_connections 2023-07-21T22:33:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.168+0000 7f5662010b80 1 --2- 172.21.15.107:0/1340125743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b15f1a50 0x5628b15f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.168+0000 7f5662010b80 1 --2- 172.21.15.107:0/1340125743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b144c6b0 0x5628b15f00a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.168+0000 7f5662010b80 1 --2- 172.21.15.107:0/1340125743 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b1600f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.168+0000 7f5662010b80 1 -- 172.21.15.107:0/1340125743 >> 172.21.15.107:0/1340125743 conn(0x5628b15e5c40 msgr2=0x5628b15f4630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.168+0000 7f5662010b80 1 -- 172.21.15.107:0/1340125743 shutdown_connections 2023-07-21T22:33:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.168+0000 7f5662010b80 1 -- 172.21.15.107:0/1340125743 wait complete. 2023-07-21T22:33:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.169+0000 7f5662010b80 1 -- start start 2023-07-21T22:33:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.169+0000 7f5662010b80 1 Processor -- start 2023-07-21T22:33:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.169+0000 7f5662010b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b144c6b0 0x5628b15fced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.169+0000 7f5662010b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b15fd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.169+0000 7f5662010b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b15f1a50 0x5628b1600430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.169+0000 7f5662010b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b1600970 con 0x5628b15f1a50 2023-07-21T22:33:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.169+0000 7f5662010b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b1600ae0 con 0x5628b15f0630 2023-07-21T22:33:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.169+0000 7f5662010b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b1600ca0 con 0x5628b144c6b0 2023-07-21T22:33:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.169+0000 7f5662010b80 10 client.0 _ll_register_callbacks cb 0x7f5648007e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.171+0000 7f564f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b144c6b0 0x5628b15fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.171+0000 7f564f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b144c6b0 0x5628b15fced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38934/0 (socket says 172.21.15.107:38934) 2023-07-21T22:33:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.171+0000 7f564f7fe700 1 -- 172.21.15.107:0/3903064853 learned_addr learned my addr 172.21.15.107:0/3903064853 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:33:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.171+0000 7f564effd700 1 --2- 172.21.15.107:0/3903064853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b15fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.172+0000 7f564ffff700 1 --2- 172.21.15.107:0/3903064853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b15f1a50 0x5628b1600430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.172+0000 7f564f7fe700 1 -- 172.21.15.107:0/3903064853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 msgr2=0x5628b15fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.172+0000 7f564f7fe700 1 --2- 172.21.15.107:0/3903064853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b15fd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.172+0000 7f564f7fe700 1 -- 172.21.15.107:0/3903064853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b15f1a50 msgr2=0x5628b1600430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.172+0000 7f564f7fe700 1 --2- 172.21.15.107:0/3903064853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b15f1a50 0x5628b1600430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.172+0000 7f564f7fe700 1 -- 172.21.15.107:0/3903064853 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628b136fd80 con 0x5628b144c6b0 2023-07-21T22:33:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.173+0000 7f564f7fe700 1 --2- 172.21.15.107:0/3903064853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b144c6b0 0x5628b15fced0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5640009030 tx=0x7f56400230e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e82eb322032dee20 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.173+0000 7f56337f6700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:33:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.173+0000 7f56337f6700 1 -- 172.21.15.107:0/3903064853 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56400137e0 con 0x5628b144c6b0 2023-07-21T22:33:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.173+0000 7f56337f6700 1 -- 172.21.15.107:0/3903064853 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5640004020 con 0x5628b144c6b0 2023-07-21T22:33:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.174+0000 7f5662010b80 1 -- 172.21.15.107:0/3903064853 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5640013470 con 0x5628b144c6b0 2023-07-21T22:33:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.197+0000 7f56337f6700 1 -- 172.21.15.107:0/3903064853 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5640013940 con 0x5628b144c6b0 2023-07-21T22:33:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.199+0000 7f5662010b80 10 client.25882 fetch_fsmap learned FSMap version 39 2023-07-21T22:33:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.199+0000 7f5662010b80 1 -- 172.21.15.107:0/3903064853 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5628b136e250 con 0x5628b144c6b0 2023-07-21T22:33:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.207+0000 7f56337f6700 1 -- 172.21.15.107:0/3903064853 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5640013470 con 0x5628b144c6b0 2023-07-21T22:33:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.207+0000 7f56337f6700 1 -- 172.21.15.107:0/3903064853 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f5640011070 con 0x5628b144c6b0 2023-07-21T22:33:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.207+0000 7f5662010b80 10 client.25882 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:33:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.207+0000 7f5662010b80 10 client.25882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.207+0000 7f5662010b80 20 client.25882 populate_metadata read hostname 'smithi107' 2023-07-21T22:33:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.207+0000 7f5662010b80 1 --2- 172.21.15.107:0/3903064853 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5628b165a2b0 0x5628b165c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.207+0000 7f5662010b80 4 client.25882 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:33:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.207+0000 7f5662010b80 1 -- 172.21.15.107:0/3903064853 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5628b14fcff0 con 0x5628b165a2b0 2023-07-21T22:33:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.209+0000 7f564effd700 1 --2- 172.21.15.107:0/3903064853 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5628b165a2b0 0x5628b165c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.231+0000 7f564effd700 1 --2- 172.21.15.107:0/3903064853 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5628b165a2b0 0x5628b165c6c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cfd2bb2ee8e6c5 server_cookie=976f36a22f1d595a in_seq=0 out_seq=0 2023-07-21T22:33:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.239+0000 7f56337f6700 10 client.25882 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.927+0000 7f56337f6700 1 -- 172.21.15.107:0/3903064853 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5628b14fcff0 con 0x5628b165a2b0 2023-07-21T22:33:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.927+0000 7f56337f6700 10 client.25882 handle_command_reply: tid=0 2023-07-21T22:33:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.927+0000 7f5662010b80 1 client.25882 shutdown 2023-07-21T22:33:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.928+0000 7f5662010b80 1 -- 172.21.15.107:0/3903064853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b144c6b0 msgr2=0x5628b15fced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.928+0000 7f5662010b80 1 --2- 172.21.15.107:0/3903064853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b144c6b0 0x5628b15fced0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5640009030 tx=0x7f56400230e0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.931+0000 7f5662010b80 1 -- 172.21.15.107:0/3903064853 shutdown_connections 2023-07-21T22:33:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.931+0000 7f5662010b80 1 --2- 172.21.15.107:0/3903064853 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5628b165a2b0 0x5628b165c6c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.931+0000 7f5662010b80 1 --2- 172.21.15.107:0/3903064853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628b15f1a50 0x5628b1600430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.931+0000 7f5662010b80 1 --2- 172.21.15.107:0/3903064853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5628b144c6b0 0x5628b15fced0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.931+0000 7f5662010b80 1 --2- 172.21.15.107:0/3903064853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5628b15f0630 0x5628b15fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.931+0000 7f5662010b80 1 -- 172.21.15.107:0/3903064853 >> 172.21.15.107:0/3903064853 conn(0x5628b15e5c40 msgr2=0x5628b15f7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.931+0000 7f56337f6700 0 client.25882 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:33:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.931+0000 7f5662010b80 1 -- 172.21.15.107:0/3903064853 shutdown_connections 2023-07-21T22:33:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.931+0000 7f5662010b80 1 -- 172.21.15.107:0/3903064853 wait complete. 2023-07-21T22:33:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:50.932+0000 7f5662010b80 20 client.25882 trim_cache size 0 max 16384 2023-07-21T22:33:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:33:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:33:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:33:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:33:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:51.020+0000 7f5654c29700 1 -- 172.21.15.107:0/2496484436 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56380a1350 msgr2=0x7f56380a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:51.020+0000 7f5654c29700 1 --2- 172.21.15.107:0/2496484436 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56380a1350 0x7f56380a3800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5648007bd0 tx=0x7f5648006210 comp rx=0 tx=0).stop 2023-07-21T22:33:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:51.020+0000 7f5654c29700 1 -- 172.21.15.107:0/2496484436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5650116410 msgr2=0x7f5650116880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:51.020+0000 7f5654c29700 1 --2- 172.21.15.107:0/2496484436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5650116410 0x7f5650116880 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f564400b4a0 tx=0x7f564400b7b0 comp rx=0 tx=0).stop 2023-07-21T22:33:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:51.021+0000 7f5654c29700 1 -- 172.21.15.107:0/2496484436 shutdown_connections 2023-07-21T22:33:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:51.021+0000 7f5654c29700 1 --2- 172.21.15.107:0/2496484436 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56380a1350 0x7f56380a3800 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:51.021+0000 7f5654c29700 1 --2- 172.21.15.107:0/2496484436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5650116410 0x7f5650116880 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:51.021+0000 7f5654c29700 1 --2- 172.21.15.107:0/2496484436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5650070b30 0x7f5650114e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:51.021+0000 7f5654c29700 1 --2- 172.21.15.107:0/2496484436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565010bd10 0x7f56501153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:51.021+0000 7f5654c29700 1 -- 172.21.15.107:0/2496484436 >> 172.21.15.107:0/2496484436 conn(0x7f565006c910 msgr2=0x7f565006e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:51.060+0000 7f5654c29700 1 -- 172.21.15.107:0/2496484436 shutdown_connections 2023-07-21T22:33:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:33:51.060+0000 7f5654c29700 1 -- 172.21.15.107:0/2496484436 wait complete. 2023-07-21T22:33:51.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:51 smithi190 ceph-mon[98334]: pgmap v437: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 222 op/s 2023-07-21T22:33:51.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:51 smithi132 ceph-mon[97863]: pgmap v437: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 222 op/s 2023-07-21T22:33:51.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:51 smithi107 ceph-mon[96543]: pgmap v437: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 222 op/s 2023-07-21T22:33:53.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:53 smithi190 ceph-mon[98334]: pgmap v438: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2023-07-21T22:33:53.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:53 smithi132 ceph-mon[97863]: pgmap v438: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2023-07-21T22:33:53.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:53 smithi107 ceph-mon[96543]: pgmap v438: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2023-07-21T22:33:55.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:55 smithi190 ceph-mon[98334]: pgmap v439: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 343 op/s 2023-07-21T22:33:55.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:55 smithi132 ceph-mon[97863]: pgmap v439: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 343 op/s 2023-07-21T22:33:55.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:55 smithi107 ceph-mon[96543]: pgmap v439: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 343 op/s 2023-07-21T22:33:56.544 INFO:tasks.workunit.client.0.smithi107.stdout: 297 2689024 719 1856559 547 1005461 414 2023-07-21T22:33:57.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:57 smithi132 ceph-mon[97863]: pgmap v440: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 405 op/s 2023-07-21T22:33:57.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:57 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:33:57.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:57 smithi107 ceph-mon[96543]: pgmap v440: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 405 op/s 2023-07-21T22:33:57.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:57 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:33:57.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:57 smithi190 ceph-mon[98334]: pgmap v440: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 405 op/s 2023-07-21T22:33:57.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:57 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:33:58.052 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:33:59.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:33:59 smithi190 ceph-mon[98334]: pgmap v441: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 482 op/s 2023-07-21T22:33:59.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:33:59 smithi132 ceph-mon[97863]: pgmap v441: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 482 op/s 2023-07-21T22:33:59.955 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:33:59 smithi107 ceph-mon[96543]: pgmap v441: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 482 op/s 2023-07-21T22:34:00.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.082+0000 7fd375992700 1 -- 172.21.15.107:0/37631741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd370070fb0 msgr2=0x7fd3700713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:00.130 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.082+0000 7fd375992700 1 --2- 172.21.15.107:0/37631741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd370070fb0 0x7fd3700713c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd360009990 tx=0x7fd360009ca0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.131 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.084+0000 7fd375992700 1 -- 172.21.15.107:0/37631741 shutdown_connections 2023-07-21T22:34:00.132 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.084+0000 7fd375992700 1 --2- 172.21.15.107:0/37631741 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd37010bff0 0x7fd37010e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.132 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.084+0000 7fd375992700 1 --2- 172.21.15.107:0/37631741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd370070fb0 0x7fd3700713c0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd360009990 tx=0x7fd360009ca0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.132 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.084+0000 7fd375992700 1 --2- 172.21.15.107:0/37631741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd370109630 0x7fd37010bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.133 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.084+0000 7fd375992700 1 -- 172.21.15.107:0/37631741 >> 172.21.15.107:0/37631741 conn(0x7fd37006ca30 msgr2=0x7fd37006ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:00.165 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.118+0000 7fd375992700 1 -- 172.21.15.107:0/37631741 shutdown_connections 2023-07-21T22:34:00.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.118+0000 7fd375992700 1 -- 172.21.15.107:0/37631741 wait complete. 2023-07-21T22:34:00.166 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.119+0000 7fd375992700 1 Processor -- start 2023-07-21T22:34:00.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.120+0000 7fd375992700 1 -- start start 2023-07-21T22:34:00.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.120+0000 7fd375992700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd370109630 0x7fd370114a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.120+0000 7fd375992700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd37010bff0 0x7fd370114f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.120+0000 7fd375992700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd370115f80 0x7fd3701163f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.120+0000 7fd375992700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd370116930 con 0x7fd37010bff0 2023-07-21T22:34:00.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.120+0000 7fd375992700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd370116aa0 con 0x7fd370109630 2023-07-21T22:34:00.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.120+0000 7fd375992700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd370116c60 con 0x7fd370115f80 2023-07-21T22:34:00.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.120+0000 7fd36f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd370115f80 0x7fd3701163f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:00.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.120+0000 7fd36f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd370115f80 0x7fd3701163f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38948/0 (socket says 172.21.15.107:38948) 2023-07-21T22:34:00.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.120+0000 7fd36f7fe700 1 -- 172.21.15.107:0/2683137184 learned_addr learned my addr 172.21.15.107:0/2683137184 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:00.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.121+0000 7fd36effd700 1 --2- 172.21.15.107:0/2683137184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd370109630 0x7fd370114a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:00.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.121+0000 7fd36f7fe700 1 -- 172.21.15.107:0/2683137184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd370109630 msgr2=0x7fd370114a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:00.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.121+0000 7fd36f7fe700 1 --2- 172.21.15.107:0/2683137184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd370109630 0x7fd370114a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.121+0000 7fd36f7fe700 1 -- 172.21.15.107:0/2683137184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd37010bff0 msgr2=0x7fd370114f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:00.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.121+0000 7fd36f7fe700 1 --2- 172.21.15.107:0/2683137184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd37010bff0 0x7fd370114f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.121+0000 7fd36f7fe700 1 -- 172.21.15.107:0/2683137184 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3600095f0 con 0x7fd370115f80 2023-07-21T22:34:00.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.122+0000 7fd36f7fe700 1 --2- 172.21.15.107:0/2683137184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd370115f80 0x7fd3701163f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd36400d980 tx=0x7fd36400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:00.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.133+0000 7fd374990700 1 -- 172.21.15.107:0/2683137184 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd364004a30 con 0x7fd370115f80 2023-07-21T22:34:00.188 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.133+0000 7fd374990700 1 -- 172.21.15.107:0/2683137184 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd36400b5c0 con 0x7fd370115f80 2023-07-21T22:34:00.189 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.133+0000 7fd374990700 1 -- 172.21.15.107:0/2683137184 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd364009b50 con 0x7fd370115f80 2023-07-21T22:34:00.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.136+0000 7fd375992700 1 -- 172.21.15.107:0/2683137184 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd370116f40 con 0x7fd370115f80 2023-07-21T22:34:00.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.136+0000 7fd375992700 1 -- 172.21.15.107:0/2683137184 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3701b5b20 con 0x7fd370115f80 2023-07-21T22:34:00.204 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.144+0000 7fd381b17b80 1 Processor -- start 2023-07-21T22:34:00.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.157+0000 7fd381b17b80 1 -- start start 2023-07-21T22:34:00.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.158+0000 7fd381b17b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b51c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.205 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.158+0000 7fd381b17b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b521a0 0x565179b50430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.206 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.158+0000 7fd381b17b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b50e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.206 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.158+0000 7fd381b17b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x565179a55cb0 con 0x565179b521a0 2023-07-21T22:34:00.206 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.158+0000 7fd381b17b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x565179aa8a40 con 0x565179b50a10 2023-07-21T22:34:00.206 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.158+0000 7fd381b17b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x565179aa1ed0 con 0x565179b51830 2023-07-21T22:34:00.206 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.158+0000 7fd36e7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b51c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:00.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.158+0000 7fd36e7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b51c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38952/0 (socket says 172.21.15.107:38952) 2023-07-21T22:34:00.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.158+0000 7fd36e7fc700 1 -- 172.21.15.107:0/3067864785 learned_addr learned my addr 172.21.15.107:0/3067864785 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:00.207 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.158+0000 7fd36e7fc700 1 --2- 172.21.15.107:0/3067864785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b50e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:00.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.160+0000 7fd36e7fc700 1 -- 172.21.15.107:0/3067864785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 msgr2=0x565179b50e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:00.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.160+0000 7fd36e7fc700 1 --2- 172.21.15.107:0/3067864785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b50e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.160+0000 7fd36e7fc700 1 -- 172.21.15.107:0/3067864785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b521a0 msgr2=0x565179b50430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:00.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.160+0000 7fd36e7fc700 1 --2- 172.21.15.107:0/3067864785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b521a0 0x565179b50430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.208 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.160+0000 7fd36e7fc700 1 -- 172.21.15.107:0/3067864785 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651798cfde0 con 0x565179b51830 2023-07-21T22:34:00.209 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.161+0000 7fd36e7fc700 1 --2- 172.21.15.107:0/3067864785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b51c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd368010760 tx=0x7fd368009d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ad62da4bb61e7c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:00.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.168+0000 7fd374990700 1 -- 172.21.15.107:0/2683137184 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd364009cb0 con 0x7fd370115f80 2023-07-21T22:34:00.217 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.169+0000 7fd374990700 1 --2- 172.21.15.107:0/2683137184 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd3580a1350 0x7fd3580a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.218 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.170+0000 7fd374990700 1 -- 172.21.15.107:0/2683137184 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd3640cd600 con 0x7fd370115f80 2023-07-21T22:34:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.172+0000 7fd355ffb700 1 -- 172.21.15.107:0/3067864785 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36800e070 con 0x565179b51830 2023-07-21T22:34:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.172+0000 7fd355ffb700 1 -- 172.21.15.107:0/3067864785 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3680091d0 con 0x565179b51830 2023-07-21T22:34:00.220 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.172+0000 7fd355ffb700 1 -- 172.21.15.107:0/3067864785 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368007b50 con 0x565179b51830 2023-07-21T22:34:00.227 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.173+0000 7fd381b17b80 1 -- 172.21.15.107:0/3067864785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 msgr2=0x565179b51c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:00.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.173+0000 7fd381b17b80 1 --2- 172.21.15.107:0/3067864785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b51c40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd368010760 tx=0x7fd368009d60 comp rx=0 tx=0).stop 2023-07-21T22:34:00.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.180+0000 7fd381b17b80 1 -- 172.21.15.107:0/3067864785 shutdown_connections 2023-07-21T22:34:00.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.180+0000 7fd381b17b80 1 --2- 172.21.15.107:0/3067864785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b50e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.180+0000 7fd381b17b80 1 --2- 172.21.15.107:0/3067864785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b51c40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.180+0000 7fd381b17b80 1 --2- 172.21.15.107:0/3067864785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b521a0 0x565179b50430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.180+0000 7fd381b17b80 1 -- 172.21.15.107:0/3067864785 >> 172.21.15.107:0/3067864785 conn(0x565179b45ef0 msgr2=0x565179b4b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:00.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.185+0000 7fd36effd700 1 --2- 172.21.15.107:0/2683137184 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd3580a1350 0x7fd3580a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:00.233 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.186+0000 7fd381b17b80 1 -- 172.21.15.107:0/3067864785 shutdown_connections 2023-07-21T22:34:00.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.186+0000 7fd381b17b80 1 -- 172.21.15.107:0/3067864785 wait complete. 2023-07-21T22:34:00.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.187+0000 7fd381b17b80 1 -- start start 2023-07-21T22:34:00.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.187+0000 7fd381b17b80 1 Processor -- start 2023-07-21T22:34:00.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.187+0000 7fd381b17b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b5a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.187+0000 7fd381b17b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b5a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.187+0000 7fd381b17b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b521a0 0x565179b616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.187+0000 7fd381b17b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x565179a41ac0 con 0x565179b521a0 2023-07-21T22:34:00.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.187+0000 7fd381b17b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x565179a417c0 con 0x565179b50a10 2023-07-21T22:34:00.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.187+0000 7fd381b17b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x565179925920 con 0x565179b51830 2023-07-21T22:34:00.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.187+0000 7fd381b17b80 10 client.0 _ll_register_callbacks cb 0x565179a3f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:00.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.188+0000 7fd36e7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b5a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:00.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.188+0000 7fd36e7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b5a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38960/0 (socket says 172.21.15.107:38960) 2023-07-21T22:34:00.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.188+0000 7fd36e7fc700 1 -- 172.21.15.107:0/544295789 learned_addr learned my addr 172.21.15.107:0/544295789 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:00.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.188+0000 7fd36e7fc700 1 -- 172.21.15.107:0/544295789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 msgr2=0x565179b5a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:00.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.188+0000 7fd36e7fc700 1 --2- 172.21.15.107:0/544295789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b5a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.188+0000 7fd36e7fc700 1 -- 172.21.15.107:0/544295789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b521a0 msgr2=0x565179b616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:00.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.188+0000 7fd36e7fc700 1 --2- 172.21.15.107:0/544295789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b521a0 0x565179b616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.188+0000 7fd36e7fc700 1 -- 172.21.15.107:0/544295789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd368011040 con 0x565179b51830 2023-07-21T22:34:00.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.190+0000 7fd36e7fc700 1 --2- 172.21.15.107:0/544295789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b5a6a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd368004b10 tx=0x7fd3680093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37f65003382773d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:00.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.190+0000 7fd3537f6700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:34:00.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.190+0000 7fd3537f6700 1 -- 172.21.15.107:0/544295789 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36800e070 con 0x565179b51830 2023-07-21T22:34:00.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.190+0000 7fd3537f6700 1 -- 172.21.15.107:0/544295789 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd368007ad0 con 0x565179b51830 2023-07-21T22:34:00.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.190+0000 7fd381b17b80 1 -- 172.21.15.107:0/544295789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651799fb4f0 con 0x565179b51830 2023-07-21T22:34:00.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.190+0000 7fd3537f6700 1 -- 172.21.15.107:0/544295789 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd368007c90 con 0x565179b51830 2023-07-21T22:34:00.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.190+0000 7fd381b17b80 10 client.25894 fetch_fsmap learned FSMap version 39 2023-07-21T22:34:00.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.190+0000 7fd381b17b80 1 -- 172.21.15.107:0/544295789 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5651798b8c60 con 0x565179b51830 2023-07-21T22:34:00.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.191+0000 7fd3537f6700 1 -- 172.21.15.107:0/544295789 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5651799fb4f0 con 0x565179b51830 2023-07-21T22:34:00.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.191+0000 7fd3537f6700 1 -- 172.21.15.107:0/544295789 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fd368021410 con 0x565179b51830 2023-07-21T22:34:00.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.192+0000 7fd381b17b80 10 client.25894 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:34:00.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.192+0000 7fd381b17b80 10 client.25894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:00.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.192+0000 7fd381b17b80 20 client.25894 populate_metadata read hostname 'smithi107' 2023-07-21T22:34:00.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.192+0000 7fd381b17b80 1 --2- 172.21.15.107:0/544295789 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x565179b65e20 0x565179b68230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.192+0000 7fd381b17b80 4 client.25894 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:00.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.192+0000 7fd381b17b80 1 -- 172.21.15.107:0/544295789 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565179a9aeb0 con 0x565179b65e20 2023-07-21T22:34:00.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.193+0000 7fd36effd700 1 --2- 172.21.15.107:0/544295789 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x565179b65e20 0x565179b68230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:00.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.193+0000 7fd36effd700 1 --2- 172.21.15.107:0/544295789 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x565179b65e20 0x565179b68230 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac4a529bd5c4731 server_cookie=9913912cc2f46692 in_seq=0 out_seq=0 2023-07-21T22:34:00.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.194+0000 7fd3537f6700 10 client.25894 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:00.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.194+0000 7fd3537f6700 1 -- 172.21.15.107:0/544295789 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x565179a9aeb0 con 0x565179b65e20 2023-07-21T22:34:00.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.194+0000 7fd3537f6700 10 client.25894 handle_command_reply: tid=0 2023-07-21T22:34:00.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.194+0000 7fd381b17b80 1 client.25894 shutdown 2023-07-21T22:34:00.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.194+0000 7fd381b17b80 1 -- 172.21.15.107:0/544295789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 msgr2=0x565179b5a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:00.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.194+0000 7fd381b17b80 1 --2- 172.21.15.107:0/544295789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b5a6a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd368004b10 tx=0x7fd3680093b0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.195+0000 7fd381b17b80 1 -- 172.21.15.107:0/544295789 shutdown_connections 2023-07-21T22:34:00.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.195+0000 7fd381b17b80 1 --2- 172.21.15.107:0/544295789 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x565179b65e20 0x565179b68230 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.195+0000 7fd381b17b80 1 --2- 172.21.15.107:0/544295789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b521a0 0x565179b616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.195+0000 7fd381b17b80 1 --2- 172.21.15.107:0/544295789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b5a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.195+0000 7fd381b17b80 1 --2- 172.21.15.107:0/544295789 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b5a6a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.195+0000 7fd381b17b80 1 -- 172.21.15.107:0/544295789 >> 172.21.15.107:0/544295789 conn(0x565179b45ef0 msgr2=0x565179b4aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:00.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.195+0000 7fd3537f6700 0 client.25894 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:00.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.195+0000 7fd381b17b80 1 -- 172.21.15.107:0/544295789 shutdown_connections 2023-07-21T22:34:00.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.195+0000 7fd381b17b80 1 -- 172.21.15.107:0/544295789 wait complete. 2023-07-21T22:34:00.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.195+0000 7fd381b17b80 20 client.25894 trim_cache size 0 max 16384 2023-07-21T22:34:00.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.211+0000 7fd36effd700 1 --2- 172.21.15.107:0/2683137184 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd3580a1350 0x7fd3580a3800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd360000c00 tx=0x7fd360005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:00.279 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.232+0000 7fd381b17b80 1 Processor -- start 2023-07-21T22:34:00.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.232+0000 7fd381b17b80 1 -- start start 2023-07-21T22:34:00.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.232+0000 7fd381b17b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b5a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.232+0000 7fd381b17b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b5a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.280 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.232+0000 7fd381b17b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b521a0 0x565179b616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.232+0000 7fd381b17b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x565179b45a50 con 0x565179b521a0 2023-07-21T22:34:00.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.232+0000 7fd381b17b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x565179b59cd0 con 0x565179b50a10 2023-07-21T22:34:00.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.232+0000 7fd381b17b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x565179b595b0 con 0x565179b51830 2023-07-21T22:34:00.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.233+0000 7fd36e7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b5a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:00.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.233+0000 7fd36effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b5a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:00.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.233+0000 7fd36e7fc700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b5a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38970/0 (socket says 172.21.15.107:38970) 2023-07-21T22:34:00.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.233+0000 7fd36e7fc700 1 -- 172.21.15.107:0/989327684 learned_addr learned my addr 172.21.15.107:0/989327684 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:00.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.235+0000 7fd36e7fc700 1 -- 172.21.15.107:0/989327684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 msgr2=0x565179b5a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:00.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.235+0000 7fd36e7fc700 1 --2- 172.21.15.107:0/989327684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b5a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.235+0000 7fd36e7fc700 1 -- 172.21.15.107:0/989327684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b521a0 msgr2=0x565179b616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:00.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.235+0000 7fd36e7fc700 1 --2- 172.21.15.107:0/989327684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b521a0 0x565179b616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.235+0000 7fd36e7fc700 1 -- 172.21.15.107:0/989327684 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651798f6250 con 0x565179b51830 2023-07-21T22:34:00.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.235+0000 7fd36e7fc700 1 --2- 172.21.15.107:0/989327684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b5a6a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd368009c10 tx=0x7fd368004a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6450636c221eb05c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:00.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.236+0000 7fd3567fc700 1 -- 172.21.15.107:0/989327684 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36800e070 con 0x565179b51830 2023-07-21T22:34:00.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.236+0000 7fd3567fc700 1 -- 172.21.15.107:0/989327684 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd368019e90 con 0x565179b51830 2023-07-21T22:34:00.284 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.236+0000 7fd3567fc700 1 -- 172.21.15.107:0/989327684 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368021570 con 0x565179b51830 2023-07-21T22:34:00.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.237+0000 7fd381b17b80 1 -- 172.21.15.107:0/989327684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 msgr2=0x565179b5a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:00.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.237+0000 7fd381b17b80 1 --2- 172.21.15.107:0/989327684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b5a6a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd368009c10 tx=0x7fd368004a10 comp rx=0 tx=0).stop 2023-07-21T22:34:00.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.237+0000 7fd381b17b80 1 -- 172.21.15.107:0/989327684 shutdown_connections 2023-07-21T22:34:00.285 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.237+0000 7fd381b17b80 1 --2- 172.21.15.107:0/989327684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b521a0 0x565179b616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.237+0000 7fd381b17b80 1 --2- 172.21.15.107:0/989327684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b5a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.237+0000 7fd381b17b80 1 --2- 172.21.15.107:0/989327684 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b51830 0x565179b5a6a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.237+0000 7fd381b17b80 1 -- 172.21.15.107:0/989327684 >> 172.21.15.107:0/989327684 conn(0x565179b45ef0 msgr2=0x565179b57fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:00.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.238+0000 7fd381b17b80 1 -- 172.21.15.107:0/989327684 shutdown_connections 2023-07-21T22:34:00.286 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.238+0000 7fd381b17b80 1 -- 172.21.15.107:0/989327684 wait complete. 2023-07-21T22:34:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.238+0000 7fd381b17b80 1 -- start start 2023-07-21T22:34:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.238+0000 7fd381b17b80 1 Processor -- start 2023-07-21T22:34:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.239+0000 7fd381b17b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b5d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.239+0000 7fd381b17b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b51830 0x565179b5dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.239+0000 7fd381b17b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b521a0 0x565179b60cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.239+0000 7fd381b17b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x565179b61210 con 0x565179b51830 2023-07-21T22:34:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.239+0000 7fd381b17b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x565179b61380 con 0x565179b50a10 2023-07-21T22:34:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.239+0000 7fd381b17b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x565179b61540 con 0x565179b521a0 2023-07-21T22:34:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.239+0000 7fd381b17b80 10 client.0 _ll_register_callbacks cb 0x7fd368010640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.239+0000 7fd36effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b5d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.239+0000 7fd36f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b521a0 0x565179b60cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.239+0000 7fd36f7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b521a0 0x565179b60cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38976/0 (socket says 172.21.15.107:38976) 2023-07-21T22:34:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.239+0000 7fd36f7fe700 1 -- 172.21.15.107:0/4062296222 learned_addr learned my addr 172.21.15.107:0/4062296222 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.240+0000 7fd36f7fe700 1 -- 172.21.15.107:0/4062296222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 msgr2=0x565179b5d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.240+0000 7fd36f7fe700 1 --2- 172.21.15.107:0/4062296222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b5d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.240+0000 7fd36f7fe700 1 -- 172.21.15.107:0/4062296222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b51830 msgr2=0x565179b5dcb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:34:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.240+0000 7fd36f7fe700 1 --2- 172.21.15.107:0/4062296222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b51830 0x565179b5dcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.240+0000 7fd36f7fe700 1 -- 172.21.15.107:0/4062296222 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651798b8c60 con 0x565179b521a0 2023-07-21T22:34:00.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.241+0000 7fd36f7fe700 1 --2- 172.21.15.107:0/4062296222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b521a0 0x565179b60cd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd364097400 tx=0x7fd36409a5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=733be8927755f04b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:00.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.241+0000 7fd3537f6700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:34:00.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.241+0000 7fd3537f6700 1 -- 172.21.15.107:0/4062296222 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36409e9c0 con 0x565179b521a0 2023-07-21T22:34:00.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.241+0000 7fd3537f6700 1 -- 172.21.15.107:0/4062296222 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd3640a2de0 con 0x565179b521a0 2023-07-21T22:34:00.290 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.241+0000 7fd381b17b80 1 -- 172.21.15.107:0/4062296222 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3680091d0 con 0x565179b521a0 2023-07-21T22:34:00.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.242+0000 7fd3537f6700 1 -- 172.21.15.107:0/4062296222 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd364099e80 con 0x565179b521a0 2023-07-21T22:34:00.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.242+0000 7fd381b17b80 10 client.25900 fetch_fsmap learned FSMap version 39 2023-07-21T22:34:00.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.242+0000 7fd381b17b80 1 -- 172.21.15.107:0/4062296222 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5651798ce2b0 con 0x565179b521a0 2023-07-21T22:34:00.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.243+0000 7fd3537f6700 1 -- 172.21.15.107:0/4062296222 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3680091d0 con 0x565179b521a0 2023-07-21T22:34:00.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.244+0000 7fd3537f6700 1 -- 172.21.15.107:0/4062296222 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fd364096780 con 0x565179b521a0 2023-07-21T22:34:00.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.244+0000 7fd381b17b80 10 client.25900 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:34:00.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.244+0000 7fd381b17b80 10 client.25900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:00.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.244+0000 7fd381b17b80 20 client.25900 populate_metadata read hostname 'smithi107' 2023-07-21T22:34:00.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.244+0000 7fd381b17b80 1 --2- 172.21.15.107:0/4062296222 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x565179bbae20 0x565179bbd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:00.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.244+0000 7fd381b17b80 4 client.25900 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:34:00.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.244+0000 7fd381b17b80 1 -- 172.21.15.107:0/4062296222 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565179a5d050 con 0x565179bbae20 2023-07-21T22:34:00.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.245+0000 7fd36effd700 1 --2- 172.21.15.107:0/4062296222 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x565179bbae20 0x565179bbd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:00.293 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.245+0000 7fd36effd700 1 --2- 172.21.15.107:0/4062296222 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x565179bbae20 0x565179bbd200 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6902ec56da0543a server_cookie=14787db09325fd2b in_seq=0 out_seq=0 2023-07-21T22:34:00.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.245+0000 7fd3537f6700 10 client.25900 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:00.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.245+0000 7fd3537f6700 1 -- 172.21.15.107:0/4062296222 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x565179a5d050 con 0x565179bbae20 2023-07-21T22:34:00.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.245+0000 7fd3537f6700 10 client.25900 handle_command_reply: tid=0 2023-07-21T22:34:00.294 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.245+0000 7fd381b17b80 1 client.25900 shutdown 2023-07-21T22:34:00.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.284+0000 7fd381b17b80 1 -- 172.21.15.107:0/4062296222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b521a0 msgr2=0x565179b60cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:00.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.284+0000 7fd381b17b80 1 --2- 172.21.15.107:0/4062296222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b521a0 0x565179b60cd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd364097400 tx=0x7fd36409a5c0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.285+0000 7fd381b17b80 1 -- 172.21.15.107:0/4062296222 shutdown_connections 2023-07-21T22:34:00.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.285+0000 7fd381b17b80 1 --2- 172.21.15.107:0/4062296222 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x565179bbae20 0x565179bbd200 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.285+0000 7fd381b17b80 1 --2- 172.21.15.107:0/4062296222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x565179b521a0 0x565179b60cd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.285+0000 7fd381b17b80 1 --2- 172.21.15.107:0/4062296222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x565179b50a10 0x565179b5d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.285+0000 7fd381b17b80 1 --2- 172.21.15.107:0/4062296222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565179b51830 0x565179b5dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.285+0000 7fd381b17b80 1 -- 172.21.15.107:0/4062296222 >> 172.21.15.107:0/4062296222 conn(0x565179b45ef0 msgr2=0x565179b54ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:00.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.285+0000 7fd3537f6700 0 client.25900 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:00.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.285+0000 7fd381b17b80 1 -- 172.21.15.107:0/4062296222 shutdown_connections 2023-07-21T22:34:00.334 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.285+0000 7fd381b17b80 1 -- 172.21.15.107:0/4062296222 wait complete. 2023-07-21T22:34:00.334 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.285+0000 7fd381b17b80 20 client.25900 trim_cache size 0 max 16384 2023-07-21T22:34:00.334 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:34:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:34:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:34:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "d037412e-e4e0-4d2c-8d6e-c7193371fe0f": { 2023-07-21T22:34:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:34:00.335 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d037412e-e4e0-4d2c-8d6e-c7193371fe0f", 2023-07-21T22:34:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:34:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:34:00.336 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:34:00.336 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:34:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.323+0000 7fd375992700 1 -- 172.21.15.107:0/2683137184 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd3580a1350 msgr2=0x7fd3580a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.323+0000 7fd375992700 1 --2- 172.21.15.107:0/2683137184 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd3580a1350 0x7fd3580a3800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd360000c00 tx=0x7fd360005c30 comp rx=0 tx=0).stop 2023-07-21T22:34:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.323+0000 7fd375992700 1 -- 172.21.15.107:0/2683137184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd370115f80 msgr2=0x7fd3701163f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.323+0000 7fd375992700 1 --2- 172.21.15.107:0/2683137184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd370115f80 0x7fd3701163f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd36400d980 tx=0x7fd36400dc90 comp rx=0 tx=0).stop 2023-07-21T22:34:00.387 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.340+0000 7fd375992700 1 -- 172.21.15.107:0/2683137184 shutdown_connections 2023-07-21T22:34:00.387 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.340+0000 7fd375992700 1 --2- 172.21.15.107:0/2683137184 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd3580a1350 0x7fd3580a3800 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.387 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.340+0000 7fd375992700 1 --2- 172.21.15.107:0/2683137184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd370115f80 0x7fd3701163f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.387 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.340+0000 7fd375992700 1 --2- 172.21.15.107:0/2683137184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd370109630 0x7fd370114a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.340+0000 7fd375992700 1 --2- 172.21.15.107:0/2683137184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd37010bff0 0x7fd370114f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:00.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.340+0000 7fd375992700 1 -- 172.21.15.107:0/2683137184 >> 172.21.15.107:0/2683137184 conn(0x7fd37006ca30 msgr2=0x7fd37010af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:00.388 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.340+0000 7fd375992700 1 -- 172.21.15.107:0/2683137184 shutdown_connections 2023-07-21T22:34:00.389 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:00.340+0000 7fd375992700 1 -- 172.21.15.107:0/2683137184 wait complete. 2023-07-21T22:34:00.457 INFO:tasks.cephfs.filesystem:scrub status for tag:d037412e-e4e0-4d2c-8d6e-c7193371fe0f - {'path': '/', 'tag': 'd037412e-e4e0-4d2c-8d6e-c7193371fe0f', 'options': 'recursive,force'} 2023-07-21T22:34:01.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:01 smithi190 ceph-mon[98334]: pgmap v442: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 477 op/s 2023-07-21T22:34:01.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:01 smithi132 ceph-mon[97863]: pgmap v442: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 477 op/s 2023-07-21T22:34:01.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:01 smithi107 ceph-mon[96543]: pgmap v442: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 477 op/s 2023-07-21T22:34:02.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:02 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:34:02.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:02 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:34:02.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:02 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:34:02.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:02 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:34:02.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:02 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:34:02.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:02 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:34:02.942 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:02 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:34:02.943 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:02 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:34:02.943 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:02 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:34:03.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:03 smithi190 ceph-mon[98334]: pgmap v443: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 832 op/s 2023-07-21T22:34:03.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:03 smithi132 ceph-mon[97863]: pgmap v443: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 832 op/s 2023-07-21T22:34:03.944 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:03 smithi107 ceph-mon[96543]: pgmap v443: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 832 op/s 2023-07-21T22:34:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.602+0000 7f131efb7700 1 -- 172.21.15.107:0/3488205606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13180711a0 msgr2=0x7f13180715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.602+0000 7f131efb7700 1 --2- 172.21.15.107:0/3488205606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13180711a0 0x7f13180715b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1308009de0 tx=0x7f130800b540 comp rx=0 tx=0).stop 2023-07-21T22:34:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.602+0000 7f131efb7700 1 -- 172.21.15.107:0/3488205606 shutdown_connections 2023-07-21T22:34:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.602+0000 7f131efb7700 1 --2- 172.21.15.107:0/3488205606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1318077240 0x7f13180796c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.602+0000 7f131efb7700 1 --2- 172.21.15.107:0/3488205606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13180711a0 0x7f13180715b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.602+0000 7f131efb7700 1 --2- 172.21.15.107:0/3488205606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1318074880 0x7f1318076d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.602+0000 7f131efb7700 1 -- 172.21.15.107:0/3488205606 >> 172.21.15.107:0/3488205606 conn(0x7f131806ca50 msgr2=0x7f131806eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.602+0000 7f131efb7700 1 -- 172.21.15.107:0/3488205606 shutdown_connections 2023-07-21T22:34:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.602+0000 7f131efb7700 1 -- 172.21.15.107:0/3488205606 wait complete. 2023-07-21T22:34:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.603+0000 7f131efb7700 1 Processor -- start 2023-07-21T22:34:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.604+0000 7f131efb7700 1 -- start start 2023-07-21T22:34:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.605+0000 7f131efb7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13180711a0 0x7f13180825b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.605+0000 7f131efb7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1318074880 0x7f1318082af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.605+0000 7f131efb7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1318077240 0x7f13181ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.605+0000 7f131efb7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13181aeab0 con 0x7f1318074880 2023-07-21T22:34:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.605+0000 7f131efb7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13181aec20 con 0x7f13180711a0 2023-07-21T22:34:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.605+0000 7f131efb7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13181aede0 con 0x7f1318077240 2023-07-21T22:34:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.611+0000 7f1317fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1318074880 0x7f1318082af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.611+0000 7f1317fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1318074880 0x7f1318082af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50428/0 (socket says 172.21.15.107:50428) 2023-07-21T22:34:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.611+0000 7f1317fff700 1 -- 172.21.15.107:0/2087619773 learned_addr learned my addr 172.21.15.107:0/2087619773 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.611+0000 7f131d554700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1318077240 0x7f13181ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.611+0000 7f131cd53700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13180711a0 0x7f13180825b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.613+0000 7f131cd53700 1 -- 172.21.15.107:0/2087619773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1318077240 msgr2=0x7f13181ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.613+0000 7f131cd53700 1 --2- 172.21.15.107:0/2087619773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1318077240 0x7f13181ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.613+0000 7f131cd53700 1 -- 172.21.15.107:0/2087619773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1318074880 msgr2=0x7f1318082af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.613+0000 7f131cd53700 1 --2- 172.21.15.107:0/2087619773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1318074880 0x7f1318082af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.613+0000 7f131cd53700 1 -- 172.21.15.107:0/2087619773 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13080095f0 con 0x7f13180711a0 2023-07-21T22:34:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.613+0000 7f131cd53700 1 --2- 172.21.15.107:0/2087619773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13180711a0 0x7f13180825b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1308005950 tx=0x7f1308009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.632+0000 7f1315ffb700 1 -- 172.21.15.107:0/2087619773 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308024070 con 0x7f13180711a0 2023-07-21T22:34:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.632+0000 7f1315ffb700 1 -- 172.21.15.107:0/2087619773 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1308004460 con 0x7f13180711a0 2023-07-21T22:34:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.632+0000 7f1315ffb700 1 -- 172.21.15.107:0/2087619773 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130800fa00 con 0x7f13180711a0 2023-07-21T22:34:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.632+0000 7f131efb7700 1 -- 172.21.15.107:0/2087619773 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13181af060 con 0x7f13180711a0 2023-07-21T22:34:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.632+0000 7f131efb7700 1 -- 172.21.15.107:0/2087619773 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13181076f0 con 0x7f13180711a0 2023-07-21T22:34:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.640+0000 7f132b13cb80 1 Processor -- start 2023-07-21T22:34:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.644+0000 7f132b13cb80 1 -- start start 2023-07-21T22:34:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.644+0000 7f132b13cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4df26b0 0x555fd4f962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.644+0000 7f132b13cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4f96830 0x555fd4f97040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.644+0000 7f132b13cb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4f97580 0x555fd4f979f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.644+0000 7f132b13cb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4e9bc50 con 0x555fd4f97580 2023-07-21T22:34:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.644+0000 7f132b13cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4eee9e0 con 0x555fd4f96830 2023-07-21T22:34:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.644+0000 7f132b13cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4ee7e70 con 0x555fd4df26b0 2023-07-21T22:34:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.645+0000 7f131d554700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4df26b0 0x555fd4f962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.645+0000 7f131d554700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4df26b0 0x555fd4f962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:38994/0 (socket says 172.21.15.107:38994) 2023-07-21T22:34:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.645+0000 7f131d554700 1 -- 172.21.15.107:0/4144785512 learned_addr learned my addr 172.21.15.107:0/4144785512 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.645+0000 7f131cd53700 1 --2- 172.21.15.107:0/4144785512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4f96830 0x555fd4f97040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.645+0000 7f131d554700 1 -- 172.21.15.107:0/4144785512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4f96830 msgr2=0x555fd4f97040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.645+0000 7f131d554700 1 --2- 172.21.15.107:0/4144785512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4f96830 0x555fd4f97040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.645+0000 7f131d554700 1 -- 172.21.15.107:0/4144785512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4f97580 msgr2=0x555fd4f979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.645+0000 7f131d554700 1 --2- 172.21.15.107:0/4144785512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4f97580 0x555fd4f979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.645+0000 7f131d554700 1 -- 172.21.15.107:0/4144785512 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fd4d15d80 con 0x555fd4df26b0 2023-07-21T22:34:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.646+0000 7f131d554700 1 --2- 172.21.15.107:0/4144785512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4df26b0 0x555fd4f962f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1310013700 tx=0x7f1310009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7781d1f9f9fb6318 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.648+0000 7f1315ffb700 1 -- 172.21.15.107:0/2087619773 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f130800f460 con 0x7f13180711a0 2023-07-21T22:34:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.649+0000 7f1315ffb700 1 --2- 172.21.15.107:0/2087619773 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f13000a1320 0x7f13000a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.650+0000 7f1315ffb700 1 -- 172.21.15.107:0/2087619773 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f13080cd510 con 0x7f13180711a0 2023-07-21T22:34:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.652+0000 7f131cd53700 1 --2- 172.21.15.107:0/2087619773 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f13000a1320 0x7f13000a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.673+0000 7f131cd53700 1 --2- 172.21.15.107:0/2087619773 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f13000a1320 0x7f13000a37d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1308097430 tx=0x7f13080d0f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.770+0000 7f12feffd700 1 -- 172.21.15.107:0/4144785512 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310013250 con 0x555fd4df26b0 2023-07-21T22:34:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.770+0000 7f12feffd700 1 -- 172.21.15.107:0/4144785512 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13100093e0 con 0x555fd4df26b0 2023-07-21T22:34:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.770+0000 7f12feffd700 1 -- 172.21.15.107:0/4144785512 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13100128a0 con 0x555fd4df26b0 2023-07-21T22:34:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.771+0000 7f132b13cb80 1 -- 172.21.15.107:0/4144785512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4df26b0 msgr2=0x555fd4f962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.771+0000 7f132b13cb80 1 --2- 172.21.15.107:0/4144785512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4df26b0 0x555fd4f962f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1310013700 tx=0x7f1310009920 comp rx=0 tx=0).stop 2023-07-21T22:34:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.780+0000 7f132b13cb80 1 -- 172.21.15.107:0/4144785512 shutdown_connections 2023-07-21T22:34:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.780+0000 7f132b13cb80 1 --2- 172.21.15.107:0/4144785512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4f97580 0x555fd4f979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.780+0000 7f132b13cb80 1 --2- 172.21.15.107:0/4144785512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4df26b0 0x555fd4f962f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.780+0000 7f132b13cb80 1 --2- 172.21.15.107:0/4144785512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4f96830 0x555fd4f97040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.780+0000 7f132b13cb80 1 -- 172.21.15.107:0/4144785512 >> 172.21.15.107:0/4144785512 conn(0x555fd4f8bae0 msgr2=0x555fd4f90c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.781+0000 7f132b13cb80 1 -- 172.21.15.107:0/4144785512 shutdown_connections 2023-07-21T22:34:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.781+0000 7f132b13cb80 1 -- 172.21.15.107:0/4144785512 wait complete. 2023-07-21T22:34:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.781+0000 7f132b13cb80 1 -- start start 2023-07-21T22:34:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.781+0000 7f132b13cb80 1 Processor -- start 2023-07-21T22:34:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.782+0000 7f132b13cb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4df26b0 0x555fd4fa2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.782+0000 7f132b13cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f96830 0x555fd4fa32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.782+0000 7f132b13cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4f97580 0x555fd4f9f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.782+0000 7f132b13cb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4e87a60 con 0x555fd4df26b0 2023-07-21T22:34:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.782+0000 7f132b13cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4e87760 con 0x555fd4f97580 2023-07-21T22:34:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.782+0000 7f132b13cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4d6b8c0 con 0x555fd4f96830 2023-07-21T22:34:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.782+0000 7f132b13cb80 10 client.0 _ll_register_callbacks cb 0x555fd4e85700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.783+0000 7f1317fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f96830 0x555fd4fa32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.783+0000 7f1317fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f96830 0x555fd4fa32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39000/0 (socket says 172.21.15.107:39000) 2023-07-21T22:34:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.783+0000 7f131d554700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4f97580 0x555fd4f9f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.783+0000 7f1317fff700 1 -- 172.21.15.107:0/433028210 learned_addr learned my addr 172.21.15.107:0/433028210 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.784+0000 7f1317fff700 1 -- 172.21.15.107:0/433028210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4f97580 msgr2=0x555fd4f9f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.784+0000 7f1317fff700 1 --2- 172.21.15.107:0/433028210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4f97580 0x555fd4f9f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.784+0000 7f1317fff700 1 -- 172.21.15.107:0/433028210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4df26b0 msgr2=0x555fd4fa2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.784+0000 7f1317fff700 1 --2- 172.21.15.107:0/433028210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4df26b0 0x555fd4fa2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.784+0000 7f1317fff700 1 -- 172.21.15.107:0/433028210 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1310005e70 con 0x555fd4f96830 2023-07-21T22:34:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.784+0000 7f1317fff700 1 --2- 172.21.15.107:0/433028210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f96830 0x555fd4fa32f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f130c009f70 tx=0x7f130c054770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aed77c58cf075ed server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.826+0000 7f12f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:34:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.826+0000 7f12f7fff700 1 -- 172.21.15.107:0/433028210 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c060680 con 0x555fd4f96830 2023-07-21T22:34:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.826+0000 7f12f7fff700 1 -- 172.21.15.107:0/433028210 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f130c05b070 con 0x555fd4f96830 2023-07-21T22:34:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.826+0000 7f132b13cb80 1 -- 172.21.15.107:0/433028210 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fd4ee0e50 con 0x555fd4f96830 2023-07-21T22:34:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.836+0000 7f12f7fff700 1 -- 172.21.15.107:0/433028210 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f130c0607e0 con 0x555fd4f96830 2023-07-21T22:34:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.836+0000 7f132b13cb80 10 client.25912 fetch_fsmap learned FSMap version 39 2023-07-21T22:34:05.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.836+0000 7f132b13cb80 1 -- 172.21.15.107:0/433028210 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x555fd4cfec00 con 0x555fd4f96830 2023-07-21T22:34:05.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:05 smithi190 ceph-mon[98334]: pgmap v444: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 850 op/s 2023-07-21T22:34:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.847+0000 7f12f7fff700 1 -- 172.21.15.107:0/433028210 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555fd4ee0e50 con 0x555fd4f96830 2023-07-21T22:34:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.847+0000 7f12f7fff700 1 -- 172.21.15.107:0/433028210 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f130c055b80 con 0x555fd4f96830 2023-07-21T22:34:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.848+0000 7f132b13cb80 10 client.25912 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:34:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.849+0000 7f132b13cb80 10 client.25912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.849+0000 7f132b13cb80 20 client.25912 populate_metadata read hostname 'smithi107' 2023-07-21T22:34:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.849+0000 7f132b13cb80 1 --2- 172.21.15.107:0/433028210 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555fd4fa7650 0x555fd4fa9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.849+0000 7f132b13cb80 4 client.25912 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.849+0000 7f132b13cb80 1 -- 172.21.15.107:0/433028210 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fd4ea2ff0 con 0x555fd4fa7650 2023-07-21T22:34:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.849+0000 7f131d554700 1 --2- 172.21.15.107:0/433028210 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555fd4fa7650 0x555fd4fa9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.879+0000 7f131d554700 1 --2- 172.21.15.107:0/433028210 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555fd4fa7650 0x555fd4fa9a60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2becff8079414d00 server_cookie=24d25120444f7c51 in_seq=0 out_seq=0 2023-07-21T22:34:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.879+0000 7f12f7fff700 10 client.25912 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.879+0000 7f12f7fff700 1 -- 172.21.15.107:0/433028210 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555fd4ea2ff0 con 0x555fd4fa7650 2023-07-21T22:34:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.879+0000 7f12f7fff700 10 client.25912 handle_command_reply: tid=0 2023-07-21T22:34:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:05.879+0000 7f132b13cb80 1 client.25912 shutdown 2023-07-21T22:34:05.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:05 smithi132 ceph-mon[97863]: pgmap v444: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 850 op/s 2023-07-21T22:34:05.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:05 smithi107 ceph-mon[96543]: pgmap v444: 97 pgs: 97 active+clean; 1.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 850 op/s 2023-07-21T22:34:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.004+0000 7f132b13cb80 1 -- 172.21.15.107:0/433028210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f96830 msgr2=0x555fd4fa32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.004+0000 7f132b13cb80 1 --2- 172.21.15.107:0/433028210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f96830 0x555fd4fa32f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f130c009f70 tx=0x7f130c054770 comp rx=0 tx=0).stop 2023-07-21T22:34:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.014+0000 7f132b13cb80 1 -- 172.21.15.107:0/433028210 shutdown_connections 2023-07-21T22:34:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.014+0000 7f132b13cb80 1 --2- 172.21.15.107:0/433028210 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555fd4fa7650 0x555fd4fa9a60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.014+0000 7f132b13cb80 1 --2- 172.21.15.107:0/433028210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4f97580 0x555fd4f9f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.014+0000 7f132b13cb80 1 --2- 172.21.15.107:0/433028210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4df26b0 0x555fd4fa2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.014+0000 7f132b13cb80 1 --2- 172.21.15.107:0/433028210 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f96830 0x555fd4fa32f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.014+0000 7f132b13cb80 1 -- 172.21.15.107:0/433028210 >> 172.21.15.107:0/433028210 conn(0x555fd4f8bae0 msgr2=0x555fd4f903e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.015+0000 7f12f7fff700 0 client.25912 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.015+0000 7f132b13cb80 1 -- 172.21.15.107:0/433028210 shutdown_connections 2023-07-21T22:34:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.015+0000 7f132b13cb80 1 -- 172.21.15.107:0/433028210 wait complete. 2023-07-21T22:34:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.015+0000 7f132b13cb80 20 client.25912 trim_cache size 0 max 16384 2023-07-21T22:34:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.090+0000 7f132b13cb80 1 Processor -- start 2023-07-21T22:34:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.090+0000 7f132b13cb80 1 -- start start 2023-07-21T22:34:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.091+0000 7f132b13cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4df26b0 0x555fd4f962b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.091+0000 7f132b13cb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4f96830 0x555fd4fc5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.091+0000 7f132b13cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f97580 0x555fd4f957c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.091+0000 7f132b13cb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4ee0e80 con 0x555fd4f96830 2023-07-21T22:34:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.091+0000 7f132b13cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4fa4b60 con 0x555fd4df26b0 2023-07-21T22:34:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.091+0000 7f132b13cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4fa0560 con 0x555fd4f97580 2023-07-21T22:34:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.091+0000 7f131d554700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4df26b0 0x555fd4f962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.091+0000 7f131d554700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4df26b0 0x555fd4f962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41278/0 (socket says 172.21.15.107:41278) 2023-07-21T22:34:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.091+0000 7f131d554700 1 -- 172.21.15.107:0/2869340116 learned_addr learned my addr 172.21.15.107:0/2869340116 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.091+0000 7f131d554700 1 --2- 172.21.15.107:0/2869340116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f97580 0x555fd4f957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.092+0000 7f131d554700 1 -- 172.21.15.107:0/2869340116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f97580 msgr2=0x555fd4f957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.092+0000 7f131d554700 1 --2- 172.21.15.107:0/2869340116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f97580 0x555fd4f957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.092+0000 7f131d554700 1 -- 172.21.15.107:0/2869340116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4f96830 msgr2=0x555fd4fc5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.092+0000 7f131d554700 1 --2- 172.21.15.107:0/2869340116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4f96830 0x555fd4fc5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.092+0000 7f131d554700 1 -- 172.21.15.107:0/2869340116 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fd4d3c1f0 con 0x555fd4df26b0 2023-07-21T22:34:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.092+0000 7f131d554700 1 --2- 172.21.15.107:0/2869340116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4df26b0 0x555fd4f962b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x555fd4f96050 tx=0x7f1310000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68b2d531c6cba872 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.103+0000 7f12ff7fe700 1 -- 172.21.15.107:0/2869340116 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131000e040 con 0x555fd4df26b0 2023-07-21T22:34:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.104+0000 7f12ff7fe700 1 -- 172.21.15.107:0/2869340116 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1310020040 con 0x555fd4df26b0 2023-07-21T22:34:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.104+0000 7f12ff7fe700 1 -- 172.21.15.107:0/2869340116 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310007940 con 0x555fd4df26b0 2023-07-21T22:34:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.105+0000 7f132b13cb80 1 -- 172.21.15.107:0/2869340116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4df26b0 msgr2=0x555fd4f962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.105+0000 7f132b13cb80 1 --2- 172.21.15.107:0/2869340116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4df26b0 0x555fd4f962b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x555fd4f96050 tx=0x7f1310000f40 comp rx=0 tx=0).stop 2023-07-21T22:34:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.107+0000 7f132b13cb80 1 -- 172.21.15.107:0/2869340116 shutdown_connections 2023-07-21T22:34:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.107+0000 7f132b13cb80 1 --2- 172.21.15.107:0/2869340116 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f97580 0x555fd4f957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.107+0000 7f132b13cb80 1 --2- 172.21.15.107:0/2869340116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4df26b0 0x555fd4f962b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.107+0000 7f132b13cb80 1 --2- 172.21.15.107:0/2869340116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4f96830 0x555fd4fc5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.107+0000 7f132b13cb80 1 -- 172.21.15.107:0/2869340116 >> 172.21.15.107:0/2869340116 conn(0x555fd4f8bae0 msgr2=0x555fd4f90be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.107+0000 7f132b13cb80 1 -- 172.21.15.107:0/2869340116 shutdown_connections 2023-07-21T22:34:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.107+0000 7f132b13cb80 1 -- 172.21.15.107:0/2869340116 wait complete. 2023-07-21T22:34:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.108+0000 7f132b13cb80 1 -- start start 2023-07-21T22:34:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.108+0000 7f132b13cb80 1 Processor -- start 2023-07-21T22:34:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.108+0000 7f132b13cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4df26b0 0x555fd4fabe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.108+0000 7f132b13cb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4f96830 0x555fd4fac340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.108+0000 7f132b13cb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f97580 0x555fd4faf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.108+0000 7f132b13cb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4faf8a0 con 0x555fd4f96830 2023-07-21T22:34:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.108+0000 7f132b13cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4fafa10 con 0x555fd4df26b0 2023-07-21T22:34:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.108+0000 7f132b13cb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4fafbd0 con 0x555fd4f97580 2023-07-21T22:34:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.109+0000 7f132b13cb80 10 client.0 _ll_register_callbacks cb 0x7f130c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.110+0000 7f131d554700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4df26b0 0x555fd4fabe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.110+0000 7f131d554700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f97580 0x555fd4faf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.110+0000 7f131d554700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4df26b0 0x555fd4fabe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41286/0 (socket says 172.21.15.107:41286) 2023-07-21T22:34:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.110+0000 7f131d554700 1 -- 172.21.15.107:0/1176985880 learned_addr learned my addr 172.21.15.107:0/1176985880 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.112+0000 7f131d554700 1 -- 172.21.15.107:0/1176985880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4df26b0 msgr2=0x555fd4fabe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.112+0000 7f131d554700 1 --2- 172.21.15.107:0/1176985880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4df26b0 0x555fd4fabe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.112+0000 7f131d554700 1 -- 172.21.15.107:0/1176985880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4f96830 msgr2=0x555fd4fac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.112+0000 7f131d554700 1 --2- 172.21.15.107:0/1176985880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4f96830 0x555fd4fac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.112+0000 7f131d554700 1 -- 172.21.15.107:0/1176985880 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131002f040 con 0x555fd4f97580 2023-07-21T22:34:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.114+0000 7f131d554700 1 --2- 172.21.15.107:0/1176985880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f97580 0x555fd4faf360 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1310007840 tx=0x7f13100132f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b6d69ffeae10ef9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.114+0000 7f12f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:34:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.119+0000 7f12f7fff700 1 -- 172.21.15.107:0/1176985880 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131000e040 con 0x555fd4f97580 2023-07-21T22:34:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.119+0000 7f12f7fff700 1 -- 172.21.15.107:0/1176985880 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1310007d30 con 0x555fd4f97580 2023-07-21T22:34:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.124+0000 7f132b13cb80 1 -- 172.21.15.107:0/1176985880 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fd4e41490 con 0x555fd4f97580 2023-07-21T22:34:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.124+0000 7f12f7fff700 1 -- 172.21.15.107:0/1176985880 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1310027a40 con 0x555fd4f97580 2023-07-21T22:34:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.124+0000 7f132b13cb80 10 client.25918 fetch_fsmap learned FSMap version 39 2023-07-21T22:34:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.124+0000 7f132b13cb80 1 -- 172.21.15.107:0/1176985880 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x555fd4d14250 con 0x555fd4f97580 2023-07-21T22:34:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.125+0000 7f12f7fff700 1 -- 172.21.15.107:0/1176985880 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555fd4e41490 con 0x555fd4f97580 2023-07-21T22:34:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.125+0000 7f12f7fff700 1 -- 172.21.15.107:0/1176985880 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f1310013730 con 0x555fd4f97580 2023-07-21T22:34:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.125+0000 7f132b13cb80 10 client.25918 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:34:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.125+0000 7f132b13cb80 10 client.25918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.125+0000 7f132b13cb80 20 client.25918 populate_metadata read hostname 'smithi107' 2023-07-21T22:34:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.125+0000 7f132b13cb80 1 --2- 172.21.15.107:0/1176985880 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555fd5011350 0x555fd5013760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.125+0000 7f132b13cb80 4 client.25918 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:34:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.125+0000 7f132b13cb80 1 -- 172.21.15.107:0/1176985880 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555fd4d36f10 con 0x555fd5011350 2023-07-21T22:34:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.125+0000 7f1317fff700 1 --2- 172.21.15.107:0/1176985880 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555fd5011350 0x555fd5013760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.127+0000 7f1317fff700 1 --2- 172.21.15.107:0/1176985880 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555fd5011350 0x555fd5013760 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bfcb255264e0e0c server_cookie=69bad225a98ba409 in_seq=0 out_seq=0 2023-07-21T22:34:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.127+0000 7f12f7fff700 10 client.25918 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:06.595 INFO:tasks.workunit.client.0.smithi107.stdout: 308 2341666 666 1609019 358 871533 991 2023-07-21T22:34:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.768+0000 7f12f7fff700 1 -- 172.21.15.107:0/1176985880 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555fd4d36f10 con 0x555fd5011350 2023-07-21T22:34:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.768+0000 7f12f7fff700 10 client.25918 handle_command_reply: tid=0 2023-07-21T22:34:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.768+0000 7f132b13cb80 1 client.25918 shutdown 2023-07-21T22:34:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.769+0000 7f132b13cb80 1 -- 172.21.15.107:0/1176985880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f97580 msgr2=0x555fd4faf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.769+0000 7f132b13cb80 1 --2- 172.21.15.107:0/1176985880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f97580 0x555fd4faf360 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1310007840 tx=0x7f13100132f0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.832+0000 7f132b13cb80 1 -- 172.21.15.107:0/1176985880 shutdown_connections 2023-07-21T22:34:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.832+0000 7f132b13cb80 1 --2- 172.21.15.107:0/1176985880 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555fd5011350 0x555fd5013760 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.832+0000 7f132b13cb80 1 --2- 172.21.15.107:0/1176985880 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555fd4f97580 0x555fd4faf360 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.832+0000 7f132b13cb80 1 --2- 172.21.15.107:0/1176985880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555fd4df26b0 0x555fd4fabe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.832+0000 7f132b13cb80 1 --2- 172.21.15.107:0/1176985880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fd4f96830 0x555fd4fac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.832+0000 7f132b13cb80 1 -- 172.21.15.107:0/1176985880 >> 172.21.15.107:0/1176985880 conn(0x555fd4f8bae0 msgr2=0x555fd4fc5170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.832+0000 7f12f7fff700 0 client.25918 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.836+0000 7f132b13cb80 1 -- 172.21.15.107:0/1176985880 shutdown_connections 2023-07-21T22:34:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.836+0000 7f132b13cb80 1 -- 172.21.15.107:0/1176985880 wait complete. 2023-07-21T22:34:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.836+0000 7f132b13cb80 20 client.25918 trim_cache size 0 max 16384 2023-07-21T22:34:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:34:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:34:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:34:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:34:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.923+0000 7f131efb7700 1 -- 172.21.15.107:0/2087619773 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f13000a1320 msgr2=0x7f13000a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.923+0000 7f131efb7700 1 --2- 172.21.15.107:0/2087619773 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f13000a1320 0x7f13000a37d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1308097430 tx=0x7f13080d0f50 comp rx=0 tx=0).stop 2023-07-21T22:34:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.923+0000 7f131efb7700 1 -- 172.21.15.107:0/2087619773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13180711a0 msgr2=0x7f13180825b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.923+0000 7f131efb7700 1 --2- 172.21.15.107:0/2087619773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13180711a0 0x7f13180825b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1308005950 tx=0x7f1308009c40 comp rx=0 tx=0).stop 2023-07-21T22:34:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.935+0000 7f131efb7700 1 -- 172.21.15.107:0/2087619773 shutdown_connections 2023-07-21T22:34:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.935+0000 7f131efb7700 1 --2- 172.21.15.107:0/2087619773 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f13000a1320 0x7f13000a37d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.935+0000 7f131efb7700 1 --2- 172.21.15.107:0/2087619773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1318077240 0x7f13181ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.935+0000 7f131efb7700 1 --2- 172.21.15.107:0/2087619773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13180711a0 0x7f13180825b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.935+0000 7f131efb7700 1 --2- 172.21.15.107:0/2087619773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1318074880 0x7f1318082af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.935+0000 7f131efb7700 1 -- 172.21.15.107:0/2087619773 >> 172.21.15.107:0/2087619773 conn(0x7f131806ca50 msgr2=0x7f1318078a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.935+0000 7f131efb7700 1 -- 172.21.15.107:0/2087619773 shutdown_connections 2023-07-21T22:34:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:06.935+0000 7f131efb7700 1 -- 172.21.15.107:0/2087619773 wait complete. 2023-07-21T22:34:07.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:07 smithi190 ceph-mon[98334]: pgmap v445: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 799 op/s 2023-07-21T22:34:07.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:07 smithi132 ceph-mon[97863]: pgmap v445: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 799 op/s 2023-07-21T22:34:08.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:07 smithi107 ceph-mon[96543]: pgmap v445: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 799 op/s 2023-07-21T22:34:09.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:09 smithi190 ceph-mon[98334]: pgmap v446: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 799 op/s 2023-07-21T22:34:09.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:09 smithi132 ceph-mon[97863]: pgmap v446: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 799 op/s 2023-07-21T22:34:09.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:09 smithi107 ceph-mon[96543]: pgmap v446: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 799 op/s 2023-07-21T22:34:11.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:11 smithi132 ceph-mon[97863]: pgmap v447: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 754 op/s 2023-07-21T22:34:11.949 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:11 smithi107 ceph-mon[96543]: pgmap v447: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 754 op/s 2023-07-21T22:34:12.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:11 smithi190 ceph-mon[98334]: pgmap v447: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 754 op/s 2023-07-21T22:34:14.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:13 smithi190 ceph-mon[98334]: pgmap v448: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 827 op/s 2023-07-21T22:34:14.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:13 smithi132 ceph-mon[97863]: pgmap v448: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 827 op/s 2023-07-21T22:34:14.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:13 smithi107 ceph-mon[96543]: pgmap v448: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 827 op/s 2023-07-21T22:34:16.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:15 smithi190 ceph-mon[98334]: pgmap v449: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 495 op/s 2023-07-21T22:34:16.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:15 smithi132 ceph-mon[97863]: pgmap v449: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 495 op/s 2023-07-21T22:34:16.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:15 smithi107 ceph-mon[96543]: pgmap v449: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 495 op/s 2023-07-21T22:34:16.568 INFO:tasks.workunit.client.0.smithi107.stdout: 319 2687779 608 1819645 545 1009945 1356 2023-07-21T22:34:17.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:17 smithi190 ceph-mon[98334]: pgmap v450: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 422 op/s 2023-07-21T22:34:17.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:17 smithi132 ceph-mon[97863]: pgmap v450: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 422 op/s 2023-07-21T22:34:17.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:17 smithi107 ceph-mon[96543]: pgmap v450: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 422 op/s 2023-07-21T22:34:19.641 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:19 smithi190 ceph-mon[98334]: pgmap v451: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 417 op/s 2023-07-21T22:34:19.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:19 smithi132 ceph-mon[97863]: pgmap v451: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 417 op/s 2023-07-21T22:34:19.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:19 smithi107 ceph-mon[96543]: pgmap v451: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 417 op/s 2023-07-21T22:34:21.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:21 smithi132 ceph-mon[97863]: pgmap v452: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 340 op/s 2023-07-21T22:34:21.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:21 smithi107 ceph-mon[96543]: pgmap v452: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 340 op/s 2023-07-21T22:34:21.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:21 smithi190 ceph-mon[98334]: pgmap v452: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 340 op/s 2023-07-21T22:34:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.189+0000 7f51a67d0700 1 -- 172.21.15.107:0/3036959493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a00710c0 msgr2=0x7f51a00714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.189+0000 7f51a67d0700 1 --2- 172.21.15.107:0/3036959493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a00710c0 0x7f51a00714d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f519c007570 tx=0x7f519c007880 comp rx=0 tx=0).stop 2023-07-21T22:34:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.190+0000 7f51a67d0700 1 -- 172.21.15.107:0/3036959493 shutdown_connections 2023-07-21T22:34:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.190+0000 7f51a67d0700 1 --2- 172.21.15.107:0/3036959493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51a0076cf0 0x7f51a00791a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.190+0000 7f51a67d0700 1 --2- 172.21.15.107:0/3036959493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a00710c0 0x7f51a00714d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.190+0000 7f51a67d0700 1 --2- 172.21.15.107:0/3036959493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a0071a10 0x7f51a0076680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.190+0000 7f51a67d0700 1 -- 172.21.15.107:0/3036959493 >> 172.21.15.107:0/3036959493 conn(0x7f51a006c910 msgr2=0x7f51a006ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.190+0000 7f51a67d0700 1 -- 172.21.15.107:0/3036959493 shutdown_connections 2023-07-21T22:34:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.191+0000 7f51a67d0700 1 -- 172.21.15.107:0/3036959493 wait complete. 2023-07-21T22:34:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.191+0000 7f51a67d0700 1 Processor -- start 2023-07-21T22:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.192+0000 7f51a67d0700 1 -- start start 2023-07-21T22:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.192+0000 7f51a67d0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a00710c0 0x7f51a0082300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.192+0000 7f51a67d0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51a0071a10 0x7f51a0082840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.192+0000 7f51a67d0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a0076cf0 0x7f51a01b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.192+0000 7f51a67d0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a01b4960 con 0x7f51a0076cf0 2023-07-21T22:34:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.192+0000 7f51a67d0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a01b4ad0 con 0x7f51a0071a10 2023-07-21T22:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.192+0000 7f51a67d0700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a01b4c90 con 0x7f51a00710c0 2023-07-21T22:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.193+0000 7f51a5fcf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a0076cf0 0x7f51a01b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.193+0000 7f51a4fcd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51a0071a10 0x7f51a0082840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.242 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.193+0000 7f51a4fcd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51a0071a10 0x7f51a0082840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41300/0 (socket says 172.21.15.107:41300) 2023-07-21T22:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.193+0000 7f51a4fcd700 1 -- 172.21.15.107:0/4200766150 learned_addr learned my addr 172.21.15.107:0/4200766150 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.194+0000 7f51a57ce700 1 --2- 172.21.15.107:0/4200766150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a00710c0 0x7f51a0082300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.194+0000 7f51a4fcd700 1 -- 172.21.15.107:0/4200766150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a00710c0 msgr2=0x7f51a0082300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.194+0000 7f51a4fcd700 1 --2- 172.21.15.107:0/4200766150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a00710c0 0x7f51a0082300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.194+0000 7f51a4fcd700 1 -- 172.21.15.107:0/4200766150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a0076cf0 msgr2=0x7f51a01b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:22.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.194+0000 7f51a4fcd700 1 --2- 172.21.15.107:0/4200766150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a0076cf0 0x7f51a01b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.194+0000 7f51a4fcd700 1 -- 172.21.15.107:0/4200766150 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c007220 con 0x7f51a0071a10 2023-07-21T22:34:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.195+0000 7f51a4fcd700 1 --2- 172.21.15.107:0/4200766150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51a0071a10 0x7f51a0082840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f519000d950 tx=0x7f519000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.195+0000 7f51967fc700 1 -- 172.21.15.107:0/4200766150 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190009940 con 0x7f51a0071a10 2023-07-21T22:34:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.196+0000 7f51a67d0700 1 -- 172.21.15.107:0/4200766150 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a01b4f10 con 0x7f51a0071a10 2023-07-21T22:34:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.197+0000 7f51a67d0700 1 -- 172.21.15.107:0/4200766150 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a01b5470 con 0x7f51a0071a10 2023-07-21T22:34:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.203+0000 7f51b3bb7b80 1 Processor -- start 2023-07-21T22:34:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.203+0000 7f51b3bb7b80 1 -- start start 2023-07-21T22:34:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.204+0000 7f51b3bb7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec41510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.204+0000 7f51b3bb7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ec41a50 0x55ef3ec400f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.204+0000 7f51b3bb7b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec40630 0x55ef3ec40ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.204+0000 7f51b3bb7b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3eb45c50 con 0x55ef3ec40630 2023-07-21T22:34:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.204+0000 7f51b3bb7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3eb989e0 con 0x55ef3ea9c6b0 2023-07-21T22:34:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.204+0000 7f51b3bb7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3eb91e70 con 0x55ef3ec41a50 2023-07-21T22:34:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.204+0000 7f51a5fcf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec41510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.204+0000 7f51a5fcf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec41510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41308/0 (socket says 172.21.15.107:41308) 2023-07-21T22:34:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.204+0000 7f51a5fcf700 1 -- 172.21.15.107:0/2284256584 learned_addr learned my addr 172.21.15.107:0/2284256584 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.204+0000 7f51a57ce700 1 --2- 172.21.15.107:0/2284256584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ec41a50 0x55ef3ec400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.207+0000 7f51a5fcf700 1 -- 172.21.15.107:0/2284256584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ec41a50 msgr2=0x55ef3ec400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.207+0000 7f51a5fcf700 1 --2- 172.21.15.107:0/2284256584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ec41a50 0x55ef3ec400f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.207+0000 7f51a5fcf700 1 -- 172.21.15.107:0/2284256584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec40630 msgr2=0x55ef3ec40ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.207+0000 7f51a5fcf700 1 --2- 172.21.15.107:0/2284256584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec40630 0x55ef3ec40ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.207+0000 7f51a5fcf700 1 -- 172.21.15.107:0/2284256584 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef3e9bfd80 con 0x55ef3ea9c6b0 2023-07-21T22:34:22.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.208+0000 7f51a5fcf700 1 --2- 172.21.15.107:0/2284256584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec41510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5198009fd0 tx=0x7f5198009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36cf3917621e42d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.208+0000 7f51967fc700 1 -- 172.21.15.107:0/4200766150 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5190010460 con 0x7f51a0071a10 2023-07-21T22:34:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.208+0000 7f51967fc700 1 -- 172.21.15.107:0/4200766150 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000f620 con 0x7f51a0071a10 2023-07-21T22:34:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.208+0000 7f51967fc700 1 -- 172.21.15.107:0/4200766150 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f519000f800 con 0x7f51a0071a10 2023-07-21T22:34:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.209+0000 7f51967fc700 1 --2- 172.21.15.107:0/4200766150 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f518c0a1300 0x7f518c0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.209+0000 7f51967fc700 1 -- 172.21.15.107:0/4200766150 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f51900cfc30 con 0x7f51a0071a10 2023-07-21T22:34:22.259 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.211+0000 7f51a57ce700 1 --2- 172.21.15.107:0/4200766150 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f518c0a1300 0x7f518c0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.218+0000 7f518b7fe700 1 -- 172.21.15.107:0/2284256584 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198009e70 con 0x55ef3ea9c6b0 2023-07-21T22:34:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.218+0000 7f518b7fe700 1 -- 172.21.15.107:0/2284256584 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f519800f050 con 0x55ef3ea9c6b0 2023-07-21T22:34:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.218+0000 7f518b7fe700 1 -- 172.21.15.107:0/2284256584 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198014670 con 0x55ef3ea9c6b0 2023-07-21T22:34:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.221+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/2284256584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ea9c6b0 msgr2=0x55ef3ec41510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.221+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/2284256584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec41510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5198009fd0 tx=0x7f5198009920 comp rx=0 tx=0).stop 2023-07-21T22:34:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.222+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/2284256584 shutdown_connections 2023-07-21T22:34:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.222+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/2284256584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec40630 0x55ef3ec40ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.222+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/2284256584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec41510 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.222+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/2284256584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ec41a50 0x55ef3ec400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.222+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/2284256584 >> 172.21.15.107:0/2284256584 conn(0x55ef3ec35c40 msgr2=0x55ef3ec3ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.223+0000 7f51a57ce700 1 --2- 172.21.15.107:0/4200766150 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f518c0a1300 0x7f518c0a37b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f519c000c00 tx=0x7f519c005040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.223+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/2284256584 shutdown_connections 2023-07-21T22:34:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.223+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/2284256584 wait complete. 2023-07-21T22:34:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.223+0000 7f51b3bb7b80 1 -- start start 2023-07-21T22:34:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.223+0000 7f51b3bb7b80 1 Processor -- start 2023-07-21T22:34:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.224+0000 7f51b3bb7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec49a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.224+0000 7f51b3bb7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ec40630 0x55ef3ec49f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.224+0000 7f51b3bb7b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec41a50 0x55ef3ec50fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.224+0000 7f51b3bb7b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3eb31a60 con 0x55ef3ec41a50 2023-07-21T22:34:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.224+0000 7f51b3bb7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3eb31760 con 0x55ef3ec40630 2023-07-21T22:34:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.224+0000 7f51b3bb7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3ea158c0 con 0x55ef3ea9c6b0 2023-07-21T22:34:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.224+0000 7f51b3bb7b80 10 client.0 _ll_register_callbacks cb 0x55ef3eb2f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.225+0000 7f51a5fcf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec41a50 0x55ef3ec50fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.225+0000 7f51a4fcd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ec40630 0x55ef3ec49f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.225+0000 7f51a4fcd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ec40630 0x55ef3ec49f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41314/0 (socket says 172.21.15.107:41314) 2023-07-21T22:34:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.225+0000 7f51a4fcd700 1 -- 172.21.15.107:0/1804334510 learned_addr learned my addr 172.21.15.107:0/1804334510 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.226+0000 7f51a4fcd700 1 -- 172.21.15.107:0/1804334510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ea9c6b0 msgr2=0x55ef3ec49a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:34:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.226+0000 7f51a4fcd700 1 --2- 172.21.15.107:0/1804334510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec49a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.226+0000 7f51a4fcd700 1 -- 172.21.15.107:0/1804334510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec41a50 msgr2=0x55ef3ec50fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.226+0000 7f51a4fcd700 1 --2- 172.21.15.107:0/1804334510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec41a50 0x55ef3ec50fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.226+0000 7f51a4fcd700 1 -- 172.21.15.107:0/1804334510 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5198009050 con 0x55ef3ec40630 2023-07-21T22:34:22.275 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.227+0000 7f51a4fcd700 1 --2- 172.21.15.107:0/1804334510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ec40630 0x55ef3ec49f60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5190099ab0 tx=0x7f51900a1070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d4068ac4a30a226 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.280+0000 7f5188ff9700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:34:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.280+0000 7f5188ff9700 1 -- 172.21.15.107:0/1804334510 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519009d4a0 con 0x55ef3ec40630 2023-07-21T22:34:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.280+0000 7f5188ff9700 1 -- 172.21.15.107:0/1804334510 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f519009c530 con 0x55ef3ec40630 2023-07-21T22:34:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.280+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1804334510 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef3eaeb490 con 0x55ef3ec40630 2023-07-21T22:34:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.294+0000 7f5188ff9700 1 -- 172.21.15.107:0/1804334510 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f519009c6f0 con 0x55ef3ec40630 2023-07-21T22:34:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.294+0000 7f51b3bb7b80 10 client.25871 fetch_fsmap learned FSMap version 39 2023-07-21T22:34:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.294+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1804334510 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ef3e9a8c00 con 0x55ef3ec40630 2023-07-21T22:34:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.298+0000 7f5188ff9700 1 -- 172.21.15.107:0/1804334510 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef3eaeb490 con 0x55ef3ec40630 2023-07-21T22:34:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.298+0000 7f5188ff9700 1 -- 172.21.15.107:0/1804334510 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f51900cdb70 con 0x55ef3ec40630 2023-07-21T22:34:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.298+0000 7f51b3bb7b80 10 client.25871 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:34:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.298+0000 7f51b3bb7b80 10 client.25871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.298+0000 7f51b3bb7b80 20 client.25871 populate_metadata read hostname 'smithi107' 2023-07-21T22:34:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.298+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/1804334510 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ef3ec55730 0x55ef3ec57b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.298+0000 7f51b3bb7b80 4 client.25871 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.298+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1804334510 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef3eb8ae50 con 0x55ef3ec55730 2023-07-21T22:34:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.300+0000 7f51a57ce700 1 --2- 172.21.15.107:0/1804334510 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ef3ec55730 0x55ef3ec57b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.304+0000 7f51a57ce700 1 --2- 172.21.15.107:0/1804334510 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ef3ec55730 0x55ef3ec57b40 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94153d7d2c9d2092 server_cookie=6f0ad9622661b679 in_seq=0 out_seq=0 2023-07-21T22:34:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.305+0000 7f5188ff9700 10 client.25871 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.315+0000 7f5188ff9700 1 -- 172.21.15.107:0/1804334510 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ef3eb8ae50 con 0x55ef3ec55730 2023-07-21T22:34:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.315+0000 7f5188ff9700 10 client.25871 handle_command_reply: tid=0 2023-07-21T22:34:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.316+0000 7f51b3bb7b80 1 client.25871 shutdown 2023-07-21T22:34:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.326+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1804334510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ec40630 msgr2=0x55ef3ec49f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.326+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/1804334510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ec40630 0x55ef3ec49f60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5190099ab0 tx=0x7f51900a1070 comp rx=0 tx=0).stop 2023-07-21T22:34:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.327+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1804334510 shutdown_connections 2023-07-21T22:34:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.327+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/1804334510 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ef3ec55730 0x55ef3ec57b40 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.327+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/1804334510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec41a50 0x55ef3ec50fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.327+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/1804334510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec49a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.327+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/1804334510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ec40630 0x55ef3ec49f60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.327+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1804334510 >> 172.21.15.107:0/1804334510 conn(0x55ef3ec35c40 msgr2=0x55ef3ec3a540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.327+0000 7f5188ff9700 0 client.25871 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.328+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1804334510 shutdown_connections 2023-07-21T22:34:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.328+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1804334510 wait complete. 2023-07-21T22:34:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.328+0000 7f51b3bb7b80 20 client.25871 trim_cache size 0 max 16384 2023-07-21T22:34:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.432+0000 7f51b3bb7b80 1 Processor -- start 2023-07-21T22:34:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.435+0000 7f51b3bb7b80 1 -- start start 2023-07-21T22:34:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.435+0000 7f51b3bb7b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec35c40 0x55ef3ec400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.435+0000 7f51b3bb7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec50f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.435+0000 7f51b3bb7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ec40630 0x55ef3ec446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.435+0000 7f51b3bb7b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3eaeb490 con 0x55ef3ec35c40 2023-07-21T22:34:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.435+0000 7f51b3bb7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3ec48560 con 0x55ef3ec40630 2023-07-21T22:34:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.435+0000 7f51b3bb7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3ec3c260 con 0x55ef3ea9c6b0 2023-07-21T22:34:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.436+0000 7f51a5fcf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ec40630 0x55ef3ec446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.436+0000 7f51a5fcf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ec40630 0x55ef3ec446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41326/0 (socket says 172.21.15.107:41326) 2023-07-21T22:34:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.436+0000 7f51a5fcf700 1 -- 172.21.15.107:0/3145008826 learned_addr learned my addr 172.21.15.107:0/3145008826 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.436+0000 7f51a4fcd700 1 --2- 172.21.15.107:0/3145008826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec50f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.441+0000 7f51a57ce700 1 --2- 172.21.15.107:0/3145008826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec35c40 0x55ef3ec400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.441+0000 7f51a4fcd700 1 -- 172.21.15.107:0/3145008826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ec40630 msgr2=0x55ef3ec446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.441+0000 7f51a4fcd700 1 --2- 172.21.15.107:0/3145008826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ec40630 0x55ef3ec446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.441+0000 7f51a4fcd700 1 -- 172.21.15.107:0/3145008826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec35c40 msgr2=0x55ef3ec400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.441+0000 7f51a4fcd700 1 --2- 172.21.15.107:0/3145008826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec35c40 0x55ef3ec400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.441+0000 7f51a4fcd700 1 -- 172.21.15.107:0/3145008826 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef3e9e61f0 con 0x55ef3ea9c6b0 2023-07-21T22:34:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.442+0000 7f51a4fcd700 1 --2- 172.21.15.107:0/3145008826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec50f60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f51900048e0 tx=0x7f519000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63779036ef5d4186 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.522+0000 7f518bfff700 1 -- 172.21.15.107:0/3145008826 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190014040 con 0x55ef3ea9c6b0 2023-07-21T22:34:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.522+0000 7f518bfff700 1 -- 172.21.15.107:0/3145008826 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f519009c0c0 con 0x55ef3ea9c6b0 2023-07-21T22:34:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.522+0000 7f518bfff700 1 -- 172.21.15.107:0/3145008826 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51900a5260 con 0x55ef3ea9c6b0 2023-07-21T22:34:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.524+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/3145008826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ea9c6b0 msgr2=0x55ef3ec50f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.524+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/3145008826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec50f60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f51900048e0 tx=0x7f519000bf60 comp rx=0 tx=0).stop 2023-07-21T22:34:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.525+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/3145008826 shutdown_connections 2023-07-21T22:34:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.525+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/3145008826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ec40630 0x55ef3ec446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.525+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/3145008826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec35c40 0x55ef3ec400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.525+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/3145008826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec50f60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.525+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/3145008826 >> 172.21.15.107:0/3145008826 conn(0x55ef3ec3afd0 msgr2=0x55ef3ec47b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.526+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/3145008826 shutdown_connections 2023-07-21T22:34:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.526+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/3145008826 wait complete. 2023-07-21T22:34:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.526+0000 7f51b3bb7b80 1 -- start start 2023-07-21T22:34:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.526+0000 7f51b3bb7b80 1 Processor -- start 2023-07-21T22:34:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.527+0000 7f51b3bb7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec4cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.527+0000 7f51b3bb7b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec35c40 0x55ef3ec4d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.527+0000 7f51b3bb7b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ec40630 0x55ef3ec50540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.527+0000 7f51b3bb7b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3ec50a80 con 0x55ef3ec35c40 2023-07-21T22:34:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.527+0000 7f51b3bb7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3ec50bf0 con 0x55ef3ea9c6b0 2023-07-21T22:34:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.527+0000 7f51b3bb7b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3ec50db0 con 0x55ef3ec40630 2023-07-21T22:34:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.527+0000 7f51b3bb7b80 10 client.0 _ll_register_callbacks cb 0x7f519009c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.528+0000 7f51a57ce700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec4cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.528+0000 7f51a57ce700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec4cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41328/0 (socket says 172.21.15.107:41328) 2023-07-21T22:34:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.528+0000 7f51a57ce700 1 -- 172.21.15.107:0/1642272132 learned_addr learned my addr 172.21.15.107:0/1642272132 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:22.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.528+0000 7f51a5fcf700 1 --2- 172.21.15.107:0/1642272132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ec40630 0x55ef3ec50540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.529+0000 7f51a4fcd700 1 --2- 172.21.15.107:0/1642272132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec35c40 0x55ef3ec4d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.529+0000 7f51a5fcf700 1 -- 172.21.15.107:0/1642272132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ea9c6b0 msgr2=0x55ef3ec4cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.529+0000 7f51a5fcf700 1 --2- 172.21.15.107:0/1642272132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec4cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.529+0000 7f51a5fcf700 1 -- 172.21.15.107:0/1642272132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec35c40 msgr2=0x55ef3ec4d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.529+0000 7f51a5fcf700 1 --2- 172.21.15.107:0/1642272132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec35c40 0x55ef3ec4d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.529+0000 7f51a5fcf700 1 -- 172.21.15.107:0/1642272132 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef3e9a8c00 con 0x55ef3ec40630 2023-07-21T22:34:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.537+0000 7f51a5fcf700 1 --2- 172.21.15.107:0/1642272132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ec40630 0x55ef3ec50540 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55ef3ec4a870 tx=0x7f51980146f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6344ce3ee79adfe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:22.653 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.606+0000 7f5188ff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:34:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.606+0000 7f5188ff9700 1 -- 172.21.15.107:0/1642272132 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198014ca0 con 0x55ef3ec40630 2023-07-21T22:34:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.606+0000 7f5188ff9700 1 -- 172.21.15.107:0/1642272132 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5198014e00 con 0x55ef3ec40630 2023-07-21T22:34:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.606+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1642272132 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f519800f050 con 0x55ef3ec40630 2023-07-21T22:34:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.613+0000 7f5188ff9700 1 -- 172.21.15.107:0/1642272132 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f519801e400 con 0x55ef3ec40630 2023-07-21T22:34:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.613+0000 7f51b3bb7b80 10 client.25933 fetch_fsmap learned FSMap version 39 2023-07-21T22:34:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.613+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1642272132 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ef3e9be250 con 0x55ef3ec40630 2023-07-21T22:34:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.618+0000 7f5188ff9700 1 -- 172.21.15.107:0/1642272132 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f519800f050 con 0x55ef3ec40630 2023-07-21T22:34:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.618+0000 7f5188ff9700 1 -- 172.21.15.107:0/1642272132 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f519802e050 con 0x55ef3ec40630 2023-07-21T22:34:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.619+0000 7f51b3bb7b80 10 client.25933 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:34:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.619+0000 7f51b3bb7b80 10 client.25933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.619+0000 7f51b3bb7b80 20 client.25933 populate_metadata read hostname 'smithi107' 2023-07-21T22:34:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.619+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/1642272132 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ef3ec41a50 0x55ef3ecac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.619+0000 7f51b3bb7b80 4 client.25933 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:34:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.619+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1642272132 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ef3eb4cff0 con 0x55ef3ec41a50 2023-07-21T22:34:22.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.619+0000 7f51a57ce700 1 --2- 172.21.15.107:0/1642272132 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ef3ec41a50 0x55ef3ecac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:22.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:22 smithi132 ceph-mon[97863]: scrub summary: active paths [/] 2023-07-21T22:34:22.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:22 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:22.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:22 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.640+0000 7f51a57ce700 1 --2- 172.21.15.107:0/1642272132 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ef3ec41a50 0x55ef3ecac720 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d83d63d56bd88a83 server_cookie=5500c6af77e69b05 in_seq=0 out_seq=0 2023-07-21T22:34:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:22.640+0000 7f5188ff9700 10 client.25933 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:22.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:22 smithi107 ceph-mon[96543]: scrub summary: active paths [/] 2023-07-21T22:34:22.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:22 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:22.691 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:22 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:22.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:22 smithi190 ceph-mon[98334]: scrub summary: active paths [/] 2023-07-21T22:34:22.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:22 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:22.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:22 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:23.693 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:23 smithi107 ceph-mon[96543]: pgmap v453: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 340 op/s 2023-07-21T22:34:23.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:23 smithi190 ceph-mon[98334]: pgmap v453: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 340 op/s 2023-07-21T22:34:23.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:23 smithi132 ceph-mon[97863]: pgmap v453: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 340 op/s 2023-07-21T22:34:25.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:34:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:34:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:34:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:34:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.035+0000 7f5188ff9700 1 -- 172.21.15.107:0/1642272132 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ef3eb4cff0 con 0x55ef3ec41a50 2023-07-21T22:34:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.035+0000 7f5188ff9700 10 client.25933 handle_command_reply: tid=0 2023-07-21T22:34:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.035+0000 7f51b3bb7b80 1 client.25933 shutdown 2023-07-21T22:34:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.035+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1642272132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ec40630 msgr2=0x55ef3ec50540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.035+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/1642272132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ec40630 0x55ef3ec50540 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55ef3ec4a870 tx=0x7f51980146f0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.035+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1642272132 shutdown_connections 2023-07-21T22:34:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.035+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/1642272132 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ef3ec41a50 0x55ef3ecac720 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.035+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/1642272132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ef3ec40630 0x55ef3ec50540 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.036+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/1642272132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ef3ea9c6b0 0x55ef3ec4cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.036+0000 7f51b3bb7b80 1 --2- 172.21.15.107:0/1642272132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ef3ec35c40 0x55ef3ec4d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.036+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1642272132 >> 172.21.15.107:0/1642272132 conn(0x55ef3ec3afd0 msgr2=0x55ef3ec446d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.036+0000 7f5188ff9700 0 client.25933 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.036+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1642272132 shutdown_connections 2023-07-21T22:34:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.036+0000 7f51b3bb7b80 1 -- 172.21.15.107:0/1642272132 wait complete. 2023-07-21T22:34:25.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.036+0000 7f51b3bb7b80 20 client.25933 trim_cache size 0 max 16384 2023-07-21T22:34:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.248+0000 7f51a67d0700 1 -- 172.21.15.107:0/4200766150 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f518c0a1300 msgr2=0x7f518c0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.248+0000 7f51a67d0700 1 --2- 172.21.15.107:0/4200766150 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f518c0a1300 0x7f518c0a37b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f519c000c00 tx=0x7f519c005040 comp rx=0 tx=0).stop 2023-07-21T22:34:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.248+0000 7f51a67d0700 1 -- 172.21.15.107:0/4200766150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51a0071a10 msgr2=0x7f51a0082840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.248+0000 7f51a67d0700 1 --2- 172.21.15.107:0/4200766150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51a0071a10 0x7f51a0082840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f519000d950 tx=0x7f519000dc60 comp rx=0 tx=0).stop 2023-07-21T22:34:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.249+0000 7f51a67d0700 1 -- 172.21.15.107:0/4200766150 shutdown_connections 2023-07-21T22:34:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.249+0000 7f51a67d0700 1 --2- 172.21.15.107:0/4200766150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51a0076cf0 0x7f51a01b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.249+0000 7f51a67d0700 1 --2- 172.21.15.107:0/4200766150 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f518c0a1300 0x7f518c0a37b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.249+0000 7f51a67d0700 1 --2- 172.21.15.107:0/4200766150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51a00710c0 0x7f51a0082300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.249+0000 7f51a67d0700 1 --2- 172.21.15.107:0/4200766150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51a0071a10 0x7f51a0082840 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.249+0000 7f51a67d0700 1 -- 172.21.15.107:0/4200766150 >> 172.21.15.107:0/4200766150 conn(0x7f51a006c910 msgr2=0x7f51a006e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.250+0000 7f51a67d0700 1 -- 172.21.15.107:0/4200766150 shutdown_connections 2023-07-21T22:34:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:25.250+0000 7f51a67d0700 1 -- 172.21.15.107:0/4200766150 wait complete. 2023-07-21T22:34:25.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:25 smithi190 ceph-mon[98334]: pgmap v454: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 250 op/s 2023-07-21T22:34:25.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:25 smithi132 ceph-mon[97863]: pgmap v454: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 250 op/s 2023-07-21T22:34:25.998 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:25 smithi107 ceph-mon[96543]: pgmap v454: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 250 op/s 2023-07-21T22:34:26.568 INFO:tasks.workunit.client.0.smithi107.stdout: 329 2683391 572 1821135 549 1019327 1376 2023-07-21T22:34:27.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:27 smithi190 ceph-mon[98334]: pgmap v455: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 335 op/s 2023-07-21T22:34:27.934 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:27 smithi132 ceph-mon[97863]: pgmap v455: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 335 op/s 2023-07-21T22:34:27.946 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:27 smithi107 ceph-mon[96543]: pgmap v455: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 335 op/s 2023-07-21T22:34:30.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:29 smithi190 ceph-mon[98334]: pgmap v456: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 441 op/s 2023-07-21T22:34:30.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:29 smithi132 ceph-mon[97863]: pgmap v456: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 441 op/s 2023-07-21T22:34:30.197 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:29 smithi107 ceph-mon[96543]: pgmap v456: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 441 op/s 2023-07-21T22:34:30.458 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:34:31.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:30 smithi132 ceph-mon[97863]: pgmap v457: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 439 op/s 2023-07-21T22:34:31.201 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:30 smithi107 ceph-mon[96543]: pgmap v457: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 439 op/s 2023-07-21T22:34:31.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:30 smithi190 ceph-mon[98334]: pgmap v457: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 439 op/s 2023-07-21T22:34:33.682 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:33 smithi132 ceph-mon[97863]: pgmap v458: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.9 MiB/s wr, 725 op/s 2023-07-21T22:34:33.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:33 smithi107 ceph-mon[96543]: pgmap v458: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.9 MiB/s wr, 725 op/s 2023-07-21T22:34:33.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:33 smithi190 ceph-mon[98334]: pgmap v458: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.9 MiB/s wr, 725 op/s 2023-07-21T22:34:35.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.222+0000 7f0d82eed700 1 -- 172.21.15.107:0/756189892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d7c0711d0 msgr2=0x7f0d7c0715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:35.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.222+0000 7f0d82eed700 1 --2- 172.21.15.107:0/756189892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d7c0711d0 0x7f0d7c0715e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009990 tx=0x7f0d78009ca0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.222+0000 7f0d82eed700 1 -- 172.21.15.107:0/756189892 shutdown_connections 2023-07-21T22:34:35.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.222+0000 7f0d82eed700 1 --2- 172.21.15.107:0/756189892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d7c10bba0 0x7f0d7c10e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.222+0000 7f0d82eed700 1 --2- 172.21.15.107:0/756189892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d7c0711d0 0x7f0d7c0715e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.222+0000 7f0d82eed700 1 --2- 172.21.15.107:0/756189892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d7c071b20 0x7f0d7c10b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.222+0000 7f0d82eed700 1 -- 172.21.15.107:0/756189892 >> 172.21.15.107:0/756189892 conn(0x7f0d7c06c8b0 msgr2=0x7f0d7c06ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:35.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.223+0000 7f0d82eed700 1 -- 172.21.15.107:0/756189892 shutdown_connections 2023-07-21T22:34:35.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.223+0000 7f0d82eed700 1 -- 172.21.15.107:0/756189892 wait complete. 2023-07-21T22:34:35.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.224+0000 7f0d82eed700 1 Processor -- start 2023-07-21T22:34:35.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.224+0000 7f0d82eed700 1 -- start start 2023-07-21T22:34:35.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.224+0000 7f0d82eed700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d7c0711d0 0x7f0d7c113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.224+0000 7f0d82eed700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d7c071b20 0x7f0d7c113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.224+0000 7f0d82eed700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d7c10bba0 0x7f0d7c1b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.224+0000 7f0d82eed700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c117050 con 0x7f0d7c071b20 2023-07-21T22:34:35.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.224+0000 7f0d82eed700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c1171c0 con 0x7f0d7c0711d0 2023-07-21T22:34:35.272 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.224+0000 7f0d82eed700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c1b4be0 con 0x7f0d7c10bba0 2023-07-21T22:34:35.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.227+0000 7f0d816ea700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d7c071b20 0x7f0d7c113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.227+0000 7f0d816ea700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d7c071b20 0x7f0d7c113f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50514/0 (socket says 172.21.15.107:50514) 2023-07-21T22:34:35.273 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.227+0000 7f0d816ea700 1 -- 172.21.15.107:0/2640722115 learned_addr learned my addr 172.21.15.107:0/2640722115 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:35.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.227+0000 7f0d81eeb700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d7c0711d0 0x7f0d7c113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.274 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.227+0000 7f0d826ec700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d7c10bba0 0x7f0d7c1b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.238+0000 7f0d81eeb700 1 -- 172.21.15.107:0/2640722115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d7c10bba0 msgr2=0x7f0d7c1b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:35.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.238+0000 7f0d81eeb700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d7c10bba0 0x7f0d7c1b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.238+0000 7f0d81eeb700 1 -- 172.21.15.107:0/2640722115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d7c071b20 msgr2=0x7f0d7c113f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:35.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.238+0000 7f0d81eeb700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d7c071b20 0x7f0d7c113f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.238+0000 7f0d81eeb700 1 -- 172.21.15.107:0/2640722115 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d780095f0 con 0x7f0d7c0711d0 2023-07-21T22:34:35.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.249+0000 7f0d81eeb700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d7c0711d0 0x7f0d7c113a30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0d78000c00 tx=0x7f0d780057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:35.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.336+0000 7f0d72ffd700 1 -- 172.21.15.107:0/2640722115 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7801d070 con 0x7f0d7c0711d0 2023-07-21T22:34:35.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.336+0000 7f0d72ffd700 1 -- 172.21.15.107:0/2640722115 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d78021410 con 0x7f0d7c0711d0 2023-07-21T22:34:35.392 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.336+0000 7f0d72ffd700 1 -- 172.21.15.107:0/2640722115 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78017650 con 0x7f0d7c0711d0 2023-07-21T22:34:35.393 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.336+0000 7f0d82eed700 1 -- 172.21.15.107:0/2640722115 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c1b4d80 con 0x7f0d7c0711d0 2023-07-21T22:34:35.393 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.337+0000 7f0d82eed700 1 -- 172.21.15.107:0/2640722115 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c1b50f0 con 0x7f0d7c0711d0 2023-07-21T22:34:35.393 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.342+0000 7f0d902d4b80 1 Processor -- start 2023-07-21T22:34:35.393 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.345+0000 7f0d902d4b80 1 -- start start 2023-07-21T22:34:35.393 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.345+0000 7f0d902d4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e58ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.393 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.345+0000 7f0d902d4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58cf70 0x55bb4e58d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.393 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.345+0000 7f0d902d4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58ddc0 0x55bb4e58e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.394 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.345+0000 7f0d902d4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4e491cb0 con 0x55bb4e58cf70 2023-07-21T22:34:35.394 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.345+0000 7f0d902d4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4e4e4a40 con 0x55bb4e58c620 2023-07-21T22:34:35.394 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.345+0000 7f0d902d4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4e4dded0 con 0x55bb4e58ddc0 2023-07-21T22:34:35.394 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.345+0000 7f0d826ec700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e58ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.394 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.345+0000 7f0d826ec700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e58ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41350/0 (socket says 172.21.15.107:41350) 2023-07-21T22:34:35.394 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.345+0000 7f0d826ec700 1 -- 172.21.15.107:0/3413405733 learned_addr learned my addr 172.21.15.107:0/3413405733 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:35.394 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.346+0000 7f0d816ea700 1 --2- 172.21.15.107:0/3413405733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58ddc0 0x55bb4e58e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.346+0000 7f0d816ea700 1 -- 172.21.15.107:0/3413405733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 msgr2=0x55bb4e58ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:35.396 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.346+0000 7f0d816ea700 1 --2- 172.21.15.107:0/3413405733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e58ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.346+0000 7f0d816ea700 1 -- 172.21.15.107:0/3413405733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58cf70 msgr2=0x55bb4e58d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:35.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.346+0000 7f0d816ea700 1 --2- 172.21.15.107:0/3413405733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58cf70 0x55bb4e58d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.346+0000 7f0d816ea700 1 -- 172.21.15.107:0/3413405733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb4e30bde0 con 0x55bb4e58ddc0 2023-07-21T22:34:35.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.346+0000 7f0d816ea700 1 --2- 172.21.15.107:0/3413405733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58ddc0 0x55bb4e58e230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00bd20 tx=0x7f0d6c0106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff66818b47ef29b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:35.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.347+0000 7f0d72ffd700 1 -- 172.21.15.107:0/2640722115 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0d780177b0 con 0x7f0d7c0711d0 2023-07-21T22:34:35.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.348+0000 7f0d72ffd700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0d680a1430 0x7f0d680a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.349+0000 7f0d72ffd700 1 -- 172.21.15.107:0/2640722115 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d780ceb70 con 0x7f0d7c0711d0 2023-07-21T22:34:35.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.350+0000 7f0d826ec700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0d680a1430 0x7f0d680a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.409 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.362+0000 7f0d67fff700 1 -- 172.21.15.107:0/3413405733 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c010e10 con 0x55bb4e58ddc0 2023-07-21T22:34:35.410 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.362+0000 7f0d67fff700 1 -- 172.21.15.107:0/3413405733 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d6c015070 con 0x55bb4e58ddc0 2023-07-21T22:34:35.410 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.362+0000 7f0d67fff700 1 -- 172.21.15.107:0/3413405733 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c01b7b0 con 0x55bb4e58ddc0 2023-07-21T22:34:35.410 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.363+0000 7f0d902d4b80 1 -- 172.21.15.107:0/3413405733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58ddc0 msgr2=0x55bb4e58e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:35.410 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.363+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/3413405733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58ddc0 0x55bb4e58e230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00bd20 tx=0x7f0d6c0106b0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.410 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.363+0000 7f0d826ec700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0d680a1430 0x7f0d680a38e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0d740075c0 tx=0x7f0d7400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.446+0000 7f0d902d4b80 1 -- 172.21.15.107:0/3413405733 shutdown_connections 2023-07-21T22:34:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.446+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/3413405733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58ddc0 0x55bb4e58e230 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.446+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/3413405733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e58ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.446+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/3413405733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58cf70 0x55bb4e58d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.446+0000 7f0d902d4b80 1 -- 172.21.15.107:0/3413405733 >> 172.21.15.107:0/3413405733 conn(0x55bb4e581ef0 msgr2=0x55bb4e587390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.448+0000 7f0d902d4b80 1 -- 172.21.15.107:0/3413405733 shutdown_connections 2023-07-21T22:34:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.448+0000 7f0d902d4b80 1 -- 172.21.15.107:0/3413405733 wait complete. 2023-07-21T22:34:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.448+0000 7f0d902d4b80 1 -- start start 2023-07-21T22:34:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.449+0000 7f0d902d4b80 1 Processor -- start 2023-07-21T22:34:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.449+0000 7f0d902d4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e5963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.449+0000 7f0d902d4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58cf70 0x55bb4e596900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.449+0000 7f0d902d4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58ddc0 0x55bb4e59d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.449+0000 7f0d902d4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4e47dac0 con 0x55bb4e58ddc0 2023-07-21T22:34:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.449+0000 7f0d902d4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4e47d7c0 con 0x55bb4e58c620 2023-07-21T22:34:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.449+0000 7f0d902d4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4e361920 con 0x55bb4e58cf70 2023-07-21T22:34:35.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.449+0000 7f0d902d4b80 10 client.0 _ll_register_callbacks cb 0x55bb4e47b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:35.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.451+0000 7f0d826ec700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58ddc0 0x55bb4e59d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.451+0000 7f0d826ec700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58ddc0 0x55bb4e59d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50532/0 (socket says 172.21.15.107:50532) 2023-07-21T22:34:35.497 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.451+0000 7f0d826ec700 1 -- 172.21.15.107:0/760595801 learned_addr learned my addr 172.21.15.107:0/760595801 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:35.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.458+0000 7f0d816ea700 1 --2- 172.21.15.107:0/760595801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58cf70 0x55bb4e596900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.504 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.458+0000 7f0d81eeb700 1 --2- 172.21.15.107:0/760595801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e5963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.515 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.469+0000 7f0d81eeb700 1 -- 172.21.15.107:0/760595801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58cf70 msgr2=0x55bb4e596900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:35.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.469+0000 7f0d81eeb700 1 --2- 172.21.15.107:0/760595801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58cf70 0x55bb4e596900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.469+0000 7f0d81eeb700 1 -- 172.21.15.107:0/760595801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58ddc0 msgr2=0x55bb4e59d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:35.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.469+0000 7f0d81eeb700 1 --2- 172.21.15.107:0/760595801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58ddc0 0x55bb4e59d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.516 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.469+0000 7f0d81eeb700 1 -- 172.21.15.107:0/760595801 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6c00b950 con 0x55bb4e58c620 2023-07-21T22:34:35.531 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.484+0000 7f0d81eeb700 1 --2- 172.21.15.107:0/760595801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e5963c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0d78098490 tx=0x7f0d7809c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f1d53c40058ead1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:35.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.493+0000 7f0d657fa700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:34:35.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.493+0000 7f0d657fa700 1 -- 172.21.15.107:0/760595801 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780a40c0 con 0x55bb4e58c620 2023-07-21T22:34:35.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.493+0000 7f0d657fa700 1 -- 172.21.15.107:0/760595801 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d7809bf90 con 0x55bb4e58c620 2023-07-21T22:34:35.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.494+0000 7f0d902d4b80 1 -- 172.21.15.107:0/760595801 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb4e4374f0 con 0x55bb4e58c620 2023-07-21T22:34:35.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.501+0000 7f0d657fa700 1 -- 172.21.15.107:0/760595801 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d7809ffc0 con 0x55bb4e58c620 2023-07-21T22:34:35.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.501+0000 7f0d902d4b80 10 client.25889 fetch_fsmap learned FSMap version 39 2023-07-21T22:34:35.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.501+0000 7f0d902d4b80 1 -- 172.21.15.107:0/760595801 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55bb4e2f4c60 con 0x55bb4e58c620 2023-07-21T22:34:35.558 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.511+0000 7f0d657fa700 1 -- 172.21.15.107:0/760595801 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb4e4374f0 con 0x55bb4e58c620 2023-07-21T22:34:35.558 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.511+0000 7f0d657fa700 1 -- 172.21.15.107:0/760595801 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f0d780ccab0 con 0x55bb4e58c620 2023-07-21T22:34:35.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.512+0000 7f0d902d4b80 10 client.25889 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:34:35.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.512+0000 7f0d902d4b80 10 client.25889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:35.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.512+0000 7f0d902d4b80 20 client.25889 populate_metadata read hostname 'smithi107' 2023-07-21T22:34:35.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.512+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/760595801 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bb4e5a1d40 0x55bb4e5a4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.512+0000 7f0d902d4b80 4 client.25889 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:35.559 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.512+0000 7f0d902d4b80 1 -- 172.21.15.107:0/760595801 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb4e4d6eb0 con 0x55bb4e5a1d40 2023-07-21T22:34:35.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.512+0000 7f0d816ea700 1 --2- 172.21.15.107:0/760595801 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bb4e5a1d40 0x55bb4e5a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.513+0000 7f0d816ea700 1 --2- 172.21.15.107:0/760595801 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bb4e5a1d40 0x55bb4e5a4150 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d3b5f1f14d8664f server_cookie=ec44bd58f8c70b37 in_seq=0 out_seq=0 2023-07-21T22:34:35.560 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.513+0000 7f0d657fa700 10 client.25889 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:35.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.514+0000 7f0d657fa700 1 -- 172.21.15.107:0/760595801 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bb4e4d6eb0 con 0x55bb4e5a1d40 2023-07-21T22:34:35.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.514+0000 7f0d657fa700 10 client.25889 handle_command_reply: tid=0 2023-07-21T22:34:35.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.514+0000 7f0d902d4b80 1 client.25889 shutdown 2023-07-21T22:34:35.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.514+0000 7f0d902d4b80 1 -- 172.21.15.107:0/760595801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 msgr2=0x55bb4e5963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:35.561 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.514+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/760595801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e5963c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0d78098490 tx=0x7f0d7809c4d0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.514+0000 7f0d902d4b80 1 -- 172.21.15.107:0/760595801 shutdown_connections 2023-07-21T22:34:35.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.514+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/760595801 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bb4e5a1d40 0x55bb4e5a4150 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.514+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/760595801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58ddc0 0x55bb4e59d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.514+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/760595801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e5963c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.514+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/760595801 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58cf70 0x55bb4e596900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.514+0000 7f0d902d4b80 1 -- 172.21.15.107:0/760595801 >> 172.21.15.107:0/760595801 conn(0x55bb4e581ef0 msgr2=0x55bb4e586a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:35.562 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.514+0000 7f0d657fa700 0 client.25889 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:35.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.521+0000 7f0d902d4b80 1 -- 172.21.15.107:0/760595801 shutdown_connections 2023-07-21T22:34:35.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.521+0000 7f0d902d4b80 1 -- 172.21.15.107:0/760595801 wait complete. 2023-07-21T22:34:35.578 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.521+0000 7f0d902d4b80 20 client.25889 trim_cache size 0 max 16384 2023-07-21T22:34:35.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.583+0000 7f0d902d4b80 1 Processor -- start 2023-07-21T22:34:35.641 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.594+0000 7f0d902d4b80 1 -- start start 2023-07-21T22:34:35.641 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.594+0000 7f0d902d4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58c620 0x55bb4e5941e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.594+0000 7f0d902d4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58cf70 0x55bb4e59d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.594+0000 7f0d902d4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58ddc0 0x55bb4e5a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.595+0000 7f0d902d4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4e584010 con 0x55bb4e58cf70 2023-07-21T22:34:35.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.595+0000 7f0d902d4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4e588e60 con 0x55bb4e58ddc0 2023-07-21T22:34:35.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.595+0000 7f0d902d4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4e597180 con 0x55bb4e58c620 2023-07-21T22:34:35.642 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.595+0000 7f0d81eeb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58c620 0x55bb4e5941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.600+0000 7f0d81eeb700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58c620 0x55bb4e5941e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39096/0 (socket says 172.21.15.107:39096) 2023-07-21T22:34:35.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.600+0000 7f0d81eeb700 1 -- 172.21.15.107:0/1066800656 learned_addr learned my addr 172.21.15.107:0/1066800656 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:35.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.600+0000 7f0d81eeb700 1 -- 172.21.15.107:0/1066800656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58ddc0 msgr2=0x55bb4e5a3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:35.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.600+0000 7f0d81eeb700 1 --2- 172.21.15.107:0/1066800656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58ddc0 0x55bb4e5a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.600+0000 7f0d81eeb700 1 -- 172.21.15.107:0/1066800656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58cf70 msgr2=0x55bb4e59d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:35.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.600+0000 7f0d81eeb700 1 --2- 172.21.15.107:0/1066800656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58cf70 0x55bb4e59d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.600+0000 7f0d81eeb700 1 -- 172.21.15.107:0/1066800656 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb4e332250 con 0x55bb4e58c620 2023-07-21T22:34:35.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.600+0000 7f0d81eeb700 1 --2- 172.21.15.107:0/1066800656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58c620 0x55bb4e5941e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0d780095c0 tx=0x7f0d78005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb790f2323dc4bf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:35.692 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:35 smithi107 ceph-mon[96543]: pgmap v459: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 855 op/s 2023-07-21T22:34:35.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.646+0000 7f0d70ff9700 1 -- 172.21.15.107:0/1066800656 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7809bf90 con 0x55bb4e58c620 2023-07-21T22:34:35.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.646+0000 7f0d70ff9700 1 -- 172.21.15.107:0/1066800656 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d7809ffc0 con 0x55bb4e58c620 2023-07-21T22:34:35.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.646+0000 7f0d70ff9700 1 -- 172.21.15.107:0/1066800656 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7809b2a0 con 0x55bb4e58c620 2023-07-21T22:34:35.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.651+0000 7f0d902d4b80 1 -- 172.21.15.107:0/1066800656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58c620 msgr2=0x55bb4e5941e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:35.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.651+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/1066800656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58c620 0x55bb4e5941e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0d780095c0 tx=0x7f0d78005c70 comp rx=0 tx=0).stop 2023-07-21T22:34:35.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.653+0000 7f0d902d4b80 1 -- 172.21.15.107:0/1066800656 shutdown_connections 2023-07-21T22:34:35.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.653+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/1066800656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58ddc0 0x55bb4e5a3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.653+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/1066800656 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58c620 0x55bb4e5941e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.653+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/1066800656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58cf70 0x55bb4e59d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.653+0000 7f0d902d4b80 1 -- 172.21.15.107:0/1066800656 >> 172.21.15.107:0/1066800656 conn(0x55bb4e581ef0 msgr2=0x55bb4e59e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:35.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.664+0000 7f0d902d4b80 1 -- 172.21.15.107:0/1066800656 shutdown_connections 2023-07-21T22:34:35.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.664+0000 7f0d902d4b80 1 -- 172.21.15.107:0/1066800656 wait complete. 2023-07-21T22:34:35.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.664+0000 7f0d902d4b80 1 -- start start 2023-07-21T22:34:35.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.665+0000 7f0d902d4b80 1 Processor -- start 2023-07-21T22:34:35.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.665+0000 7f0d902d4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e59d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.665+0000 7f0d902d4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58cf70 0x55bb4e59d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.665+0000 7f0d902d4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58ddc0 0x55bb4e599be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.665+0000 7f0d902d4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4e59a160 con 0x55bb4e58ddc0 2023-07-21T22:34:35.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.665+0000 7f0d902d4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4e59a2a0 con 0x55bb4e58c620 2023-07-21T22:34:35.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.665+0000 7f0d902d4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb4e59a460 con 0x55bb4e58cf70 2023-07-21T22:34:35.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.665+0000 7f0d902d4b80 10 client.0 _ll_register_callbacks cb 0x55bb4e47b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:35.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.666+0000 7f0d81eeb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e59d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.666+0000 7f0d81eeb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e59d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41372/0 (socket says 172.21.15.107:41372) 2023-07-21T22:34:35.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.666+0000 7f0d81eeb700 1 -- 172.21.15.107:0/1324409434 learned_addr learned my addr 172.21.15.107:0/1324409434 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:35.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.666+0000 7f0d816ea700 1 --2- 172.21.15.107:0/1324409434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58cf70 0x55bb4e59d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.667+0000 7f0d816ea700 1 -- 172.21.15.107:0/1324409434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 msgr2=0x55bb4e59d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:35.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.667+0000 7f0d816ea700 1 --2- 172.21.15.107:0/1324409434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e59d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.667+0000 7f0d816ea700 1 -- 172.21.15.107:0/1324409434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58ddc0 msgr2=0x55bb4e599be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:35.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.667+0000 7f0d816ea700 1 --2- 172.21.15.107:0/1324409434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58ddc0 0x55bb4e599be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.667+0000 7f0d816ea700 1 -- 172.21.15.107:0/1324409434 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb4e2f4c60 con 0x55bb4e58cf70 2023-07-21T22:34:35.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.678+0000 7f0d816ea700 1 --2- 172.21.15.107:0/1324409434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58cf70 0x55bb4e59d680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00bcf0 tx=0x7f0d6c012de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=339cf8333e62d1a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:35.735 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.689+0000 7f0d657fa700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:34:35.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.689+0000 7f0d657fa700 1 -- 172.21.15.107:0/1324409434 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c010ee0 con 0x55bb4e58cf70 2023-07-21T22:34:35.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.689+0000 7f0d657fa700 1 -- 172.21.15.107:0/1324409434 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d6c017070 con 0x55bb4e58cf70 2023-07-21T22:34:35.736 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.689+0000 7f0d902d4b80 1 -- 172.21.15.107:0/1324409434 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d6c015070 con 0x55bb4e58cf70 2023-07-21T22:34:35.746 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.700+0000 7f0d657fa700 1 -- 172.21.15.107:0/1324409434 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d6c02ac60 con 0x55bb4e58cf70 2023-07-21T22:34:35.747 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.700+0000 7f0d902d4b80 10 client.25951 fetch_fsmap learned FSMap version 39 2023-07-21T22:34:35.747 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.700+0000 7f0d902d4b80 1 -- 172.21.15.107:0/1324409434 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55bb4e30a2b0 con 0x55bb4e58cf70 2023-07-21T22:34:35.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.712+0000 7f0d657fa700 1 -- 172.21.15.107:0/1324409434 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d6c015070 con 0x55bb4e58cf70 2023-07-21T22:34:35.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.712+0000 7f0d657fa700 1 -- 172.21.15.107:0/1324409434 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f0d6c010720 con 0x55bb4e58cf70 2023-07-21T22:34:35.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.713+0000 7f0d902d4b80 10 client.25951 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:34:35.760 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.713+0000 7f0d902d4b80 10 client.25951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:35.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.713+0000 7f0d902d4b80 20 client.25951 populate_metadata read hostname 'smithi107' 2023-07-21T22:34:35.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.713+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/1324409434 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bb4e5f6ef0 0x55bb4e5f9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:35.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.713+0000 7f0d902d4b80 4 client.25951 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:34:35.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.713+0000 7f0d902d4b80 1 -- 172.21.15.107:0/1324409434 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bb4e499050 con 0x55bb4e5f6ef0 2023-07-21T22:34:35.761 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.714+0000 7f0d81eeb700 1 --2- 172.21.15.107:0/1324409434 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bb4e5f6ef0 0x55bb4e5f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:35.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.735+0000 7f0d81eeb700 1 --2- 172.21.15.107:0/1324409434 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bb4e5f6ef0 0x55bb4e5f9300 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85aea92a050fb8cf server_cookie=76941a73105a401e in_seq=0 out_seq=0 2023-07-21T22:34:35.781 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.735+0000 7f0d657fa700 10 client.25951 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:35.792 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.746+0000 7f0d657fa700 1 -- 172.21.15.107:0/1324409434 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bb4e499050 con 0x55bb4e5f6ef0 2023-07-21T22:34:35.792 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.746+0000 7f0d657fa700 10 client.25951 handle_command_reply: tid=0 2023-07-21T22:34:35.792 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.746+0000 7f0d902d4b80 1 client.25951 shutdown 2023-07-21T22:34:35.792 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.746+0000 7f0d902d4b80 1 -- 172.21.15.107:0/1324409434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58cf70 msgr2=0x55bb4e59d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:35.793 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.746+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/1324409434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58cf70 0x55bb4e59d680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00bcf0 tx=0x7f0d6c012de0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.751+0000 7f0d902d4b80 1 -- 172.21.15.107:0/1324409434 shutdown_connections 2023-07-21T22:34:35.797 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.751+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/1324409434 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55bb4e5f6ef0 0x55bb4e5f9300 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.751+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/1324409434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bb4e58ddc0 0x55bb4e599be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.751+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/1324409434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bb4e58c620 0x55bb4e59d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.751+0000 7f0d902d4b80 1 --2- 172.21.15.107:0/1324409434 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bb4e58cf70 0x55bb4e59d680 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.751+0000 7f0d657fa700 0 client.25951 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:35.798 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.751+0000 7f0d902d4b80 1 -- 172.21.15.107:0/1324409434 >> 172.21.15.107:0/1324409434 conn(0x55bb4e581ef0 msgr2=0x55bb4e5941e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:35.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.762+0000 7f0d902d4b80 1 -- 172.21.15.107:0/1324409434 shutdown_connections 2023-07-21T22:34:35.819 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.773+0000 7f0d902d4b80 1 -- 172.21.15.107:0/1324409434 wait complete. 2023-07-21T22:34:35.825 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.779+0000 7f0d902d4b80 20 client.25951 trim_cache size 0 max 16384 2023-07-21T22:34:35.826 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:34:35.826 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:34:35.826 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:34:35.826 INFO:teuthology.orchestra.run.smithi107.stdout: "d037412e-e4e0-4d2c-8d6e-c7193371fe0f": { 2023-07-21T22:34:35.826 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:34:35.826 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d037412e-e4e0-4d2c-8d6e-c7193371fe0f", 2023-07-21T22:34:35.827 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:34:35.827 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:34:35.827 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:34:35.827 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:34:35.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.796+0000 7f0d82eed700 1 -- 172.21.15.107:0/2640722115 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0d680a1430 msgr2=0x7f0d680a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:35.842 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.796+0000 7f0d82eed700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0d680a1430 0x7f0d680a38e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0d740075c0 tx=0x7f0d7400b040 comp rx=0 tx=0).stop 2023-07-21T22:34:35.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.796+0000 7f0d82eed700 1 -- 172.21.15.107:0/2640722115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d7c0711d0 msgr2=0x7f0d7c113a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:35.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.796+0000 7f0d82eed700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d7c0711d0 0x7f0d7c113a30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0d78000c00 tx=0x7f0d780057b0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.796+0000 7f0d82eed700 1 -- 172.21.15.107:0/2640722115 shutdown_connections 2023-07-21T22:34:35.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.796+0000 7f0d82eed700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f0d680a1430 0x7f0d680a38e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.796+0000 7f0d82eed700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d7c10bba0 0x7f0d7c1b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.796+0000 7f0d82eed700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d7c0711d0 0x7f0d7c113a30 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0d78000c00 tx=0x7f0d780057b0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.843 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.796+0000 7f0d82eed700 1 --2- 172.21.15.107:0/2640722115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d7c071b20 0x7f0d7c113f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:35.844 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.796+0000 7f0d82eed700 1 -- 172.21.15.107:0/2640722115 >> 172.21.15.107:0/2640722115 conn(0x7f0d7c06c8b0 msgr2=0x7f0d7c06db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:35.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.799+0000 7f0d82eed700 1 -- 172.21.15.107:0/2640722115 shutdown_connections 2023-07-21T22:34:35.845 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:34:35.799+0000 7f0d82eed700 1 -- 172.21.15.107:0/2640722115 wait complete. 2023-07-21T22:34:35.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:35 smithi190 ceph-mon[98334]: pgmap v459: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 855 op/s 2023-07-21T22:34:35.904 INFO:tasks.cephfs.filesystem:scrub status for tag:d037412e-e4e0-4d2c-8d6e-c7193371fe0f - {'path': '/', 'tag': 'd037412e-e4e0-4d2c-8d6e-c7193371fe0f', 'options': 'recursive,force'} 2023-07-21T22:34:35.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:35 smithi132 ceph-mon[97863]: pgmap v459: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 855 op/s 2023-07-21T22:34:36.569 INFO:tasks.workunit.client.0.smithi107.stdout: 338 2330161 442 1577848 507 913027 1297 2023-07-21T22:34:37.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:37 smithi190 ceph-mon[98334]: pgmap v460: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 914 op/s 2023-07-21T22:34:37.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:37 smithi132 ceph-mon[97863]: pgmap v460: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 914 op/s 2023-07-21T22:34:37.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:37 smithi107 ceph-mon[96543]: pgmap v460: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 914 op/s 2023-07-21T22:34:39.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:39 smithi190 ceph-mon[98334]: pgmap v461: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 903 op/s 2023-07-21T22:34:39.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:39 smithi132 ceph-mon[97863]: pgmap v461: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 903 op/s 2023-07-21T22:34:39.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:39 smithi107 ceph-mon[96543]: pgmap v461: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 903 op/s 2023-07-21T22:34:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.112+0000 7f3939a9c700 1 -- 172.21.15.107:0/2989775902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3934071480 msgr2=0x7f393410b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.112+0000 7f3939a9c700 1 --2- 172.21.15.107:0/2989775902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3934071480 0x7f393410b7d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3928009990 tx=0x7f3928009ca0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.123+0000 7f3939a9c700 1 -- 172.21.15.107:0/2989775902 shutdown_connections 2023-07-21T22:34:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.123+0000 7f3939a9c700 1 --2- 172.21.15.107:0/2989775902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f393410bd10 0x7f393410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.123+0000 7f3939a9c700 1 --2- 172.21.15.107:0/2989775902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3934070b30 0x7f3934070f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.123+0000 7f3939a9c700 1 --2- 172.21.15.107:0/2989775902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3934071480 0x7f393410b7d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.123+0000 7f3939a9c700 1 -- 172.21.15.107:0/2989775902 >> 172.21.15.107:0/2989775902 conn(0x7f393406c910 msgr2=0x7f393406ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:41.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.123+0000 7f3939a9c700 1 -- 172.21.15.107:0/2989775902 shutdown_connections 2023-07-21T22:34:41.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.123+0000 7f3939a9c700 1 -- 172.21.15.107:0/2989775902 wait complete. 2023-07-21T22:34:41.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.124+0000 7f3939a9c700 1 Processor -- start 2023-07-21T22:34:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.125+0000 7f3939a9c700 1 -- start start 2023-07-21T22:34:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.125+0000 7f3939a9c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3934070b30 0x7f3934114b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.125+0000 7f3939a9c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934071480 0x7f39341150d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.126+0000 7f3939a9c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f393410bd10 0x7f3934116110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.126+0000 7f3939a9c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39341166e0 con 0x7f3934070b30 2023-07-21T22:34:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.126+0000 7f3939a9c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3934116850 con 0x7f3934071480 2023-07-21T22:34:41.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.126+0000 7f3939a9c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3934116a10 con 0x7f393410bd10 2023-07-21T22:34:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.126+0000 7f393929b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f393410bd10 0x7f3934116110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.126+0000 7f393929b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f393410bd10 0x7f3934116110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39116/0 (socket says 172.21.15.107:39116) 2023-07-21T22:34:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.126+0000 7f393929b700 1 -- 172.21.15.107:0/2551262453 learned_addr learned my addr 172.21.15.107:0/2551262453 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.126+0000 7f3933fff700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934071480 0x7f39341150d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.126+0000 7f3938a9a700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3934070b30 0x7f3934114b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.135+0000 7f393929b700 1 -- 172.21.15.107:0/2551262453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934071480 msgr2=0x7f39341150d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:41.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.135+0000 7f393929b700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934071480 0x7f39341150d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.135+0000 7f393929b700 1 -- 172.21.15.107:0/2551262453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3934070b30 msgr2=0x7f3934114b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:41.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.135+0000 7f393929b700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3934070b30 0x7f3934114b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.135+0000 7f393929b700 1 -- 172.21.15.107:0/2551262453 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3924009520 con 0x7f393410bd10 2023-07-21T22:34:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.138+0000 7f393929b700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f393410bd10 0x7f3934116110 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f392c00ef30 tx=0x7f392c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:41.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.171+0000 7f3931ffb700 1 -- 172.21.15.107:0/2551262453 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c018070 con 0x7f393410bd10 2023-07-21T22:34:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.171+0000 7f3931ffb700 1 -- 172.21.15.107:0/2551262453 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f392c00f040 con 0x7f393410bd10 2023-07-21T22:34:41.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.171+0000 7f3931ffb700 1 -- 172.21.15.107:0/2551262453 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c00ccd0 con 0x7f393410bd10 2023-07-21T22:34:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.171+0000 7f3939a9c700 1 -- 172.21.15.107:0/2551262453 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39280095f0 con 0x7f393410bd10 2023-07-21T22:34:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.172+0000 7f3939a9c700 1 -- 172.21.15.107:0/2551262453 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3934116f60 con 0x7f393410bd10 2023-07-21T22:34:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.178+0000 7f3946e83b80 1 Processor -- start 2023-07-21T22:34:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.181+0000 7f3946e83b80 1 -- start start 2023-07-21T22:34:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.181+0000 7f3946e83b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 0x5637605b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.181+0000 7f3946e83b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b1a50 0x5637605b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.181+0000 7f3946e83b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b0630 0x5637605b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.181+0000 7f3946e83b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637604b5c50 con 0x5637605b0630 2023-07-21T22:34:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.181+0000 7f3946e83b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637605089e0 con 0x5637605b1a50 2023-07-21T22:34:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.181+0000 7f3946e83b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563760501e70 con 0x56376040c6b0 2023-07-21T22:34:41.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.181+0000 7f3931ffb700 1 -- 172.21.15.107:0/2551262453 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f392c004000 con 0x7f393410bd10 2023-07-21T22:34:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.181+0000 7f3933fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 0x5637605b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.181+0000 7f3933fff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 0x5637605b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39120/0 (socket says 172.21.15.107:39120) 2023-07-21T22:34:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.181+0000 7f3933fff700 1 -- 172.21.15.107:0/2261737645 learned_addr learned my addr 172.21.15.107:0/2261737645 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.182+0000 7f3938a9a700 1 --2- 172.21.15.107:0/2261737645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b1a50 0x5637605b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.182+0000 7f3933fff700 1 -- 172.21.15.107:0/2261737645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b1a50 msgr2=0x5637605b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.182+0000 7f3933fff700 1 --2- 172.21.15.107:0/2261737645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b1a50 0x5637605b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.182+0000 7f3933fff700 1 -- 172.21.15.107:0/2261737645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b0630 msgr2=0x5637605b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.182+0000 7f3933fff700 1 --2- 172.21.15.107:0/2261737645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b0630 0x5637605b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.182+0000 7f3933fff700 1 -- 172.21.15.107:0/2261737645 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56376032fd80 con 0x56376040c6b0 2023-07-21T22:34:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.182+0000 7f3933fff700 1 --2- 172.21.15.107:0/2261737645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 0x5637605b1510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3928009990 tx=0x7f3928005c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d819a8a998c8f92 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.189+0000 7f3931ffb700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f391c0a1350 0x7f391c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.190+0000 7f3931ffb700 1 -- 172.21.15.107:0/2551262453 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f392c0cd2a0 con 0x7f393410bd10 2023-07-21T22:34:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.191+0000 7f3938a9a700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f391c0a1350 0x7f391c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.203+0000 7f3938a9a700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f391c0a1350 0x7f391c0a3800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3934072360 tx=0x7f3924009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.273+0000 7f391affd700 1 -- 172.21.15.107:0/2261737645 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39280055c0 con 0x56376040c6b0 2023-07-21T22:34:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.273+0000 7f391affd700 1 -- 172.21.15.107:0/2261737645 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f392800bbb0 con 0x56376040c6b0 2023-07-21T22:34:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.273+0000 7f391affd700 1 -- 172.21.15.107:0/2261737645 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39280173f0 con 0x56376040c6b0 2023-07-21T22:34:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.281+0000 7f3946e83b80 1 -- 172.21.15.107:0/2261737645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 msgr2=0x5637605b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.281+0000 7f3946e83b80 1 --2- 172.21.15.107:0/2261737645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 0x5637605b1510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3928009990 tx=0x7f3928005c60 comp rx=0 tx=0).stop 2023-07-21T22:34:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.283+0000 7f3946e83b80 1 -- 172.21.15.107:0/2261737645 shutdown_connections 2023-07-21T22:34:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.283+0000 7f3946e83b80 1 --2- 172.21.15.107:0/2261737645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b0630 0x5637605b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.283+0000 7f3946e83b80 1 --2- 172.21.15.107:0/2261737645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 0x5637605b1510 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.283+0000 7f3946e83b80 1 --2- 172.21.15.107:0/2261737645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b1a50 0x5637605b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.283+0000 7f3946e83b80 1 -- 172.21.15.107:0/2261737645 >> 172.21.15.107:0/2261737645 conn(0x5637605a5c40 msgr2=0x5637605aae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.283+0000 7f3946e83b80 1 -- 172.21.15.107:0/2261737645 shutdown_connections 2023-07-21T22:34:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.284+0000 7f3946e83b80 1 -- 172.21.15.107:0/2261737645 wait complete. 2023-07-21T22:34:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.284+0000 7f3946e83b80 1 -- start start 2023-07-21T22:34:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.284+0000 7f3946e83b80 1 Processor -- start 2023-07-21T22:34:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.285+0000 7f3946e83b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56376040c6b0 0x5637605b9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.285+0000 7f3946e83b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637605b0630 0x5637605b9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.285+0000 7f3946e83b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b1a50 0x5637605c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.285+0000 7f3946e83b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637604a1a60 con 0x5637605b1a50 2023-07-21T22:34:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.285+0000 7f3946e83b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637604a1760 con 0x56376040c6b0 2023-07-21T22:34:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.285+0000 7f3946e83b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637603858c0 con 0x5637605b0630 2023-07-21T22:34:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.285+0000 7f3946e83b80 10 client.0 _ll_register_callbacks cb 0x56376049f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.290+0000 7f393929b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b1a50 0x5637605c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.291+0000 7f393929b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b1a50 0x5637605c0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50572/0 (socket says 172.21.15.107:50572) 2023-07-21T22:34:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.291+0000 7f393929b700 1 -- 172.21.15.107:0/2747606104 learned_addr learned my addr 172.21.15.107:0/2747606104 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.293+0000 7f393929b700 1 -- 172.21.15.107:0/2747606104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637605b0630 msgr2=0x5637605b9ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:34:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.293+0000 7f393929b700 1 --2- 172.21.15.107:0/2747606104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637605b0630 0x5637605b9ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.293+0000 7f393929b700 1 -- 172.21.15.107:0/2747606104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56376040c6b0 msgr2=0x5637605b9990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:34:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.293+0000 7f393929b700 1 --2- 172.21.15.107:0/2747606104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56376040c6b0 0x5637605b9990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.293+0000 7f393929b700 1 -- 172.21.15.107:0/2747606104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3928020040 con 0x5637605b1a50 2023-07-21T22:34:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.324+0000 7f393929b700 1 --2- 172.21.15.107:0/2747606104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b1a50 0x5637605c0f10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f392c00c7a0 tx=0x7f392c09a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36659e64396df1a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.330+0000 7f39187f8700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:34:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.339+0000 7f39187f8700 1 -- 172.21.15.107:0/2747606104 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c09e6f0 con 0x5637605b1a50 2023-07-21T22:34:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.339+0000 7f39187f8700 1 -- 172.21.15.107:0/2747606104 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f392c0a2a80 con 0x5637605b1a50 2023-07-21T22:34:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.340+0000 7f3946e83b80 1 -- 172.21.15.107:0/2747606104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56376045b490 con 0x5637605b1a50 2023-07-21T22:34:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.344+0000 7f39187f8700 1 -- 172.21.15.107:0/2747606104 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f392c0ba0d0 con 0x5637605b1a50 2023-07-21T22:34:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.345+0000 7f3946e83b80 10 client.16365 fetch_fsmap learned FSMap version 39 2023-07-21T22:34:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.345+0000 7f3946e83b80 1 -- 172.21.15.107:0/2747606104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x563760318c00 con 0x5637605b1a50 2023-07-21T22:34:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.358+0000 7f39187f8700 1 -- 172.21.15.107:0/2747606104 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56376045b490 con 0x5637605b1a50 2023-07-21T22:34:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.358+0000 7f39187f8700 1 -- 172.21.15.107:0/2747606104 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f392c0964b0 con 0x5637605b1a50 2023-07-21T22:34:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.359+0000 7f3946e83b80 10 client.16365 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:34:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.359+0000 7f3946e83b80 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.359+0000 7f3946e83b80 20 client.16365 populate_metadata read hostname 'smithi107' 2023-07-21T22:34:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.359+0000 7f3946e83b80 1 --2- 172.21.15.107:0/2747606104 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5637605c5740 0x5637605c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.359+0000 7f3946e83b80 4 client.16365 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.359+0000 7f3946e83b80 1 -- 172.21.15.107:0/2747606104 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637604fae50 con 0x5637605c5740 2023-07-21T22:34:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.374+0000 7f3938a9a700 1 --2- 172.21.15.107:0/2747606104 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5637605c5740 0x5637605c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.375+0000 7f3938a9a700 1 --2- 172.21.15.107:0/2747606104 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5637605c5740 0x5637605c7b50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbaffec750439079 server_cookie=dbc9aae53ac248c7 in_seq=0 out_seq=0 2023-07-21T22:34:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.375+0000 7f39187f8700 10 client.16365 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.377+0000 7f39187f8700 1 -- 172.21.15.107:0/2747606104 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5637604fae50 con 0x5637605c5740 2023-07-21T22:34:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.377+0000 7f39187f8700 10 client.16365 handle_command_reply: tid=0 2023-07-21T22:34:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.377+0000 7f3946e83b80 1 client.16365 shutdown 2023-07-21T22:34:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.378+0000 7f3946e83b80 1 -- 172.21.15.107:0/2747606104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b1a50 msgr2=0x5637605c0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.378+0000 7f3946e83b80 1 --2- 172.21.15.107:0/2747606104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b1a50 0x5637605c0f10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f392c00c7a0 tx=0x7f392c09a150 comp rx=0 tx=0).stop 2023-07-21T22:34:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.379+0000 7f3946e83b80 1 -- 172.21.15.107:0/2747606104 shutdown_connections 2023-07-21T22:34:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.379+0000 7f3946e83b80 1 --2- 172.21.15.107:0/2747606104 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5637605c5740 0x5637605c7b50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.379+0000 7f3946e83b80 1 --2- 172.21.15.107:0/2747606104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b1a50 0x5637605c0f10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.379+0000 7f3946e83b80 1 --2- 172.21.15.107:0/2747606104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56376040c6b0 0x5637605b9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.379+0000 7f3946e83b80 1 --2- 172.21.15.107:0/2747606104 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5637605b0630 0x5637605b9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.379+0000 7f3946e83b80 1 -- 172.21.15.107:0/2747606104 >> 172.21.15.107:0/2747606104 conn(0x5637605a5c40 msgr2=0x5637605aa540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:41.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.383+0000 7f39187f8700 0 client.16365 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.383+0000 7f3946e83b80 1 -- 172.21.15.107:0/2747606104 shutdown_connections 2023-07-21T22:34:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.383+0000 7f3946e83b80 1 -- 172.21.15.107:0/2747606104 wait complete. 2023-07-21T22:34:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.384+0000 7f3946e83b80 20 client.16365 trim_cache size 0 max 16384 2023-07-21T22:34:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.479+0000 7f3946e83b80 1 Processor -- start 2023-07-21T22:34:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.480+0000 7f3946e83b80 1 -- start start 2023-07-21T22:34:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.480+0000 7f3946e83b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 0x5637605b9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.480+0000 7f3946e83b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b0630 0x5637605c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.480+0000 7f3946e83b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b1a50 0x5637605b9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.480+0000 7f3946e83b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637605ab920 con 0x5637605b1a50 2023-07-21T22:34:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.480+0000 7f3946e83b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637605b2160 con 0x5637605b0630 2023-07-21T22:34:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.480+0000 7f3946e83b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637605b9510 con 0x56376040c6b0 2023-07-21T22:34:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.480+0000 7f3933fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b0630 0x5637605c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.480+0000 7f3938a9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 0x5637605b9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.481+0000 7f3933fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b0630 0x5637605c0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41408/0 (socket says 172.21.15.107:41408) 2023-07-21T22:34:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.481+0000 7f3933fff700 1 -- 172.21.15.107:0/3133949246 learned_addr learned my addr 172.21.15.107:0/3133949246 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.491+0000 7f3933fff700 1 -- 172.21.15.107:0/3133949246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 msgr2=0x5637605b9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.491+0000 7f3933fff700 1 --2- 172.21.15.107:0/3133949246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 0x5637605b9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.491+0000 7f3933fff700 1 -- 172.21.15.107:0/3133949246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b1a50 msgr2=0x5637605b9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.491+0000 7f3933fff700 1 --2- 172.21.15.107:0/3133949246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b1a50 0x5637605b9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.491+0000 7f3933fff700 1 -- 172.21.15.107:0/3133949246 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637603561f0 con 0x5637605b0630 2023-07-21T22:34:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.492+0000 7f3933fff700 1 --2- 172.21.15.107:0/3133949246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b0630 0x5637605c0ed0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f39280093a0 tx=0x7f39280058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4603656779e094d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.493+0000 7f391b7fe700 1 -- 172.21.15.107:0/3133949246 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928005680 con 0x5637605b0630 2023-07-21T22:34:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.493+0000 7f391b7fe700 1 -- 172.21.15.107:0/3133949246 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f392800bd20 con 0x5637605b0630 2023-07-21T22:34:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.493+0000 7f391b7fe700 1 -- 172.21.15.107:0/3133949246 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928025440 con 0x5637605b0630 2023-07-21T22:34:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.494+0000 7f3946e83b80 1 -- 172.21.15.107:0/3133949246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b0630 msgr2=0x5637605c0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.494+0000 7f3946e83b80 1 --2- 172.21.15.107:0/3133949246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b0630 0x5637605c0ed0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f39280093a0 tx=0x7f39280058e0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.533+0000 7f3946e83b80 1 -- 172.21.15.107:0/3133949246 shutdown_connections 2023-07-21T22:34:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.533+0000 7f3946e83b80 1 --2- 172.21.15.107:0/3133949246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b1a50 0x5637605b9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.533+0000 7f3946e83b80 1 --2- 172.21.15.107:0/3133949246 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 0x5637605b9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.533+0000 7f3946e83b80 1 --2- 172.21.15.107:0/3133949246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b0630 0x5637605c0ed0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.533+0000 7f3946e83b80 1 -- 172.21.15.107:0/3133949246 >> 172.21.15.107:0/3133949246 conn(0x5637605a5c40 msgr2=0x5637605b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.537+0000 7f3946e83b80 1 -- 172.21.15.107:0/3133949246 shutdown_connections 2023-07-21T22:34:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.537+0000 7f3946e83b80 1 -- 172.21.15.107:0/3133949246 wait complete. 2023-07-21T22:34:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.538+0000 7f3946e83b80 1 -- start start 2023-07-21T22:34:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.538+0000 7f3946e83b80 1 Processor -- start 2023-07-21T22:34:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.539+0000 7f3946e83b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 0x5637605bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.539+0000 7f3946e83b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b0630 0x5637605bd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.539+0000 7f3946e83b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b1a50 0x5637605c00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.539+0000 7f3946e83b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637605c0610 con 0x5637605b0630 2023-07-21T22:34:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.539+0000 7f3946e83b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637605c0780 con 0x5637605b1a50 2023-07-21T22:34:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.539+0000 7f3946e83b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637605c0940 con 0x56376040c6b0 2023-07-21T22:34:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.539+0000 7f3946e83b80 10 client.0 _ll_register_callbacks cb 0x7f392c09a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.540+0000 7f3933fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b0630 0x5637605bd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.540+0000 7f3933fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b0630 0x5637605bd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50586/0 (socket says 172.21.15.107:50586) 2023-07-21T22:34:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.540+0000 7f3933fff700 1 -- 172.21.15.107:0/1073517372 learned_addr learned my addr 172.21.15.107:0/1073517372 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.540+0000 7f3933fff700 1 -- 172.21.15.107:0/1073517372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 msgr2=0x5637605bcb70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:34:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.540+0000 7f3933fff700 1 --2- 172.21.15.107:0/1073517372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 0x5637605bcb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.540+0000 7f3933fff700 1 -- 172.21.15.107:0/1073517372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b1a50 msgr2=0x5637605c00d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:34:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.540+0000 7f3933fff700 1 --2- 172.21.15.107:0/1073517372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b1a50 0x5637605c00d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.540+0000 7f3933fff700 1 -- 172.21.15.107:0/1073517372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3928029040 con 0x5637605b0630 2023-07-21T22:34:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.541+0000 7f3933fff700 1 --2- 172.21.15.107:0/1073517372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b0630 0x5637605bd0b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f39280098e0 tx=0x7f392800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d08f6c95af545b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.544+0000 7f39187f8700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:34:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.544+0000 7f39187f8700 1 -- 172.21.15.107:0/1073517372 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392800fc00 con 0x5637605b0630 2023-07-21T22:34:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.544+0000 7f39187f8700 1 -- 172.21.15.107:0/1073517372 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f392800fd60 con 0x5637605b0630 2023-07-21T22:34:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.544+0000 7f3946e83b80 1 -- 172.21.15.107:0/1073517372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f392800bbb0 con 0x5637605b0630 2023-07-21T22:34:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.545+0000 7f39187f8700 1 -- 172.21.15.107:0/1073517372 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3928025440 con 0x5637605b0630 2023-07-21T22:34:41.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.545+0000 7f3946e83b80 10 client.16368 fetch_fsmap learned FSMap version 39 2023-07-21T22:34:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.545+0000 7f3946e83b80 1 -- 172.21.15.107:0/1073517372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56376032e250 con 0x5637605b0630 2023-07-21T22:34:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.546+0000 7f39187f8700 1 -- 172.21.15.107:0/1073517372 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f392800bbb0 con 0x5637605b0630 2023-07-21T22:34:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.546+0000 7f39187f8700 1 -- 172.21.15.107:0/1073517372 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f392803e070 con 0x5637605b0630 2023-07-21T22:34:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.547+0000 7f3946e83b80 10 client.16368 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:34:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.547+0000 7f3946e83b80 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.547+0000 7f3946e83b80 20 client.16368 populate_metadata read hostname 'smithi107' 2023-07-21T22:34:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.547+0000 7f3946e83b80 1 --2- 172.21.15.107:0/1073517372 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563760619ee0 0x56376061c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.547+0000 7f3946e83b80 4 client.16368 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:34:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.547+0000 7f3946e83b80 1 -- 172.21.15.107:0/1073517372 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5637604bcff0 con 0x563760619ee0 2023-07-21T22:34:41.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.547+0000 7f3938a9a700 1 --2- 172.21.15.107:0/1073517372 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563760619ee0 0x56376061c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.566+0000 7f3938a9a700 1 --2- 172.21.15.107:0/1073517372 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563760619ee0 0x56376061c2c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c639c512dcb5005 server_cookie=bb916da9a32a7ebb in_seq=0 out_seq=0 2023-07-21T22:34:41.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:41.567+0000 7f39187f8700 10 client.16368 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:41.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:41 smithi190 ceph-mon[98334]: pgmap v462: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 751 op/s 2023-07-21T22:34:41.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:41 smithi132 ceph-mon[97863]: pgmap v462: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 751 op/s 2023-07-21T22:34:41.951 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:41 smithi107 ceph-mon[96543]: pgmap v462: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 751 op/s 2023-07-21T22:34:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.300+0000 7f39187f8700 1 -- 172.21.15.107:0/1073517372 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5637604bcff0 con 0x563760619ee0 2023-07-21T22:34:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.300+0000 7f39187f8700 10 client.16368 handle_command_reply: tid=0 2023-07-21T22:34:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.300+0000 7f3946e83b80 1 client.16368 shutdown 2023-07-21T22:34:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.300+0000 7f3946e83b80 1 -- 172.21.15.107:0/1073517372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b0630 msgr2=0x5637605bd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.300+0000 7f3946e83b80 1 --2- 172.21.15.107:0/1073517372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b0630 0x5637605bd0b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f39280098e0 tx=0x7f392800bfd0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.301+0000 7f3946e83b80 1 -- 172.21.15.107:0/1073517372 shutdown_connections 2023-07-21T22:34:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.301+0000 7f3946e83b80 1 --2- 172.21.15.107:0/1073517372 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x563760619ee0 0x56376061c2c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.301+0000 7f3946e83b80 1 --2- 172.21.15.107:0/1073517372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5637605b1a50 0x5637605c00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.301+0000 7f3946e83b80 1 --2- 172.21.15.107:0/1073517372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56376040c6b0 0x5637605bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.301+0000 7f3946e83b80 1 --2- 172.21.15.107:0/1073517372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637605b0630 0x5637605bd0b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.301+0000 7f3946e83b80 1 -- 172.21.15.107:0/1073517372 >> 172.21.15.107:0/1073517372 conn(0x5637605a5c40 msgr2=0x5637605b7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.301+0000 7f39187f8700 0 client.16368 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.301+0000 7f3946e83b80 1 -- 172.21.15.107:0/1073517372 shutdown_connections 2023-07-21T22:34:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.301+0000 7f3946e83b80 1 -- 172.21.15.107:0/1073517372 wait complete. 2023-07-21T22:34:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.301+0000 7f3946e83b80 20 client.16368 trim_cache size 0 max 16384 2023-07-21T22:34:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:34:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:34:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:34:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:34:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.438+0000 7f3939a9c700 1 -- 172.21.15.107:0/2551262453 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f391c0a1350 msgr2=0x7f391c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.438+0000 7f3939a9c700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f391c0a1350 0x7f391c0a3800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3934072360 tx=0x7f3924009260 comp rx=0 tx=0).stop 2023-07-21T22:34:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.438+0000 7f3939a9c700 1 -- 172.21.15.107:0/2551262453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f393410bd10 msgr2=0x7f3934116110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.438+0000 7f3939a9c700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f393410bd10 0x7f3934116110 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f392c00ef30 tx=0x7f392c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.439+0000 7f3939a9c700 1 -- 172.21.15.107:0/2551262453 shutdown_connections 2023-07-21T22:34:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.439+0000 7f3939a9c700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f391c0a1350 0x7f391c0a3800 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.439+0000 7f3939a9c700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f393410bd10 0x7f3934116110 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.439+0000 7f3939a9c700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3934070b30 0x7f3934114b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.439+0000 7f3939a9c700 1 --2- 172.21.15.107:0/2551262453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3934071480 0x7f39341150d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.439+0000 7f3939a9c700 1 -- 172.21.15.107:0/2551262453 >> 172.21.15.107:0/2551262453 conn(0x7f393406c910 msgr2=0x7f393406db50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.440+0000 7f3939a9c700 1 -- 172.21.15.107:0/2551262453 shutdown_connections 2023-07-21T22:34:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:42.440+0000 7f3939a9c700 1 -- 172.21.15.107:0/2551262453 wait complete. 2023-07-21T22:34:43.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:43 smithi190 ceph-mon[98334]: pgmap v463: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 770 op/s 2023-07-21T22:34:43.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:43 smithi132 ceph-mon[97863]: pgmap v463: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 770 op/s 2023-07-21T22:34:43.949 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:43 smithi107 ceph-mon[96543]: pgmap v463: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 770 op/s 2023-07-21T22:34:45.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:45 smithi190 ceph-mon[98334]: pgmap v464: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 561 op/s 2023-07-21T22:34:45.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:45 smithi132 ceph-mon[97863]: pgmap v464: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 561 op/s 2023-07-21T22:34:45.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:45 smithi107 ceph-mon[96543]: pgmap v464: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 561 op/s 2023-07-21T22:34:46.576 INFO:tasks.workunit.client.0.smithi107.stdout: 351 2508449 761 1703307 647 979368 944 2023-07-21T22:34:47.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:47 smithi132 ceph-mon[97863]: pgmap v465: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 462 op/s 2023-07-21T22:34:47.954 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:47 smithi107 ceph-mon[96543]: pgmap v465: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 462 op/s 2023-07-21T22:34:48.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:47 smithi190 ceph-mon[98334]: pgmap v465: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 462 op/s 2023-07-21T22:34:49.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:49 smithi132 ceph-mon[97863]: pgmap v466: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 440 op/s 2023-07-21T22:34:49.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:49 smithi107 ceph-mon[96543]: pgmap v466: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 440 op/s 2023-07-21T22:34:50.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:49 smithi190 ceph-mon[98334]: pgmap v466: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 440 op/s 2023-07-21T22:34:52.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:51 smithi190 ceph-mon[98334]: pgmap v467: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 334 op/s 2023-07-21T22:34:52.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:51 smithi132 ceph-mon[97863]: pgmap v467: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 334 op/s 2023-07-21T22:34:52.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:51 smithi107 ceph-mon[96543]: pgmap v467: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 334 op/s 2023-07-21T22:34:54.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:53 smithi190 ceph-mon[98334]: pgmap v468: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 362 op/s 2023-07-21T22:34:54.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:53 smithi132 ceph-mon[97863]: pgmap v468: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 362 op/s 2023-07-21T22:34:54.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:53 smithi107 ceph-mon[96543]: pgmap v468: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 362 op/s 2023-07-21T22:34:56.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:55 smithi190 ceph-mon[98334]: pgmap v469: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 307 op/s 2023-07-21T22:34:56.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:55 smithi132 ceph-mon[97863]: pgmap v469: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 307 op/s 2023-07-21T22:34:56.193 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:55 smithi107 ceph-mon[96543]: pgmap v469: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 307 op/s 2023-07-21T22:34:56.578 INFO:tasks.workunit.client.0.smithi107.stdout: 373 2209530 1147 1515813 1161 873746 996 2023-07-21T22:34:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:56.988+0000 7f67e4eae700 1 -- 172.21.15.107:0/4292209444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e0071480 msgr2=0x7f67e010b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:56.988+0000 7f67e4eae700 1 --2- 172.21.15.107:0/4292209444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e0071480 0x7f67e010b7d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f67d800b0a0 tx=0x7f67d800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:56.994+0000 7f67e4eae700 1 -- 172.21.15.107:0/4292209444 shutdown_connections 2023-07-21T22:34:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:56.994+0000 7f67e4eae700 1 --2- 172.21.15.107:0/4292209444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67e010bd10 0x7f67e010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:56.994+0000 7f67e4eae700 1 --2- 172.21.15.107:0/4292209444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67e0070b30 0x7f67e0070f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:56.994+0000 7f67e4eae700 1 --2- 172.21.15.107:0/4292209444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e0071480 0x7f67e010b7d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:56.994+0000 7f67e4eae700 1 -- 172.21.15.107:0/4292209444 >> 172.21.15.107:0/4292209444 conn(0x7f67e006c910 msgr2=0x7f67e006ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.004+0000 7f67e4eae700 1 -- 172.21.15.107:0/4292209444 shutdown_connections 2023-07-21T22:34:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.004+0000 7f67e4eae700 1 -- 172.21.15.107:0/4292209444 wait complete. 2023-07-21T22:34:57.052 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.005+0000 7f67e4eae700 1 Processor -- start 2023-07-21T22:34:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.006+0000 7f67e4eae700 1 -- start start 2023-07-21T22:34:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.006+0000 7f67e4eae700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e0070b30 0x7f67e01a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.006+0000 7f67e4eae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67e010bd10 0x7f67e01a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.006+0000 7f67e4eae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67e01a98e0 0x7f67e01abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.006+0000 7f67e4eae700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e01ac290 con 0x7f67e010bd10 2023-07-21T22:34:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.006+0000 7f67e4eae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e010ec40 con 0x7f67e01a98e0 2023-07-21T22:34:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.006+0000 7f67e4eae700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e010edd0 con 0x7f67e0070b30 2023-07-21T22:34:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.007+0000 7f67deffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67e010bd10 0x7f67e01a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.007+0000 7f67df7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e0070b30 0x7f67e01a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.007+0000 7f67dffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67e01a98e0 0x7f67e01abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.007+0000 7f67dffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67e01a98e0 0x7f67e01abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41430/0 (socket says 172.21.15.107:41430) 2023-07-21T22:34:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.007+0000 7f67dffff700 1 -- 172.21.15.107:0/1134095991 learned_addr learned my addr 172.21.15.107:0/1134095991 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.009+0000 7f67df7fe700 1 -- 172.21.15.107:0/1134095991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67e01a98e0 msgr2=0x7f67e01abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.009+0000 7f67df7fe700 1 --2- 172.21.15.107:0/1134095991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67e01a98e0 0x7f67e01abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.009+0000 7f67df7fe700 1 -- 172.21.15.107:0/1134095991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67e010bd10 msgr2=0x7f67e01a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.009+0000 7f67df7fe700 1 --2- 172.21.15.107:0/1134095991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67e010bd10 0x7f67e01a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.009+0000 7f67df7fe700 1 -- 172.21.15.107:0/1134095991 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d8009d00 con 0x7f67e0070b30 2023-07-21T22:34:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.016+0000 7f67df7fe700 1 --2- 172.21.15.107:0/1134095991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e0070b30 0x7f67e01a4340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f67d000e9c0 tx=0x7f67d000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.076+0000 7f67dcff9700 1 -- 172.21.15.107:0/1134095991 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d000cbd0 con 0x7f67e0070b30 2023-07-21T22:34:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.076+0000 7f67dcff9700 1 -- 172.21.15.107:0/1134095991 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d000eeb0 con 0x7f67e0070b30 2023-07-21T22:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.076+0000 7f67dcff9700 1 -- 172.21.15.107:0/1134095991 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0003c70 con 0x7f67e0070b30 2023-07-21T22:34:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.077+0000 7f67e4eae700 1 -- 172.21.15.107:0/1134095991 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e010f0b0 con 0x7f67e0070b30 2023-07-21T22:34:57.130 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.077+0000 7f67e4eae700 1 -- 172.21.15.107:0/1134095991 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e010f570 con 0x7f67e0070b30 2023-07-21T22:34:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.085+0000 7f67f2295b80 1 Processor -- start 2023-07-21T22:34:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.085+0000 7f67f2295b80 1 -- start start 2023-07-21T22:34:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.086+0000 7f67f2295b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a5ee26b0 0x5571a6087510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.086+0000 7f67f2295b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 0x5571a60860f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.086+0000 7f67f2295b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a6086630 0x5571a6086ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.086+0000 7f67f2295b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a5f8bc50 con 0x5571a6086630 2023-07-21T22:34:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.086+0000 7f67f2295b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a5fde9e0 con 0x5571a6087a50 2023-07-21T22:34:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.086+0000 7f67f2295b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a5fd7e70 con 0x5571a5ee26b0 2023-07-21T22:34:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.086+0000 7f67dffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a5ee26b0 0x5571a6087510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.086+0000 7f67dffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a5ee26b0 0x5571a6087510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39164/0 (socket says 172.21.15.107:39164) 2023-07-21T22:34:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.086+0000 7f67dffff700 1 -- 172.21.15.107:0/1680920746 learned_addr learned my addr 172.21.15.107:0/1680920746 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.087+0000 7f67deffd700 1 --2- 172.21.15.107:0/1680920746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a6086630 0x5571a6086ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.087+0000 7f67dffff700 1 -- 172.21.15.107:0/1680920746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 msgr2=0x5571a60860f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:34:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.087+0000 7f67dffff700 1 --2- 172.21.15.107:0/1680920746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 0x5571a60860f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.087+0000 7f67dffff700 1 -- 172.21.15.107:0/1680920746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a6086630 msgr2=0x5571a6086ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.087+0000 7f67dffff700 1 --2- 172.21.15.107:0/1680920746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a6086630 0x5571a6086ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.087+0000 7f67dffff700 1 -- 172.21.15.107:0/1680920746 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a5e05d80 con 0x5571a5ee26b0 2023-07-21T22:34:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.089+0000 7f67dffff700 1 --2- 172.21.15.107:0/1680920746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a5ee26b0 0x5571a6087510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f67d400ad10 tx=0x7f67d400f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e0caed4ed23521c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.100+0000 7f67dcff9700 1 -- 172.21.15.107:0/1134095991 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f67d0018430 con 0x7f67e0070b30 2023-07-21T22:34:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.101+0000 7f67dcff9700 1 --2- 172.21.15.107:0/1134095991 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f67c80a1350 0x7f67c80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.101+0000 7f67dcff9700 1 -- 172.21.15.107:0/1134095991 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f67d0014070 con 0x7f67e0070b30 2023-07-21T22:34:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.108+0000 7f67df7fe700 1 --2- 172.21.15.107:0/1134095991 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f67c80a1350 0x7f67c80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.112+0000 7f67c5ffb700 1 -- 172.21.15.107:0/1680920746 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d400a810 con 0x5571a5ee26b0 2023-07-21T22:34:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.112+0000 7f67c5ffb700 1 -- 172.21.15.107:0/1680920746 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d400fec0 con 0x5571a5ee26b0 2023-07-21T22:34:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.112+0000 7f67c5ffb700 1 -- 172.21.15.107:0/1680920746 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d40186a0 con 0x5571a5ee26b0 2023-07-21T22:34:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.114+0000 7f67f2295b80 1 -- 172.21.15.107:0/1680920746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a5ee26b0 msgr2=0x5571a6087510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.114+0000 7f67f2295b80 1 --2- 172.21.15.107:0/1680920746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a5ee26b0 0x5571a6087510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f67d400ad10 tx=0x7f67d400f6a0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.115+0000 7f67f2295b80 1 -- 172.21.15.107:0/1680920746 shutdown_connections 2023-07-21T22:34:57.162 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.115+0000 7f67f2295b80 1 --2- 172.21.15.107:0/1680920746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a6086630 0x5571a6086ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.115+0000 7f67f2295b80 1 --2- 172.21.15.107:0/1680920746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a5ee26b0 0x5571a6087510 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.115+0000 7f67f2295b80 1 --2- 172.21.15.107:0/1680920746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 0x5571a60860f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.115+0000 7f67f2295b80 1 -- 172.21.15.107:0/1680920746 >> 172.21.15.107:0/1680920746 conn(0x5571a607bc40 msgr2=0x5571a6080e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.115+0000 7f67df7fe700 1 --2- 172.21.15.107:0/1134095991 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f67c80a1350 0x7f67c80a3800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f67d009ad60 tx=0x7f67d0096ac0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.168+0000 7f67f2295b80 1 -- 172.21.15.107:0/1680920746 shutdown_connections 2023-07-21T22:34:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.168+0000 7f67f2295b80 1 -- 172.21.15.107:0/1680920746 wait complete. 2023-07-21T22:34:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.168+0000 7f67f2295b80 1 -- start start 2023-07-21T22:34:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.169+0000 7f67f2295b80 1 Processor -- start 2023-07-21T22:34:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.169+0000 7f67f2295b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a5ee26b0 0x5571a608f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.169+0000 7f67f2295b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a6086630 0x5571a608ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.169+0000 7f67f2295b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 0x5571a6096f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.169+0000 7f67f2295b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a5f77a60 con 0x5571a6086630 2023-07-21T22:34:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.169+0000 7f67f2295b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a5f77760 con 0x5571a6087a50 2023-07-21T22:34:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.169+0000 7f67f2295b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a5e5b8c0 con 0x5571a5ee26b0 2023-07-21T22:34:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.169+0000 7f67f2295b80 10 client.0 _ll_register_callbacks cb 0x5571a5f75700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.170+0000 7f67dffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a5ee26b0 0x5571a608f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.170+0000 7f67dffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a5ee26b0 0x5571a608f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39172/0 (socket says 172.21.15.107:39172) 2023-07-21T22:34:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.170+0000 7f67dffff700 1 -- 172.21.15.107:0/761143294 learned_addr learned my addr 172.21.15.107:0/761143294 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.170+0000 7f67dffff700 1 --2- 172.21.15.107:0/761143294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 0x5571a6096f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.171+0000 7f67deffd700 1 --2- 172.21.15.107:0/761143294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a6086630 0x5571a608ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.171+0000 7f67dffff700 1 -- 172.21.15.107:0/761143294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a5ee26b0 msgr2=0x5571a608f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.171+0000 7f67dffff700 1 --2- 172.21.15.107:0/761143294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a5ee26b0 0x5571a608f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.171+0000 7f67dffff700 1 -- 172.21.15.107:0/761143294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a6086630 msgr2=0x5571a608ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.171+0000 7f67dffff700 1 --2- 172.21.15.107:0/761143294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a6086630 0x5571a608ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.171+0000 7f67dffff700 1 -- 172.21.15.107:0/761143294 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d400a970 con 0x5571a6087a50 2023-07-21T22:34:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.176+0000 7f67dffff700 1 --2- 172.21.15.107:0/761143294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 0x5571a6096f50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f67d4005950 tx=0x7f67d400a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15b3df54d147657c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.214+0000 7f67beffd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:34:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.214+0000 7f67beffd700 1 -- 172.21.15.107:0/761143294 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4004960 con 0x5571a6087a50 2023-07-21T22:34:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.214+0000 7f67beffd700 1 -- 172.21.15.107:0/761143294 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f67d4004ac0 con 0x5571a6087a50 2023-07-21T22:34:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.214+0000 7f67f2295b80 1 -- 172.21.15.107:0/761143294 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571a5fd0e50 con 0x5571a6087a50 2023-07-21T22:34:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.219+0000 7f67beffd700 1 -- 172.21.15.107:0/761143294 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67d4004c80 con 0x5571a6087a50 2023-07-21T22:34:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.219+0000 7f67f2295b80 10 client.25910 fetch_fsmap learned FSMap version 39 2023-07-21T22:34:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.219+0000 7f67f2295b80 1 -- 172.21.15.107:0/761143294 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5571a5deec00 con 0x5571a6087a50 2023-07-21T22:34:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.230+0000 7f67beffd700 1 -- 172.21.15.107:0/761143294 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5571a5fd0e50 con 0x5571a6087a50 2023-07-21T22:34:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.230+0000 7f67beffd700 1 -- 172.21.15.107:0/761143294 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f67d402f070 con 0x5571a6087a50 2023-07-21T22:34:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.231+0000 7f67f2295b80 10 client.25910 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:34:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.231+0000 7f67f2295b80 10 client.25910 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.231+0000 7f67f2295b80 20 client.25910 populate_metadata read hostname 'smithi107' 2023-07-21T22:34:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.231+0000 7f67f2295b80 1 --2- 172.21.15.107:0/761143294 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5571a609b650 0x5571a609da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.231+0000 7f67f2295b80 4 client.25910 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.231+0000 7f67f2295b80 1 -- 172.21.15.107:0/761143294 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571a5f92ff0 con 0x5571a609b650 2023-07-21T22:34:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.235+0000 7f67deffd700 1 --2- 172.21.15.107:0/761143294 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5571a609b650 0x5571a609da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.246+0000 7f67deffd700 1 --2- 172.21.15.107:0/761143294 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5571a609b650 0x5571a609da60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2cba7f605d4c769 server_cookie=e91319eea5ce4ba7 in_seq=0 out_seq=0 2023-07-21T22:34:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.246+0000 7f67beffd700 10 client.25910 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.257+0000 7f67beffd700 1 -- 172.21.15.107:0/761143294 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5571a5f92ff0 con 0x5571a609b650 2023-07-21T22:34:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.257+0000 7f67beffd700 10 client.25910 handle_command_reply: tid=0 2023-07-21T22:34:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.257+0000 7f67f2295b80 1 client.25910 shutdown 2023-07-21T22:34:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.258+0000 7f67f2295b80 1 -- 172.21.15.107:0/761143294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 msgr2=0x5571a6096f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.258+0000 7f67f2295b80 1 --2- 172.21.15.107:0/761143294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 0x5571a6096f50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f67d4005950 tx=0x7f67d400a300 comp rx=0 tx=0).stop 2023-07-21T22:34:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.258+0000 7f67f2295b80 1 -- 172.21.15.107:0/761143294 shutdown_connections 2023-07-21T22:34:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.258+0000 7f67f2295b80 1 --2- 172.21.15.107:0/761143294 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5571a609b650 0x5571a609da60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.258+0000 7f67f2295b80 1 --2- 172.21.15.107:0/761143294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 0x5571a6096f50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.258+0000 7f67f2295b80 1 --2- 172.21.15.107:0/761143294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a5ee26b0 0x5571a608f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.258+0000 7f67f2295b80 1 --2- 172.21.15.107:0/761143294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a6086630 0x5571a608ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.258+0000 7f67f2295b80 1 -- 172.21.15.107:0/761143294 >> 172.21.15.107:0/761143294 conn(0x5571a607bc40 msgr2=0x5571a6080540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.258+0000 7f67beffd700 0 client.25910 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.258+0000 7f67f2295b80 1 -- 172.21.15.107:0/761143294 shutdown_connections 2023-07-21T22:34:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.258+0000 7f67f2295b80 1 -- 172.21.15.107:0/761143294 wait complete. 2023-07-21T22:34:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.259+0000 7f67f2295b80 20 client.25910 trim_cache size 0 max 16384 2023-07-21T22:34:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.331+0000 7f67f2295b80 1 Processor -- start 2023-07-21T22:34:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.340+0000 7f67f2295b80 1 -- start start 2023-07-21T22:34:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.340+0000 7f67f2295b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a5ee26b0 0x5571a60860a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.340+0000 7f67f2295b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a6086630 0x5571a6096e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.340+0000 7f67f2295b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 0x5571a608db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.340+0000 7f67f2295b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a6082280 con 0x5571a5ee26b0 2023-07-21T22:34:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.340+0000 7f67f2295b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a6090bc0 con 0x5571a6087a50 2023-07-21T22:34:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.340+0000 7f67f2295b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a60820c0 con 0x5571a6086630 2023-07-21T22:34:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.341+0000 7f67dffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 0x5571a608db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.341+0000 7f67dffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 0x5571a608db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41450/0 (socket says 172.21.15.107:41450) 2023-07-21T22:34:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.341+0000 7f67dffff700 1 -- 172.21.15.107:0/833756796 learned_addr learned my addr 172.21.15.107:0/833756796 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.341+0000 7f67deffd700 1 --2- 172.21.15.107:0/833756796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a6086630 0x5571a6096e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.341+0000 7f67dffff700 1 --2- 172.21.15.107:0/833756796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a5ee26b0 0x5571a60860a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.342+0000 7f67deffd700 1 -- 172.21.15.107:0/833756796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 msgr2=0x5571a608db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.342+0000 7f67deffd700 1 --2- 172.21.15.107:0/833756796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 0x5571a608db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.342+0000 7f67deffd700 1 -- 172.21.15.107:0/833756796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a5ee26b0 msgr2=0x5571a60860a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.342+0000 7f67deffd700 1 --2- 172.21.15.107:0/833756796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a5ee26b0 0x5571a60860a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.342+0000 7f67deffd700 1 -- 172.21.15.107:0/833756796 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571a5e2c1f0 con 0x5571a6086630 2023-07-21T22:34:57.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.342+0000 7f67deffd700 1 --2- 172.21.15.107:0/833756796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a6086630 0x5571a6096e70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67d8000f80 tx=0x7f67d8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5c064201cdcac95 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.371+0000 7f67c67fc700 1 -- 172.21.15.107:0/833756796 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d800e040 con 0x5571a6086630 2023-07-21T22:34:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.371+0000 7f67c67fc700 1 -- 172.21.15.107:0/833756796 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d8022070 con 0x5571a6086630 2023-07-21T22:34:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.371+0000 7f67c67fc700 1 -- 172.21.15.107:0/833756796 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8005050 con 0x5571a6086630 2023-07-21T22:34:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.372+0000 7f67f2295b80 1 -- 172.21.15.107:0/833756796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a6086630 msgr2=0x5571a6096e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:57.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.372+0000 7f67f2295b80 1 --2- 172.21.15.107:0/833756796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a6086630 0x5571a6096e70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67d8000f80 tx=0x7f67d8009b80 comp rx=0 tx=0).stop 2023-07-21T22:34:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.372+0000 7f67f2295b80 1 -- 172.21.15.107:0/833756796 shutdown_connections 2023-07-21T22:34:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.372+0000 7f67f2295b80 1 --2- 172.21.15.107:0/833756796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6087a50 0x5571a608db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.372+0000 7f67f2295b80 1 --2- 172.21.15.107:0/833756796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a5ee26b0 0x5571a60860a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.372+0000 7f67f2295b80 1 --2- 172.21.15.107:0/833756796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a6086630 0x5571a6096e70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.372+0000 7f67f2295b80 1 -- 172.21.15.107:0/833756796 >> 172.21.15.107:0/833756796 conn(0x5571a607bc40 msgr2=0x5571a608a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.372+0000 7f67f2295b80 1 -- 172.21.15.107:0/833756796 shutdown_connections 2023-07-21T22:34:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.372+0000 7f67f2295b80 1 -- 172.21.15.107:0/833756796 wait complete. 2023-07-21T22:34:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.373+0000 7f67f2295b80 1 -- start start 2023-07-21T22:34:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.373+0000 7f67f2295b80 1 Processor -- start 2023-07-21T22:34:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.374+0000 7f67f2295b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a5ee26b0 0x5571a60930a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.374+0000 7f67f2295b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6086630 0x5571a60935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.374+0000 7f67f2295b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a6087a50 0x5571a6096430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.374+0000 7f67f2295b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a6096970 con 0x5571a5ee26b0 2023-07-21T22:34:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.374+0000 7f67f2295b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a6096ae0 con 0x5571a6086630 2023-07-21T22:34:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.374+0000 7f67f2295b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571a6096ca0 con 0x5571a6087a50 2023-07-21T22:34:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.374+0000 7f67f2295b80 10 client.0 _ll_register_callbacks cb 0x7f67d4003f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.374+0000 7f67deffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6086630 0x5571a60935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.374+0000 7f67dffff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a6087a50 0x5571a6096430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.374+0000 7f67deffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6086630 0x5571a60935e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41454/0 (socket says 172.21.15.107:41454) 2023-07-21T22:34:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.374+0000 7f67deffd700 1 -- 172.21.15.107:0/901514110 learned_addr learned my addr 172.21.15.107:0/901514110 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:34:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.375+0000 7f67dffff700 1 --2- 172.21.15.107:0/901514110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a5ee26b0 0x5571a60930a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.375+0000 7f67dffff700 1 -- 172.21.15.107:0/901514110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6086630 msgr2=0x5571a60935e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.375+0000 7f67dffff700 1 --2- 172.21.15.107:0/901514110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6086630 0x5571a60935e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.375+0000 7f67dffff700 1 -- 172.21.15.107:0/901514110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a5ee26b0 msgr2=0x5571a60930a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.375+0000 7f67dffff700 1 --2- 172.21.15.107:0/901514110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a5ee26b0 0x5571a60930a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.375+0000 7f67dffff700 1 -- 172.21.15.107:0/901514110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e0117450 con 0x5571a6087a50 2023-07-21T22:34:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.375+0000 7f67dffff700 1 --2- 172.21.15.107:0/901514110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a6087a50 0x5571a6096430 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f67d4001eb0 tx=0x7f67d400ae60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74af7d6e87eb8d60 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.394+0000 7f67beffd700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:34:57.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.394+0000 7f67beffd700 1 -- 172.21.15.107:0/901514110 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4018ba0 con 0x5571a6087a50 2023-07-21T22:34:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.394+0000 7f67beffd700 1 -- 172.21.15.107:0/901514110 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f67d4000e40 con 0x5571a6087a50 2023-07-21T22:34:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.394+0000 7f67f2295b80 1 -- 172.21.15.107:0/901514110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571a5f31490 con 0x5571a6087a50 2023-07-21T22:34:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.403+0000 7f67beffd700 1 -- 172.21.15.107:0/901514110 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67d4018760 con 0x5571a6087a50 2023-07-21T22:34:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.403+0000 7f67f2295b80 10 client.25981 fetch_fsmap learned FSMap version 39 2023-07-21T22:34:57.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.403+0000 7f67f2295b80 1 -- 172.21.15.107:0/901514110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5571a5e04250 con 0x5571a6087a50 2023-07-21T22:34:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.405+0000 7f67beffd700 1 -- 172.21.15.107:0/901514110 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d4042070 con 0x5571a6087a50 2023-07-21T22:34:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.405+0000 7f67beffd700 1 -- 172.21.15.107:0/901514110 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f67d402f070 con 0x5571a6087a50 2023-07-21T22:34:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.407+0000 7f67f2295b80 10 client.25981 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:34:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.407+0000 7f67f2295b80 10 client.25981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.407+0000 7f67f2295b80 20 client.25981 populate_metadata read hostname 'smithi107' 2023-07-21T22:34:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.407+0000 7f67f2295b80 1 --2- 172.21.15.107:0/901514110 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5571a60f03a0 0x5571a60f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.407+0000 7f67f2295b80 4 client.25981 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:34:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.407+0000 7f67f2295b80 1 -- 172.21.15.107:0/901514110 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5571a5e26f10 con 0x5571a60f03a0 2023-07-21T22:34:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.418+0000 7f67deffd700 1 --2- 172.21.15.107:0/901514110 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5571a60f03a0 0x5571a60f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.438+0000 7f67deffd700 1 --2- 172.21.15.107:0/901514110 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5571a60f03a0 0x5571a60f2780 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3c06bd0cccd90ed server_cookie=6429b7554ce12c8e in_seq=0 out_seq=0 2023-07-21T22:34:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:34:57.438+0000 7f67beffd700 10 client.25981 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:34:58.144 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:57 smithi190 ceph-mon[98334]: pgmap v470: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2023-07-21T22:34:58.184 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:57 smithi132 ceph-mon[97863]: pgmap v470: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2023-07-21T22:34:58.197 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:57 smithi107 ceph-mon[96543]: pgmap v470: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2023-07-21T22:34:59.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:34:58 smithi190 ceph-mon[98334]: pgmap v471: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 208 op/s 2023-07-21T22:34:59.432 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:34:58 smithi132 ceph-mon[97863]: pgmap v471: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 208 op/s 2023-07-21T22:34:59.448 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:34:58 smithi107 ceph-mon[96543]: pgmap v471: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 208 op/s 2023-07-21T22:35:01.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:01 smithi190 ceph-mon[98334]: pgmap v472: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2023-07-21T22:35:01.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:01 smithi132 ceph-mon[97863]: pgmap v472: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2023-07-21T22:35:01.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:01 smithi107 ceph-mon[96543]: pgmap v472: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2023-07-21T22:35:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.940+0000 7f67beffd700 1 -- 172.21.15.107:0/901514110 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5571a5e26f10 con 0x5571a60f03a0 2023-07-21T22:35:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.940+0000 7f67beffd700 10 client.25981 handle_command_reply: tid=0 2023-07-21T22:35:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.941+0000 7f67f2295b80 1 client.25981 shutdown 2023-07-21T22:35:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.941+0000 7f67f2295b80 1 -- 172.21.15.107:0/901514110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a6087a50 msgr2=0x5571a6096430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.941+0000 7f67f2295b80 1 --2- 172.21.15.107:0/901514110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a6087a50 0x5571a6096430 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f67d4001eb0 tx=0x7f67d400ae60 comp rx=0 tx=0).stop 2023-07-21T22:35:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.941+0000 7f67f2295b80 1 -- 172.21.15.107:0/901514110 shutdown_connections 2023-07-21T22:35:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.941+0000 7f67f2295b80 1 --2- 172.21.15.107:0/901514110 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5571a60f03a0 0x5571a60f2780 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.941+0000 7f67f2295b80 1 --2- 172.21.15.107:0/901514110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5571a6087a50 0x5571a6096430 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.941+0000 7f67f2295b80 1 --2- 172.21.15.107:0/901514110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571a5ee26b0 0x5571a60930a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.941+0000 7f67f2295b80 1 --2- 172.21.15.107:0/901514110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5571a6086630 0x5571a60935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.941+0000 7f67f2295b80 1 -- 172.21.15.107:0/901514110 >> 172.21.15.107:0/901514110 conn(0x5571a607bc40 msgr2=0x5571a608db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.941+0000 7f67beffd700 0 client.25981 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:35:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:35:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:35:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:35:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.942+0000 7f67f2295b80 1 -- 172.21.15.107:0/901514110 shutdown_connections 2023-07-21T22:35:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.942+0000 7f67f2295b80 1 -- 172.21.15.107:0/901514110 wait complete. 2023-07-21T22:35:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.943+0000 7f67f2295b80 20 client.25981 trim_cache size 0 max 16384 2023-07-21T22:35:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.987+0000 7f67e4eae700 1 -- 172.21.15.107:0/1134095991 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f67c80a1350 msgr2=0x7f67c80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.987+0000 7f67e4eae700 1 --2- 172.21.15.107:0/1134095991 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f67c80a1350 0x7f67c80a3800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f67d009ad60 tx=0x7f67d0096ac0 comp rx=0 tx=0).stop 2023-07-21T22:35:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.987+0000 7f67e4eae700 1 -- 172.21.15.107:0/1134095991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e0070b30 msgr2=0x7f67e01a4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.987+0000 7f67e4eae700 1 --2- 172.21.15.107:0/1134095991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e0070b30 0x7f67e01a4340 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f67d000e9c0 tx=0x7f67d000ecd0 comp rx=0 tx=0).stop 2023-07-21T22:35:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.987+0000 7f67e4eae700 1 -- 172.21.15.107:0/1134095991 shutdown_connections 2023-07-21T22:35:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.987+0000 7f67e4eae700 1 --2- 172.21.15.107:0/1134095991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67e01a98e0 0x7f67e01abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.987+0000 7f67e4eae700 1 --2- 172.21.15.107:0/1134095991 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f67c80a1350 0x7f67c80a3800 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f67d009ad60 tx=0x7f67d0096ac0 comp rx=0 tx=0).stop 2023-07-21T22:35:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.987+0000 7f67e4eae700 1 --2- 172.21.15.107:0/1134095991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e0070b30 0x7f67e01a4340 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f67d000e9c0 tx=0x7f67d000ecd0 comp rx=0 tx=0).stop 2023-07-21T22:35:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.987+0000 7f67e4eae700 1 --2- 172.21.15.107:0/1134095991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67e010bd10 0x7f67e01a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:02.987+0000 7f67e4eae700 1 -- 172.21.15.107:0/1134095991 >> 172.21.15.107:0/1134095991 conn(0x7f67e006c910 msgr2=0x7f67e006d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:03.009+0000 7f67e4eae700 1 -- 172.21.15.107:0/1134095991 shutdown_connections 2023-07-21T22:35:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:03.020+0000 7f67e4eae700 1 -- 172.21.15.107:0/1134095991 wait complete. 2023-07-21T22:35:03.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:03 smithi190 ceph-mon[98334]: pgmap v473: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.7 MiB/s wr, 636 op/s 2023-07-21T22:35:03.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:03 smithi132 ceph-mon[97863]: pgmap v473: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.7 MiB/s wr, 636 op/s 2023-07-21T22:35:03.949 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:03 smithi107 ceph-mon[96543]: pgmap v473: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.7 MiB/s wr, 636 op/s 2023-07-21T22:35:04.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:04 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:35:04.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:04 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:35:04.948 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:04 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:35:05.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:05 smithi190 ceph-mon[98334]: pgmap v474: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 12 MiB/s wr, 876 op/s 2023-07-21T22:35:05.906 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:35:05.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:05 smithi132 ceph-mon[97863]: pgmap v474: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 12 MiB/s wr, 876 op/s 2023-07-21T22:35:06.052 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:05 smithi107 ceph-mon[96543]: pgmap v474: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 12 MiB/s wr, 876 op/s 2023-07-21T22:35:06.579 INFO:tasks.workunit.client.0.smithi107.stdout: 384 2220016 707 1520036 483 856671 320 2023-07-21T22:35:07.137 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.090+0000 7f930c47f700 1 -- 172.21.15.107:0/1738998432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9304070ee0 msgr2=0x7f93040712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:07.138 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.090+0000 7f930c47f700 1 --2- 172.21.15.107:0/1738998432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9304070ee0 0x7f93040712f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f93000098f0 tx=0x7f9300009c00 comp rx=0 tx=0).stop 2023-07-21T22:35:07.141 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.094+0000 7f930c47f700 1 -- 172.21.15.107:0/1738998432 shutdown_connections 2023-07-21T22:35:07.142 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.094+0000 7f930c47f700 1 --2- 172.21.15.107:0/1738998432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f930410bd60 0x7f930410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.142 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.094+0000 7f930c47f700 1 --2- 172.21.15.107:0/1738998432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9304070ee0 0x7f93040712f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.142 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.094+0000 7f930c47f700 1 --2- 172.21.15.107:0/1738998432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9304071830 0x7f930410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.142 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.094+0000 7f930c47f700 1 -- 172.21.15.107:0/1738998432 >> 172.21.15.107:0/1738998432 conn(0x7f930406ca90 msgr2=0x7f930406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:07.142 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.094+0000 7f930c47f700 1 -- 172.21.15.107:0/1738998432 shutdown_connections 2023-07-21T22:35:07.142 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.094+0000 7f930c47f700 1 -- 172.21.15.107:0/1738998432 wait complete. 2023-07-21T22:35:07.143 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.095+0000 7f930c47f700 1 Processor -- start 2023-07-21T22:35:07.143 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.095+0000 7f930c47f700 1 -- start start 2023-07-21T22:35:07.143 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.096+0000 7f930c47f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9304070ee0 0x7f9304113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.096+0000 7f930c47f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9304071830 0x7f9304113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.096+0000 7f930c47f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f930410bd60 0x7f93041b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.096+0000 7f930c47f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041170c0 con 0x7f9304071830 2023-07-21T22:35:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.096+0000 7f930c47f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9304117230 con 0x7f9304070ee0 2023-07-21T22:35:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.096+0000 7f930c47f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041b4e70 con 0x7f930410bd60 2023-07-21T22:35:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.096+0000 7f930aa1c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f930410bd60 0x7f93041b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.096+0000 7f930aa1c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f930410bd60 0x7f93041b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39200/0 (socket says 172.21.15.107:39200) 2023-07-21T22:35:07.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.096+0000 7f930aa1c700 1 -- 172.21.15.107:0/4143662204 learned_addr learned my addr 172.21.15.107:0/4143662204 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:07.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.096+0000 7f930a21b700 1 --2- 172.21.15.107:0/4143662204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9304070ee0 0x7f9304113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.097+0000 7f930a21b700 1 -- 172.21.15.107:0/4143662204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f930410bd60 msgr2=0x7f93041b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:07.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.097+0000 7f930a21b700 1 --2- 172.21.15.107:0/4143662204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f930410bd60 0x7f93041b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.097+0000 7f930a21b700 1 -- 172.21.15.107:0/4143662204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9304071830 msgr2=0x7f9304113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.097+0000 7f930a21b700 1 --2- 172.21.15.107:0/4143662204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9304071830 0x7f9304113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.097+0000 7f930a21b700 1 -- 172.21.15.107:0/4143662204 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9300009520 con 0x7f9304070ee0 2023-07-21T22:35:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.097+0000 7f930a21b700 1 --2- 172.21.15.107:0/4143662204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9304070ee0 0x7f9304113aa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9300009810 tx=0x7f930000bb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:07.154 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.107+0000 7f92fb7fe700 1 -- 172.21.15.107:0/4143662204 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300005520 con 0x7f9304070ee0 2023-07-21T22:35:07.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.107+0000 7f92fb7fe700 1 -- 172.21.15.107:0/4143662204 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9300005680 con 0x7f9304070ee0 2023-07-21T22:35:07.155 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.107+0000 7f92fb7fe700 1 -- 172.21.15.107:0/4143662204 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93000178e0 con 0x7f9304070ee0 2023-07-21T22:35:07.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.107+0000 7f930c47f700 1 -- 172.21.15.107:0/4143662204 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93041b5070 con 0x7f9304070ee0 2023-07-21T22:35:07.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.107+0000 7f930c47f700 1 -- 172.21.15.107:0/4143662204 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93041b5460 con 0x7f9304070ee0 2023-07-21T22:35:07.160 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.112+0000 7f9318604b80 1 Processor -- start 2023-07-21T22:35:07.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.113+0000 7f9318604b80 1 -- start start 2023-07-21T22:35:07.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.113+0000 7f9318604b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac79100830 0x55ac79100c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.114+0000 7f9318604b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac791011a0 0x55ac790ff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.114+0000 7f9318604b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 0x55ac790ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.114+0000 7f9318604b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac79004cb0 con 0x55ac790ffa10 2023-07-21T22:35:07.167 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.114+0000 7f9318604b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac79057a40 con 0x55ac79100830 2023-07-21T22:35:07.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.114+0000 7f9318604b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac79050ed0 con 0x55ac791011a0 2023-07-21T22:35:07.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.119+0000 7f930aa1c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac79100830 0x55ac79100c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.119+0000 7f930aa1c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac79100830 0x55ac79100c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41476/0 (socket says 172.21.15.107:41476) 2023-07-21T22:35:07.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.119+0000 7f930aa1c700 1 -- 172.21.15.107:0/1681420533 learned_addr learned my addr 172.21.15.107:0/1681420533 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:07.168 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.121+0000 7f9309a1a700 1 --2- 172.21.15.107:0/1681420533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 0x55ac790ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.121+0000 7f930aa1c700 1 -- 172.21.15.107:0/1681420533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac791011a0 msgr2=0x55ac790ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:07.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.121+0000 7f930aa1c700 1 --2- 172.21.15.107:0/1681420533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac791011a0 0x55ac790ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.121+0000 7f930aa1c700 1 -- 172.21.15.107:0/1681420533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 msgr2=0x55ac790ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:07.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.121+0000 7f930aa1c700 1 --2- 172.21.15.107:0/1681420533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 0x55ac790ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.169 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.121+0000 7f930aa1c700 1 -- 172.21.15.107:0/1681420533 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac78e7ede0 con 0x55ac79100830 2023-07-21T22:35:07.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.122+0000 7f92fb7fe700 1 -- 172.21.15.107:0/4143662204 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9300017a40 con 0x7f9304070ee0 2023-07-21T22:35:07.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.123+0000 7f92fb7fe700 1 --2- 172.21.15.107:0/4143662204 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f92f00a1350 0x7f92f00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.123+0000 7f92fb7fe700 1 -- 172.21.15.107:0/4143662204 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f93000cea10 con 0x7f9304070ee0 2023-07-21T22:35:07.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.123+0000 7f930aa1c700 1 --2- 172.21.15.107:0/1681420533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac79100830 0x55ac79100c40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009df0 tx=0x7f92f4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d1fd6c9161f4d67 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:07.173 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.125+0000 7f92f8dac700 1 -- 172.21.15.107:0/1681420533 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4019630 con 0x55ac79100830 2023-07-21T22:35:07.173 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.125+0000 7f92f8dac700 1 -- 172.21.15.107:0/1681420533 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92f4019c50 con 0x55ac79100830 2023-07-21T22:35:07.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.125+0000 7f92f8dac700 1 -- 172.21.15.107:0/1681420533 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f400ec80 con 0x55ac79100830 2023-07-21T22:35:07.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.126+0000 7f9318604b80 1 -- 172.21.15.107:0/1681420533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac79100830 msgr2=0x55ac79100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:07.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.126+0000 7f9318604b80 1 --2- 172.21.15.107:0/1681420533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac79100830 0x55ac79100c40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009df0 tx=0x7f92f4011720 comp rx=0 tx=0).stop 2023-07-21T22:35:07.177 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.129+0000 7f9318604b80 1 -- 172.21.15.107:0/1681420533 shutdown_connections 2023-07-21T22:35:07.177 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.129+0000 7f9318604b80 1 --2- 172.21.15.107:0/1681420533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 0x55ac790ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.177 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.129+0000 7f9318604b80 1 --2- 172.21.15.107:0/1681420533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac79100830 0x55ac79100c40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.177 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.129+0000 7f9318604b80 1 --2- 172.21.15.107:0/1681420533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac791011a0 0x55ac790ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.177 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.129+0000 7f9318604b80 1 -- 172.21.15.107:0/1681420533 >> 172.21.15.107:0/1681420533 conn(0x55ac790f4ef0 msgr2=0x55ac790fa360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:07.186 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.136+0000 7f930a21b700 1 --2- 172.21.15.107:0/4143662204 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f92f00a1350 0x7f92f00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.190 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.143+0000 7f9318604b80 1 -- 172.21.15.107:0/1681420533 shutdown_connections 2023-07-21T22:35:07.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.143+0000 7f9318604b80 1 -- 172.21.15.107:0/1681420533 wait complete. 2023-07-21T22:35:07.191 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.144+0000 7f9318604b80 1 -- start start 2023-07-21T22:35:07.192 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.144+0000 7f9318604b80 1 Processor -- start 2023-07-21T22:35:07.192 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.144+0000 7f9318604b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 0x55ac791091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.192 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.144+0000 7f9318604b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac79100830 0x55ac79109730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.192 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.144+0000 7f9318604b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac79110770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.193 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.144+0000 7f9318604b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac78ff0ac0 con 0x55ac790ffa10 2023-07-21T22:35:07.193 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.144+0000 7f9318604b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac78ff07c0 con 0x55ac791011a0 2023-07-21T22:35:07.193 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.144+0000 7f9318604b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac78ed4920 con 0x55ac79100830 2023-07-21T22:35:07.193 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.145+0000 7f9318604b80 10 client.0 _ll_register_callbacks cb 0x55ac78fee760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:07.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.150+0000 7f930aa1c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 0x55ac791091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.150+0000 7f930aa1c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 0x55ac791091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50656/0 (socket says 172.21.15.107:50656) 2023-07-21T22:35:07.199 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.150+0000 7f930aa1c700 1 -- 172.21.15.107:0/1728258098 learned_addr learned my addr 172.21.15.107:0/1728258098 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:07.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.152+0000 7f930aa1c700 1 --2- 172.21.15.107:0/1728258098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac79110770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.200 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.152+0000 7f9309a1a700 1 --2- 172.21.15.107:0/1728258098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac79100830 0x55ac79109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.163+0000 7f9309a1a700 1 -- 172.21.15.107:0/1728258098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 msgr2=0x55ac79110770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:07.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.163+0000 7f9309a1a700 1 --2- 172.21.15.107:0/1728258098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac79110770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.163+0000 7f9309a1a700 1 -- 172.21.15.107:0/1728258098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 msgr2=0x55ac791091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:07.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.163+0000 7f9309a1a700 1 --2- 172.21.15.107:0/1728258098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 0x55ac791091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.211 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.163+0000 7f9309a1a700 1 -- 172.21.15.107:0/1728258098 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f4009ad0 con 0x55ac79100830 2023-07-21T22:35:07.212 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.165+0000 7f930a21b700 1 --2- 172.21.15.107:0/4143662204 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f92f00a1350 0x7f92f00a3800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9300098330 tx=0x7f930009b840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:07.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.174+0000 7f9309a1a700 1 --2- 172.21.15.107:0/1728258098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac79100830 0x55ac79109730 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00ca10 tx=0x7f92fc00ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea71619b49bc4a6b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:07.221 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.174+0000 7f92edffb700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:35:07.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.174+0000 7f92edffb700 1 -- 172.21.15.107:0/1728258098 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc011070 con 0x55ac79100830 2023-07-21T22:35:07.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.174+0000 7f92edffb700 1 -- 172.21.15.107:0/1728258098 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f92fc015410 con 0x55ac79100830 2023-07-21T22:35:07.222 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.174+0000 7f9318604b80 1 -- 172.21.15.107:0/1728258098 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac78faa4f0 con 0x55ac79100830 2023-07-21T22:35:07.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.185+0000 7f92edffb700 1 -- 172.21.15.107:0/1728258098 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92fc0098d0 con 0x55ac79100830 2023-07-21T22:35:07.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.185+0000 7f9318604b80 10 client.25990 fetch_fsmap learned FSMap version 39 2023-07-21T22:35:07.232 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.185+0000 7f9318604b80 1 -- 172.21.15.107:0/1728258098 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ac78e67c60 con 0x55ac79100830 2023-07-21T22:35:07.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.185+0000 7f92edffb700 1 -- 172.21.15.107:0/1728258098 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac78faa4f0 con 0x55ac79100830 2023-07-21T22:35:07.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.186+0000 7f92edffb700 1 -- 172.21.15.107:0/1728258098 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f92fc01d070 con 0x55ac79100830 2023-07-21T22:35:07.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.186+0000 7f9318604b80 10 client.25990 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:35:07.234 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.186+0000 7f9318604b80 10 client.25990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:07.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.186+0000 7f9318604b80 20 client.25990 populate_metadata read hostname 'smithi107' 2023-07-21T22:35:07.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.187+0000 7f9318604b80 1 --2- 172.21.15.107:0/1728258098 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ac79114f10 0x55ac79117320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.187+0000 7f9318604b80 4 client.25990 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:07.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.187+0000 7f9318604b80 1 -- 172.21.15.107:0/1728258098 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac79049eb0 con 0x55ac79114f10 2023-07-21T22:35:07.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.187+0000 7f930aa1c700 1 --2- 172.21.15.107:0/1728258098 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ac79114f10 0x55ac79117320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.187+0000 7f930aa1c700 1 --2- 172.21.15.107:0/1728258098 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ac79114f10 0x55ac79117320 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f45a4042d101888 server_cookie=19c6b792a361a596 in_seq=0 out_seq=0 2023-07-21T22:35:07.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.187+0000 7f92edffb700 10 client.25990 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:07.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.189+0000 7f92edffb700 1 -- 172.21.15.107:0/1728258098 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ac79049eb0 con 0x55ac79114f10 2023-07-21T22:35:07.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.190+0000 7f92edffb700 10 client.25990 handle_command_reply: tid=0 2023-07-21T22:35:07.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.190+0000 7f9318604b80 1 client.25990 shutdown 2023-07-21T22:35:07.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.190+0000 7f9318604b80 1 -- 172.21.15.107:0/1728258098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac79100830 msgr2=0x55ac79109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:07.238 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.190+0000 7f9318604b80 1 --2- 172.21.15.107:0/1728258098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac79100830 0x55ac79109730 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00ca10 tx=0x7f92fc00ca40 comp rx=0 tx=0).stop 2023-07-21T22:35:07.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.193+0000 7f9318604b80 1 -- 172.21.15.107:0/1728258098 shutdown_connections 2023-07-21T22:35:07.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.193+0000 7f9318604b80 1 --2- 172.21.15.107:0/1728258098 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ac79114f10 0x55ac79117320 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.193+0000 7f9318604b80 1 --2- 172.21.15.107:0/1728258098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac79110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.193+0000 7f9318604b80 1 --2- 172.21.15.107:0/1728258098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 0x55ac791091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.193+0000 7f9318604b80 1 --2- 172.21.15.107:0/1728258098 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac79100830 0x55ac79109730 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.193+0000 7f9318604b80 1 -- 172.21.15.107:0/1728258098 >> 172.21.15.107:0/1728258098 conn(0x55ac790f4ef0 msgr2=0x55ac790f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:07.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.193+0000 7f92edffb700 0 client.25990 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:07.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.194+0000 7f9318604b80 1 -- 172.21.15.107:0/1728258098 shutdown_connections 2023-07-21T22:35:07.242 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.194+0000 7f9318604b80 1 -- 172.21.15.107:0/1728258098 wait complete. 2023-07-21T22:35:07.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.194+0000 7f9318604b80 20 client.25990 trim_cache size 0 max 16384 2023-07-21T22:35:07.310 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.263+0000 7f9318604b80 1 Processor -- start 2023-07-21T22:35:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.271+0000 7f9318604b80 1 -- start start 2023-07-21T22:35:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.271+0000 7f9318604b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 0x55ac790ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.271+0000 7f9318604b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac79100830 0x55ac79110690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.271+0000 7f9318604b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac79103b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.271+0000 7f9318604b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7910a3e0 con 0x55ac790ffa10 2023-07-21T22:35:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.271+0000 7f9318604b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac791084f0 con 0x55ac791011a0 2023-07-21T22:35:07.319 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.271+0000 7f9318604b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac790f49b0 con 0x55ac79100830 2023-07-21T22:35:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.271+0000 7f9309a1a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac79100830 0x55ac79110690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.271+0000 7f9309a1a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac79100830 0x55ac79110690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39220/0 (socket says 172.21.15.107:39220) 2023-07-21T22:35:07.320 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.271+0000 7f9309a1a700 1 -- 172.21.15.107:0/1473663536 learned_addr learned my addr 172.21.15.107:0/1473663536 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.276+0000 7f930aa1c700 1 --2- 172.21.15.107:0/1473663536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac79103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.323 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.276+0000 7f930aa1c700 1 --2- 172.21.15.107:0/1473663536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 0x55ac790ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.276+0000 7f930aa1c700 1 -- 172.21.15.107:0/1473663536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac79100830 msgr2=0x55ac79110690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:07.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.276+0000 7f930aa1c700 1 --2- 172.21.15.107:0/1473663536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac79100830 0x55ac79110690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.324 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.276+0000 7f930aa1c700 1 -- 172.21.15.107:0/1473663536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 msgr2=0x55ac790ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:07.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.276+0000 7f930aa1c700 1 --2- 172.21.15.107:0/1473663536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 0x55ac790ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.276+0000 7f930aa1c700 1 -- 172.21.15.107:0/1473663536 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac78ea5250 con 0x55ac791011a0 2023-07-21T22:35:07.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.277+0000 7f930aa1c700 1 --2- 172.21.15.107:0/1473663536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac79103b10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55ac79117bc0 tx=0x7f92f4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7590a92e826573c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:07.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.277+0000 7f92f95ad700 1 -- 172.21.15.107:0/1473663536 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f400e700 con 0x55ac791011a0 2023-07-21T22:35:07.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.277+0000 7f92f95ad700 1 -- 172.21.15.107:0/1473663536 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92f400ed90 con 0x55ac791011a0 2023-07-21T22:35:07.325 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.277+0000 7f92f95ad700 1 -- 172.21.15.107:0/1473663536 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4004970 con 0x55ac791011a0 2023-07-21T22:35:07.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.278+0000 7f9318604b80 1 -- 172.21.15.107:0/1473663536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 msgr2=0x55ac79103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:07.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.278+0000 7f9318604b80 1 --2- 172.21.15.107:0/1473663536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac79103b10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55ac79117bc0 tx=0x7f92f4009840 comp rx=0 tx=0).stop 2023-07-21T22:35:07.327 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.280+0000 7f9318604b80 1 -- 172.21.15.107:0/1473663536 shutdown_connections 2023-07-21T22:35:07.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.280+0000 7f9318604b80 1 --2- 172.21.15.107:0/1473663536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac79103b10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.280+0000 7f9318604b80 1 --2- 172.21.15.107:0/1473663536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac790ffa10 0x55ac790ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.280+0000 7f9318604b80 1 --2- 172.21.15.107:0/1473663536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac79100830 0x55ac79110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.280+0000 7f9318604b80 1 -- 172.21.15.107:0/1473663536 >> 172.21.15.107:0/1473663536 conn(0x55ac790f4ef0 msgr2=0x55ac79111060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:07.328 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.280+0000 7f9318604b80 1 -- 172.21.15.107:0/1473663536 shutdown_connections 2023-07-21T22:35:07.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.280+0000 7f9318604b80 1 -- 172.21.15.107:0/1473663536 wait complete. 2023-07-21T22:35:07.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.281+0000 7f9318604b80 1 -- start start 2023-07-21T22:35:07.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.281+0000 7f9318604b80 1 Processor -- start 2023-07-21T22:35:07.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.281+0000 7f9318604b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac790ffa10 0x55ac79110100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.329 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.281+0000 7f9318604b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac79100830 0x55ac79110640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.281+0000 7f9318604b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac7910cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.281+0000 7f9318604b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7910d370 con 0x55ac79100830 2023-07-21T22:35:07.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.281+0000 7f9318604b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7910d4e0 con 0x55ac791011a0 2023-07-21T22:35:07.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.281+0000 7f9318604b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7910d6a0 con 0x55ac790ffa10 2023-07-21T22:35:07.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.281+0000 7f9318604b80 10 client.0 _ll_register_callbacks cb 0x7f92fc009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:07.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.282+0000 7f930aa1c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac7910cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.282+0000 7f930aa1c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac7910cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41500/0 (socket says 172.21.15.107:41500) 2023-07-21T22:35:07.330 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.282+0000 7f930aa1c700 1 -- 172.21.15.107:0/795210468 learned_addr learned my addr 172.21.15.107:0/795210468 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:07.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.282+0000 7f930aa1c700 1 --2- 172.21.15.107:0/795210468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac790ffa10 0x55ac79110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.282+0000 7f930aa1c700 1 -- 172.21.15.107:0/795210468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac790ffa10 msgr2=0x55ac79110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:07.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.282+0000 7f930aa1c700 1 --2- 172.21.15.107:0/795210468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac790ffa10 0x55ac79110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.331 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.282+0000 7f930aa1c700 1 -- 172.21.15.107:0/795210468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac79100830 msgr2=0x55ac79110640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:07.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.282+0000 7f930aa1c700 1 --2- 172.21.15.107:0/795210468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac79100830 0x55ac79110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.282+0000 7f930aa1c700 1 -- 172.21.15.107:0/795210468 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f4031040 con 0x55ac791011a0 2023-07-21T22:35:07.332 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.283+0000 7f930aa1c700 1 --2- 172.21.15.107:0/795210468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac7910cc90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f92f4029040 tx=0x7f92f4009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b34bd48521ae428c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:07.351 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.294+0000 7f92edffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:35:07.351 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.294+0000 7f92edffb700 1 -- 172.21.15.107:0/795210468 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4019430 con 0x55ac791011a0 2023-07-21T22:35:07.352 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.294+0000 7f92edffb700 1 -- 172.21.15.107:0/795210468 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f92f4019590 con 0x55ac791011a0 2023-07-21T22:35:07.352 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.304+0000 7f9318604b80 1 -- 172.21.15.107:0/795210468 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92f4019c50 con 0x55ac791011a0 2023-07-21T22:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.316+0000 7f92edffb700 1 -- 172.21.15.107:0/795210468 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92f4019750 con 0x55ac791011a0 2023-07-21T22:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.316+0000 7f9318604b80 10 client.25934 fetch_fsmap learned FSMap version 39 2023-07-21T22:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.316+0000 7f9318604b80 1 -- 172.21.15.107:0/795210468 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ac78e7d2b0 con 0x55ac791011a0 2023-07-21T22:35:07.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.324+0000 7f92edffb700 1 -- 172.21.15.107:0/795210468 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92f404a030 con 0x55ac791011a0 2023-07-21T22:35:07.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.324+0000 7f92edffb700 1 -- 172.21.15.107:0/795210468 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f92f4005480 con 0x55ac791011a0 2023-07-21T22:35:07.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.326+0000 7f9318604b80 10 client.25934 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:35:07.374 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.326+0000 7f9318604b80 10 client.25934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:07.374 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.326+0000 7f9318604b80 20 client.25934 populate_metadata read hostname 'smithi107' 2023-07-21T22:35:07.374 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.326+0000 7f9318604b80 1 --2- 172.21.15.107:0/795210468 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ac7916a090 0x55ac7916c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:07.374 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.326+0000 7f9318604b80 4 client.25934 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:35:07.374 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.326+0000 7f9318604b80 1 -- 172.21.15.107:0/795210468 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ac7900c050 con 0x55ac7916a090 2023-07-21T22:35:07.374 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.326+0000 7f9309a1a700 1 --2- 172.21.15.107:0/795210468 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ac7916a090 0x55ac7916c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:07.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.365+0000 7f9309a1a700 1 --2- 172.21.15.107:0/795210468 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ac7916a090 0x55ac7916c4a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0cf4dc150b9930d server_cookie=431abdf194eec7bd in_seq=0 out_seq=0 2023-07-21T22:35:07.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.365+0000 7f92edffb700 10 client.25934 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:07.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.365+0000 7f92edffb700 1 -- 172.21.15.107:0/795210468 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ac7900c050 con 0x55ac7916a090 2023-07-21T22:35:07.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.365+0000 7f92edffb700 10 client.25934 handle_command_reply: tid=0 2023-07-21T22:35:07.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.365+0000 7f9318604b80 1 client.25934 shutdown 2023-07-21T22:35:07.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.365+0000 7f9318604b80 1 -- 172.21.15.107:0/795210468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 msgr2=0x55ac7910cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:07.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.365+0000 7f9318604b80 1 --2- 172.21.15.107:0/795210468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac7910cc90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f92f4029040 tx=0x7f92f4009290 comp rx=0 tx=0).stop 2023-07-21T22:35:07.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.366+0000 7f9318604b80 1 -- 172.21.15.107:0/795210468 shutdown_connections 2023-07-21T22:35:07.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.366+0000 7f9318604b80 1 --2- 172.21.15.107:0/795210468 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ac7916a090 0x55ac7916c4a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.366+0000 7f9318604b80 1 --2- 172.21.15.107:0/795210468 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ac791011a0 0x55ac7910cc90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.366+0000 7f9318604b80 1 --2- 172.21.15.107:0/795210468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ac790ffa10 0x55ac79110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.366+0000 7f9318604b80 1 --2- 172.21.15.107:0/795210468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac79100830 0x55ac79110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.366+0000 7f9318604b80 1 -- 172.21.15.107:0/795210468 >> 172.21.15.107:0/795210468 conn(0x55ac790f4ef0 msgr2=0x55ac790ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:07.414 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.366+0000 7f92edffb700 0 client.25934 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:07.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.366+0000 7f9318604b80 1 -- 172.21.15.107:0/795210468 shutdown_connections 2023-07-21T22:35:07.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.366+0000 7f9318604b80 1 -- 172.21.15.107:0/795210468 wait complete. 2023-07-21T22:35:07.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.366+0000 7f9318604b80 20 client.25934 trim_cache size 0 max 16384 2023-07-21T22:35:07.416 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:35:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:35:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:35:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "d037412e-e4e0-4d2c-8d6e-c7193371fe0f": { 2023-07-21T22:35:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:35:07.416 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d037412e-e4e0-4d2c-8d6e-c7193371fe0f", 2023-07-21T22:35:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:35:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:35:07.417 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:35:07.417 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:35:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.374+0000 7f930c47f700 1 -- 172.21.15.107:0/4143662204 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f92f00a1350 msgr2=0x7f92f00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.374+0000 7f930c47f700 1 --2- 172.21.15.107:0/4143662204 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f92f00a1350 0x7f92f00a3800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9300098330 tx=0x7f930009b840 comp rx=0 tx=0).stop 2023-07-21T22:35:07.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.374+0000 7f930c47f700 1 -- 172.21.15.107:0/4143662204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9304070ee0 msgr2=0x7f9304113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:07.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.374+0000 7f930c47f700 1 --2- 172.21.15.107:0/4143662204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9304070ee0 0x7f9304113aa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9300009810 tx=0x7f930000bb00 comp rx=0 tx=0).stop 2023-07-21T22:35:07.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.374+0000 7f930c47f700 1 -- 172.21.15.107:0/4143662204 shutdown_connections 2023-07-21T22:35:07.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.374+0000 7f930c47f700 1 --2- 172.21.15.107:0/4143662204 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f92f00a1350 0x7f92f00a3800 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.374+0000 7f930c47f700 1 --2- 172.21.15.107:0/4143662204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f930410bd60 0x7f93041b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.374+0000 7f930c47f700 1 --2- 172.21.15.107:0/4143662204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9304070ee0 0x7f9304113aa0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.374+0000 7f930c47f700 1 --2- 172.21.15.107:0/4143662204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9304071830 0x7f9304113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.374+0000 7f930c47f700 1 -- 172.21.15.107:0/4143662204 >> 172.21.15.107:0/4143662204 conn(0x7f930406ca90 msgr2=0x7f930406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.375+0000 7f930c47f700 1 -- 172.21.15.107:0/4143662204 shutdown_connections 2023-07-21T22:35:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:07.375+0000 7f930c47f700 1 -- 172.21.15.107:0/4143662204 wait complete. 2023-07-21T22:35:07.538 INFO:tasks.cephfs.filesystem:scrub status for tag:d037412e-e4e0-4d2c-8d6e-c7193371fe0f - {'path': '/', 'tag': 'd037412e-e4e0-4d2c-8d6e-c7193371fe0f', 'options': 'recursive,force'} 2023-07-21T22:35:07.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:07 smithi190 ceph-mon[98334]: pgmap v475: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 881 op/s 2023-07-21T22:35:07.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:07 smithi132 ceph-mon[97863]: pgmap v475: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 881 op/s 2023-07-21T22:35:07.948 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:07 smithi107 ceph-mon[96543]: pgmap v475: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 881 op/s 2023-07-21T22:35:08.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:08 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:35:08.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:08 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:35:08.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:08 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:35:08.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:08 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:35:08.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:08 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:35:08.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:08 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:35:08.945 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:35:08.946 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:35:08.946 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:35:09.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:09 smithi132 ceph-mon[97863]: pgmap v476: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 997 op/s 2023-07-21T22:35:09.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:09 smithi107 ceph-mon[96543]: pgmap v476: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 997 op/s 2023-07-21T22:35:10.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:09 smithi190 ceph-mon[98334]: pgmap v476: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 997 op/s 2023-07-21T22:35:11.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:11 smithi132 ceph-mon[97863]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-07-21T22:35:11.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:11 smithi107 ceph-mon[96543]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-07-21T22:35:12.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:11 smithi190 ceph-mon[98334]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 14 MiB/s wr, 1.02k op/s 2023-07-21T22:35:14.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:13 smithi190 ceph-mon[98334]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 954 op/s 2023-07-21T22:35:14.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:13 smithi132 ceph-mon[97863]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 954 op/s 2023-07-21T22:35:14.195 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:13 smithi107 ceph-mon[96543]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 954 op/s 2023-07-21T22:35:16.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:15 smithi190 ceph-mon[98334]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.5 MiB/s wr, 564 op/s 2023-07-21T22:35:16.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:15 smithi132 ceph-mon[97863]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.5 MiB/s wr, 564 op/s 2023-07-21T22:35:16.194 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:15 smithi107 ceph-mon[96543]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.5 MiB/s wr, 564 op/s 2023-07-21T22:35:16.586 INFO:tasks.workunit.client.0.smithi107.stdout: 399 2615567 578 1786437 955 991241 864 2023-07-21T22:35:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.623+0000 7fd6423ec700 1 -- 172.21.15.107:0/259798572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd63c0fea00 msgr2=0x7fd63c0fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.623+0000 7fd6423ec700 1 --2- 172.21.15.107:0/259798572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd63c0fea00 0x7fd63c0fee10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009990 tx=0x7fd62c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.623+0000 7fd6423ec700 1 -- 172.21.15.107:0/259798572 shutdown_connections 2023-07-21T22:35:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.623+0000 7fd6423ec700 1 --2- 172.21.15.107:0/259798572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd63c100580 0x7fd63c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.623+0000 7fd6423ec700 1 --2- 172.21.15.107:0/259798572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd63c0fea00 0x7fd63c0fee10 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.623+0000 7fd6423ec700 1 --2- 172.21.15.107:0/259798572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd63c0ffbf0 0x7fd63c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.623+0000 7fd6423ec700 1 -- 172.21.15.107:0/259798572 >> 172.21.15.107:0/259798572 conn(0x7fd63c0f9fe0 msgr2=0x7fd63c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.623+0000 7fd6423ec700 1 -- 172.21.15.107:0/259798572 shutdown_connections 2023-07-21T22:35:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.624+0000 7fd6423ec700 1 -- 172.21.15.107:0/259798572 wait complete. 2023-07-21T22:35:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.624+0000 7fd6423ec700 1 Processor -- start 2023-07-21T22:35:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.624+0000 7fd6423ec700 1 -- start start 2023-07-21T22:35:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.624+0000 7fd6423ec700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd63c0fea00 0x7fd63c194860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.624+0000 7fd6423ec700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd63c0ffbf0 0x7fd63c194da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.624+0000 7fd6423ec700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd63c100580 0x7fd63c195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd6423ec700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c196280 con 0x7fd63c100580 2023-07-21T22:35:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd6423ec700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c1963f0 con 0x7fd63c0ffbf0 2023-07-21T22:35:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd6423ec700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c1965b0 con 0x7fd63c0fea00 2023-07-21T22:35:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd640989700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd63c100580 0x7fd63c195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd63bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd63c0fea00 0x7fd63c194860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd640989700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd63c100580 0x7fd63c195c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50686/0 (socket says 172.21.15.107:50686) 2023-07-21T22:35:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd640989700 1 -- 172.21.15.107:0/2508771159 learned_addr learned my addr 172.21.15.107:0/2508771159 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd63b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd63c0ffbf0 0x7fd63c194da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd640989700 1 -- 172.21.15.107:0/2508771159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd63c0fea00 msgr2=0x7fd63c194860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd640989700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd63c0fea00 0x7fd63c194860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd640989700 1 -- 172.21.15.107:0/2508771159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd63c0ffbf0 msgr2=0x7fd63c194da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd640989700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd63c0ffbf0 0x7fd63c194da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd640989700 1 -- 172.21.15.107:0/2508771159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c0095f0 con 0x7fd63c100580 2023-07-21T22:35:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd63bfff700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd63c0fea00 0x7fd63c194860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd63b7fe700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd63c0ffbf0 0x7fd63c194da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd640989700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd63c100580 0x7fd63c195c20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd63400ef30 tx=0x7fd63400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd6397fa700 1 -- 172.21.15.107:0/2508771159 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634018070 con 0x7fd63c100580 2023-07-21T22:35:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd6423ec700 1 -- 172.21.15.107:0/2508771159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c196830 con 0x7fd63c100580 2023-07-21T22:35:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd6397fa700 1 -- 172.21.15.107:0/2508771159 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd63400f040 con 0x7fd63c100580 2023-07-21T22:35:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd6397fa700 1 -- 172.21.15.107:0/2508771159 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63400cd00 con 0x7fd63c100580 2023-07-21T22:35:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.625+0000 7fd6423ec700 1 -- 172.21.15.107:0/2508771159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c068660 con 0x7fd63c100580 2023-07-21T22:35:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.627+0000 7fd6397fa700 1 -- 172.21.15.107:0/2508771159 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd634003a00 con 0x7fd63c100580 2023-07-21T22:35:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.628+0000 7fd6397fa700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd6240a1150 0x7fd6240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.628+0000 7fd63bfff700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd6240a1150 0x7fd6240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.628+0000 7fd6397fa700 1 -- 172.21.15.107:0/2508771159 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd6340cc480 con 0x7fd63c100580 2023-07-21T22:35:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.628+0000 7fd63bfff700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd6240a1150 0x7fd6240a3600 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006010 tx=0x7fd62c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.629+0000 7fd64e571b80 1 Processor -- start 2023-07-21T22:35:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.629+0000 7fd64e571b80 1 -- start start 2023-07-21T22:35:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.629+0000 7fd64e571b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 0x559dff6e9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.629+0000 7fd64e571b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 0x559dff6ea520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.629+0000 7fd64e571b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6eaa60 0x559dff6eaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.629+0000 7fd64e571b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff5eac50 con 0x559dff6e9ba0 2023-07-21T22:35:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.629+0000 7fd64e571b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff63d9e0 con 0x559dff6eaa60 2023-07-21T22:35:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.629+0000 7fd64e571b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff636e70 con 0x559dff5416b0 2023-07-21T22:35:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.629+0000 7fd63b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 0x559dff6ea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.629+0000 7fd63b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 0x559dff6ea520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50692/0 (socket says 172.21.15.107:50692) 2023-07-21T22:35:16.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.629+0000 7fd63bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 0x559dff6e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.629+0000 7fd63b7fe700 1 -- 172.21.15.107:0/367749130 learned_addr learned my addr 172.21.15.107:0/367749130 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.629+0000 7fd640989700 1 --2- 172.21.15.107:0/367749130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6eaa60 0x559dff6eaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.630+0000 7fd63b7fe700 1 -- 172.21.15.107:0/367749130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 msgr2=0x559dff6e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.630+0000 7fd63b7fe700 1 --2- 172.21.15.107:0/367749130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 0x559dff6e9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.630+0000 7fd63b7fe700 1 -- 172.21.15.107:0/367749130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6eaa60 msgr2=0x559dff6eaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.630+0000 7fd63b7fe700 1 --2- 172.21.15.107:0/367749130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6eaa60 0x559dff6eaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.630+0000 7fd63b7fe700 1 -- 172.21.15.107:0/367749130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dff464d80 con 0x559dff6e9ba0 2023-07-21T22:35:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.630+0000 7fd640989700 1 --2- 172.21.15.107:0/367749130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6eaa60 0x559dff6eaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.630+0000 7fd63bfff700 1 --2- 172.21.15.107:0/367749130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 0x559dff6e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.630+0000 7fd63b7fe700 1 --2- 172.21.15.107:0/367749130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 0x559dff6ea520 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd63000e480 tx=0x7fd630009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed6dc5ebce26ce0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.630+0000 7fd6227fc700 1 -- 172.21.15.107:0/367749130 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630013040 con 0x559dff6e9ba0 2023-07-21T22:35:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.630+0000 7fd6227fc700 1 -- 172.21.15.107:0/367749130 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd630017ac0 con 0x559dff6e9ba0 2023-07-21T22:35:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.630+0000 7fd6227fc700 1 -- 172.21.15.107:0/367749130 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630016a90 con 0x559dff6e9ba0 2023-07-21T22:35:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 -- 172.21.15.107:0/367749130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 msgr2=0x559dff6ea520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 --2- 172.21.15.107:0/367749130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 0x559dff6ea520 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd63000e480 tx=0x7fd630009260 comp rx=0 tx=0).stop 2023-07-21T22:35:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 -- 172.21.15.107:0/367749130 shutdown_connections 2023-07-21T22:35:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 --2- 172.21.15.107:0/367749130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6eaa60 0x559dff6eaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 --2- 172.21.15.107:0/367749130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 0x559dff6e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 --2- 172.21.15.107:0/367749130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 0x559dff6ea520 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 -- 172.21.15.107:0/367749130 >> 172.21.15.107:0/367749130 conn(0x559dff6daae0 msgr2=0x559dff6dfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 -- 172.21.15.107:0/367749130 shutdown_connections 2023-07-21T22:35:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 -- 172.21.15.107:0/367749130 wait complete. 2023-07-21T22:35:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 -- start start 2023-07-21T22:35:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 Processor -- start 2023-07-21T22:35:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff5416b0 0x559dff6f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6e9ba0 0x559dff6f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff6eaa60 0x559dff6fa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff5d6a60 con 0x559dff5416b0 2023-07-21T22:35:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff5d6760 con 0x559dff6e9ba0 2023-07-21T22:35:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.631+0000 7fd64e571b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff4ba8c0 con 0x559dff6eaa60 2023-07-21T22:35:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd64e571b80 10 client.0 _ll_register_callbacks cb 0x559dff5d4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd63bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff5416b0 0x559dff6f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd63bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff5416b0 0x559dff6f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50696/0 (socket says 172.21.15.107:50696) 2023-07-21T22:35:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd63bfff700 1 -- 172.21.15.107:0/2753328835 learned_addr learned my addr 172.21.15.107:0/2753328835 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd640989700 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff6eaa60 0x559dff6fa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd63b7fe700 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6e9ba0 0x559dff6f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd63bfff700 1 -- 172.21.15.107:0/2753328835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff6eaa60 msgr2=0x559dff6fa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd63bfff700 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff6eaa60 0x559dff6fa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd63bfff700 1 -- 172.21.15.107:0/2753328835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6e9ba0 msgr2=0x559dff6f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd63bfff700 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6e9ba0 0x559dff6f30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd63bfff700 1 -- 172.21.15.107:0/2753328835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c107770 con 0x559dff5416b0 2023-07-21T22:35:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd640989700 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff6eaa60 0x559dff6fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd63b7fe700 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6e9ba0 0x559dff6f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd63bfff700 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff5416b0 0x559dff6f2b60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005850 tx=0x7fd62c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e6345b59590aaf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd61b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:35:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd61b7fe700 1 -- 172.21.15.107:0/2753328835 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c02b050 con 0x559dff5416b0 2023-07-21T22:35:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd61b7fe700 1 -- 172.21.15.107:0/2753328835 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd62c026a10 con 0x559dff5416b0 2023-07-21T22:35:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd64e571b80 1 -- 172.21.15.107:0/2753328835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dff590490 con 0x559dff5416b0 2023-07-21T22:35:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd61b7fe700 1 -- 172.21.15.107:0/2753328835 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd62c026bd0 con 0x559dff5416b0 2023-07-21T22:35:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd64e571b80 10 client.16404 fetch_fsmap learned FSMap version 39 2023-07-21T22:35:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.632+0000 7fd64e571b80 1 -- 172.21.15.107:0/2753328835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fd63c068660 con 0x559dff5416b0 2023-07-21T22:35:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.633+0000 7fd61b7fe700 1 -- 172.21.15.107:0/2753328835 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd62c03f020 con 0x559dff5416b0 2023-07-21T22:35:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.633+0000 7fd61b7fe700 1 -- 172.21.15.107:0/2753328835 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fd62c024cd0 con 0x559dff5416b0 2023-07-21T22:35:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.633+0000 7fd64e571b80 10 client.16404 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:35:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.633+0000 7fd64e571b80 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.633+0000 7fd64e571b80 20 client.16404 populate_metadata read hostname 'smithi107' 2023-07-21T22:35:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.633+0000 7fd64e571b80 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559dff6fe8e0 0x559dff700cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.633+0000 7fd64e571b80 4 client.16404 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.633+0000 7fd64e571b80 1 -- 172.21.15.107:0/2753328835 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559dff62fe50 con 0x559dff6fe8e0 2023-07-21T22:35:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.633+0000 7fd63b7fe700 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559dff6fe8e0 0x559dff700cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.633+0000 7fd61b7fe700 10 client.16404 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.633+0000 7fd63b7fe700 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559dff6fe8e0 0x559dff700cf0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=911a46f863ec0db7 server_cookie=250f76b6a0385d91 in_seq=0 out_seq=0 2023-07-21T22:35:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.634+0000 7fd61b7fe700 1 -- 172.21.15.107:0/2753328835 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559dff62fe50 con 0x559dff6fe8e0 2023-07-21T22:35:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.634+0000 7fd61b7fe700 10 client.16404 handle_command_reply: tid=0 2023-07-21T22:35:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.635+0000 7fd64e571b80 1 client.16404 shutdown 2023-07-21T22:35:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.635+0000 7fd64e571b80 1 -- 172.21.15.107:0/2753328835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff5416b0 msgr2=0x559dff6f2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.635+0000 7fd64e571b80 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff5416b0 0x559dff6f2b60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005850 tx=0x7fd62c01a050 comp rx=0 tx=0).stop 2023-07-21T22:35:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.635+0000 7fd64e571b80 1 -- 172.21.15.107:0/2753328835 shutdown_connections 2023-07-21T22:35:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.635+0000 7fd64e571b80 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559dff6fe8e0 0x559dff700cf0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.635+0000 7fd64e571b80 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff6eaa60 0x559dff6fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.635+0000 7fd61b7fe700 0 client.16404 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.635+0000 7fd64e571b80 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff5416b0 0x559dff6f2b60 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.635+0000 7fd64e571b80 1 --2- 172.21.15.107:0/2753328835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6e9ba0 0x559dff6f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.635+0000 7fd64e571b80 1 -- 172.21.15.107:0/2753328835 >> 172.21.15.107:0/2753328835 conn(0x559dff6daae0 msgr2=0x559dff6df3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.635+0000 7fd64e571b80 1 -- 172.21.15.107:0/2753328835 shutdown_connections 2023-07-21T22:35:16.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.635+0000 7fd64e571b80 1 -- 172.21.15.107:0/2753328835 wait complete. 2023-07-21T22:35:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.635+0000 7fd64e571b80 20 client.16404 trim_cache size 0 max 16384 2023-07-21T22:35:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd64e571b80 1 Processor -- start 2023-07-21T22:35:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd64e571b80 1 -- start start 2023-07-21T22:35:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd64e571b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 0x559dff6f2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd64e571b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6e9ba0 0x559dff6fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd64e571b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6eaa60 0x559dff6eae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd64e571b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff6eb370 con 0x559dff6eaa60 2023-07-21T22:35:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd64e571b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff6e9f70 con 0x559dff6e9ba0 2023-07-21T22:35:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd64e571b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff6ee410 con 0x559dff5416b0 2023-07-21T22:35:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd640989700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6eaa60 0x559dff6eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd63bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 0x559dff6f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd640989700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6eaa60 0x559dff6eae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50708/0 (socket says 172.21.15.107:50708) 2023-07-21T22:35:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd640989700 1 -- 172.21.15.107:0/455770158 learned_addr learned my addr 172.21.15.107:0/455770158 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd63b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6e9ba0 0x559dff6fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd640989700 1 -- 172.21.15.107:0/455770158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 msgr2=0x559dff6f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.651+0000 7fd640989700 1 --2- 172.21.15.107:0/455770158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 0x559dff6f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.652+0000 7fd640989700 1 -- 172.21.15.107:0/455770158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6e9ba0 msgr2=0x559dff6fa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.652+0000 7fd640989700 1 --2- 172.21.15.107:0/455770158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6e9ba0 0x559dff6fa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.652+0000 7fd640989700 1 -- 172.21.15.107:0/455770158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd630009dd0 con 0x559dff6eaa60 2023-07-21T22:35:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.652+0000 7fd63b7fe700 1 --2- 172.21.15.107:0/455770158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6e9ba0 0x559dff6fa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.652+0000 7fd640989700 1 --2- 172.21.15.107:0/455770158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6eaa60 0x559dff6eae30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd6340c9fb0 tx=0x7fd63409f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b2fc58e4dfceec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.652+0000 7fd622ffd700 1 -- 172.21.15.107:0/455770158 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340b7ea0 con 0x559dff6eaa60 2023-07-21T22:35:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.652+0000 7fd622ffd700 1 -- 172.21.15.107:0/455770158 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd63400ce60 con 0x559dff6eaa60 2023-07-21T22:35:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.652+0000 7fd622ffd700 1 -- 172.21.15.107:0/455770158 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634097dc0 con 0x559dff6eaa60 2023-07-21T22:35:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.652+0000 7fd64e571b80 1 -- 172.21.15.107:0/455770158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6eaa60 msgr2=0x559dff6eae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.652+0000 7fd64e571b80 1 --2- 172.21.15.107:0/455770158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6eaa60 0x559dff6eae30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd6340c9fb0 tx=0x7fd63409f040 comp rx=0 tx=0).stop 2023-07-21T22:35:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.653+0000 7fd64e571b80 1 -- 172.21.15.107:0/455770158 shutdown_connections 2023-07-21T22:35:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.653+0000 7fd64e571b80 1 --2- 172.21.15.107:0/455770158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6eaa60 0x559dff6eae30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.653+0000 7fd64e571b80 1 --2- 172.21.15.107:0/455770158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 0x559dff6f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.653+0000 7fd64e571b80 1 --2- 172.21.15.107:0/455770158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6e9ba0 0x559dff6fa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.653+0000 7fd64e571b80 1 -- 172.21.15.107:0/455770158 >> 172.21.15.107:0/455770158 conn(0x559dff6daae0 msgr2=0x559dff705650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.653+0000 7fd64e571b80 1 -- 172.21.15.107:0/455770158 shutdown_connections 2023-07-21T22:35:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.653+0000 7fd64e571b80 1 -- 172.21.15.107:0/455770158 wait complete. 2023-07-21T22:35:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.653+0000 7fd64e571b80 1 -- start start 2023-07-21T22:35:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.653+0000 7fd64e571b80 1 Processor -- start 2023-07-21T22:35:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.653+0000 7fd64e571b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 0x559dff6f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.653+0000 7fd64e571b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 0x559dff6f6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd64e571b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6eaa60 0x559dff6f9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd64e571b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff6f99c0 con 0x559dff6e9ba0 2023-07-21T22:35:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd64e571b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff6f9b30 con 0x559dff6eaa60 2023-07-21T22:35:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd64e571b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff6f9cf0 con 0x559dff5416b0 2023-07-21T22:35:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd64e571b80 10 client.0 _ll_register_callbacks cb 0x7fd62c00f7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd63b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 0x559dff6f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd63b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 0x559dff6f6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50712/0 (socket says 172.21.15.107:50712) 2023-07-21T22:35:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd63b7fe700 1 -- 172.21.15.107:0/3754641877 learned_addr learned my addr 172.21.15.107:0/3754641877 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd63bfff700 1 --2- 172.21.15.107:0/3754641877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 0x559dff6f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd640989700 1 --2- 172.21.15.107:0/3754641877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6eaa60 0x559dff6f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd63b7fe700 1 -- 172.21.15.107:0/3754641877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 msgr2=0x559dff6f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd63b7fe700 1 --2- 172.21.15.107:0/3754641877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 0x559dff6f5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd63b7fe700 1 -- 172.21.15.107:0/3754641877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6eaa60 msgr2=0x559dff6f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd63b7fe700 1 --2- 172.21.15.107:0/3754641877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6eaa60 0x559dff6f9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd63b7fe700 1 -- 172.21.15.107:0/3754641877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559dff464d80 con 0x559dff6e9ba0 2023-07-21T22:35:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd63b7fe700 1 --2- 172.21.15.107:0/3754641877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 0x559dff6f6460 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd63000e420 tx=0x7fd630005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0d72463b603f1b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd61b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:35:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd61b7fe700 1 -- 172.21.15.107:0/3754641877 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630013040 con 0x559dff6e9ba0 2023-07-21T22:35:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd61b7fe700 1 -- 172.21.15.107:0/3754641877 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd63002b2d0 con 0x559dff6e9ba0 2023-07-21T22:35:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd64e571b80 1 -- 172.21.15.107:0/3754641877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd630017ac0 con 0x559dff6e9ba0 2023-07-21T22:35:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd61b7fe700 1 -- 172.21.15.107:0/3754641877 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd63002b490 con 0x559dff6e9ba0 2023-07-21T22:35:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.654+0000 7fd64e571b80 10 client.16410 fetch_fsmap learned FSMap version 39 2023-07-21T22:35:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.655+0000 7fd64e571b80 1 -- 172.21.15.107:0/3754641877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fd63c196830 con 0x559dff6e9ba0 2023-07-21T22:35:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.655+0000 7fd61b7fe700 1 -- 172.21.15.107:0/3754641877 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd630017ac0 con 0x559dff6e9ba0 2023-07-21T22:35:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.655+0000 7fd61b7fe700 1 -- 172.21.15.107:0/3754641877 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fd63001e070 con 0x559dff6e9ba0 2023-07-21T22:35:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.655+0000 7fd64e571b80 10 client.16410 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:35:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.655+0000 7fd64e571b80 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.655+0000 7fd64e571b80 20 client.16410 populate_metadata read hostname 'smithi107' 2023-07-21T22:35:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.655+0000 7fd64e571b80 1 --2- 172.21.15.107:0/3754641877 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559dff7536b0 0x559dff755ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.655+0000 7fd64e571b80 4 client.16410 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:35:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.655+0000 7fd64e571b80 1 -- 172.21.15.107:0/3754641877 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559dff5f1ff0 con 0x559dff7536b0 2023-07-21T22:35:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.655+0000 7fd63bfff700 1 --2- 172.21.15.107:0/3754641877 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559dff7536b0 0x559dff755ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.655+0000 7fd61b7fe700 10 client.16410 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:16.655+0000 7fd63bfff700 1 --2- 172.21.15.107:0/3754641877 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559dff7536b0 0x559dff755ac0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a6473c24847c383 server_cookie=970c32b7e77d37ab in_seq=0 out_seq=0 2023-07-21T22:35:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd61b7fe700 1 -- 172.21.15.107:0/3754641877 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559dff5f1ff0 con 0x559dff7536b0 2023-07-21T22:35:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd61b7fe700 10 client.16410 handle_command_reply: tid=0 2023-07-21T22:35:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd64e571b80 1 client.16410 shutdown 2023-07-21T22:35:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd64e571b80 1 -- 172.21.15.107:0/3754641877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 msgr2=0x559dff6f6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd64e571b80 1 --2- 172.21.15.107:0/3754641877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 0x559dff6f6460 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd63000e420 tx=0x7fd630005b20 comp rx=0 tx=0).stop 2023-07-21T22:35:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd64e571b80 1 -- 172.21.15.107:0/3754641877 shutdown_connections 2023-07-21T22:35:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd64e571b80 1 --2- 172.21.15.107:0/3754641877 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559dff7536b0 0x559dff755ac0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd64e571b80 1 --2- 172.21.15.107:0/3754641877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559dff6eaa60 0x559dff6f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd64e571b80 1 --2- 172.21.15.107:0/3754641877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559dff5416b0 0x559dff6f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd64e571b80 1 --2- 172.21.15.107:0/3754641877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559dff6e9ba0 0x559dff6f6460 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd64e571b80 1 -- 172.21.15.107:0/3754641877 >> 172.21.15.107:0/3754641877 conn(0x559dff6daae0 msgr2=0x559dff6ed8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd61b7fe700 0 client.16410 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd64e571b80 1 -- 172.21.15.107:0/3754641877 shutdown_connections 2023-07-21T22:35:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.158+0000 7fd64e571b80 1 -- 172.21.15.107:0/3754641877 wait complete. 2023-07-21T22:35:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.159+0000 7fd64e571b80 20 client.16410 trim_cache size 0 max 16384 2023-07-21T22:35:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:35:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:35:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:35:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:35:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.159+0000 7fd6423ec700 1 -- 172.21.15.107:0/2508771159 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd6240a1150 msgr2=0x7fd6240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.159+0000 7fd6423ec700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd6240a1150 0x7fd6240a3600 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006010 tx=0x7fd62c005960 comp rx=0 tx=0).stop 2023-07-21T22:35:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.159+0000 7fd6423ec700 1 -- 172.21.15.107:0/2508771159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd63c100580 msgr2=0x7fd63c195c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.159+0000 7fd6423ec700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd63c100580 0x7fd63c195c20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd63400ef30 tx=0x7fd63400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:35:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.159+0000 7fd6423ec700 1 -- 172.21.15.107:0/2508771159 shutdown_connections 2023-07-21T22:35:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.159+0000 7fd6423ec700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd63c100580 0x7fd63c195c20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.159+0000 7fd6423ec700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd6240a1150 0x7fd6240a3600 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.159+0000 7fd6423ec700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd63c0fea00 0x7fd63c194860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.159+0000 7fd6423ec700 1 --2- 172.21.15.107:0/2508771159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd63c0ffbf0 0x7fd63c194da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.159+0000 7fd6423ec700 1 -- 172.21.15.107:0/2508771159 >> 172.21.15.107:0/2508771159 conn(0x7fd63c0f9fe0 msgr2=0x7fd63c0fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.159+0000 7fd6423ec700 1 -- 172.21.15.107:0/2508771159 shutdown_connections 2023-07-21T22:35:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:17.159+0000 7fd6423ec700 1 -- 172.21.15.107:0/2508771159 wait complete. 2023-07-21T22:35:18.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:17 smithi190 ceph-mon[98334]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 314 op/s 2023-07-21T22:35:18.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:17 smithi132 ceph-mon[97863]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 314 op/s 2023-07-21T22:35:18.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:17 smithi107 ceph-mon[96543]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 314 op/s 2023-07-21T22:35:20.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:19 smithi190 ceph-mon[98334]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 457 op/s 2023-07-21T22:35:20.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:19 smithi132 ceph-mon[97863]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 457 op/s 2023-07-21T22:35:20.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:19 smithi107 ceph-mon[96543]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 457 op/s 2023-07-21T22:35:22.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:21 smithi190 ceph-mon[98334]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 359 op/s 2023-07-21T22:35:22.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:21 smithi132 ceph-mon[97863]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 359 op/s 2023-07-21T22:35:22.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:21 smithi107 ceph-mon[96543]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 359 op/s 2023-07-21T22:35:24.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:23 smithi190 ceph-mon[98334]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 400 op/s 2023-07-21T22:35:24.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:23 smithi132 ceph-mon[97863]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 400 op/s 2023-07-21T22:35:24.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:23 smithi107 ceph-mon[96543]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 400 op/s 2023-07-21T22:35:26.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:25 smithi190 ceph-mon[98334]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 372 op/s 2023-07-21T22:35:26.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:25 smithi132 ceph-mon[97863]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 372 op/s 2023-07-21T22:35:26.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:25 smithi107 ceph-mon[96543]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 372 op/s 2023-07-21T22:35:28.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:27 smithi190 ceph-mon[98334]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 357 op/s 2023-07-21T22:35:28.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:27 smithi132 ceph-mon[97863]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 357 op/s 2023-07-21T22:35:28.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:27 smithi107 ceph-mon[96543]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 357 op/s 2023-07-21T22:35:30.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:29 smithi190 ceph-mon[98334]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1013 KiB/s wr, 343 op/s 2023-07-21T22:35:30.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:29 smithi132 ceph-mon[97863]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1013 KiB/s wr, 343 op/s 2023-07-21T22:35:30.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:29 smithi107 ceph-mon[96543]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1013 KiB/s wr, 343 op/s 2023-07-21T22:35:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.267+0000 7f6af92b8700 1 -- 172.21.15.107:0/3636362166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6af4100600 msgr2=0x7f6af4106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.267+0000 7f6af92b8700 1 --2- 172.21.15.107:0/3636362166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6af4100600 0x7f6af4106ec0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00b0a0 tx=0x7f6aec00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.267+0000 7f6af92b8700 1 -- 172.21.15.107:0/3636362166 shutdown_connections 2023-07-21T22:35:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.267+0000 7f6af92b8700 1 --2- 172.21.15.107:0/3636362166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6af4100600 0x7f6af4106ec0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.267+0000 7f6af92b8700 1 --2- 172.21.15.107:0/3636362166 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af40fea80 0x7f6af40fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.267+0000 7f6af92b8700 1 --2- 172.21.15.107:0/3636362166 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6af40ffc70 0x7f6af41000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.267+0000 7f6af92b8700 1 -- 172.21.15.107:0/3636362166 >> 172.21.15.107:0/3636362166 conn(0x7f6af40fa0c0 msgr2=0x7f6af40fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.267+0000 7f6af92b8700 1 -- 172.21.15.107:0/3636362166 shutdown_connections 2023-07-21T22:35:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.267+0000 7f6af92b8700 1 -- 172.21.15.107:0/3636362166 wait complete. 2023-07-21T22:35:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.267+0000 7f6af92b8700 1 Processor -- start 2023-07-21T22:35:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af92b8700 1 -- start start 2023-07-21T22:35:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af92b8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af40fea80 0x7f6af4194910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af92b8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6af40ffc70 0x7f6af4194e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af92b8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6af4100600 0x7f6af4195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af92b8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af4196330 con 0x7f6af40ffc70 2023-07-21T22:35:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af92b8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af41964a0 con 0x7f6af4100600 2023-07-21T22:35:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af92b8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af4196660 con 0x7f6af40fea80 2023-07-21T22:35:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af27fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6af40ffc70 0x7f6af4194e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af27fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6af40ffc70 0x7f6af4194e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50726/0 (socket says 172.21.15.107:50726) 2023-07-21T22:35:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af27fc700 1 -- 172.21.15.107:0/1555162712 learned_addr learned my addr 172.21.15.107:0/1555162712 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af2ffd700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af40fea80 0x7f6af4194910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af37fe700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6af4100600 0x7f6af4195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af27fc700 1 -- 172.21.15.107:0/1555162712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af40fea80 msgr2=0x7f6af4194910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af27fc700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af40fea80 0x7f6af4194910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af27fc700 1 -- 172.21.15.107:0/1555162712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6af4100600 msgr2=0x7f6af4195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af27fc700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6af4100600 0x7f6af4195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af27fc700 1 -- 172.21.15.107:0/1555162712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec009d00 con 0x7f6af40ffc70 2023-07-21T22:35:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af27fc700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6af40ffc70 0x7f6af4194e50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400e9a0 tx=0x7f6ae400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6adbfff700 1 -- 172.21.15.107:0/1555162712 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4009790 con 0x7f6af40ffc70 2023-07-21T22:35:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6af92b8700 1 -- 172.21.15.107:0/1555162712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af4196940 con 0x7f6af40ffc70 2023-07-21T22:35:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6adbfff700 1 -- 172.21.15.107:0/1555162712 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ae4010430 con 0x7f6af40ffc70 2023-07-21T22:35:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.268+0000 7f6adbfff700 1 -- 172.21.15.107:0/1555162712 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4018680 con 0x7f6af40ffc70 2023-07-21T22:35:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.269+0000 7f6af92b8700 1 -- 172.21.15.107:0/1555162712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af419d100 con 0x7f6af40ffc70 2023-07-21T22:35:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.270+0000 7f6adbfff700 1 -- 172.21.15.107:0/1555162712 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6ae40105a0 con 0x7f6af40ffc70 2023-07-21T22:35:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.271+0000 7f6adbfff700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6adc0a1150 0x7f6adc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.271+0000 7f6af2ffd700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6adc0a1150 0x7f6adc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.271+0000 7f6adbfff700 1 -- 172.21.15.107:0/1555162712 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6ae4014070 con 0x7f6af40ffc70 2023-07-21T22:35:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6af2ffd700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6adc0a1150 0x7f6adc0a3600 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6af40ffad0 tx=0x7f6ae8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6b0543db80 1 Processor -- start 2023-07-21T22:35:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6b0543db80 1 -- start start 2023-07-21T22:35:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6b0543db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 0x561c1cd66460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6b0543db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd669a0 0x561c1cd67320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6b0543db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd67860 0x561c1cd67cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6b0543db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1cc69c50 con 0x561c1cbc06b0 2023-07-21T22:35:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6b0543db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1ccbc9e0 con 0x561c1cd67860 2023-07-21T22:35:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6b0543db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1ccb5e70 con 0x561c1cd669a0 2023-07-21T22:35:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6af2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 0x561c1cd66460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6af2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 0x561c1cd66460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50732/0 (socket says 172.21.15.107:50732) 2023-07-21T22:35:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6af2ffd700 1 -- 172.21.15.107:0/3788091664 learned_addr learned my addr 172.21.15.107:0/3788091664 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6af37fe700 1 --2- 172.21.15.107:0/3788091664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd67860 0x561c1cd67cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.272+0000 7f6af27fc700 1 --2- 172.21.15.107:0/3788091664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd669a0 0x561c1cd67320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.273+0000 7f6af2ffd700 1 -- 172.21.15.107:0/3788091664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd669a0 msgr2=0x561c1cd67320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.273+0000 7f6af2ffd700 1 --2- 172.21.15.107:0/3788091664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd669a0 0x561c1cd67320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.273+0000 7f6af2ffd700 1 -- 172.21.15.107:0/3788091664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd67860 msgr2=0x561c1cd67cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.273+0000 7f6af2ffd700 1 --2- 172.21.15.107:0/3788091664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd67860 0x561c1cd67cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.273+0000 7f6af2ffd700 1 -- 172.21.15.107:0/3788091664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c1cae3d80 con 0x561c1cbc06b0 2023-07-21T22:35:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.273+0000 7f6af2ffd700 1 --2- 172.21.15.107:0/3788091664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 0x561c1cd66460 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8011d50 tx=0x7f6ae8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f788dd3d16c9faad server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.273+0000 7f6ad97fa700 1 -- 172.21.15.107:0/3788091664 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae800ed20 con 0x561c1cbc06b0 2023-07-21T22:35:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.273+0000 7f6ad97fa700 1 -- 172.21.15.107:0/3788091664 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ae8021070 con 0x561c1cbc06b0 2023-07-21T22:35:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.273+0000 7f6ad97fa700 1 -- 172.21.15.107:0/3788091664 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae801d410 con 0x561c1cbc06b0 2023-07-21T22:35:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 -- 172.21.15.107:0/3788091664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 msgr2=0x561c1cd66460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 --2- 172.21.15.107:0/3788091664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 0x561c1cd66460 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8011d50 tx=0x7f6ae8014000 comp rx=0 tx=0).stop 2023-07-21T22:35:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 -- 172.21.15.107:0/3788091664 shutdown_connections 2023-07-21T22:35:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 --2- 172.21.15.107:0/3788091664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd67860 0x561c1cd67cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 --2- 172.21.15.107:0/3788091664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 0x561c1cd66460 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 --2- 172.21.15.107:0/3788091664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd669a0 0x561c1cd67320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 -- 172.21.15.107:0/3788091664 >> 172.21.15.107:0/3788091664 conn(0x561c1cd59ae0 msgr2=0x561c1cd5ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 -- 172.21.15.107:0/3788091664 shutdown_connections 2023-07-21T22:35:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 -- 172.21.15.107:0/3788091664 wait complete. 2023-07-21T22:35:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 -- start start 2023-07-21T22:35:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 Processor -- start 2023-07-21T22:35:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cbc06b0 0x561c1cd6f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd669a0 0x561c1cd6ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd67860 0x561c1cd76f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1cc55a60 con 0x561c1cd669a0 2023-07-21T22:35:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1cc55760 con 0x561c1cbc06b0 2023-07-21T22:35:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1cb398c0 con 0x561c1cd67860 2023-07-21T22:35:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6b0543db80 10 client.0 _ll_register_callbacks cb 0x561c1cc53700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6af27fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd669a0 0x561c1cd6ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6af2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cbc06b0 0x561c1cd6f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6af2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cbc06b0 0x561c1cd6f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41566/0 (socket says 172.21.15.107:41566) 2023-07-21T22:35:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6af27fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd669a0 0x561c1cd6ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50742/0 (socket says 172.21.15.107:50742) 2023-07-21T22:35:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6af2ffd700 1 -- 172.21.15.107:0/2081729873 learned_addr learned my addr 172.21.15.107:0/2081729873 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.274+0000 7f6af37fe700 1 --2- 172.21.15.107:0/2081729873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd67860 0x561c1cd76f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6af2ffd700 1 -- 172.21.15.107:0/2081729873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd67860 msgr2=0x561c1cd76f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6af2ffd700 1 --2- 172.21.15.107:0/2081729873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd67860 0x561c1cd76f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6af2ffd700 1 -- 172.21.15.107:0/2081729873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd669a0 msgr2=0x561c1cd6ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6af2ffd700 1 --2- 172.21.15.107:0/2081729873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd669a0 0x561c1cd6ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6af2ffd700 1 -- 172.21.15.107:0/2081729873 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae8017070 con 0x561c1cbc06b0 2023-07-21T22:35:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6af27fc700 1 --2- 172.21.15.107:0/2081729873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd669a0 0x561c1cd6ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6af2ffd700 1 --2- 172.21.15.107:0/2081729873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cbc06b0 0x561c1cd6f9d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8011b30 tx=0x7f6ae801cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b19aeda79e7beb4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6ad6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:35:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6ad6ff5700 1 -- 172.21.15.107:0/2081729873 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae800ed30 con 0x561c1cbc06b0 2023-07-21T22:35:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6b0543db80 1 -- 172.21.15.107:0/2081729873 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c1cc0f490 con 0x561c1cbc06b0 2023-07-21T22:35:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6ad6ff5700 1 -- 172.21.15.107:0/2081729873 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6ae801c410 con 0x561c1cbc06b0 2023-07-21T22:35:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6ad6ff5700 1 -- 172.21.15.107:0/2081729873 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ae801c5d0 con 0x561c1cbc06b0 2023-07-21T22:35:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6b0543db80 10 client.25964 fetch_fsmap learned FSMap version 39 2023-07-21T22:35:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.275+0000 7f6b0543db80 1 -- 172.21.15.107:0/2081729873 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x561c1caccc00 con 0x561c1cbc06b0 2023-07-21T22:35:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.276+0000 7f6ad6ff5700 1 -- 172.21.15.107:0/2081729873 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c1cc0f490 con 0x561c1cbc06b0 2023-07-21T22:35:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.276+0000 7f6ad6ff5700 1 -- 172.21.15.107:0/2081729873 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f6ae8004950 con 0x561c1cbc06b0 2023-07-21T22:35:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.276+0000 7f6b0543db80 10 client.25964 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:35:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.276+0000 7f6b0543db80 10 client.25964 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.276+0000 7f6b0543db80 20 client.25964 populate_metadata read hostname 'smithi107' 2023-07-21T22:35:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.276+0000 7f6b0543db80 1 --2- 172.21.15.107:0/2081729873 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561c1cd7b7a0 0x561c1cd7dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.276+0000 7f6b0543db80 4 client.25964 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.276+0000 7f6b0543db80 1 -- 172.21.15.107:0/2081729873 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c1ccaee50 con 0x561c1cd7b7a0 2023-07-21T22:35:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.276+0000 7f6af27fc700 1 --2- 172.21.15.107:0/2081729873 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561c1cd7b7a0 0x561c1cd7dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.276+0000 7f6ad6ff5700 10 client.25964 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.276+0000 7f6af27fc700 1 --2- 172.21.15.107:0/2081729873 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561c1cd7b7a0 0x561c1cd7dbb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06a344677764417 server_cookie=c197605fec16112b in_seq=0 out_seq=0 2023-07-21T22:35:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6ad6ff5700 1 -- 172.21.15.107:0/2081729873 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561c1ccaee50 con 0x561c1cd7b7a0 2023-07-21T22:35:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6ad6ff5700 10 client.25964 handle_command_reply: tid=0 2023-07-21T22:35:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6b0543db80 1 client.25964 shutdown 2023-07-21T22:35:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6b0543db80 1 -- 172.21.15.107:0/2081729873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cbc06b0 msgr2=0x561c1cd6f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6b0543db80 1 --2- 172.21.15.107:0/2081729873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cbc06b0 0x561c1cd6f9d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8011b30 tx=0x7f6ae801cd80 comp rx=0 tx=0).stop 2023-07-21T22:35:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6b0543db80 1 -- 172.21.15.107:0/2081729873 shutdown_connections 2023-07-21T22:35:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6b0543db80 1 --2- 172.21.15.107:0/2081729873 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561c1cd7b7a0 0x561c1cd7dbb0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6b0543db80 1 --2- 172.21.15.107:0/2081729873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd67860 0x561c1cd76f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6ad6ff5700 0 client.25964 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6b0543db80 1 --2- 172.21.15.107:0/2081729873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cbc06b0 0x561c1cd6f9d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6b0543db80 1 --2- 172.21.15.107:0/2081729873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd669a0 0x561c1cd6ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6b0543db80 1 -- 172.21.15.107:0/2081729873 >> 172.21.15.107:0/2081729873 conn(0x561c1cd59ae0 msgr2=0x561c1cd5e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6b0543db80 1 -- 172.21.15.107:0/2081729873 shutdown_connections 2023-07-21T22:35:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6b0543db80 1 -- 172.21.15.107:0/2081729873 wait complete. 2023-07-21T22:35:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.277+0000 7f6b0543db80 20 client.25964 trim_cache size 0 max 16384 2023-07-21T22:35:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 Processor -- start 2023-07-21T22:35:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 -- start start 2023-07-21T22:35:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cbc06b0 0x561c1cd6e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd669a0 0x561c1cd824d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd67860 0x561c1cd6a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1cd6f5a0 con 0x561c1cd67860 2023-07-21T22:35:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1ccaee80 con 0x561c1cd669a0 2023-07-21T22:35:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1cd5f950 con 0x561c1cbc06b0 2023-07-21T22:35:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6af37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd67860 0x561c1cd6a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6af37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd67860 0x561c1cd6a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50750/0 (socket says 172.21.15.107:50750) 2023-07-21T22:35:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6af2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cbc06b0 0x561c1cd6e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6af37fe700 1 -- 172.21.15.107:0/2313077775 learned_addr learned my addr 172.21.15.107:0/2313077775 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6af27fc700 1 --2- 172.21.15.107:0/2313077775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd669a0 0x561c1cd824d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6af37fe700 1 -- 172.21.15.107:0/2313077775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cbc06b0 msgr2=0x561c1cd6e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6af37fe700 1 --2- 172.21.15.107:0/2313077775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cbc06b0 0x561c1cd6e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6af37fe700 1 -- 172.21.15.107:0/2313077775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd669a0 msgr2=0x561c1cd824d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6af37fe700 1 --2- 172.21.15.107:0/2313077775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd669a0 0x561c1cd824d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6af37fe700 1 -- 172.21.15.107:0/2313077775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c1cb0a1f0 con 0x561c1cd67860 2023-07-21T22:35:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6af37fe700 1 --2- 172.21.15.107:0/2313077775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd67860 0x561c1cd6a710 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00d010 tx=0x7f6aec009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ae62acf6e9d388 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6ad9ffb700 1 -- 172.21.15.107:0/2313077775 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00e040 con 0x561c1cd67860 2023-07-21T22:35:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6ad9ffb700 1 -- 172.21.15.107:0/2313077775 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aec007dc0 con 0x561c1cd67860 2023-07-21T22:35:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6ad9ffb700 1 -- 172.21.15.107:0/2313077775 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec027480 con 0x561c1cd67860 2023-07-21T22:35:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 -- 172.21.15.107:0/2313077775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd67860 msgr2=0x561c1cd6a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 --2- 172.21.15.107:0/2313077775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd67860 0x561c1cd6a710 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00d010 tx=0x7f6aec009470 comp rx=0 tx=0).stop 2023-07-21T22:35:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 -- 172.21.15.107:0/2313077775 shutdown_connections 2023-07-21T22:35:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 --2- 172.21.15.107:0/2313077775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cd67860 0x561c1cd6a710 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 --2- 172.21.15.107:0/2313077775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cbc06b0 0x561c1cd6e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 --2- 172.21.15.107:0/2313077775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd669a0 0x561c1cd824d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 -- 172.21.15.107:0/2313077775 >> 172.21.15.107:0/2313077775 conn(0x561c1cd59ae0 msgr2=0x561c1cd5ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 -- 172.21.15.107:0/2313077775 shutdown_connections 2023-07-21T22:35:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 -- 172.21.15.107:0/2313077775 wait complete. 2023-07-21T22:35:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 -- start start 2023-07-21T22:35:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.293+0000 7f6b0543db80 1 Processor -- start 2023-07-21T22:35:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6b0543db80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 0x561c1cd63780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6b0543db80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd669a0 0x561c1cd63cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6b0543db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd67860 0x561c1cd64b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6b0543db80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1cd65090 con 0x561c1cbc06b0 2023-07-21T22:35:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6b0543db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1cd65200 con 0x561c1cd67860 2023-07-21T22:35:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6b0543db80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c1cd653c0 con 0x561c1cd669a0 2023-07-21T22:35:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6b0543db80 10 client.0 _ll_register_callbacks cb 0x7f6ae4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6af2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 0x561c1cd63780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6af2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 0x561c1cd63780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50752/0 (socket says 172.21.15.107:50752) 2023-07-21T22:35:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6af2ffd700 1 -- 172.21.15.107:0/3999840914 learned_addr learned my addr 172.21.15.107:0/3999840914 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6af37fe700 1 --2- 172.21.15.107:0/3999840914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd67860 0x561c1cd64b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6af27fc700 1 --2- 172.21.15.107:0/3999840914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd669a0 0x561c1cd63cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6af2ffd700 1 -- 172.21.15.107:0/3999840914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd669a0 msgr2=0x561c1cd63cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6af2ffd700 1 --2- 172.21.15.107:0/3999840914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd669a0 0x561c1cd63cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6af2ffd700 1 -- 172.21.15.107:0/3999840914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd67860 msgr2=0x561c1cd64b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6af2ffd700 1 --2- 172.21.15.107:0/3999840914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd67860 0x561c1cd64b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6af2ffd700 1 -- 172.21.15.107:0/3999840914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af41077f0 con 0x561c1cbc06b0 2023-07-21T22:35:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6af2ffd700 1 --2- 172.21.15.107:0/3999840914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 0x561c1cd63780 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009410 tx=0x7f6ae801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b03151662aef95d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6ad6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:35:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6ad6ff5700 1 -- 172.21.15.107:0/3999840914 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8005b00 con 0x561c1cbc06b0 2023-07-21T22:35:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6ad6ff5700 1 -- 172.21.15.107:0/3999840914 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6ae801c410 con 0x561c1cbc06b0 2023-07-21T22:35:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6b0543db80 1 -- 172.21.15.107:0/3999840914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ae8021070 con 0x561c1cbc06b0 2023-07-21T22:35:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6ad6ff5700 1 -- 172.21.15.107:0/3999840914 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ae801c5d0 con 0x561c1cbc06b0 2023-07-21T22:35:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6b0543db80 10 client.16428 fetch_fsmap learned FSMap version 39 2023-07-21T22:35:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.294+0000 7f6b0543db80 1 -- 172.21.15.107:0/3999840914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x561c1cae2250 con 0x561c1cbc06b0 2023-07-21T22:35:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.296+0000 7f6ad6ff5700 1 -- 172.21.15.107:0/3999840914 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ae8021070 con 0x561c1cbc06b0 2023-07-21T22:35:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.296+0000 7f6ad6ff5700 1 -- 172.21.15.107:0/3999840914 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f6ae8003980 con 0x561c1cbc06b0 2023-07-21T22:35:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.296+0000 7f6b0543db80 10 client.16428 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:35:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.296+0000 7f6b0543db80 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.296+0000 7f6b0543db80 20 client.16428 populate_metadata read hostname 'smithi107' 2023-07-21T22:35:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.296+0000 7f6b0543db80 1 --2- 172.21.15.107:0/3999840914 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561c1cd7b7a0 0x561c1cdd3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.296+0000 7f6b0543db80 4 client.16428 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:35:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.296+0000 7f6b0543db80 1 -- 172.21.15.107:0/3999840914 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561c1cc70ff0 con 0x561c1cd7b7a0 2023-07-21T22:35:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.296+0000 7f6af27fc700 1 --2- 172.21.15.107:0/3999840914 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561c1cd7b7a0 0x561c1cdd3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.296+0000 7f6ad6ff5700 10 client.16428 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.296+0000 7f6af27fc700 1 --2- 172.21.15.107:0/3999840914 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561c1cd7b7a0 0x561c1cdd3f60 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b158342b66341a2d server_cookie=38330f52df9aa3f0 in_seq=0 out_seq=0 2023-07-21T22:35:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6ad6ff5700 1 -- 172.21.15.107:0/3999840914 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561c1cc70ff0 con 0x561c1cd7b7a0 2023-07-21T22:35:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6ad6ff5700 10 client.16428 handle_command_reply: tid=0 2023-07-21T22:35:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6b0543db80 1 client.16428 shutdown 2023-07-21T22:35:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6b0543db80 1 -- 172.21.15.107:0/3999840914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 msgr2=0x561c1cd63780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6b0543db80 1 --2- 172.21.15.107:0/3999840914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 0x561c1cd63780 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009410 tx=0x7f6ae801c9e0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6b0543db80 1 -- 172.21.15.107:0/3999840914 shutdown_connections 2023-07-21T22:35:30.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6b0543db80 1 --2- 172.21.15.107:0/3999840914 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561c1cd7b7a0 0x561c1cdd3f60 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6b0543db80 1 --2- 172.21.15.107:0/3999840914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561c1cd67860 0x561c1cd64b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6ad6ff5700 0 client.16428 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6b0543db80 1 --2- 172.21.15.107:0/3999840914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c1cbc06b0 0x561c1cd63780 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6b0543db80 1 --2- 172.21.15.107:0/3999840914 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561c1cd669a0 0x561c1cd63cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6b0543db80 1 -- 172.21.15.107:0/3999840914 >> 172.21.15.107:0/3999840914 conn(0x561c1cd59ae0 msgr2=0x561c1cd824d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6b0543db80 1 -- 172.21.15.107:0/3999840914 shutdown_connections 2023-07-21T22:35:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6b0543db80 1 -- 172.21.15.107:0/3999840914 wait complete. 2023-07-21T22:35:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.410+0000 7f6b0543db80 20 client.16428 trim_cache size 0 max 16384 2023-07-21T22:35:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:35:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:35:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:35:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:35:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.411+0000 7f6af92b8700 1 -- 172.21.15.107:0/1555162712 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6adc0a1150 msgr2=0x7f6adc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.411+0000 7f6af92b8700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6adc0a1150 0x7f6adc0a3600 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6af40ffad0 tx=0x7f6ae8009260 comp rx=0 tx=0).stop 2023-07-21T22:35:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.411+0000 7f6af92b8700 1 -- 172.21.15.107:0/1555162712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6af40ffc70 msgr2=0x7f6af4194e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.411+0000 7f6af92b8700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6af40ffc70 0x7f6af4194e50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400e9a0 tx=0x7f6ae400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.411+0000 7f6af92b8700 1 -- 172.21.15.107:0/1555162712 shutdown_connections 2023-07-21T22:35:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.411+0000 7f6af92b8700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6af4100600 0x7f6af4195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.411+0000 7f6af92b8700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6adc0a1150 0x7f6adc0a3600 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.411+0000 7f6af92b8700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af40fea80 0x7f6af4194910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.411+0000 7f6af92b8700 1 --2- 172.21.15.107:0/1555162712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6af40ffc70 0x7f6af4194e50 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.411+0000 7f6af92b8700 1 -- 172.21.15.107:0/1555162712 >> 172.21.15.107:0/1555162712 conn(0x7f6af40fa0c0 msgr2=0x7f6af4103100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.411+0000 7f6af92b8700 1 -- 172.21.15.107:0/1555162712 shutdown_connections 2023-07-21T22:35:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:30.411+0000 7f6af92b8700 1 -- 172.21.15.107:0/1555162712 wait complete. 2023-07-21T22:35:32.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:31 smithi190 ceph-mon[98334]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-07-21T22:35:32.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:31 smithi132 ceph-mon[97863]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-07-21T22:35:32.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:31 smithi107 ceph-mon[96543]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 1.4 MiB/s wr, 209 op/s 2023-07-21T22:35:34.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:33 smithi190 ceph-mon[98334]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 4.8 MiB/s wr, 363 op/s 2023-07-21T22:35:34.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:33 smithi132 ceph-mon[97863]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 4.8 MiB/s wr, 363 op/s 2023-07-21T22:35:34.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:33 smithi107 ceph-mon[96543]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 4.8 MiB/s wr, 363 op/s 2023-07-21T22:35:36.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:35 smithi190 ceph-mon[98334]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 7.5 MiB/s wr, 401 op/s 2023-07-21T22:35:36.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:35 smithi132 ceph-mon[97863]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 7.5 MiB/s wr, 401 op/s 2023-07-21T22:35:36.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:35 smithi107 ceph-mon[96543]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 7.5 MiB/s wr, 401 op/s 2023-07-21T22:35:37.539 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:35:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.639+0000 7fa56f4fe700 1 -- 172.21.15.107:0/77306455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5680fe9e0 msgr2=0x7fa5680fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.639+0000 7fa56f4fe700 1 --2- 172.21.15.107:0/77306455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5680fe9e0 0x7fa5680fedf0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa564009990 tx=0x7fa564009ca0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.639+0000 7fa56f4fe700 1 -- 172.21.15.107:0/77306455 shutdown_connections 2023-07-21T22:35:37.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.639+0000 7fa56f4fe700 1 --2- 172.21.15.107:0/77306455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa568100560 0x7fa568106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.639+0000 7fa56f4fe700 1 --2- 172.21.15.107:0/77306455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5680fe9e0 0x7fa5680fedf0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.639+0000 7fa56f4fe700 1 --2- 172.21.15.107:0/77306455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5680ffbd0 0x7fa568100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.639+0000 7fa56f4fe700 1 -- 172.21.15.107:0/77306455 >> 172.21.15.107:0/77306455 conn(0x7fa5680fa000 msgr2=0x7fa5680fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:37.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.639+0000 7fa56f4fe700 1 -- 172.21.15.107:0/77306455 shutdown_connections 2023-07-21T22:35:37.688 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.639+0000 7fa56f4fe700 1 -- 172.21.15.107:0/77306455 wait complete. 2023-07-21T22:35:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56f4fe700 1 Processor -- start 2023-07-21T22:35:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56f4fe700 1 -- start start 2023-07-21T22:35:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56f4fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5680fe9e0 0x7fa568194750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56f4fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5680ffbd0 0x7fa568194c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56f4fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa568100560 0x7fa56819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56f4fe700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56819c2d0 con 0x7fa568100560 2023-07-21T22:35:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56f4fe700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56819c410 con 0x7fa5680fe9e0 2023-07-21T22:35:37.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56f4fe700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56819c5a0 con 0x7fa5680ffbd0 2023-07-21T22:35:37.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56d29a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5680fe9e0 0x7fa568194750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56d29a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5680fe9e0 0x7fa568194750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41594/0 (socket says 172.21.15.107:41594) 2023-07-21T22:35:37.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56d29a700 1 -- 172.21.15.107:0/2753025967 learned_addr learned my addr 172.21.15.107:0/2753025967 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:37.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56da9b700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa568100560 0x7fa56819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56ca99700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5680ffbd0 0x7fa568194c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56d29a700 1 -- 172.21.15.107:0/2753025967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5680ffbd0 msgr2=0x7fa568194c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:37.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56d29a700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5680ffbd0 0x7fa568194c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56d29a700 1 -- 172.21.15.107:0/2753025967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa568100560 msgr2=0x7fa56819bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:37.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56d29a700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa568100560 0x7fa56819bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56d29a700 1 -- 172.21.15.107:0/2753025967 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5640095f0 con 0x7fa5680fe9e0 2023-07-21T22:35:37.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.640+0000 7fa56da9b700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa568100560 0x7fa56819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:37.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.641+0000 7fa56d29a700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5680fe9e0 0x7fa568194750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa56400b620 tx=0x7fa564004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:37.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.641+0000 7fa55a7fc700 1 -- 172.21.15.107:0/2753025967 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56401d070 con 0x7fa5680fe9e0 2023-07-21T22:35:37.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.641+0000 7fa56f4fe700 1 -- 172.21.15.107:0/2753025967 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56819c820 con 0x7fa5680fe9e0 2023-07-21T22:35:37.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.641+0000 7fa56f4fe700 1 -- 172.21.15.107:0/2753025967 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56819cca0 con 0x7fa5680fe9e0 2023-07-21T22:35:37.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.641+0000 7fa55a7fc700 1 -- 172.21.15.107:0/2753025967 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa564005030 con 0x7fa5680fe9e0 2023-07-21T22:35:37.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.641+0000 7fa55a7fc700 1 -- 172.21.15.107:0/2753025967 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564004ba0 con 0x7fa5680fe9e0 2023-07-21T22:35:37.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.643+0000 7fa55a7fc700 1 -- 172.21.15.107:0/2753025967 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa564004d00 con 0x7fa5680fe9e0 2023-07-21T22:35:37.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.644+0000 7fa55a7fc700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa5540a1150 0x7fa5540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.644+0000 7fa55a7fc700 1 -- 172.21.15.107:0/2753025967 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa5640cd740 con 0x7fa5680fe9e0 2023-07-21T22:35:37.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.644+0000 7fa56ca99700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa5540a1150 0x7fa5540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.644+0000 7fa56ca99700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa5540a1150 0x7fa5540a3600 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009c60 tx=0x7fa55c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:37.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.644+0000 7fa57b683b80 1 Processor -- start 2023-07-21T22:35:37.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa57b683b80 1 -- start start 2023-07-21T22:35:37.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa57b683b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d1a60 0x55579d0d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa57b683b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa57b683b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa57b683b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579cfd3cb0 con 0x55579d0d23f0 2023-07-21T22:35:37.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa57b683b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579d026a40 con 0x55579d0d1a60 2023-07-21T22:35:37.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa57b683b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579d01fed0 con 0x55579d0d0b90 2023-07-21T22:35:37.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa56ca99700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa56ca99700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0d05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50776/0 (socket says 172.21.15.107:50776) 2023-07-21T22:35:37.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa56ca99700 1 -- 172.21.15.107:0/1222894410 learned_addr learned my addr 172.21.15.107:0/1222894410 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:37.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa56d29a700 1 --2- 172.21.15.107:0/1222894410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d1a60 0x55579d0d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa56da9b700 1 --2- 172.21.15.107:0/1222894410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa56ca99700 1 -- 172.21.15.107:0/1222894410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 msgr2=0x55579d0d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:37.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa56ca99700 1 --2- 172.21.15.107:0/1222894410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa56ca99700 1 -- 172.21.15.107:0/1222894410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d1a60 msgr2=0x55579d0d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:37.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa56ca99700 1 --2- 172.21.15.107:0/1222894410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d1a60 0x55579d0d1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa56ca99700 1 -- 172.21.15.107:0/1222894410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55579ce4dde0 con 0x55579d0d23f0 2023-07-21T22:35:37.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa56ca99700 1 --2- 172.21.15.107:0/1222894410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0d05b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00e4e0 tx=0x7fa55c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ed2a2b3d24354f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:37.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa5537fe700 1 -- 172.21.15.107:0/1222894410 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00ee00 con 0x55579d0d23f0 2023-07-21T22:35:37.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa5537fe700 1 -- 172.21.15.107:0/1222894410 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa55c021070 con 0x55579d0d23f0 2023-07-21T22:35:37.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.645+0000 7fa5537fe700 1 -- 172.21.15.107:0/1222894410 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c01d810 con 0x55579d0d23f0 2023-07-21T22:35:37.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.646+0000 7fa57b683b80 1 -- 172.21.15.107:0/1222894410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 msgr2=0x55579d0d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:37.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.646+0000 7fa57b683b80 1 --2- 172.21.15.107:0/1222894410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0d05b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00e4e0 tx=0x7fa55c014000 comp rx=0 tx=0).stop 2023-07-21T22:35:37.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.646+0000 7fa57b683b80 1 -- 172.21.15.107:0/1222894410 shutdown_connections 2023-07-21T22:35:37.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.646+0000 7fa57b683b80 1 --2- 172.21.15.107:0/1222894410 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.646+0000 7fa57b683b80 1 --2- 172.21.15.107:0/1222894410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d1a60 0x55579d0d1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.646+0000 7fa57b683b80 1 --2- 172.21.15.107:0/1222894410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0d05b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.646+0000 7fa57b683b80 1 -- 172.21.15.107:0/1222894410 >> 172.21.15.107:0/1222894410 conn(0x55579d0c3ef0 msgr2=0x55579d0c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:37.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.646+0000 7fa57b683b80 1 -- 172.21.15.107:0/1222894410 shutdown_connections 2023-07-21T22:35:37.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.646+0000 7fa57b683b80 1 -- 172.21.15.107:0/1222894410 wait complete. 2023-07-21T22:35:37.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.646+0000 7fa57b683b80 1 -- start start 2023-07-21T22:35:37.701 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.647+0000 7fa57b683b80 1 Processor -- start 2023-07-21T22:35:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.647+0000 7fa57b683b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0da2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.647+0000 7fa57b683b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d1a60 0x55579d0da7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.647+0000 7fa57b683b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.647+0000 7fa57b683b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579cfbfac0 con 0x55579d0d23f0 2023-07-21T22:35:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.647+0000 7fa57b683b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579cfbf7c0 con 0x55579d0d1a60 2023-07-21T22:35:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.647+0000 7fa57b683b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579cea3920 con 0x55579d0d0b90 2023-07-21T22:35:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.647+0000 7fa57b683b80 10 client.0 _ll_register_callbacks cb 0x55579cfbd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:37.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.647+0000 7fa56da9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.647+0000 7fa56d29a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0da2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.647+0000 7fa56ca99700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d1a60 0x55579d0da7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.647+0000 7fa56da9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0e1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50784/0 (socket says 172.21.15.107:50784) 2023-07-21T22:35:37.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.647+0000 7fa56da9b700 1 -- 172.21.15.107:0/2887238585 learned_addr learned my addr 172.21.15.107:0/2887238585 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:37.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa56d29a700 1 -- 172.21.15.107:0/2887238585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d1a60 msgr2=0x55579d0da7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:37.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa56d29a700 1 --2- 172.21.15.107:0/2887238585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d1a60 0x55579d0da7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa56d29a700 1 -- 172.21.15.107:0/2887238585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 msgr2=0x55579d0e1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:37.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa56d29a700 1 --2- 172.21.15.107:0/2887238585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0e1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa56d29a700 1 -- 172.21.15.107:0/2887238585 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c017070 con 0x55579d0d0b90 2023-07-21T22:35:37.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa56da9b700 1 --2- 172.21.15.107:0/2887238585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0e1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:37.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa56ca99700 1 --2- 172.21.15.107:0/2887238585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d1a60 0x55579d0da7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:37.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa56d29a700 1 --2- 172.21.15.107:0/2887238585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0da2b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5640cb270 tx=0x7fa564000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ec15e57972e17b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:37.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa550ff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:35:37.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa550ff9700 1 -- 172.21.15.107:0/2887238585 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640a2cf0 con 0x55579d0d0b90 2023-07-21T22:35:37.704 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa57b683b80 1 -- 172.21.15.107:0/2887238585 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55579cf794f0 con 0x55579d0d0b90 2023-07-21T22:35:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa550ff9700 1 -- 172.21.15.107:0/2887238585 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa5640a2f80 con 0x55579d0d0b90 2023-07-21T22:35:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa550ff9700 1 -- 172.21.15.107:0/2887238585 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa56400fa90 con 0x55579d0d0b90 2023-07-21T22:35:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa57b683b80 10 client.26044 fetch_fsmap learned FSMap version 39 2023-07-21T22:35:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa57b683b80 1 -- 172.21.15.107:0/2887238585 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55579ce36c60 con 0x55579d0d0b90 2023-07-21T22:35:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.648+0000 7fa550ff9700 1 -- 172.21.15.107:0/2887238585 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55579cf794f0 con 0x55579d0d0b90 2023-07-21T22:35:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.649+0000 7fa550ff9700 1 -- 172.21.15.107:0/2887238585 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fa564096a40 con 0x55579d0d0b90 2023-07-21T22:35:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.649+0000 7fa57b683b80 10 client.26044 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:35:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.649+0000 7fa57b683b80 10 client.26044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.649+0000 7fa57b683b80 20 client.26044 populate_metadata read hostname 'smithi107' 2023-07-21T22:35:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.649+0000 7fa57b683b80 1 --2- 172.21.15.107:0/2887238585 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55579d0e5a10 0x55579d0e7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.649+0000 7fa57b683b80 4 client.26044 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.649+0000 7fa57b683b80 1 -- 172.21.15.107:0/2887238585 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55579d018eb0 con 0x55579d0e5a10 2023-07-21T22:35:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.649+0000 7fa56ca99700 1 --2- 172.21.15.107:0/2887238585 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55579d0e5a10 0x55579d0e7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.649+0000 7fa56ca99700 1 --2- 172.21.15.107:0/2887238585 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55579d0e5a10 0x55579d0e7df0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ad9d8f1bd116d1 server_cookie=51175e946628ae34 in_seq=0 out_seq=0 2023-07-21T22:35:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.649+0000 7fa550ff9700 10 client.26044 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:37.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa550ff9700 1 -- 172.21.15.107:0/2887238585 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55579d018eb0 con 0x55579d0e5a10 2023-07-21T22:35:37.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa550ff9700 10 client.26044 handle_command_reply: tid=0 2023-07-21T22:35:37.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa57b683b80 1 client.26044 shutdown 2023-07-21T22:35:37.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa57b683b80 1 -- 172.21.15.107:0/2887238585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 msgr2=0x55579d0da2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:37.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa57b683b80 1 --2- 172.21.15.107:0/2887238585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0da2b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5640cb270 tx=0x7fa564000bc0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa57b683b80 1 -- 172.21.15.107:0/2887238585 shutdown_connections 2023-07-21T22:35:37.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa57b683b80 1 --2- 172.21.15.107:0/2887238585 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55579d0e5a10 0x55579d0e7df0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa57b683b80 1 --2- 172.21.15.107:0/2887238585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0e1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa57b683b80 1 --2- 172.21.15.107:0/2887238585 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0da2b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa550ff9700 0 client.26044 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:37.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa57b683b80 1 --2- 172.21.15.107:0/2887238585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d1a60 0x55579d0da7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa57b683b80 1 -- 172.21.15.107:0/2887238585 >> 172.21.15.107:0/2887238585 conn(0x55579d0c3ef0 msgr2=0x55579d0c8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:37.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa57b683b80 1 -- 172.21.15.107:0/2887238585 shutdown_connections 2023-07-21T22:35:37.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa57b683b80 1 -- 172.21.15.107:0/2887238585 wait complete. 2023-07-21T22:35:37.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.650+0000 7fa57b683b80 20 client.26044 trim_cache size 0 max 16384 2023-07-21T22:35:37.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa57b683b80 1 Processor -- start 2023-07-21T22:35:37.715 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa57b683b80 1 -- start start 2023-07-21T22:35:37.715 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa57b683b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.715 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa57b683b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d1a60 0x55579d0e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.715 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa57b683b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d23f0 0x55579d0d4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.715 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa57b683b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579d0d9e80 con 0x55579d0d1a60 2023-07-21T22:35:37.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa57b683b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579d0ca470 con 0x55579d0d23f0 2023-07-21T22:35:37.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa57b683b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579d0d9790 con 0x55579d0d0b90 2023-07-21T22:35:37.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa56ca99700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d1a60 0x55579d0e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa56ca99700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d1a60 0x55579d0e17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50790/0 (socket says 172.21.15.107:50790) 2023-07-21T22:35:37.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa56ca99700 1 -- 172.21.15.107:0/3638658331 learned_addr learned my addr 172.21.15.107:0/3638658331 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:37.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa56d29a700 1 --2- 172.21.15.107:0/3638658331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa56da9b700 1 --2- 172.21.15.107:0/3638658331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d23f0 0x55579d0d4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa56ca99700 1 -- 172.21.15.107:0/3638658331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 msgr2=0x55579d0d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa56ca99700 1 --2- 172.21.15.107:0/3638658331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa56ca99700 1 -- 172.21.15.107:0/3638658331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d23f0 msgr2=0x55579d0d4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa56ca99700 1 --2- 172.21.15.107:0/3638658331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d23f0 0x55579d0d4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa56ca99700 1 -- 172.21.15.107:0/3638658331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55579ce74250 con 0x55579d0d1a60 2023-07-21T22:35:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa56d29a700 1 --2- 172.21.15.107:0/3638658331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:35:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa56ca99700 1 --2- 172.21.15.107:0/3638658331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d1a60 0x55579d0e17f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa55c005950 tx=0x7fa55c011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fab64041bbb0201 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa553fff700 1 -- 172.21.15.107:0/3638658331 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00ec90 con 0x55579d0d1a60 2023-07-21T22:35:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa553fff700 1 -- 172.21.15.107:0/3638658331 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa55c0335e0 con 0x55579d0d1a60 2023-07-21T22:35:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.667+0000 7fa553fff700 1 -- 172.21.15.107:0/3638658331 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0038e0 con 0x55579d0d1a60 2023-07-21T22:35:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.668+0000 7fa57b683b80 1 -- 172.21.15.107:0/3638658331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d1a60 msgr2=0x55579d0e17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.668+0000 7fa57b683b80 1 --2- 172.21.15.107:0/3638658331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d1a60 0x55579d0e17f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa55c005950 tx=0x7fa55c011be0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.668+0000 7fa57b683b80 1 -- 172.21.15.107:0/3638658331 shutdown_connections 2023-07-21T22:35:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.668+0000 7fa57b683b80 1 --2- 172.21.15.107:0/3638658331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d23f0 0x55579d0d4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.668+0000 7fa57b683b80 1 --2- 172.21.15.107:0/3638658331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d0b90 0x55579d0d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.668+0000 7fa57b683b80 1 --2- 172.21.15.107:0/3638658331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d1a60 0x55579d0e17f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.668+0000 7fa57b683b80 1 -- 172.21.15.107:0/3638658331 >> 172.21.15.107:0/3638658331 conn(0x55579d0c3ef0 msgr2=0x55579d0d9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.668+0000 7fa57b683b80 1 -- 172.21.15.107:0/3638658331 shutdown_connections 2023-07-21T22:35:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.668+0000 7fa57b683b80 1 -- 172.21.15.107:0/3638658331 wait complete. 2023-07-21T22:35:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.668+0000 7fa57b683b80 1 -- start start 2023-07-21T22:35:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.668+0000 7fa57b683b80 1 Processor -- start 2023-07-21T22:35:37.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa57b683b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d0b90 0x55579d0dd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa57b683b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d1a60 0x55579d0ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa57b683b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa57b683b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579d0e10e0 con 0x55579d0d23f0 2023-07-21T22:35:37.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa57b683b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579d0e1250 con 0x55579d0d0b90 2023-07-21T22:35:37.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa57b683b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579d0e1410 con 0x55579d0d1a60 2023-07-21T22:35:37.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa57b683b80 10 client.0 _ll_register_callbacks cb 0x7fa56409a7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:37.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56da9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56d29a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d0b90 0x55579d0dd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56ca99700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d1a60 0x55579d0ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56da9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0e0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50798/0 (socket says 172.21.15.107:50798) 2023-07-21T22:35:37.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56d29a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d0b90 0x55579d0dd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41622/0 (socket says 172.21.15.107:41622) 2023-07-21T22:35:37.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56da9b700 1 -- 172.21.15.107:0/71306775 learned_addr learned my addr 172.21.15.107:0/71306775 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:37.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56da9b700 1 -- 172.21.15.107:0/71306775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d1a60 msgr2=0x55579d0ddb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:37.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56da9b700 1 --2- 172.21.15.107:0/71306775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d1a60 0x55579d0ddb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56da9b700 1 -- 172.21.15.107:0/71306775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d0b90 msgr2=0x55579d0dd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:37.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56da9b700 1 --2- 172.21.15.107:0/71306775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d0b90 0x55579d0dd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56da9b700 1 -- 172.21.15.107:0/71306775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c02a040 con 0x55579d0d23f0 2023-07-21T22:35:37.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56ca99700 1 --2- 172.21.15.107:0/71306775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d1a60 0x55579d0ddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:37.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56d29a700 1 --2- 172.21.15.107:0/71306775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d0b90 0x55579d0dd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:37.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa56da9b700 1 --2- 172.21.15.107:0/71306775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0e0ba0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa560004970 tx=0x7fa5600077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea4a7da7d80b91d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:37.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.669+0000 7fa550ff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:35:37.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa550ff9700 1 -- 172.21.15.107:0/71306775 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56001c070 con 0x55579d0d23f0 2023-07-21T22:35:37.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa550ff9700 1 -- 172.21.15.107:0/71306775 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa560007dd0 con 0x55579d0d23f0 2023-07-21T22:35:37.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa57b683b80 1 -- 172.21.15.107:0/71306775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa55c021070 con 0x55579d0d23f0 2023-07-21T22:35:37.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa550ff9700 1 -- 172.21.15.107:0/71306775 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa560004bb0 con 0x55579d0d23f0 2023-07-21T22:35:37.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa57b683b80 10 client.16446 fetch_fsmap learned FSMap version 39 2023-07-21T22:35:37.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa57b683b80 1 -- 172.21.15.107:0/71306775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55579ce4c2b0 con 0x55579d0d23f0 2023-07-21T22:35:37.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa550ff9700 1 -- 172.21.15.107:0/71306775 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa55c021070 con 0x55579d0d23f0 2023-07-21T22:35:37.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa550ff9700 1 -- 172.21.15.107:0/71306775 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fa560020410 con 0x55579d0d23f0 2023-07-21T22:35:37.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa57b683b80 10 client.16446 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:35:37.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa57b683b80 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:37.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa57b683b80 20 client.16446 populate_metadata read hostname 'smithi107' 2023-07-21T22:35:37.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa57b683b80 1 --2- 172.21.15.107:0/71306775 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55579d0e5a10 0x55579d13ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:37.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa57b683b80 4 client.16446 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:35:37.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa57b683b80 1 -- 172.21.15.107:0/71306775 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55579cfdb050 con 0x55579d0e5a10 2023-07-21T22:35:37.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.670+0000 7fa56d29a700 1 --2- 172.21.15.107:0/71306775 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55579d0e5a10 0x55579d13ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:37.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa550ff9700 10 client.16446 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:37.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa56d29a700 1 --2- 172.21.15.107:0/71306775 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55579d0e5a10 0x55579d13ca30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f42e07812328fae4 server_cookie=fb37af661cddfaf4 in_seq=0 out_seq=0 2023-07-21T22:35:37.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa550ff9700 1 -- 172.21.15.107:0/71306775 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55579cfdb050 con 0x55579d0e5a10 2023-07-21T22:35:37.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa550ff9700 10 client.16446 handle_command_reply: tid=0 2023-07-21T22:35:37.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa57b683b80 1 client.16446 shutdown 2023-07-21T22:35:37.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa57b683b80 1 -- 172.21.15.107:0/71306775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 msgr2=0x55579d0e0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:37.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa57b683b80 1 --2- 172.21.15.107:0/71306775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0e0ba0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa560004970 tx=0x7fa5600077e0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa57b683b80 1 -- 172.21.15.107:0/71306775 shutdown_connections 2023-07-21T22:35:37.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa57b683b80 1 --2- 172.21.15.107:0/71306775 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55579d0e5a10 0x55579d13ca30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa57b683b80 1 --2- 172.21.15.107:0/71306775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55579d0d23f0 0x55579d0e0ba0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa550ff9700 0 client.16446 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:37.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa57b683b80 1 --2- 172.21.15.107:0/71306775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55579d0d0b90 0x55579d0dd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa57b683b80 1 --2- 172.21.15.107:0/71306775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55579d0d1a60 0x55579d0ddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa57b683b80 1 -- 172.21.15.107:0/71306775 >> 172.21.15.107:0/71306775 conn(0x55579d0c3ef0 msgr2=0x55579d0d4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:37.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa57b683b80 1 -- 172.21.15.107:0/71306775 shutdown_connections 2023-07-21T22:35:37.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa57b683b80 1 -- 172.21.15.107:0/71306775 wait complete. 2023-07-21T22:35:37.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.671+0000 7fa57b683b80 20 client.16446 trim_cache size 0 max 16384 2023-07-21T22:35:37.729 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:35:37.730 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:35:37.730 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:35:37.730 INFO:teuthology.orchestra.run.smithi107.stdout: "d037412e-e4e0-4d2c-8d6e-c7193371fe0f": { 2023-07-21T22:35:37.730 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:35:37.730 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d037412e-e4e0-4d2c-8d6e-c7193371fe0f", 2023-07-21T22:35:37.730 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:35:37.730 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:35:37.730 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:35:37.731 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:35:37.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.672+0000 7fa56f4fe700 1 -- 172.21.15.107:0/2753025967 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa5540a1150 msgr2=0x7fa5540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:37.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.672+0000 7fa56f4fe700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa5540a1150 0x7fa5540a3600 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009c60 tx=0x7fa55c009260 comp rx=0 tx=0).stop 2023-07-21T22:35:37.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.672+0000 7fa56f4fe700 1 -- 172.21.15.107:0/2753025967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5680fe9e0 msgr2=0x7fa568194750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:37.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.672+0000 7fa56f4fe700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5680fe9e0 0x7fa568194750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa56400b620 tx=0x7fa564004730 comp rx=0 tx=0).stop 2023-07-21T22:35:37.732 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.672+0000 7fa56f4fe700 1 -- 172.21.15.107:0/2753025967 shutdown_connections 2023-07-21T22:35:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.672+0000 7fa56f4fe700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa568100560 0x7fa56819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.672+0000 7fa56f4fe700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5680fe9e0 0x7fa568194750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.672+0000 7fa56f4fe700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fa5540a1150 0x7fa5540a3600 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.672+0000 7fa56f4fe700 1 --2- 172.21.15.107:0/2753025967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5680ffbd0 0x7fa568194c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.672+0000 7fa56f4fe700 1 -- 172.21.15.107:0/2753025967 >> 172.21.15.107:0/2753025967 conn(0x7fa5680fa000 msgr2=0x7fa5680fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.672+0000 7fa56f4fe700 1 -- 172.21.15.107:0/2753025967 shutdown_connections 2023-07-21T22:35:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:35:37.672+0000 7fa56f4fe700 1 -- 172.21.15.107:0/2753025967 wait complete. 2023-07-21T22:35:37.734 INFO:tasks.cephfs.filesystem:scrub status for tag:d037412e-e4e0-4d2c-8d6e-c7193371fe0f - {'path': '/', 'tag': 'd037412e-e4e0-4d2c-8d6e-c7193371fe0f', 'options': 'recursive,force'} 2023-07-21T22:35:37.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:37 smithi107 ceph-mon[96543]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 7.5 MiB/s wr, 410 op/s 2023-07-21T22:35:38.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:37 smithi190 ceph-mon[98334]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 7.5 MiB/s wr, 410 op/s 2023-07-21T22:35:38.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:37 smithi132 ceph-mon[97863]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 7.5 MiB/s wr, 410 op/s 2023-07-21T22:35:40.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:39 smithi190 ceph-mon[98334]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 7.5 MiB/s wr, 416 op/s 2023-07-21T22:35:40.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:39 smithi132 ceph-mon[97863]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 7.5 MiB/s wr, 416 op/s 2023-07-21T22:35:40.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:39 smithi107 ceph-mon[96543]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 7.5 MiB/s wr, 416 op/s 2023-07-21T22:35:42.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:41 smithi190 ceph-mon[98334]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 7.5 MiB/s wr, 407 op/s 2023-07-21T22:35:42.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:41 smithi132 ceph-mon[97863]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 7.5 MiB/s wr, 407 op/s 2023-07-21T22:35:42.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:41 smithi107 ceph-mon[96543]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 7.5 MiB/s wr, 407 op/s 2023-07-21T22:35:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.528+0000 7f4577fc3700 1 -- 172.21.15.107:0/2918596657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45700ffd00 msgr2=0x7f45701065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.528+0000 7f4577fc3700 1 --2- 172.21.15.107:0/2918596657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45700ffd00 0x7f45701065c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f456800c210 tx=0x7f456800c520 comp rx=0 tx=0).stop 2023-07-21T22:35:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.528+0000 7f4577fc3700 1 -- 172.21.15.107:0/2918596657 shutdown_connections 2023-07-21T22:35:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.528+0000 7f4577fc3700 1 --2- 172.21.15.107:0/2918596657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45700ffd00 0x7f45701065c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.528+0000 7f4577fc3700 1 --2- 172.21.15.107:0/2918596657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45700fe180 0x7f45700fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.528+0000 7f4577fc3700 1 --2- 172.21.15.107:0/2918596657 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45700ff370 0x7f45700ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.528+0000 7f4577fc3700 1 -- 172.21.15.107:0/2918596657 >> 172.21.15.107:0/2918596657 conn(0x7f45700f9780 msgr2=0x7f45700fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.528+0000 7f4577fc3700 1 -- 172.21.15.107:0/2918596657 shutdown_connections 2023-07-21T22:35:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.528+0000 7f4577fc3700 1 -- 172.21.15.107:0/2918596657 wait complete. 2023-07-21T22:35:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f4577fc3700 1 Processor -- start 2023-07-21T22:35:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f4577fc3700 1 -- start start 2023-07-21T22:35:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f4577fc3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45700fe180 0x7f4570100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f4577fc3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45700ff370 0x7f4570100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f4577fc3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45700ffd00 0x7f45701011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f4577fc3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457019f9b0 con 0x7f45700ffd00 2023-07-21T22:35:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f4577fc3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457019faf0 con 0x7f45700fe180 2023-07-21T22:35:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f4577fc3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457019fc30 con 0x7f45700ff370 2023-07-21T22:35:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f457555e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45700ff370 0x7f4570100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f4576560700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45700ffd00 0x7f45701011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f457555e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45700ff370 0x7f4570100c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39368/0 (socket says 172.21.15.107:39368) 2023-07-21T22:35:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f457555e700 1 -- 172.21.15.107:0/2272409942 learned_addr learned my addr 172.21.15.107:0/2272409942 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f4576560700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45700ffd00 0x7f45701011d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50812/0 (socket says 172.21.15.107:50812) 2023-07-21T22:35:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f4575d5f700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45700fe180 0x7f4570100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f457555e700 1 -- 172.21.15.107:0/2272409942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45700fe180 msgr2=0x7f4570100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f457555e700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45700fe180 0x7f4570100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f457555e700 1 -- 172.21.15.107:0/2272409942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45700ffd00 msgr2=0x7f45701011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f457555e700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45700ffd00 0x7f45701011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f457555e700 1 -- 172.21.15.107:0/2272409942 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456800ae10 con 0x7f45700ff370 2023-07-21T22:35:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f4576560700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45700ffd00 0x7f45701011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.529+0000 7f457555e700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45700ff370 0x7f4570100c90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f456c00e9a0 tx=0x7f456c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.530+0000 7f4566ffd700 1 -- 172.21.15.107:0/2272409942 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c009790 con 0x7f45700ff370 2023-07-21T22:35:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.530+0000 7f4577fc3700 1 -- 172.21.15.107:0/2272409942 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457019fec0 con 0x7f45700ff370 2023-07-21T22:35:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.530+0000 7f4566ffd700 1 -- 172.21.15.107:0/2272409942 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f456c010430 con 0x7f45700ff370 2023-07-21T22:35:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.530+0000 7f4566ffd700 1 -- 172.21.15.107:0/2272409942 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c018680 con 0x7f45700ff370 2023-07-21T22:35:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.530+0000 7f4577fc3700 1 -- 172.21.15.107:0/2272409942 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45701a0390 con 0x7f45700ff370 2023-07-21T22:35:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.532+0000 7f4566ffd700 1 -- 172.21.15.107:0/2272409942 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f456c0105a0 con 0x7f45700ff370 2023-07-21T22:35:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.533+0000 7f4566ffd700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f455c0a1150 0x7f455c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.533+0000 7f4575d5f700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f455c0a1150 0x7f455c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.533+0000 7f4566ffd700 1 -- 172.21.15.107:0/2272409942 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f456c014070 con 0x7f45700ff370 2023-07-21T22:35:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.533+0000 7f4575d5f700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f455c0a1150 0x7f455c0a3600 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4570102060 tx=0x7f4560009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.533+0000 7f4584148b80 1 Processor -- start 2023-07-21T22:35:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f4584148b80 1 -- start start 2023-07-21T22:35:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f4584148b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e5c06b0 0x55c94e7642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f4584148b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 0x55c94e765040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f4584148b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e765580 0x55c94e7659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f4584148b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94e669c50 con 0x55c94e764830 2023-07-21T22:35:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f4584148b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94e6bc9e0 con 0x55c94e765580 2023-07-21T22:35:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f4584148b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94e6b5e70 con 0x55c94e5c06b0 2023-07-21T22:35:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f457555e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 0x55c94e765040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f4575d5f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e5c06b0 0x55c94e7642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f4576560700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e765580 0x55c94e7659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f457555e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 0x55c94e765040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50818/0 (socket says 172.21.15.107:50818) 2023-07-21T22:35:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f4575d5f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e5c06b0 0x55c94e7642f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39374/0 (socket says 172.21.15.107:39374) 2023-07-21T22:35:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f457555e700 1 -- 172.21.15.107:0/87366230 learned_addr learned my addr 172.21.15.107:0/87366230 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f457555e700 1 -- 172.21.15.107:0/87366230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e5c06b0 msgr2=0x55c94e7642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f457555e700 1 --2- 172.21.15.107:0/87366230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e5c06b0 0x55c94e7642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f457555e700 1 -- 172.21.15.107:0/87366230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e765580 msgr2=0x55c94e7659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f457555e700 1 --2- 172.21.15.107:0/87366230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e765580 0x55c94e7659f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f457555e700 1 -- 172.21.15.107:0/87366230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c94e4e3d80 con 0x55c94e764830 2023-07-21T22:35:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f4575d5f700 1 --2- 172.21.15.107:0/87366230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e5c06b0 0x55c94e7642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.534+0000 7f457555e700 1 --2- 172.21.15.107:0/87366230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 0x55c94e765040 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f456c099e60 tx=0x7f456c099930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fbc1b7faebc90 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.535+0000 7f455bfff700 1 -- 172.21.15.107:0/87366230 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c0a2320 con 0x55c94e764830 2023-07-21T22:35:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.535+0000 7f455bfff700 1 -- 172.21.15.107:0/87366230 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f456c098270 con 0x55c94e764830 2023-07-21T22:35:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.535+0000 7f455bfff700 1 -- 172.21.15.107:0/87366230 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c0a20f0 con 0x55c94e764830 2023-07-21T22:35:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.535+0000 7f4584148b80 1 -- 172.21.15.107:0/87366230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 msgr2=0x55c94e765040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.535+0000 7f4584148b80 1 --2- 172.21.15.107:0/87366230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 0x55c94e765040 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f456c099e60 tx=0x7f456c099930 comp rx=0 tx=0).stop 2023-07-21T22:35:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.535+0000 7f4584148b80 1 -- 172.21.15.107:0/87366230 shutdown_connections 2023-07-21T22:35:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.535+0000 7f4584148b80 1 --2- 172.21.15.107:0/87366230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e765580 0x55c94e7659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.535+0000 7f4584148b80 1 --2- 172.21.15.107:0/87366230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e5c06b0 0x55c94e7642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.535+0000 7f4584148b80 1 --2- 172.21.15.107:0/87366230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 0x55c94e765040 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.535+0000 7f4584148b80 1 -- 172.21.15.107:0/87366230 >> 172.21.15.107:0/87366230 conn(0x55c94e759ae0 msgr2=0x55c94e75ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.535+0000 7f4584148b80 1 -- 172.21.15.107:0/87366230 shutdown_connections 2023-07-21T22:35:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.535+0000 7f4584148b80 1 -- 172.21.15.107:0/87366230 wait complete. 2023-07-21T22:35:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f4584148b80 1 -- start start 2023-07-21T22:35:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f4584148b80 1 Processor -- start 2023-07-21T22:35:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f4584148b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 0x55c94e76d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f4584148b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e764830 0x55c94e76ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f4584148b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 0x55c94e774e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f4584148b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94e655a60 con 0x55c94e5c06b0 2023-07-21T22:35:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f4584148b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94e655760 con 0x55c94e764830 2023-07-21T22:35:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f4584148b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94e5398c0 con 0x55c94e765580 2023-07-21T22:35:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f4584148b80 10 client.0 _ll_register_callbacks cb 0x55c94e653700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f4575d5f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 0x55c94e76d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f4575d5f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 0x55c94e76d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50822/0 (socket says 172.21.15.107:50822) 2023-07-21T22:35:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f4575d5f700 1 -- 172.21.15.107:0/1840188907 learned_addr learned my addr 172.21.15.107:0/1840188907 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f457555e700 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e764830 0x55c94e76ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.536+0000 7f4576560700 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 0x55c94e774e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f4575d5f700 1 -- 172.21.15.107:0/1840188907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 msgr2=0x55c94e774e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f4575d5f700 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 0x55c94e774e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f4575d5f700 1 -- 172.21.15.107:0/1840188907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e764830 msgr2=0x55c94e76ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f4575d5f700 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e764830 0x55c94e76ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f4575d5f700 1 -- 172.21.15.107:0/1840188907 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4560017070 con 0x55c94e5c06b0 2023-07-21T22:35:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f457555e700 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e764830 0x55c94e76ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f4575d5f700 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 0x55c94e76d880 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4560004440 tx=0x7f45600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b17fda21bbce9c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f45597fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:35:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f45597fa700 1 -- 172.21.15.107:0/1840188907 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560022070 con 0x55c94e5c06b0 2023-07-21T22:35:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f4584148b80 1 -- 172.21.15.107:0/1840188907 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c94e60f490 con 0x55c94e5c06b0 2023-07-21T22:35:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f45597fa700 1 -- 172.21.15.107:0/1840188907 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4560003d70 con 0x55c94e5c06b0 2023-07-21T22:35:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f45597fa700 1 -- 172.21.15.107:0/1840188907 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4560027420 con 0x55c94e5c06b0 2023-07-21T22:35:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f4584148b80 10 client.16458 fetch_fsmap learned FSMap version 39 2023-07-21T22:35:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f4584148b80 1 -- 172.21.15.107:0/1840188907 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f45701a0390 con 0x55c94e5c06b0 2023-07-21T22:35:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.537+0000 7f45597fa700 1 -- 172.21.15.107:0/1840188907 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c94e60f490 con 0x55c94e5c06b0 2023-07-21T22:35:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.538+0000 7f45597fa700 1 -- 172.21.15.107:0/1840188907 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f456002f020 con 0x55c94e5c06b0 2023-07-21T22:35:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.538+0000 7f4584148b80 10 client.16458 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:35:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.538+0000 7f4584148b80 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.538+0000 7f4584148b80 20 client.16458 populate_metadata read hostname 'smithi107' 2023-07-21T22:35:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.538+0000 7f4584148b80 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c94e7796e0 0x55c94e77baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.538+0000 7f4584148b80 4 client.16458 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.538+0000 7f4584148b80 1 -- 172.21.15.107:0/1840188907 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c94e6aee50 con 0x55c94e7796e0 2023-07-21T22:35:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.538+0000 7f457555e700 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c94e7796e0 0x55c94e77baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.538+0000 7f45597fa700 10 client.16458 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.538+0000 7f457555e700 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c94e7796e0 0x55c94e77baf0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f7f34d132cee835 server_cookie=801682f38b86c3d1 in_seq=0 out_seq=0 2023-07-21T22:35:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f45597fa700 1 -- 172.21.15.107:0/1840188907 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c94e6aee50 con 0x55c94e7796e0 2023-07-21T22:35:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f45597fa700 10 client.16458 handle_command_reply: tid=0 2023-07-21T22:35:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f4584148b80 1 client.16458 shutdown 2023-07-21T22:35:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f4584148b80 1 -- 172.21.15.107:0/1840188907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 msgr2=0x55c94e76d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f4584148b80 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 0x55c94e76d880 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4560004440 tx=0x7f45600047c0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f4584148b80 1 -- 172.21.15.107:0/1840188907 shutdown_connections 2023-07-21T22:35:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f4584148b80 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c94e7796e0 0x55c94e77baf0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f4584148b80 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 0x55c94e774e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f45597fa700 0 client.16458 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f4584148b80 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 0x55c94e76d880 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f4584148b80 1 --2- 172.21.15.107:0/1840188907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e764830 0x55c94e76ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f4584148b80 1 -- 172.21.15.107:0/1840188907 >> 172.21.15.107:0/1840188907 conn(0x55c94e759ae0 msgr2=0x55c94e75e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f4584148b80 1 -- 172.21.15.107:0/1840188907 shutdown_connections 2023-07-21T22:35:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f4584148b80 1 -- 172.21.15.107:0/1840188907 wait complete. 2023-07-21T22:35:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.539+0000 7f4584148b80 20 client.16458 trim_cache size 0 max 16384 2023-07-21T22:35:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f4584148b80 1 Processor -- start 2023-07-21T22:35:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f4584148b80 1 -- start start 2023-07-21T22:35:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f4584148b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e5c06b0 0x55c94e780450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f4584148b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 0x55c94e774d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f4584148b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 0x55c94e76c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f4584148b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94e76d450 con 0x55c94e764830 2023-07-21T22:35:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f4584148b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94e76cd30 con 0x55c94e5c06b0 2023-07-21T22:35:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f4584148b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94e75bc00 con 0x55c94e765580 2023-07-21T22:35:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f457555e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 0x55c94e774d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f457555e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 0x55c94e774d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50832/0 (socket says 172.21.15.107:50832) 2023-07-21T22:35:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f457555e700 1 -- 172.21.15.107:0/3511292539 learned_addr learned my addr 172.21.15.107:0/3511292539 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f4576560700 1 --2- 172.21.15.107:0/3511292539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 0x55c94e76c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f4575d5f700 1 --2- 172.21.15.107:0/3511292539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e5c06b0 0x55c94e780450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f457555e700 1 -- 172.21.15.107:0/3511292539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 msgr2=0x55c94e76c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f457555e700 1 --2- 172.21.15.107:0/3511292539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 0x55c94e76c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f457555e700 1 -- 172.21.15.107:0/3511292539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e5c06b0 msgr2=0x55c94e780450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f457555e700 1 --2- 172.21.15.107:0/3511292539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e5c06b0 0x55c94e780450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f457555e700 1 -- 172.21.15.107:0/3511292539 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c94e4ccc00 con 0x55c94e764830 2023-07-21T22:35:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f4575d5f700 1 --2- 172.21.15.107:0/3511292539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e5c06b0 0x55c94e780450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:35:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.554+0000 7f457555e700 1 --2- 172.21.15.107:0/3511292539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 0x55c94e774d90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f456c00e970 tx=0x7f456c010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da49068477752032 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.555+0000 7f4564ff9700 1 -- 172.21.15.107:0/3511292539 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c0a2320 con 0x55c94e764830 2023-07-21T22:35:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.555+0000 7f4564ff9700 1 -- 172.21.15.107:0/3511292539 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f456c099580 con 0x55c94e764830 2023-07-21T22:35:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.555+0000 7f4564ff9700 1 -- 172.21.15.107:0/3511292539 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c095cc0 con 0x55c94e764830 2023-07-21T22:35:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.555+0000 7f4584148b80 1 -- 172.21.15.107:0/3511292539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 msgr2=0x55c94e774d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.555+0000 7f4584148b80 1 --2- 172.21.15.107:0/3511292539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 0x55c94e774d90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f456c00e970 tx=0x7f456c010c80 comp rx=0 tx=0).stop 2023-07-21T22:35:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.555+0000 7f4584148b80 1 -- 172.21.15.107:0/3511292539 shutdown_connections 2023-07-21T22:35:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.555+0000 7f4584148b80 1 --2- 172.21.15.107:0/3511292539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 0x55c94e76c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.555+0000 7f4584148b80 1 --2- 172.21.15.107:0/3511292539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e5c06b0 0x55c94e780450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.555+0000 7f4584148b80 1 --2- 172.21.15.107:0/3511292539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e764830 0x55c94e774d90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.555+0000 7f4584148b80 1 -- 172.21.15.107:0/3511292539 >> 172.21.15.107:0/3511292539 conn(0x55c94e759ae0 msgr2=0x55c94e75ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.555+0000 7f4584148b80 1 -- 172.21.15.107:0/3511292539 shutdown_connections 2023-07-21T22:35:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.555+0000 7f4584148b80 1 -- 172.21.15.107:0/3511292539 wait complete. 2023-07-21T22:35:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.555+0000 7f4584148b80 1 -- start start 2023-07-21T22:35:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4584148b80 1 Processor -- start 2023-07-21T22:35:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4584148b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 0x55c94e7703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4584148b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e764830 0x55c94e7708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4584148b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 0x55c94e773900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4584148b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94e773e40 con 0x55c94e5c06b0 2023-07-21T22:35:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4584148b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94e773fb0 con 0x55c94e764830 2023-07-21T22:35:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4584148b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94e774170 con 0x55c94e765580 2023-07-21T22:35:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4584148b80 10 client.0 _ll_register_callbacks cb 0x7f4560004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4575d5f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 0x55c94e7703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4575d5f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 0x55c94e7703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50836/0 (socket says 172.21.15.107:50836) 2023-07-21T22:35:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4575d5f700 1 -- 172.21.15.107:0/2999074263 learned_addr learned my addr 172.21.15.107:0/2999074263 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4576560700 1 --2- 172.21.15.107:0/2999074263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 0x55c94e773900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f457555e700 1 --2- 172.21.15.107:0/2999074263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e764830 0x55c94e7708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4575d5f700 1 -- 172.21.15.107:0/2999074263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 msgr2=0x55c94e773900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4575d5f700 1 --2- 172.21.15.107:0/2999074263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 0x55c94e773900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4575d5f700 1 -- 172.21.15.107:0/2999074263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e764830 msgr2=0x55c94e7708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4575d5f700 1 --2- 172.21.15.107:0/2999074263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e764830 0x55c94e7708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4575d5f700 1 -- 172.21.15.107:0/2999074263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c94e4e3d80 con 0x55c94e5c06b0 2023-07-21T22:35:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f4575d5f700 1 --2- 172.21.15.107:0/2999074263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 0x55c94e7703a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f45600093f0 tx=0x7f4560031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ca4c8bffa76fff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.556+0000 7f45597fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:35:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f45597fa700 1 -- 172.21.15.107:0/2999074263 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560022070 con 0x55c94e5c06b0 2023-07-21T22:35:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f45597fa700 1 -- 172.21.15.107:0/2999074263 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4560031df0 con 0x55c94e5c06b0 2023-07-21T22:35:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f4584148b80 1 -- 172.21.15.107:0/2999074263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f456c098270 con 0x55c94e5c06b0 2023-07-21T22:35:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f45597fa700 1 -- 172.21.15.107:0/2999074263 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4560004830 con 0x55c94e5c06b0 2023-07-21T22:35:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f4584148b80 10 client.16464 fetch_fsmap learned FSMap version 39 2023-07-21T22:35:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f4584148b80 1 -- 172.21.15.107:0/2999074263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c94e50a1f0 con 0x55c94e5c06b0 2023-07-21T22:35:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f45597fa700 1 -- 172.21.15.107:0/2999074263 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f456c098270 con 0x55c94e5c06b0 2023-07-21T22:35:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f45597fa700 1 -- 172.21.15.107:0/2999074263 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f456002f050 con 0x55c94e5c06b0 2023-07-21T22:35:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f4584148b80 10 client.16464 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:35:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f4584148b80 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f4584148b80 20 client.16464 populate_metadata read hostname 'smithi107' 2023-07-21T22:35:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f4584148b80 1 --2- 172.21.15.107:0/2999074263 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c94e7cdd60 0x55c94e7d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f4584148b80 4 client.16464 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:35:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f4584148b80 1 -- 172.21.15.107:0/2999074263 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c94e670ff0 con 0x55c94e7cdd60 2023-07-21T22:35:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.557+0000 7f457555e700 1 --2- 172.21.15.107:0/2999074263 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c94e7cdd60 0x55c94e7d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.558+0000 7f45597fa700 10 client.16464 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.558+0000 7f457555e700 1 --2- 172.21.15.107:0/2999074263 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c94e7cdd60 0x55c94e7d0140 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43de79fff6be9037 server_cookie=d26ca6b8ecbc44cf in_seq=0 out_seq=0 2023-07-21T22:35:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f45597fa700 1 -- 172.21.15.107:0/2999074263 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c94e670ff0 con 0x55c94e7cdd60 2023-07-21T22:35:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f45597fa700 10 client.16464 handle_command_reply: tid=0 2023-07-21T22:35:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f4584148b80 1 client.16464 shutdown 2023-07-21T22:35:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f4584148b80 1 -- 172.21.15.107:0/2999074263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 msgr2=0x55c94e7703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f4584148b80 1 --2- 172.21.15.107:0/2999074263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 0x55c94e7703a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f45600093f0 tx=0x7f4560031650 comp rx=0 tx=0).stop 2023-07-21T22:35:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f4584148b80 1 -- 172.21.15.107:0/2999074263 shutdown_connections 2023-07-21T22:35:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f4584148b80 1 --2- 172.21.15.107:0/2999074263 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c94e7cdd60 0x55c94e7d0140 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f4584148b80 1 --2- 172.21.15.107:0/2999074263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c94e765580 0x55c94e773900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f45597fa700 0 client.16464 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f4584148b80 1 --2- 172.21.15.107:0/2999074263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c94e5c06b0 0x55c94e7703a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f4584148b80 1 --2- 172.21.15.107:0/2999074263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c94e764830 0x55c94e7708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f4584148b80 1 -- 172.21.15.107:0/2999074263 >> 172.21.15.107:0/2999074263 conn(0x55c94e759ae0 msgr2=0x55c94e76c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f4584148b80 1 -- 172.21.15.107:0/2999074263 shutdown_connections 2023-07-21T22:35:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.588+0000 7f4584148b80 1 -- 172.21.15.107:0/2999074263 wait complete. 2023-07-21T22:35:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.589+0000 7f4584148b80 20 client.16464 trim_cache size 0 max 16384 2023-07-21T22:35:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:35:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:35:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:35:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:35:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.589+0000 7f4577fc3700 1 -- 172.21.15.107:0/2272409942 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f455c0a1150 msgr2=0x7f455c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.589+0000 7f4577fc3700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f455c0a1150 0x7f455c0a3600 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4570102060 tx=0x7f4560009260 comp rx=0 tx=0).stop 2023-07-21T22:35:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.589+0000 7f4577fc3700 1 -- 172.21.15.107:0/2272409942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45700ff370 msgr2=0x7f4570100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.589+0000 7f4577fc3700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45700ff370 0x7f4570100c90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f456c00e9a0 tx=0x7f456c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.589+0000 7f4577fc3700 1 -- 172.21.15.107:0/2272409942 shutdown_connections 2023-07-21T22:35:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.589+0000 7f4577fc3700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45700ffd00 0x7f45701011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.589+0000 7f4577fc3700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45700fe180 0x7f4570100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.589+0000 7f4577fc3700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f455c0a1150 0x7f455c0a3600 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.589+0000 7f4577fc3700 1 --2- 172.21.15.107:0/2272409942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45700ff370 0x7f4570100c90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.589+0000 7f4577fc3700 1 -- 172.21.15.107:0/2272409942 >> 172.21.15.107:0/2272409942 conn(0x7f45700f9780 msgr2=0x7f45701049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.589+0000 7f4577fc3700 1 -- 172.21.15.107:0/2272409942 shutdown_connections 2023-07-21T22:35:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:43.589+0000 7f4577fc3700 1 -- 172.21.15.107:0/2272409942 wait complete. 2023-07-21T22:35:44.067 INFO:tasks.workunit.client.0.smithi107.stdout: 2023-07-21T22:35:44.067 INFO:tasks.workunit.client.0.smithi107.stdout:Final score for writes: 399 2023-07-21T22:35:44.068 INFO:tasks.workunit.client.0.smithi107.stdout:Final score for reads : 1102407 2023-07-21T22:35:44.068 INFO:tasks.workunit.client.0.smithi107.stdout: 2023-07-21T22:35:44.069 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-07-21T22:35:44.069 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-07-21T22:35:44.093 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:43 smithi107 ceph-mon[96543]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 6.6 MiB/s wr, 377 op/s 2023-07-21T22:35:44.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:43 smithi190 ceph-mon[98334]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 6.6 MiB/s wr, 377 op/s 2023-07-21T22:35:44.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:43 smithi132 ceph-mon[97863]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 6.6 MiB/s wr, 377 op/s 2023-07-21T22:35:46.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:45 smithi190 ceph-mon[98334]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 2.9 MiB/s wr, 179 op/s 2023-07-21T22:35:46.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:45 smithi132 ceph-mon[97863]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 2.9 MiB/s wr, 179 op/s 2023-07-21T22:35:46.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:45 smithi107 ceph-mon[96543]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 2.9 MiB/s wr, 179 op/s 2023-07-21T22:35:48.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:47 smithi190 ceph-mon[98334]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 58 KiB/s wr, 37 op/s 2023-07-21T22:35:48.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:47 smithi132 ceph-mon[97863]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 58 KiB/s wr, 37 op/s 2023-07-21T22:35:48.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:47 smithi107 ceph-mon[96543]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 58 KiB/s wr, 37 op/s 2023-07-21T22:35:50.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:49 smithi190 ceph-mon[98334]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 447 KiB/s wr, 98 op/s 2023-07-21T22:35:50.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:49 smithi132 ceph-mon[97863]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 447 KiB/s wr, 98 op/s 2023-07-21T22:35:50.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:49 smithi107 ceph-mon[96543]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 447 KiB/s wr, 98 op/s 2023-07-21T22:35:52.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:51 smithi190 ceph-mon[98334]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 446 KiB/s wr, 106 op/s 2023-07-21T22:35:52.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:51 smithi132 ceph-mon[97863]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 446 KiB/s wr, 106 op/s 2023-07-21T22:35:52.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:51 smithi107 ceph-mon[96543]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 446 KiB/s wr, 106 op/s 2023-07-21T22:35:53.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:52 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:53.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:52 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:53.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:52 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:54.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:53 smithi190 ceph-mon[98334]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 178 op/s 2023-07-21T22:35:54.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:53 smithi132 ceph-mon[97863]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 178 op/s 2023-07-21T22:35:54.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:53 smithi107 ceph-mon[96543]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 178 op/s 2023-07-21T22:35:56.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:55 smithi190 ceph-mon[98334]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2023-07-21T22:35:56.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:55 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:56.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:55 smithi132 ceph-mon[97863]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2023-07-21T22:35:56.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:55 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:56.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:55 smithi107 ceph-mon[96543]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2023-07-21T22:35:56.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:55 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.700+0000 7fe93da2c700 1 -- 172.21.15.107:0/4248029368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9380ffc30 msgr2=0x7fe938100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.700+0000 7fe93da2c700 1 --2- 172.21.15.107:0/4248029368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9380ffc30 0x7fe938100080 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe928009990 tx=0x7fe928009ca0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.700+0000 7fe93da2c700 1 -- 172.21.15.107:0/4248029368 shutdown_connections 2023-07-21T22:35:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.700+0000 7fe93da2c700 1 --2- 172.21.15.107:0/4248029368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9381005c0 0x7fe938106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.700+0000 7fe93da2c700 1 --2- 172.21.15.107:0/4248029368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9380fea40 0x7fe9380fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.700+0000 7fe93da2c700 1 --2- 172.21.15.107:0/4248029368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9380ffc30 0x7fe938100080 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.700+0000 7fe93da2c700 1 -- 172.21.15.107:0/4248029368 >> 172.21.15.107:0/4248029368 conn(0x7fe9380fa060 msgr2=0x7fe9380fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.700+0000 7fe93da2c700 1 -- 172.21.15.107:0/4248029368 shutdown_connections 2023-07-21T22:35:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.700+0000 7fe93da2c700 1 -- 172.21.15.107:0/4248029368 wait complete. 2023-07-21T22:35:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.700+0000 7fe93da2c700 1 Processor -- start 2023-07-21T22:35:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.700+0000 7fe93da2c700 1 -- start start 2023-07-21T22:35:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe93da2c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9380fea40 0x7fe938195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe93da2c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9380ffc30 0x7fe938195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe93da2c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9381005c0 0x7fe938196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe93da2c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe938197390 con 0x7fe9381005c0 2023-07-21T22:35:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe93da2c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe938197500 con 0x7fe9380fea40 2023-07-21T22:35:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe93da2c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9381976c0 con 0x7fe9380ffc30 2023-07-21T22:35:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe936ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9380fea40 0x7fe938195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe9377fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9381005c0 0x7fe938196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe936ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9380fea40 0x7fe938195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41678/0 (socket says 172.21.15.107:41678) 2023-07-21T22:35:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe936ffd700 1 -- 172.21.15.107:0/3377960149 learned_addr learned my addr 172.21.15.107:0/3377960149 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe9367fc700 1 --2- 172.21.15.107:0/3377960149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9380ffc30 0x7fe938195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe9377fe700 1 -- 172.21.15.107:0/3377960149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9380ffc30 msgr2=0x7fe938195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe9377fe700 1 --2- 172.21.15.107:0/3377960149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9380ffc30 0x7fe938195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe9377fe700 1 -- 172.21.15.107:0/3377960149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9380fea40 msgr2=0x7fe938195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe9377fe700 1 --2- 172.21.15.107:0/3377960149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9380fea40 0x7fe938195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe9377fe700 1 -- 172.21.15.107:0/3377960149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9280095f0 con 0x7fe9381005c0 2023-07-21T22:35:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe9377fe700 1 --2- 172.21.15.107:0/3377960149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9381005c0 0x7fe938196d30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe93000efc0 tx=0x7fe93000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe93ca2a700 1 -- 172.21.15.107:0/3377960149 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe930018070 con 0x7fe9381005c0 2023-07-21T22:35:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe93ca2a700 1 -- 172.21.15.107:0/3377960149 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe93000f040 con 0x7fe9381005c0 2023-07-21T22:35:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe93ca2a700 1 -- 172.21.15.107:0/3377960149 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93000cdf0 con 0x7fe9381005c0 2023-07-21T22:35:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe93da2c700 1 -- 172.21.15.107:0/3377960149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe938197940 con 0x7fe9381005c0 2023-07-21T22:35:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.701+0000 7fe93da2c700 1 -- 172.21.15.107:0/3377960149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93819e160 con 0x7fe9381005c0 2023-07-21T22:35:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.703+0000 7fe93ca2a700 1 -- 172.21.15.107:0/3377960149 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe930003a00 con 0x7fe9381005c0 2023-07-21T22:35:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.704+0000 7fe93ca2a700 1 --2- 172.21.15.107:0/3377960149 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe9200a11d0 0x7fe9200a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.704+0000 7fe936ffd700 1 --2- 172.21.15.107:0/3377960149 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe9200a11d0 0x7fe9200a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.704+0000 7fe93ca2a700 1 -- 172.21.15.107:0/3377960149 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe9300cc5c0 con 0x7fe9381005c0 2023-07-21T22:35:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.704+0000 7fe936ffd700 1 --2- 172.21.15.107:0/3377960149 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe9200a11d0 0x7fe9200a3680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe92c005950 tx=0x7fe92c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe949bb1b80 1 Processor -- start 2023-07-21T22:35:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe949bb1b80 1 -- start start 2023-07-21T22:35:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe949bb1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7b6e6b0 0x5647f7d122f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe949bb1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7d12830 0x5647f7d13040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe949bb1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d13580 0x5647f7d139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe949bb1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f7c17c50 con 0x5647f7d13580 2023-07-21T22:35:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe949bb1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f7c6a9e0 con 0x5647f7d12830 2023-07-21T22:35:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe949bb1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f7c63e70 con 0x5647f7b6e6b0 2023-07-21T22:35:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe936ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7b6e6b0 0x5647f7d122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe936ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7b6e6b0 0x5647f7d122f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39416/0 (socket says 172.21.15.107:39416) 2023-07-21T22:35:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe936ffd700 1 -- 172.21.15.107:0/1748420597 learned_addr learned my addr 172.21.15.107:0/1748420597 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe9377fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d13580 0x5647f7d139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe9367fc700 1 --2- 172.21.15.107:0/1748420597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7d12830 0x5647f7d13040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe936ffd700 1 -- 172.21.15.107:0/1748420597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7d12830 msgr2=0x5647f7d13040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe936ffd700 1 --2- 172.21.15.107:0/1748420597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7d12830 0x5647f7d13040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe936ffd700 1 -- 172.21.15.107:0/1748420597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d13580 msgr2=0x5647f7d139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe936ffd700 1 --2- 172.21.15.107:0/1748420597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d13580 0x5647f7d139f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.705+0000 7fe936ffd700 1 -- 172.21.15.107:0/1748420597 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647f7a91d80 con 0x5647f7b6e6b0 2023-07-21T22:35:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.706+0000 7fe936ffd700 1 --2- 172.21.15.107:0/1748420597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7b6e6b0 0x5647f7d122f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009800 tx=0x7fe92c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=341651b47a43b467 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.706+0000 7fe91dffb700 1 -- 172.21.15.107:0/1748420597 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c026430 con 0x5647f7b6e6b0 2023-07-21T22:35:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.706+0000 7fe91dffb700 1 -- 172.21.15.107:0/1748420597 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe92c021070 con 0x5647f7b6e6b0 2023-07-21T22:35:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.706+0000 7fe91dffb700 1 -- 172.21.15.107:0/1748420597 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c01d650 con 0x5647f7b6e6b0 2023-07-21T22:35:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.706+0000 7fe949bb1b80 1 -- 172.21.15.107:0/1748420597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7b6e6b0 msgr2=0x5647f7d122f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.706+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/1748420597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7b6e6b0 0x5647f7d122f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009800 tx=0x7fe92c014000 comp rx=0 tx=0).stop 2023-07-21T22:35:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.706+0000 7fe949bb1b80 1 -- 172.21.15.107:0/1748420597 shutdown_connections 2023-07-21T22:35:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.706+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/1748420597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d13580 0x5647f7d139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.706+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/1748420597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7b6e6b0 0x5647f7d122f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.706+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/1748420597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7d12830 0x5647f7d13040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.706+0000 7fe949bb1b80 1 -- 172.21.15.107:0/1748420597 >> 172.21.15.107:0/1748420597 conn(0x5647f7d07ae0 msgr2=0x5647f7d0cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.706+0000 7fe949bb1b80 1 -- 172.21.15.107:0/1748420597 shutdown_connections 2023-07-21T22:35:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.706+0000 7fe949bb1b80 1 -- 172.21.15.107:0/1748420597 wait complete. 2023-07-21T22:35:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe949bb1b80 1 -- start start 2023-07-21T22:35:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe949bb1b80 1 Processor -- start 2023-07-21T22:35:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe949bb1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7b6e6b0 0x5647f7d1b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe949bb1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7d12830 0x5647f7d1bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe949bb1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 0x5647f7d22d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe949bb1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f7c03a60 con 0x5647f7b6e6b0 2023-07-21T22:35:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe949bb1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f7c03760 con 0x5647f7d12830 2023-07-21T22:35:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe949bb1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f7ae78c0 con 0x5647f7d13580 2023-07-21T22:35:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe949bb1b80 10 client.0 _ll_register_callbacks cb 0x5647f7c01700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe936ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7b6e6b0 0x5647f7d1b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe936ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7b6e6b0 0x5647f7d1b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50864/0 (socket says 172.21.15.107:50864) 2023-07-21T22:35:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe9367fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7d12830 0x5647f7d1bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe936ffd700 1 -- 172.21.15.107:0/2661548486 learned_addr learned my addr 172.21.15.107:0/2661548486 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.707+0000 7fe9377fe700 1 --2- 172.21.15.107:0/2661548486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 0x5647f7d22d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe936ffd700 1 -- 172.21.15.107:0/2661548486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 msgr2=0x5647f7d22d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe936ffd700 1 --2- 172.21.15.107:0/2661548486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 0x5647f7d22d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe936ffd700 1 -- 172.21.15.107:0/2661548486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7d12830 msgr2=0x5647f7d1bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe936ffd700 1 --2- 172.21.15.107:0/2661548486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7d12830 0x5647f7d1bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe936ffd700 1 -- 172.21.15.107:0/2661548486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c017070 con 0x5647f7b6e6b0 2023-07-21T22:35:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe936ffd700 1 --2- 172.21.15.107:0/2661548486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7b6e6b0 0x5647f7d1b7f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe92c01cbe0 tx=0x7fe92c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ad10bad59896fe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe916ffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:35:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe916ffd700 1 -- 172.21.15.107:0/2661548486 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c01c410 con 0x5647f7b6e6b0 2023-07-21T22:35:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe916ffd700 1 -- 172.21.15.107:0/2661548486 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe92c01c570 con 0x5647f7b6e6b0 2023-07-21T22:35:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe949bb1b80 1 -- 172.21.15.107:0/2661548486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647f7bbd490 con 0x5647f7b6e6b0 2023-07-21T22:35:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe916ffd700 1 -- 172.21.15.107:0/2661548486 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe92c01c730 con 0x5647f7b6e6b0 2023-07-21T22:35:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe949bb1b80 10 client.16476 fetch_fsmap learned FSMap version 41 2023-07-21T22:35:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe949bb1b80 1 -- 172.21.15.107:0/2661548486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5647f7a7ac00 con 0x5647f7b6e6b0 2023-07-21T22:35:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe916ffd700 1 -- 172.21.15.107:0/2661548486 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe92c03e080 con 0x5647f7b6e6b0 2023-07-21T22:35:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe916ffd700 1 -- 172.21.15.107:0/2661548486 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7fe92c038030 con 0x5647f7b6e6b0 2023-07-21T22:35:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe949bb1b80 10 client.16476 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:35:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe949bb1b80 10 client.16476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.708+0000 7fe949bb1b80 20 client.16476 populate_metadata read hostname 'smithi107' 2023-07-21T22:35:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.709+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/2661548486 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5647f7d27570 0x5647f7d29980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.709+0000 7fe949bb1b80 4 client.16476 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.709+0000 7fe949bb1b80 1 -- 172.21.15.107:0/2661548486 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647f7c5ce50 con 0x5647f7d27570 2023-07-21T22:35:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.709+0000 7fe9367fc700 1 --2- 172.21.15.107:0/2661548486 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5647f7d27570 0x5647f7d29980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.709+0000 7fe916ffd700 10 client.16476 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.709+0000 7fe9367fc700 1 --2- 172.21.15.107:0/2661548486 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5647f7d27570 0x5647f7d29980 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb2650f534876bc server_cookie=6ef26f97a1308257 in_seq=0 out_seq=0 2023-07-21T22:35:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe916ffd700 1 -- 172.21.15.107:0/2661548486 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5647f7c5ce50 con 0x5647f7d27570 2023-07-21T22:35:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe916ffd700 10 client.16476 handle_command_reply: tid=0 2023-07-21T22:35:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe949bb1b80 1 client.16476 shutdown 2023-07-21T22:35:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe949bb1b80 1 -- 172.21.15.107:0/2661548486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7b6e6b0 msgr2=0x5647f7d1b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/2661548486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7b6e6b0 0x5647f7d1b7f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe92c01cbe0 tx=0x7fe92c01cc40 comp rx=0 tx=0).stop 2023-07-21T22:35:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe949bb1b80 1 -- 172.21.15.107:0/2661548486 shutdown_connections 2023-07-21T22:35:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/2661548486 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5647f7d27570 0x5647f7d29980 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/2661548486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 0x5647f7d22d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe916ffd700 0 client.16476 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/2661548486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7b6e6b0 0x5647f7d1b7f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/2661548486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7d12830 0x5647f7d1bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe949bb1b80 1 -- 172.21.15.107:0/2661548486 >> 172.21.15.107:0/2661548486 conn(0x5647f7d07ae0 msgr2=0x5647f7d0c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe949bb1b80 1 -- 172.21.15.107:0/2661548486 shutdown_connections 2023-07-21T22:35:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe949bb1b80 1 -- 172.21.15.107:0/2661548486 wait complete. 2023-07-21T22:35:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.710+0000 7fe949bb1b80 20 client.16476 trim_cache size 0 max 16384 2023-07-21T22:35:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe949bb1b80 1 Processor -- start 2023-07-21T22:35:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe949bb1b80 1 -- start start 2023-07-21T22:35:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe949bb1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7b6e6b0 0x5647f7d1aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe949bb1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 0x5647f7d2e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe949bb1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 0x5647f7d19860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe949bb1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f7d23190 con 0x5647f7d12830 2023-07-21T22:35:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe949bb1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f7d19610 con 0x5647f7b6e6b0 2023-07-21T22:35:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe949bb1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f7d16f00 con 0x5647f7d13580 2023-07-21T22:35:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe9367fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 0x5647f7d2e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe9367fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 0x5647f7d2e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50874/0 (socket says 172.21.15.107:50874) 2023-07-21T22:35:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe9367fc700 1 -- 172.21.15.107:0/3748818530 learned_addr learned my addr 172.21.15.107:0/3748818530 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe936ffd700 1 --2- 172.21.15.107:0/3748818530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7b6e6b0 0x5647f7d1aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe9377fe700 1 --2- 172.21.15.107:0/3748818530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 0x5647f7d19860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe9367fc700 1 -- 172.21.15.107:0/3748818530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 msgr2=0x5647f7d19860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe9367fc700 1 --2- 172.21.15.107:0/3748818530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 0x5647f7d19860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe9367fc700 1 -- 172.21.15.107:0/3748818530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7b6e6b0 msgr2=0x5647f7d1aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.725+0000 7fe9367fc700 1 --2- 172.21.15.107:0/3748818530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7b6e6b0 0x5647f7d1aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.726+0000 7fe9367fc700 1 -- 172.21.15.107:0/3748818530 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647f7ab81f0 con 0x5647f7d12830 2023-07-21T22:35:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.726+0000 7fe936ffd700 1 --2- 172.21.15.107:0/3748818530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7b6e6b0 0x5647f7d1aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.726+0000 7fe9367fc700 1 --2- 172.21.15.107:0/3748818530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 0x5647f7d2e2a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe928009990 tx=0x7fe928005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3e71c7dc489a4a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.726+0000 7fe91e7fc700 1 -- 172.21.15.107:0/3748818530 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9280056e0 con 0x5647f7d12830 2023-07-21T22:35:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.726+0000 7fe91e7fc700 1 -- 172.21.15.107:0/3748818530 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe92801e040 con 0x5647f7d12830 2023-07-21T22:35:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.726+0000 7fe91e7fc700 1 -- 172.21.15.107:0/3748818530 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928004d20 con 0x5647f7d12830 2023-07-21T22:35:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.727+0000 7fe949bb1b80 1 -- 172.21.15.107:0/3748818530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 msgr2=0x5647f7d2e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.727+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/3748818530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 0x5647f7d2e2a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe928009990 tx=0x7fe928005960 comp rx=0 tx=0).stop 2023-07-21T22:35:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.727+0000 7fe949bb1b80 1 -- 172.21.15.107:0/3748818530 shutdown_connections 2023-07-21T22:35:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.727+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/3748818530 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 0x5647f7d19860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.727+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/3748818530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7b6e6b0 0x5647f7d1aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.727+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/3748818530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 0x5647f7d2e2a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.727+0000 7fe949bb1b80 1 -- 172.21.15.107:0/3748818530 >> 172.21.15.107:0/3748818530 conn(0x5647f7d07ae0 msgr2=0x5647f7d1a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.727+0000 7fe949bb1b80 1 -- 172.21.15.107:0/3748818530 shutdown_connections 2023-07-21T22:35:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.727+0000 7fe949bb1b80 1 -- 172.21.15.107:0/3748818530 wait complete. 2023-07-21T22:35:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.727+0000 7fe949bb1b80 1 -- start start 2023-07-21T22:35:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe949bb1b80 1 Processor -- start 2023-07-21T22:35:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe949bb1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7b6e6b0 0x5647f7d77610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe949bb1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 0x5647f7d77b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe949bb1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 0x5647f7d7e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe949bb1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f7d7ed60 con 0x5647f7d12830 2023-07-21T22:35:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe949bb1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f7d7eed0 con 0x5647f7b6e6b0 2023-07-21T22:35:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe949bb1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f7d7f090 con 0x5647f7d13580 2023-07-21T22:35:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe949bb1b80 10 client.0 _ll_register_callbacks cb 0x7fe930009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe9367fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 0x5647f7d77b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe9367fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 0x5647f7d77b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50878/0 (socket says 172.21.15.107:50878) 2023-07-21T22:35:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe936ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7b6e6b0 0x5647f7d77610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe9367fc700 1 -- 172.21.15.107:0/3602895390 learned_addr learned my addr 172.21.15.107:0/3602895390 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:35:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe9377fe700 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 0x5647f7d7e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe9367fc700 1 -- 172.21.15.107:0/3602895390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 msgr2=0x5647f7d7e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe9367fc700 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 0x5647f7d7e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe9367fc700 1 -- 172.21.15.107:0/3602895390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7b6e6b0 msgr2=0x5647f7d77610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe9367fc700 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7b6e6b0 0x5647f7d77610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe9367fc700 1 -- 172.21.15.107:0/3602895390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647f7a7ac00 con 0x5647f7d12830 2023-07-21T22:35:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe9377fe700 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 0x5647f7d7e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:35:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.728+0000 7fe936ffd700 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7b6e6b0 0x5647f7d77610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe9367fc700 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 0x5647f7d77b50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe92800b2e0 tx=0x7fe928004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d286a301c0fa1c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe916ffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:35:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe916ffd700 1 -- 172.21.15.107:0/3602895390 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928004ca0 con 0x5647f7d12830 2023-07-21T22:35:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe949bb1b80 1 -- 172.21.15.107:0/3602895390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe92c021070 con 0x5647f7d12830 2023-07-21T22:35:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe916ffd700 1 -- 172.21.15.107:0/3602895390 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe928004e00 con 0x5647f7d12830 2023-07-21T22:35:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe916ffd700 1 -- 172.21.15.107:0/3602895390 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe92800b7f0 con 0x5647f7d12830 2023-07-21T22:35:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe949bb1b80 10 client.16482 fetch_fsmap learned FSMap version 41 2023-07-21T22:35:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe949bb1b80 1 -- 172.21.15.107:0/3602895390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fe9381077b0 con 0x5647f7d12830 2023-07-21T22:35:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe916ffd700 1 -- 172.21.15.107:0/3602895390 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe92c021070 con 0x5647f7d12830 2023-07-21T22:35:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe916ffd700 1 -- 172.21.15.107:0/3602895390 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7fe92800fad0 con 0x5647f7d12830 2023-07-21T22:35:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe949bb1b80 10 client.16482 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:35:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe949bb1b80 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe949bb1b80 20 client.16482 populate_metadata read hostname 'smithi107' 2023-07-21T22:35:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5647f7d805c0 0x5647f7d829a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe949bb1b80 4 client.16482 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:35:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.729+0000 7fe949bb1b80 1 -- 172.21.15.107:0/3602895390 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5647f7c1eff0 con 0x5647f7d805c0 2023-07-21T22:35:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.730+0000 7fe936ffd700 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5647f7d805c0 0x5647f7d829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.730+0000 7fe916ffd700 10 client.16482 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:56.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:56.730+0000 7fe936ffd700 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5647f7d805c0 0x5647f7d829a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a7ccfc36667729c server_cookie=496719a8f9f037be in_seq=0 out_seq=0 2023-07-21T22:35:57.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.296+0000 7fe916ffd700 1 -- 172.21.15.107:0/3602895390 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5647f7c1eff0 con 0x5647f7d805c0 2023-07-21T22:35:57.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.296+0000 7fe916ffd700 10 client.16482 handle_command_reply: tid=0 2023-07-21T22:35:57.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.296+0000 7fe949bb1b80 1 client.16482 shutdown 2023-07-21T22:35:57.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.297+0000 7fe949bb1b80 1 -- 172.21.15.107:0/3602895390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 msgr2=0x5647f7d77b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:57.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.297+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 0x5647f7d77b50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe92800b2e0 tx=0x7fe928004680 comp rx=0 tx=0).stop 2023-07-21T22:35:57.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.297+0000 7fe949bb1b80 1 -- 172.21.15.107:0/3602895390 shutdown_connections 2023-07-21T22:35:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.297+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5647f7d805c0 0x5647f7d829a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.297+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5647f7d13580 0x5647f7d7e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.297+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5647f7b6e6b0 0x5647f7d77610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.297+0000 7fe949bb1b80 1 --2- 172.21.15.107:0/3602895390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5647f7d12830 0x5647f7d77b50 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.297+0000 7fe949bb1b80 1 -- 172.21.15.107:0/3602895390 >> 172.21.15.107:0/3602895390 conn(0x5647f7d07ae0 msgr2=0x5647f7d19860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.297+0000 7fe916ffd700 0 client.16482 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:35:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.297+0000 7fe949bb1b80 1 -- 172.21.15.107:0/3602895390 shutdown_connections 2023-07-21T22:35:57.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.297+0000 7fe949bb1b80 1 -- 172.21.15.107:0/3602895390 wait complete. 2023-07-21T22:35:57.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.297+0000 7fe949bb1b80 20 client.16482 trim_cache size 0 max 16384 2023-07-21T22:35:57.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:35:57.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:35:57.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:35:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:35:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.298+0000 7fe93da2c700 1 -- 172.21.15.107:0/3377960149 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe9200a11d0 msgr2=0x7fe9200a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.298+0000 7fe93da2c700 1 --2- 172.21.15.107:0/3377960149 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe9200a11d0 0x7fe9200a3680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe92c005950 tx=0x7fe92c009260 comp rx=0 tx=0).stop 2023-07-21T22:35:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.298+0000 7fe93da2c700 1 -- 172.21.15.107:0/3377960149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9381005c0 msgr2=0x7fe938196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.298+0000 7fe93da2c700 1 --2- 172.21.15.107:0/3377960149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9381005c0 0x7fe938196d30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe93000efc0 tx=0x7fe93000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:35:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.298+0000 7fe93da2c700 1 -- 172.21.15.107:0/3377960149 shutdown_connections 2023-07-21T22:35:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.298+0000 7fe93da2c700 1 --2- 172.21.15.107:0/3377960149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9381005c0 0x7fe938196d30 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.298+0000 7fe93da2c700 1 --2- 172.21.15.107:0/3377960149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9380fea40 0x7fe938195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.298+0000 7fe93da2c700 1 --2- 172.21.15.107:0/3377960149 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe9200a11d0 0x7fe9200a3680 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.298+0000 7fe93da2c700 1 --2- 172.21.15.107:0/3377960149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9380ffc30 0x7fe938195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.298+0000 7fe93da2c700 1 -- 172.21.15.107:0/3377960149 >> 172.21.15.107:0/3377960149 conn(0x7fe9380fa060 msgr2=0x7fe938068890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.298+0000 7fe93da2c700 1 -- 172.21.15.107:0/3377960149 shutdown_connections 2023-07-21T22:35:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:35:57.298+0000 7fe93da2c700 1 -- 172.21.15.107:0/3377960149 wait complete. 2023-07-21T22:35:57.479 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.179+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000112a [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-72.xml auth v1262 ap=1 s=25908 n(v0 rc2023-07-21T22:30:29.079206+0000 b25908 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd621a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000112a:[<0x40000001129/article-72.xml v787>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000112a:[<0x40000001129/article-72.xml v1262>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.479 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.179+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000114f [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-21.jpg auth v1256 ap=1 s=38967 n(v0 rc2023-07-21T22:30:30.909175+0000 b38967 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd38dcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000114f:[<0x40000001129/picture-21.jpg v781>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000114f:[<0x40000001129/picture-21.jpg v1256>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.479 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.179+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e3 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-35.xml auth v1264 ap=1 s=31187 n(v0 rc2023-07-21T22:30:39.095040+0000 b31187 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd88ae580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005e3:[<0x40000001129/article-35.xml v783>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x100000005e3:[<0x40000001129/article-35.xml v1264>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.480 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.179+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b6e [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-73.xml auth v1242 ap=1 s=6779 n(v0 rc2023-07-21T22:34:42.893045+0000 b6779 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd8cb8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b6e:[<0x40000001129/comment-73.xml v1087>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b6e:[<0x40000001129/comment-73.xml v1242>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.480 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.180+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b7e [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-60.xml auth v1238 ap=1 s=2514 n(v0 rc2023-07-21T22:34:43.608033+0000 b2514 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd44e7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b7e:[<0x40000001129/comment-60.xml v1111>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b7e:[<0x40000001129/comment-60.xml v1238>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.480 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.180+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bbf [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-1.xml auth v1226 ap=1 s=5650 n(v0 rc2023-07-21T22:34:45.967995+0000 b5650 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd44e7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bbf:[<0x40000001129/comment-1.xml v1221>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bbf:[<0x40000001129/comment-1.xml v1226>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.480 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.182+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001015 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-40.xml auth v1258 ap=1 s=32295 n(v0 rc2023-07-21T22:30:29.764194+0000 b32295 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd44e6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001015:[<0x40000001129/article-40.xml v258>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001015:[<0x40000001129/article-40.xml v1258>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.480 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.182+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b69 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-78.xml auth v1244 ap=1 s=7254 n(v0 rc2023-07-21T22:34:42.811046+0000 b7254 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd44e7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b69:[<0x40000001129/comment-78.xml v1076>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b69:[<0x40000001129/comment-78.xml v1244>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.480 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.182+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b68 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-79.xml auth v1246 ap=1 s=3777 n(v0 rc2023-07-21T22:34:42.780047+0000 b3777 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd44e6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b68:[<0x40000001129/comment-79.xml v1073>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b68:[<0x40000001129/comment-79.xml v1246>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.481 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.183+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000038 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-15.xml auth v1269 ap=1 s=27393 n(v0 rc2023-07-21T22:31:24.870288+0000 b27393 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd4c02c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000038:[<0x40000001129/article-15.xml v826>,<0x4000000033b/blog-21 v115405>,<0x400000001fb/blogtest_in v1074>,<0x400000001f6/src v1808>,<0x40000000000/blogbench-1.0 v1570>,<0x100000005e2/tmp v170>,<0x100000001fb/client.0 v125>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v42>,<0x10000000001/sv_1 v66>,<0x10000000000/_nogroup v51>,<0x1/volumes v49>]//[]","memoryvalue":"(3)0x10000000038:[<0x40000001129/article-15.xml v1269>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.481 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.183+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b95 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-40.xml auth v1234 ap=1 s=9269 n(v0 rc2023-07-21T22:34:44.381021+0000 b9269 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd67f2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b95:[<0x40000001129/comment-40.xml v1144>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b95:[<0x40000001129/comment-40.xml v1234>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.481 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.183+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b9c [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-33.xml auth v1230 ap=1 s=6592 n(v0 rc2023-07-21T22:34:44.632016+0000 b6592 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd430f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b9c:[<0x40000001129/comment-33.xml v1156>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b9c:[<0x40000001129/comment-33.xml v1230>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.481 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.183+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b96 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-39.xml auth v1232 ap=1 s=9497 n(v0 rc2023-07-21T22:34:44.450019+0000 b9497 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd4fa2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b96:[<0x40000001129/comment-39.xml v1148>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b96:[<0x40000001129/comment-39.xml v1232>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.481 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.183+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001084 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-19.xml auth v1252 ap=1 s=4981 n(v0 rc2023-07-21T22:30:30.208187+0000 b4981 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd430ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001084:[<0x40000001129/article-19.xml v426>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001084:[<0x40000001129/article-19.xml v1252>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.481 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.184+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000115c [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-20.jpg auth v1254 ap=1 s=63046 n(v0 rc2023-07-21T22:30:30.931175+0000 b63046 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd430e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000115c:[<0x40000001129/picture-20.jpg v779>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000115c:[<0x40000001129/picture-20.jpg v1254>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.481 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.184+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c9d [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-82.xml auth v1053 ap=1 s=7684 n(v0 rc2023-07-21T22:32:50.530882+0000 b7684 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd430e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c9d:[<0x40000001129/comment-82.xml v892>,<0x4000000033b/blog-21 v134411>,<0x400000001fb/blogtest_in v1820>,<0x400000001f6/src v1957>,<0x40000000000/blogbench-1.0 v1705>,<0x100000005e2/tmp v298>,<0x100000001fb/client.0 v297>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v56>,<0x10000000001/sv_1 v80>,<0x10000000000/_nogroup v65>,<0x1/volumes v63>]//[]","memoryvalue":"(3)0x10000000c9d:[<0x40000001129/comment-82.xml v1053>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.482 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.184+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b9d [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-32.xml auth v1228 ap=1 s=5035 n(v0 rc2023-07-21T22:34:44.657016+0000 b5035 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd430db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b9d:[<0x40000001129/comment-32.xml v1159>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b9d:[<0x40000001129/comment-32.xml v1228>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.482 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.184+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b7b [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-63.xml auth v1240 ap=1 s=6579 n(v0 rc2023-07-21T22:34:43.406036+0000 b6579 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd430d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b7b:[<0x40000001129/comment-63.xml v1106>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b7b:[<0x40000001129/comment-63.xml v1240>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.482 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.184+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b85 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-54.xml auth v1236 ap=1 s=3557 n(v0 rc2023-07-21T22:34:43.820030+0000 b3557 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd430d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b85:[<0x40000001129/comment-54.xml v1116>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b85:[<0x40000001129/comment-54.xml v1236>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.483 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.185+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001009 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-42.xml auth v1260 ap=1 s=3687 n(v0 rc2023-07-21T22:30:29.712195+0000 b3687 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd430cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001009:[<0x40000001129/article-42.xml v242>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001009:[<0x40000001129/article-42.xml v1260>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.483 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.185+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007d8 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-38.jpg auth v1266 ap=1 s=32111 n(v0 rc2023-07-21T22:30:39.097040+0000 b32111 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd430c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007d8:[<0x40000001129/picture-38.jpg v785>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x100000007d8:[<0x40000001129/picture-38.jpg v1266>,<0x4000000033b/blog-21 v177841>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.483 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.185+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000f9e [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-64.xml auth v1274 ap=1 s=24787 n(v0 rc2023-07-21T22:30:29.227203+0000 b24787 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd430c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000f9e:[<0x40000001129/article-64.xml v68>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000f9e:[<0x40000001129/article-64.xml v1274>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.484 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.186+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bb3 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-13.xml auth v1230 ap=1 s=5994 n(v0 rc2023-07-21T22:34:45.478003+0000 b5994 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd42ff700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bb3:[<0x40000001129/comment-13.xml v1209>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bb3:[<0x40000001129/comment-13.xml v1230>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.484 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.186+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ba9 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-21.xml auth v1234 ap=1 s=5748 n(v0 rc2023-07-21T22:34:44.937012+0000 b5748 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd42ff180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ba9:[<0x40000001129/comment-21.xml v1183>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001ba9:[<0x40000001129/comment-21.xml v1234>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.484 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.186+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ba3 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-26.xml auth v1236 ap=1 s=2581 n(v0 rc2023-07-21T22:34:44.818013+0000 b2581 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd42fec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ba3:[<0x40000001129/comment-26.xml v1174>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001ba3:[<0x40000001129/comment-26.xml v1236>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.484 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.186+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001025 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-37.xml auth v1270 ap=1 s=13179 n(v0 rc2023-07-21T22:30:29.823193+0000 b13179 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd42fe680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001025:[<0x40000001129/article-37.xml v282>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001025:[<0x40000001129/article-37.xml v1270>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.484 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.186+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001101 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-27.jpg auth v1266 ap=1 s=64093 n(v0 rc2023-07-21T22:30:30.732178+0000 b64093 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd42fe100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001101:[<0x40000001129/picture-27.jpg v781>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001101:[<0x40000001129/picture-27.jpg v1266>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.484 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.187+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000101b [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-39.xml auth v1272 ap=1 s=19708 n(v0 rc2023-07-21T22:30:29.789194+0000 b19708 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd42fdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000101b:[<0x40000001129/article-39.xml v266>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000101b:[<0x40000001129/article-39.xml v1272>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.484 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.187+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000102a [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-36.xml auth v1268 ap=1 s=19857 n(v0 rc2023-07-21T22:30:29.840193+0000 b19857 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd42fd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000102a:[<0x40000001129/article-36.xml v290>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000102a:[<0x40000001129/article-36.xml v1268>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.485 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.187+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010aa [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-12.xml auth v1258 ap=1 s=29385 n(v0 rc2023-07-21T22:30:30.380184+0000 b29385 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd42fd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010aa:[<0x40000001129/article-12.xml v482>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010aa:[<0x40000001129/article-12.xml v1258>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.485 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.187+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010e5 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-1.xml auth v1256 ap=1 s=6305 n(v0 rc2023-07-21T22:30:30.618180+0000 b6305 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd42fcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010e5:[<0x40000001129/article-1.xml v570>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010e5:[<0x40000001129/article-1.xml v1256>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.485 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.187+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b9f [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-30.xml auth v1238 ap=1 s=5706 n(v0 rc2023-07-21T22:34:44.719015+0000 b5706 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd42fc580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b9f:[<0x40000001129/comment-30.xml v1167>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b9f:[<0x40000001129/comment-30.xml v1238>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.485 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.187+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ee4 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-90.jpg auth v1067 ap=1 s=33948 n(v0 rc2023-07-21T22:33:04.060660+0000 b33948 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd42fc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ee4:[<0x40000001129/picture-90.jpg v1057>,<0x4000000033b/blog-21 v139069>,<0x400000001fb/blogtest_in v1930>,<0x400000001f6/src v1995>,<0x40000000000/blogbench-1.0 v1722>,<0x100000005e2/tmp v328>,<0x100000001fb/client.0 v313>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v58>,<0x10000000001/sv_1 v82>,<0x10000000000/_nogroup v67>,<0x1/volumes v65>]//[]","memoryvalue":"(3)0x10000000ee4:[<0x40000001129/picture-90.jpg v1067>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.485 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.188+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bbd [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-3.xml auth v1226 ap=1 s=9017 n(v0 rc2023-07-21T22:34:45.915996+0000 b9017 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd4335700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bbd:[<0x40000001129/comment-3.xml v1217>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bbd:[<0x40000001129/comment-3.xml v1226>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.485 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.188+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bad [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-18.xml auth v1232 ap=1 s=8353 n(v0 rc2023-07-21T22:34:45.231007+0000 b8353 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd462a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bad:[<0x40000001129/comment-18.xml v1196>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bad:[<0x40000001129/comment-18.xml v1232>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.486 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.188+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001185 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-16.jpg auth v1260 ap=1 s=61828 n(v0 rc2023-07-21T22:30:31.047173+0000 b61828 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd3a2c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001185:[<0x40000001129/picture-16.jpg v779>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001185:[<0x40000001129/picture-16.jpg v1260>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.486 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.188+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000f94 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-66.xml auth v1276 ap=1 s=16031 n(v0 rc2023-07-21T22:30:29.192204+0000 b16031 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd4334c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000f94:[<0x40000001129/article-66.xml v52>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000f94:[<0x40000001129/article-66.xml v1276>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.486 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.188+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bb4 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-12.xml auth v1228 ap=1 s=8029 n(v0 rc2023-07-21T22:34:45.507002+0000 b8029 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd4333b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bb4:[<0x40000001129/comment-12.xml v1214>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bb4:[<0x40000001129/comment-12.xml v1228>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.486 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.188+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b8b [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-49.xml auth v1242 ap=1 s=2991 n(v0 rc2023-07-21T22:34:44.049026+0000 b2991 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd4333600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b8b:[<0x40000001129/comment-49.xml v1124>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b8b:[<0x40000001129/comment-49.xml v1242>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.486 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.189+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b92 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-42.xml auth v1240 ap=1 s=7740 n(v0 rc2023-07-21T22:34:44.239023+0000 b7740 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd4333080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b92:[<0x40000001129/comment-42.xml v1135>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b92:[<0x40000001129/comment-42.xml v1240>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.486 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.189+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b77 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-65.xml auth v1246 ap=1 s=4909 n(v0 rc2023-07-21T22:34:43.350037+0000 b4909 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd4332b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b77:[<0x40000001129/comment-65.xml v1102>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b77:[<0x40000001129/comment-65.xml v1246>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.486 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.189+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001072 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-23.xml auth v1264 ap=1 s=15519 n(v0 rc2023-07-21T22:30:30.132188+0000 b15519 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd4332580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001072:[<0x40000001129/article-23.xml v394>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001072:[<0x40000001129/article-23.xml v1264>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.487 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.189+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000108f [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-17.xml auth v1262 ap=1 s=30647 n(v0 rc2023-07-21T22:30:30.264186+0000 b30647 1=1+0) (iversion lock) | replicated=0 dirty=1 authpin=1 scrubqueue=0 0x556fd4332000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000108f:[<0x40000001129/article-17.xml v442>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000108f:[<0x40000001129/article-17.xml v1262>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.487 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.189+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001829 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-63.xml auth v1085 ap=1 s=18303 n(v0 rc2023-07-21T22:34:18.680439+0000 b18303 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd432b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001829:[<0x40000001129/article-63.xml v1080>,<0x4000000033b/blog-21 v155439>,<0x400000001fb/blogtest_in v2345>,<0x400000001f6/src v2153>,<0x40000000000/blogbench-1.0 v1861>,<0x100000005e2/tmp v456>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v76>,<0x10000000001/sv_1 v100>,<0x10000000000/_nogroup v85>,<0x1/volumes v83>]//[]","memoryvalue":"(3)0x10000001829:[<0x40000001129/article-63.xml v1085>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.487 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.189+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b89 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-51.xml auth v1244 ap=1 s=2466 n(v0 rc2023-07-21T22:34:43.988027+0000 b2466 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd50e1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b89:[<0x40000001129/comment-51.xml v1119>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b89:[<0x40000001129/comment-51.xml v1244>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2772>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.487 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.335+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000c9f [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-81.xml auth v1035 ap=1 s=2477 n(v0 rc2023-07-21T22:32:50.555882+0000 b2477 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd944ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000c9f:[<0x40000001129/comment-81.xml v896>,<0x4000000033b/blog-21 v134411>,<0x400000001fb/blogtest_in v1820>,<0x400000001f6/src v1957>,<0x40000000000/blogbench-1.0 v1705>,<0x100000005e2/tmp v298>,<0x100000001fb/client.0 v297>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v56>,<0x10000000001/sv_1 v80>,<0x10000000000/_nogroup v65>,<0x1/volumes v63>]//[]","memoryvalue":"(3)0x10000000c9f:[<0x40000001129/comment-81.xml v1035>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.487 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.335+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001225 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-3.jpg auth v1228 ap=1 s=23251 n(v0 rc2023-07-21T22:30:31.815160+0000 b23251 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd94dd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001225:[<0x40000001129/picture-3.jpg v738>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001225:[<0x40000001129/picture-3.jpg v1228>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.487 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.335+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bb6 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-10.xml auth v1210 ap=1 s=6319 n(v0 rc2023-07-21T22:34:45.554001+0000 b6319 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd92e4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bb6:[<0x40000001129/comment-10.xml v1187>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bb6:[<0x40000001129/comment-10.xml v1210>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.488 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.336+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b86 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-53.xml auth v1218 ap=1 s=9300 n(v0 rc2023-07-21T22:34:43.852029+0000 b9300 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd92e3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b86:[<0x40000001129/comment-53.xml v1127>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b86:[<0x40000001129/comment-53.xml v1218>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.488 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.336+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bba [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-6.xml auth v1206 ap=1 s=5902 n(v0 rc2023-07-21T22:34:45.694999+0000 b5902 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd432b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bba:[<0x40000001129/comment-6.xml v1201>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bba:[<0x40000001129/comment-6.xml v1206>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.488 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.336+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000011c3 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-11.jpg auth v1232 ap=1 s=61019 n(v0 rc2023-07-21T22:30:31.303169+0000 b61019 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd7aed600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000011c3:[<0x40000001129/picture-11.jpg v779>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000011c3:[<0x40000001129/picture-11.jpg v1232>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.488 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.336+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010cf [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-5.xml auth v1230 ap=1 s=8677 n(v0 rc2023-07-21T22:30:30.530182+0000 b8677 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd80c7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010cf:[<0x40000001129/article-5.xml v538>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010cf:[<0x40000001129/article-5.xml v1230>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.489 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.336+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001038 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-34.xml auth v1238 ap=1 s=26365 n(v0 rc2023-07-21T22:30:29.898192+0000 b26365 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd432ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001038:[<0x40000001129/article-34.xml v306>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001038:[<0x40000001129/article-34.xml v1238>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.489 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.337+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b9a [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-35.xml auth v1214 ap=1 s=4126 n(v0 rc2023-07-21T22:34:44.577017+0000 b4126 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd432a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b9a:[<0x40000001129/comment-35.xml v1152>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b9a:[<0x40000001129/comment-35.xml v1214>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.489 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.337+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ba1 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-28.xml auth v1212 ap=1 s=5806 n(v0 rc2023-07-21T22:34:44.764014+0000 b5806 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd432a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ba1:[<0x40000001129/comment-28.xml v1160>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001ba1:[<0x40000001129/comment-28.xml v1212>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.489 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.337+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b82 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-57.xml auth v1220 ap=1 s=5481 n(v0 rc2023-07-21T22:34:43.743031+0000 b5481 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd4329b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b82:[<0x40000001129/comment-57.xml v1115>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b82:[<0x40000001129/comment-57.xml v1220>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.490 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.337+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000197b [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-78.xml auth v1069 ap=1 s=31539 n(v0 rc2023-07-21T22:34:20.984402+0000 b31539 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd4329600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000197b:[<0x40000001129/article-78.xml v1065>,<0x4000000033b/blog-21 v155439>,<0x400000001fb/blogtest_in v2345>,<0x400000001f6/src v2153>,<0x40000000000/blogbench-1.0 v1861>,<0x100000005e2/tmp v456>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v76>,<0x10000000001/sv_1 v100>,<0x10000000000/_nogroup v85>,<0x1/volumes v83>]//[]","memoryvalue":"(3)0x1000000197b:[<0x40000001129/article-78.xml v1069>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.493 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.337+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001054 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-29.xml auth v1236 ap=1 s=15048 n(v0 rc2023-07-21T22:30:30.014190+0000 b15048 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd4329080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001054:[<0x40000001129/article-29.xml v346>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001054:[<0x40000001129/article-29.xml v1236>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.493 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.337+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fac [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-60.xml auth v1242 ap=1 s=4522 n(v0 rc2023-07-21T22:30:29.297202+0000 b4522 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd4328b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fac:[<0x40000001129/article-60.xml v100>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fac:[<0x40000001129/article-60.xml v1242>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.493 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.338+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b7d [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-61.xml auth v1112 ap=1 s=4438 n(v0 rc2023-07-21T22:34:43.568034+0000 b4438 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd4328580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b7d:[<0x40000001129/comment-61.xml v1112>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b7d:[<0x40000001129/comment-61.xml v1112>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.493 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.338+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001077 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-22.xml auth v1234 ap=1 s=7330 n(v0 rc2023-07-21T22:30:30.151188+0000 b7330 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd58bf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001077:[<0x40000001129/article-22.xml v402>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001077:[<0x40000001129/article-22.xml v1234>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.493 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.338+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b97 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-38.xml auth v1216 ap=1 s=7345 n(v0 rc2023-07-21T22:34:44.510018+0000 b7345 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd4328000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b97:[<0x40000001129/comment-38.xml v1142>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b97:[<0x40000001129/comment-38.xml v1216>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.494 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.338+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b74 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-68.xml auth v1222 ap=1 s=4964 n(v0 rc2023-07-21T22:34:43.277039+0000 b4964 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd382a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b74:[<0x40000001129/comment-68.xml v1095>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b74:[<0x40000001129/comment-68.xml v1222>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.494 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.338+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fdb [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-49.xml auth v1240 ap=1 s=10047 n(v0 rc2023-07-21T22:30:29.521198+0000 b10047 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd3ff8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fdb:[<0x40000001129/article-49.xml v188>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fdb:[<0x40000001129/article-49.xml v1240>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.494 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.338+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bb8 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-8.xml auth v1208 ap=1 s=5633 n(v0 rc2023-07-21T22:34:45.619000+0000 b5633 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd3ff7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bb8:[<0x40000001129/comment-8.xml v1196>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bb8:[<0x40000001129/comment-8.xml v1208>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.494 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.339+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ca0 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-80.xml auth v1033 ap=1 s=4711 n(v0 rc2023-07-21T22:32:50.597881+0000 b4711 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd3ff7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ca0:[<0x40000001129/comment-80.xml v898>,<0x4000000033b/blog-21 v134411>,<0x400000001fb/blogtest_in v1820>,<0x400000001f6/src v1957>,<0x40000000000/blogbench-1.0 v1705>,<0x100000005e2/tmp v298>,<0x100000001fb/client.0 v297>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v56>,<0x10000000001/sv_1 v80>,<0x10000000000/_nogroup v65>,<0x1/volumes v63>]//[]","memoryvalue":"(3)0x10000000ca0:[<0x40000001129/comment-80.xml v1033>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.494 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.339+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000113b [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-23.jpg auth v1212 ap=1 s=26311 n(v0 rc2023-07-21T22:30:30.857176+0000 b26311 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd6159700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000113b:[<0x40000001129/picture-23.jpg v785>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000113b:[<0x40000001129/picture-23.jpg v1212>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.494 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.339+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000109e [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-14.xml auth v1204 ap=1 s=19868 n(v0 rc2023-07-21T22:30:30.329185+0000 b19868 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd79f4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000109e:[<0x40000001129/article-14.xml v466>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000109e:[<0x40000001129/article-14.xml v1204>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.494 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.339+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fee [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-46.xml auth v1220 ap=1 s=23532 n(v0 rc2023-07-21T22:30:29.625197+0000 b23532 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd3851600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fee:[<0x40000001129/article-46.xml v787>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fee:[<0x40000001129/article-46.xml v1220>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.495 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.340+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b7c [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-62.xml auth v1188 ap=1 s=6139 n(v0 rc2023-07-21T22:34:43.482035+0000 b6139 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd753cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b7c:[<0x40000001129/comment-62.xml v1095>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b7c:[<0x40000001129/comment-62.xml v1188>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.495 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.340+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000545 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-91.xml auth v896 ap=1 s=9872 n(v0 rc2023-07-21T22:32:21.364360+0000 b9872 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd3e4d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000545:[<0x40000001129/article-91.xml v896>,<0x4000000033b/blog-21 v134411>,<0x400000001fb/blogtest_in v1717>,<0x400000001f6/src v1936>,<0x40000000000/blogbench-1.0 v1673>,<0x100000005e2/tmp v281>,<0x100000001fb/client.0 v264>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v52>,<0x10000000001/sv_1 v76>,<0x10000000000/_nogroup v61>,<0x1/volumes v59>]//[]","memoryvalue":"(3)0x10000000545:[<0x40000001129/article-91.xml v896>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.495 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.340+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000544 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-66.jpg auth v892 ap=1 s=24767 n(v0 rc2023-07-21T22:32:21.332361+0000 b24767 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd38bac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000544:[<0x40000001129/picture-66.jpg v892>,<0x4000000033b/blog-21 v134411>,<0x400000001fb/blogtest_in v1717>,<0x400000001f6/src v1936>,<0x40000000000/blogbench-1.0 v1673>,<0x100000005e2/tmp v281>,<0x100000001fb/client.0 v264>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v52>,<0x10000000001/sv_1 v76>,<0x10000000000/_nogroup v61>,<0x1/volumes v59>]//[]","memoryvalue":"(3)0x10000000544:[<0x40000001129/picture-66.jpg v892>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.495 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.340+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fe8 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-47.xml auth v1222 ap=1 s=1964 n(v0 rc2023-07-21T22:30:29.568198+0000 b1964 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd38f9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fe8:[<0x40000001129/article-47.xml v204>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fe8:[<0x40000001129/article-47.xml v1222>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.495 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.340+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000f77 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-71.xml auth v1228 ap=1 s=15798 n(v0 rc2023-07-21T22:30:29.089205+0000 b15798 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd7b57b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000f77:[<0x40000001129/article-71.xml v12>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000f77:[<0x40000001129/article-71.xml v1228>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.495 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.340+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010b1 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-11.xml auth v1200 ap=1 s=27944 n(v0 rc2023-07-21T22:30:30.416184+0000 b27944 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd6158c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010b1:[<0x40000001129/article-11.xml v490>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010b1:[<0x40000001129/article-11.xml v1200>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.496 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.341+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b7f [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-59.xml auth v1186 ap=1 s=8646 n(v0 rc2023-07-21T22:34:43.631033+0000 b8646 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd4490c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b7f:[<0x40000001129/comment-59.xml v1099>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b7f:[<0x40000001129/comment-59.xml v1186>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.496 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.341+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001148 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-22.jpg auth v1210 ap=1 s=43413 n(v0 rc2023-07-21T22:30:30.879176+0000 b43413 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd343b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001148:[<0x40000001129/picture-22.jpg v783>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001148:[<0x40000001129/picture-22.jpg v1210>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.496 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.341+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001094 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-16.xml auth v1208 ap=1 s=21799 n(v0 rc2023-07-21T22:30:30.286186+0000 b21799 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd9175700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001094:[<0x40000001129/article-16.xml v450>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001094:[<0x40000001129/article-16.xml v1208>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.496 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.341+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b6a [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-77.xml auth v1190 ap=1 s=4127 n(v0 rc2023-07-21T22:34:42.830046+0000 b4127 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd4490100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b6a:[<0x40000001129/comment-77.xml v1077>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b6a:[<0x40000001129/comment-77.xml v1190>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.496 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.341+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fbc [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-54.xml auth v1226 ap=1 s=4539 n(v0 rc2023-07-21T22:30:29.400200+0000 b4539 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd42dbb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fbc:[<0x40000001129/article-54.xml v148>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fbc:[<0x40000001129/article-54.xml v1226>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.496 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.341+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001064 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-26.xml auth v1216 ap=1 s=15615 n(v0 rc2023-07-21T22:30:30.070189+0000 b15615 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd79f5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001064:[<0x40000001129/article-26.xml v370>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001064:[<0x40000001129/article-26.xml v1216>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.496 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.342+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000118f [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-15.jpg auth v1206 ap=1 s=61651 n(v0 rc2023-07-21T22:30:31.094172+0000 b61651 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd7c52680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000118f:[<0x40000001129/picture-15.jpg v781>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000118f:[<0x40000001129/picture-15.jpg v1206>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.497 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.342+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010b4 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-10.xml auth v1198 ap=1 s=6825 n(v0 rc2023-07-21T22:30:30.434183+0000 b6825 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd7aef180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010b4:[<0x40000001129/article-10.xml v498>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010b4:[<0x40000001129/article-10.xml v1198>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.497 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.342+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000011b6 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-12.jpg auth v1202 ap=1 s=40785 n(v0 rc2023-07-21T22:30:31.230170+0000 b40785 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd79f5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000011b6:[<0x40000001129/picture-12.jpg v779>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000011b6:[<0x40000001129/picture-12.jpg v1202>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.497 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.342+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b8a [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-50.xml auth v1184 ap=1 s=4265 n(v0 rc2023-07-21T22:34:44.014027+0000 b4265 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd7561080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b8a:[<0x40000001129/comment-50.xml v1120>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b8a:[<0x40000001129/comment-50.xml v1184>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.497 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.342+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fe3 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-48.xml auth v1224 ap=1 s=9302 n(v0 rc2023-07-21T22:30:29.536198+0000 b9302 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd9449080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fe3:[<0x40000001129/article-48.xml v196>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fe3:[<0x40000001129/article-48.xml v1224>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.497 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.342+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b8c [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-48.xml auth v1182 ap=1 s=5860 n(v0 rc2023-07-21T22:34:44.078025+0000 b5860 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd6157600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b8c:[<0x40000001129/comment-48.xml v1123>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b8c:[<0x40000001129/comment-48.xml v1182>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.498 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.342+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010d5 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-4.xml auth v1196 ap=1 s=13037 n(v0 rc2023-07-21T22:30:30.551181+0000 b13037 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd944b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010d5:[<0x40000001129/article-4.xml v546>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010d5:[<0x40000001129/article-4.xml v1196>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.498 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.342+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001068 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-25.xml auth v1214 ap=1 s=14482 n(v0 rc2023-07-21T22:30:30.090189+0000 b14482 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd5f5f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001068:[<0x40000001129/article-25.xml v378>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001068:[<0x40000001129/article-25.xml v1214>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.498 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.343+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000101f [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-38.xml auth v1218 ap=1 s=7540 n(v0 rc2023-07-21T22:30:29.806194+0000 b7540 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd9173b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000101f:[<0x40000001129/article-38.xml v274>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000101f:[<0x40000001129/article-38.xml v1218>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.498 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.343+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b9e [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-31.xml auth v1180 ap=1 s=8199 n(v0 rc2023-07-21T22:34:44.688016+0000 b8199 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd79f3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b9e:[<0x40000001129/comment-31.xml v1163>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b9e:[<0x40000001129/comment-31.xml v1180>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.499 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.424+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000011a9 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-13.jpg auth v1243 ap=1 s=62782 n(v0 rc2023-07-21T22:30:31.183171+0000 b62782 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd80c7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000011a9:[<0x40000001129/picture-13.jpg v783>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000011a9:[<0x40000001129/picture-13.jpg v1243>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.499 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.424+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fad [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-59.xml auth v1263 ap=1 s=12616 n(v0 rc2023-07-21T22:30:29.307202+0000 b12616 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd715c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fad:[<0x40000001129/article-59.xml v108>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fad:[<0x40000001129/article-59.xml v1263>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.499 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.424+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b83 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-56.xml auth v1229 ap=1 s=2063 n(v0 rc2023-07-21T22:34:43.771030+0000 b2063 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd7025600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b83:[<0x40000001129/comment-56.xml v1120>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b83:[<0x40000001129/comment-56.xml v1229>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.500 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.424+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000116f [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-18.jpg auth v1247 ap=1 s=33144 n(v0 rc2023-07-21T22:30:30.982174+0000 b33144 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd3929b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000116f:[<0x40000001129/picture-18.jpg v787>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000116f:[<0x40000001129/picture-18.jpg v1247>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.500 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.424+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000108a [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-18.xml auth v1249 ap=1 s=11865 n(v0 rc2023-07-21T22:30:30.238186+0000 b11865 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd5cc5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000108a:[<0x40000001129/article-18.xml v434>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000108a:[<0x40000001129/article-18.xml v1249>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.500 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.424+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bbb [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-5.xml auth v1221 ap=1 s=9939 n(v0 rc2023-07-21T22:34:45.812997+0000 b9939 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd7003180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bbb:[<0x40000001129/comment-5.xml v1212>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bbb:[<0x40000001129/comment-5.xml v1221>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.501 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.424+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001167 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-19.jpg auth v1251 ap=1 s=56482 n(v0 rc2023-07-21T22:30:30.954175+0000 b56482 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd4790c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001167:[<0x40000001129/picture-19.jpg v789>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001167:[<0x40000001129/picture-19.jpg v1251>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.501 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.424+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ba0 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-29.xml auth v1227 ap=1 s=4565 n(v0 rc2023-07-21T22:34:44.743015+0000 b4565 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd8fec580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ba0:[<0x40000001129/comment-29.xml v1171>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001ba0:[<0x40000001129/comment-29.xml v1227>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.501 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.425+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fce [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-51.xml auth v1261 ap=1 s=19690 n(v0 rc2023-07-21T22:30:29.490199+0000 b19690 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd6e93b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fce:[<0x40000001129/article-51.xml v172>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fce:[<0x40000001129/article-51.xml v1261>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.502 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.425+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000105d [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-27.xml auth v1255 ap=1 s=23809 n(v0 rc2023-07-21T22:30:30.050190+0000 b23809 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd934ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000105d:[<0x40000001129/article-27.xml v362>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000105d:[<0x40000001129/article-27.xml v1255>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.502 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.425+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000121f [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-4.jpg auth v1237 ap=1 s=16075 n(v0 rc2023-07-21T22:30:31.771161+0000 b16075 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd85b6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000121f:[<0x40000001129/picture-4.jpg v730>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000121f:[<0x40000001129/picture-4.jpg v1237>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.502 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.426+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000122e [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-2.jpg auth v1235 ap=1 s=53423 n(v0 rc2023-07-21T22:30:31.856160+0000 b53423 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd8325700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000122e:[<0x40000001129/picture-2.jpg v746>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000122e:[<0x40000001129/picture-2.jpg v1235>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.503 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.426+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000f7c [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-70.xml auth v1269 ap=1 s=26770 n(v0 rc2023-07-21T22:30:29.110205+0000 b26770 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd6ad3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000f7c:[<0x40000001129/article-70.xml v20>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000f7c:[<0x40000001129/article-70.xml v1269>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.503 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.426+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001080 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-20.xml auth v1253 ap=1 s=15175 n(v0 rc2023-07-21T22:30:30.190187+0000 b15175 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd9659700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001080:[<0x40000001129/article-20.xml v418>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001080:[<0x40000001129/article-20.xml v1253>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.503 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.426+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000119c [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-14.jpg auth v1245 ap=1 s=15302 n(v0 rc2023-07-21T22:30:31.135172+0000 b15302 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd4791700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000119c:[<0x40000001129/picture-14.jpg v785>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000119c:[<0x40000001129/picture-14.jpg v1245>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.504 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.427+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000150f [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-83.xml auth v1083 ap=1 s=1487 n(v0 rc2023-07-21T22:33:58.722764+0000 b1487 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd80b4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000150f:[<0x40000001129/article-83.xml v1071>,<0x4000000033b/blog-21 v155439>,<0x400000001fb/blogtest_in v2268>,<0x400000001f6/src v2111>,<0x40000000000/blogbench-1.0 v1829>,<0x100000005e2/tmp v424>,<0x100000001fb/client.0 v420>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v72>,<0x10000000001/sv_1 v96>,<0x10000000000/_nogroup v81>,<0x1/volumes v79>]//[]","memoryvalue":"(3)0x1000000150f:[<0x40000001129/article-83.xml v1083>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.504 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.427+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000faa [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-61.xml auth v1265 ap=1 s=28901 n(v0 rc2023-07-21T22:30:29.277202+0000 b28901 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd7000b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000faa:[<0x40000001129/article-61.xml v92>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000faa:[<0x40000001129/article-61.xml v1265>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.504 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.427+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ee5 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-41.xml auth v1053 ap=1 s=5106 n(v0 rc2023-07-21T22:33:04.103659+0000 b5106 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd80c3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ee5:[<0x40000001129/article-41.xml v1040>,<0x4000000033b/blog-21 v139069>,<0x400000001fb/blogtest_in v1930>,<0x400000001f6/src v1995>,<0x40000000000/blogbench-1.0 v1722>,<0x100000005e2/tmp v328>,<0x100000001fb/client.0 v313>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v58>,<0x10000000001/sv_1 v82>,<0x10000000000/_nogroup v67>,<0x1/volumes v65>]//[]","memoryvalue":"(3)0x10000000ee5:[<0x40000001129/article-41.xml v1053>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.505 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.428+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000ff7 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-45.xml auth v1259 ap=1 s=6587 n(v0 rc2023-07-21T22:30:29.645196+0000 b6587 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd501d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000ff7:[<0x40000001129/article-45.xml v218>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000ff7:[<0x40000001129/article-45.xml v1259>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.505 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.428+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000f84 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-69.xml auth v1267 ap=1 s=1061 n(v0 rc2023-07-21T22:30:29.133205+0000 b1061 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd9d30c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000f84:[<0x40000001129/article-69.xml v28>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000f84:[<0x40000001129/article-69.xml v1267>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.505 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.428+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001828 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-61.jpg auth v1081 ap=1 s=44397 n(v0 rc2023-07-21T22:34:18.644440+0000 b44397 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd8132100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001828:[<0x40000001129/picture-61.jpg v1077>,<0x4000000033b/blog-21 v155439>,<0x400000001fb/blogtest_in v2345>,<0x400000001f6/src v2153>,<0x40000000000/blogbench-1.0 v1861>,<0x100000005e2/tmp v456>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v76>,<0x10000000001/sv_1 v100>,<0x10000000000/_nogroup v85>,<0x1/volumes v83>]//[]","memoryvalue":"(3)0x10000001828:[<0x40000001129/picture-61.jpg v1081>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.505 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.428+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000011f3 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-7.jpg auth v1239 ap=1 s=51575 n(v0 rc2023-07-21T22:30:31.599164+0000 b51575 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd8bf2b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000011f3:[<0x40000001129/picture-7.jpg v779>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000011f3:[<0x40000001129/picture-7.jpg v1239>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.506 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.428+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000104c [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-31.xml auth v1257 ap=1 s=20991 n(v0 rc2023-07-21T22:30:29.970191+0000 b20991 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd9058580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000104c:[<0x40000001129/article-31.xml v330>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000104c:[<0x40000001129/article-31.xml v1257>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.506 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.428+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000011dc [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-9.jpg auth v1241 ap=1 s=53526 n(v0 rc2023-07-21T22:30:31.449166+0000 b53526 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd3a17180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000011dc:[<0x40000001129/picture-9.jpg v781>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000011dc:[<0x40000001129/picture-9.jpg v1241>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.506 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.429+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bb5 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-11.xml auth v1223 ap=1 s=3731 n(v0 rc2023-07-21T22:34:45.534002+0000 b3731 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd7528680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bb5:[<0x40000001129/comment-11.xml v1190>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bb5:[<0x40000001129/comment-11.xml v1223>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.506 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.429+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001baa [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-20.xml auth v1225 ap=1 s=5541 n(v0 rc2023-07-21T22:34:45.003010+0000 b5541 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd7000000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001baa:[<0x40000001129/comment-20.xml v1187>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001baa:[<0x40000001129/comment-20.xml v1225>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.507 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.429+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fb7 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-55.xml auth v1265 ap=1 s=5743 n(v0 rc2023-07-21T22:30:29.385201+0000 b5743 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd5a0f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fb7:[<0x40000001129/article-55.xml v140>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fb7:[<0x40000001129/article-55.xml v1265>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.507 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.429+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000111a [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-25.jpg auth v1255 ap=1 s=55882 n(v0 rc2023-07-21T22:30:30.798177+0000 b55882 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd8c7ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000111a:[<0x40000001129/picture-25.jpg v781>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000111a:[<0x40000001129/picture-25.jpg v1255>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.507 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.429+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010bf [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-8.xml auth v1249 ap=1 s=19855 n(v0 rc2023-07-21T22:30:30.464183+0000 b19855 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd7ace100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010bf:[<0x40000001129/article-8.xml v514>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010bf:[<0x40000001129/article-8.xml v1249>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.508 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.429+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b8d [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-47.xml auth v1227 ap=1 s=2263 n(v0 rc2023-07-21T22:34:44.110025+0000 b2263 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd637db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b8d:[<0x40000001129/comment-47.xml v1124>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b8d:[<0x40000001129/comment-47.xml v1227>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.508 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.429+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b8e [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-46.xml auth v1225 ap=1 s=8236 n(v0 rc2023-07-21T22:34:44.136025+0000 b8236 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd70ad180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b8e:[<0x40000001129/comment-46.xml v1127>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b8e:[<0x40000001129/comment-46.xml v1225>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.509 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.430+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000f88 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-68.xml auth v1269 ap=1 s=11987 n(v0 rc2023-07-21T22:30:29.151204+0000 b11987 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd98be100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000f88:[<0x40000001129/article-68.xml v36>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000f88:[<0x40000001129/article-68.xml v1269>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.509 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.430+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b72 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-70.xml auth v1233 ap=1 s=7161 n(v0 rc2023-07-21T22:34:43.224039+0000 b7161 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd7563700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b72:[<0x40000001129/comment-70.xml v1095>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b72:[<0x40000001129/comment-70.xml v1233>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.509 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.430+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000011e9 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-8.jpg auth v1247 ap=1 s=18013 n(v0 rc2023-07-21T22:30:31.520165+0000 b18013 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd79b2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000011e9:[<0x40000001129/picture-8.jpg v779>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000011e9:[<0x40000001129/picture-8.jpg v1247>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.509 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.430+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ef6 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-26.jpg auth v1067 ap=1 s=18566 n(v0 rc2023-07-21T22:33:10.516554+0000 b18566 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd67f9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ef6:[<0x40000001129/picture-26.jpg v1062>,<0x4000000033b/blog-21 v143371>,<0x400000001fb/blogtest_in v1989>,<0x400000001f6/src v2012>,<0x40000000000/blogbench-1.0 v1739>,<0x100000005e2/tmp v343>,<0x100000001fb/client.0 v344>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v62>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v69>]//[]","memoryvalue":"(3)0x10000000ef6:[<0x40000001129/picture-26.jpg v1067>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.509 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.430+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001044 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-32.xml auth v1261 ap=1 s=26018 n(v0 rc2023-07-21T22:30:29.949191+0000 b26018 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd6f7c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001044:[<0x40000001129/article-32.xml v322>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001044:[<0x40000001129/article-32.xml v1261>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.509 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.430+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b73 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-69.xml auth v1231 ap=1 s=6869 n(v0 rc2023-07-21T22:34:43.258039+0000 b6869 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd67f2b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b73:[<0x40000001129/comment-69.xml v1098>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b73:[<0x40000001129/comment-69.xml v1231>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.510 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.430+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000107b [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-21.xml auth v1253 ap=1 s=1738 n(v0 rc2023-07-21T22:30:30.170188+0000 b1738 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd7a32c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000107b:[<0x40000001129/article-21.xml v410>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000107b:[<0x40000001129/article-21.xml v1253>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.510 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.430+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001050 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-30.xml auth v1259 ap=1 s=32457 n(v0 rc2023-07-21T22:30:29.994191+0000 b32457 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd38df180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001050:[<0x40000001129/article-30.xml v338>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001050:[<0x40000001129/article-30.xml v1259>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.510 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.431+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b6d [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-74.xml auth v1237 ap=1 s=4474 n(v0 rc2023-07-21T22:34:42.876045+0000 b4474 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd8616000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b6d:[<0x40000001129/comment-74.xml v1089>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b6d:[<0x40000001129/comment-74.xml v1237>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.510 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.431+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b99 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-36.xml auth v1223 ap=1 s=10205 n(v0 rc2023-07-21T22:34:44.558018+0000 b10205 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd61e1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b99:[<0x40000001129/comment-36.xml v1149>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b99:[<0x40000001129/comment-36.xml v1223>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.510 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.431+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010a4 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-13.xml auth v1251 ap=1 s=21292 n(v0 rc2023-07-21T22:30:30.360184+0000 b21292 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd8323600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010a4:[<0x40000001129/article-13.xml v474>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010a4:[<0x40000001129/article-13.xml v1251>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.510 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.431+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b7a [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-64.xml auth v1229 ap=1 s=10014 n(v0 rc2023-07-21T22:34:43.384037+0000 b10014 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd81bd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b7a:[<0x40000001129/comment-64.xml v1104>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b7a:[<0x40000001129/comment-64.xml v1229>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.511 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.431+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b6f [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-72.xml auth v1235 ap=1 s=5337 n(v0 rc2023-07-21T22:34:43.099041+0000 b5337 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd9740b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b6f:[<0x40000001129/comment-72.xml v1092>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b6f:[<0x40000001129/comment-72.xml v1235>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.511 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.431+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bbc [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-4.xml auth v1217 ap=1 s=2943 n(v0 rc2023-07-21T22:34:45.889996+0000 b2943 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd98bcb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bbc:[<0x40000001129/comment-4.xml v1212>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bbc:[<0x40000001129/comment-4.xml v1217>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.511 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.432+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fd3 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-50.xml auth v1263 ap=1 s=12608 n(v0 rc2023-07-21T22:30:29.507199+0000 b12608 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd391e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fd3:[<0x40000001129/article-50.xml v180>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fd3:[<0x40000001129/article-50.xml v1263>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.511 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.432+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010c9 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-6.xml auth v1245 ap=1 s=22206 n(v0 rc2023-07-21T22:30:30.503182+0000 b22206 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd38b2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010c9:[<0x40000001129/article-6.xml v530>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010c9:[<0x40000001129/article-6.xml v1245>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.511 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.432+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010df [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-2.xml auth v1243 ap=1 s=24056 n(v0 rc2023-07-21T22:30:30.601180+0000 b24056 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd8a8b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010df:[<0x40000001129/article-2.xml v562>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010df:[<0x40000001129/article-2.xml v1243>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.511 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.432+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bb2 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-14.xml auth v1219 ap=1 s=7413 n(v0 rc2023-07-21T22:34:45.457003+0000 b7413 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd71d7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bb2:[<0x40000001129/comment-14.xml v1194>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bb2:[<0x40000001129/comment-14.xml v1219>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.511 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.433+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b75 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-67.xml auth v1101 ap=1 s=8831 n(v0 rc2023-07-21T22:34:43.300038+0000 b8831 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd752cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b75:[<0x40000001129/comment-67.xml v1101>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b75:[<0x40000001129/comment-67.xml v1101>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.512 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.433+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010eb [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-29.jpg auth v1257 ap=1 s=30864 n(v0 rc2023-07-21T22:30:30.651180+0000 b30864 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd7025b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010eb:[<0x40000001129/picture-29.jpg v578>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010eb:[<0x40000001129/picture-29.jpg v1257>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.512 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.433+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bac [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-19.xml auth v1221 ap=1 s=7886 n(v0 rc2023-07-21T22:34:45.095009+0000 b7886 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd6e93080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bac:[<0x40000001129/comment-19.xml v1191>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bac:[<0x40000001129/comment-19.xml v1221>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.512 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.433+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fb4 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-56.xml auth v1267 ap=1 s=16309 n(v0 rc2023-07-21T22:30:29.363201+0000 b16309 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd66c8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fb4:[<0x40000001129/article-56.xml v132>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fb4:[<0x40000001129/article-56.xml v1267>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.512 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.433+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ba8 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-22.xml auth v1231 ap=1 s=7207 n(v0 rc2023-07-21T22:34:44.912012+0000 b7207 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd5f5ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ba8:[<0x40000001129/comment-22.xml v1180>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001ba8:[<0x40000001129/comment-22.xml v1231>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.512 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.433+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bbe [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-2.xml auth v1221 ap=1 s=4304 n(v0 rc2023-07-21T22:34:45.939995+0000 b4304 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd79f3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bbe:[<0x40000001129/comment-2.xml v1216>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bbe:[<0x40000001129/comment-2.xml v1221>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.512 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.434+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b90 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-44.xml auth v1237 ap=1 s=8714 n(v0 rc2023-07-21T22:34:44.194024+0000 b8714 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd7c52100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b90:[<0x40000001129/comment-44.xml v1137>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b90:[<0x40000001129/comment-44.xml v1237>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.513 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.434+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001003 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-43.xml auth v1265 ap=1 s=23295 n(v0 rc2023-07-21T22:30:29.690196+0000 b23295 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd7563180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001003:[<0x40000001129/article-43.xml v234>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001003:[<0x40000001129/article-43.xml v1265>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.513 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.434+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000103f [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-33.xml auth v1263 ap=1 s=21120 n(v0 rc2023-07-21T22:30:29.926192+0000 b21120 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd7562100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000103f:[<0x40000001129/article-33.xml v314>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000103f:[<0x40000001129/article-33.xml v1263>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.513 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.434+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001212 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-5.jpg auth v1259 ap=1 s=47018 n(v0 rc2023-07-21T22:30:31.755161+0000 b47018 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd5364680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001212:[<0x40000001129/picture-5.jpg v779>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001212:[<0x40000001129/picture-5.jpg v1259>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.513 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.434+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ba2 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-27.xml auth v1233 ap=1 s=5070 n(v0 rc2023-07-21T22:34:44.787014+0000 b5070 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd8c3cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ba2:[<0x40000001129/comment-27.xml v1164>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001ba2:[<0x40000001129/comment-27.xml v1233>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.513 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.435+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b6b [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-76.xml auth v1245 ap=1 s=4675 n(v0 rc2023-07-21T22:34:42.832046+0000 b4675 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd80c8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b6b:[<0x40000001129/comment-76.xml v1068>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b6b:[<0x40000001129/comment-76.xml v1245>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.513 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.435+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fb2 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-57.xml auth v1271 ap=1 s=12654 n(v0 rc2023-07-21T22:30:29.351201+0000 b12654 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd63d1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fb2:[<0x40000001129/article-57.xml v124>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fb2:[<0x40000001129/article-57.xml v1271>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.514 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.435+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b87 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-52.xml auth v1241 ap=1 s=5780 n(v0 rc2023-07-21T22:34:43.922028+0000 b5780 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd7c50b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b87:[<0x40000001129/comment-52.xml v1131>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b87:[<0x40000001129/comment-52.xml v1241>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.514 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.435+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b76 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-66.xml auth v1243 ap=1 s=2907 n(v0 rc2023-07-21T22:34:43.325038+0000 b2907 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd9471600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b76:[<0x40000001129/comment-66.xml v1073>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b76:[<0x40000001129/comment-66.xml v1243>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.514 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.435+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bb9 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-7.xml auth v1223 ap=1 s=3140 n(v0 rc2023-07-21T22:34:45.634000+0000 b3140 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd6156b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bb9:[<0x40000001129/comment-7.xml v1208>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bb9:[<0x40000001129/comment-7.xml v1223>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.514 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.436+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bb1 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-15.xml auth v1227 ap=1 s=3701 n(v0 rc2023-07-21T22:34:45.426004+0000 b3701 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd80c9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bb1:[<0x40000001129/comment-15.xml v1190>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bb1:[<0x40000001129/comment-15.xml v1227>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.514 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.436+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000caa [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-58.xml auth v1028 ap=1 s=14131 n(v0 rc2023-07-21T22:32:50.916876+0000 b14131 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd88f3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000caa:[<0x40000001129/article-58.xml v1028>,<0x4000000033b/blog-21 v139069>,<0x400000001fb/blogtest_in v1930>,<0x400000001f6/src v1995>,<0x40000000000/blogbench-1.0 v1722>,<0x100000005e2/tmp v328>,<0x100000001fb/client.0 v313>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v58>,<0x10000000001/sv_1 v82>,<0x10000000000/_nogroup v67>,<0x1/volumes v65>]//[]","memoryvalue":"(3)0x10000000caa:[<0x40000001129/article-58.xml v1028>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.514 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.436+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fc1 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-53.xml auth v1269 ap=1 s=14584 n(v0 rc2023-07-21T22:30:29.422200+0000 b14584 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd5582b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fc1:[<0x40000001129/article-53.xml v156>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fc1:[<0x40000001129/article-53.xml v1269>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.514 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.436+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001baf [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-17.xml auth v1229 ap=1 s=5538 n(v0 rc2023-07-21T22:34:45.330005+0000 b5538 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd6b37600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001baf:[<0x40000001129/comment-17.xml v1187>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001baf:[<0x40000001129/comment-17.xml v1229>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.515 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.437+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b91 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-43.xml auth v1235 ap=1 s=8450 n(v0 rc2023-07-21T22:34:44.225023+0000 b8450 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd7001b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b91:[<0x40000001129/comment-43.xml v1140>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b91:[<0x40000001129/comment-43.xml v1235>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.515 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.437+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bb7 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-9.xml auth v1225 ap=1 s=5890 n(v0 rc2023-07-21T22:34:45.591001+0000 b5890 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd715a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bb7:[<0x40000001129/comment-9.xml v1197>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bb7:[<0x40000001129/comment-9.xml v1225>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.515 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.437+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fc6 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-52.xml auth v1267 ap=1 s=29197 n(v0 rc2023-07-21T22:30:29.459199+0000 b29197 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd8be4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fc6:[<0x40000001129/article-52.xml v164>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fc6:[<0x40000001129/article-52.xml v1267>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.515 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.437+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b8f [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-45.xml auth v1239 ap=1 s=6813 n(v0 rc2023-07-21T22:34:44.161024+0000 b6813 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd7526580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b8f:[<0x40000001129/comment-45.xml v1134>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b8f:[<0x40000001129/comment-45.xml v1239>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.515 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.437+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000021c9 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-46.jpg auth v1256 ap=1 s=57435 n(v0 rc2023-07-21T22:35:15.107520+0000 b57435 1=1+0) (iversion lock) caps={15357=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd9b9cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000021c9:[<0x40000001129/picture-46.jpg v1250>,<0x4000000033b/blog-21 v174133>,<0x400000001fb/blogtest_in v2617>,<0x400000001f6/src v2246>,<0x40000000000/blogbench-1.0 v1936>,<0x100000005e2/tmp v528>,<0x100000001fb/client.0 v524>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v86>,<0x10000000001/sv_1 v110>,<0x10000000000/_nogroup v95>,<0x1/volumes v93>]//[]","memoryvalue":"(3)0x100000021c9:[<0x40000001129/picture-46.jpg v1256>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.515 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.438+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b98 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-37.xml auth v1225 ap=1 s=5118 n(v0 rc2023-07-21T22:34:44.538018+0000 b5118 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd8463180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b98:[<0x40000001129/comment-37.xml v1145>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b98:[<0x40000001129/comment-37.xml v1225>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.516 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.438+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b70 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-71.xml auth v1233 ap=1 s=8566 n(v0 rc2023-07-21T22:34:43.205040+0000 b8566 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd6e95180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b70:[<0x40000001129/comment-71.xml v1093>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b70:[<0x40000001129/comment-71.xml v1233>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.516 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.438+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000011d0 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-10.jpg auth v1265 ap=1 s=40524 n(v0 rc2023-07-21T22:30:31.373168+0000 b40524 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd4790100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000011d0:[<0x40000001129/picture-10.jpg v781>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000011d0:[<0x40000001129/picture-10.jpg v1265>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.516 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.438+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001235 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-1.jpg auth{3=1} v758 ap=1 s=51509 n(v0 rc2023-07-21T22:30:31.896159+0000 b51509 1=1+0) (iversion lock) | replicated=1 authpin=1 scrubqueue=0 0x556fd6bb0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001235:[<0x40000001129/picture-1.jpg v758>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001235:[<0x40000001129/picture-1.jpg v758>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.516 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.438+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001bb0 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-16.xml auth v1204 ap=1 s=5785 n(v0 rc2023-07-21T22:34:45.386004+0000 b5785 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd9117b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001bb0:[<0x40000001129/comment-16.xml v1204>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001bb0:[<0x40000001129/comment-16.xml v1204>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.516 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.438+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010b9 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-9.xml auth v1263 ap=1 s=30034 n(v0 rc2023-07-21T22:30:30.454183+0000 b30034 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd392ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010b9:[<0x40000001129/article-9.xml v506>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010b9:[<0x40000001129/article-9.xml v1263>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.516 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.439+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000ef7 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-44.xml auth v1050 ap=1 s=4176 n(v0 rc2023-07-21T22:33:10.536554+0000 b4176 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd8c7a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000ef7:[<0x40000001129/article-44.xml v1050>,<0x4000000033b/blog-21 v143371>,<0x400000001fb/blogtest_in v1989>,<0x400000001f6/src v2012>,<0x40000000000/blogbench-1.0 v1739>,<0x100000005e2/tmp v343>,<0x100000001fb/client.0 v344>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v62>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v69>]//[]","memoryvalue":"(3)0x10000000ef7:[<0x40000001129/article-44.xml v1050>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.517 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.439+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001201 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-6.jpg auth v1259 ap=1 s=25532 n(v0 rc2023-07-21T22:30:31.663163+0000 b25532 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd391f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001201:[<0x40000001129/picture-6.jpg v779>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001201:[<0x40000001129/picture-6.jpg v1259>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.517 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.439+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007d9 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-99.xml auth v1281 ap=1 s=9767 n(v0 rc2023-07-21T22:30:39.131040+0000 b9767 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd8d01700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007d9:[<0x40000001129/article-99.xml v775>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x100000007d9:[<0x40000001129/article-99.xml v1281>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.517 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.439+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000f98 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-65.xml auth v1277 ap=1 s=21429 n(v0 rc2023-07-21T22:30:29.212203+0000 b21429 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd960ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000f98:[<0x40000001129/article-65.xml v60>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000f98:[<0x40000001129/article-65.xml v1277>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.517 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.439+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b80 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-58.xml auth v1231 ap=1 s=10214 n(v0 rc2023-07-21T22:34:43.666032+0000 b10214 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd66c7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b80:[<0x40000001129/comment-58.xml v1116>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b80:[<0x40000001129/comment-58.xml v1231>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.517 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.439+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000fa6 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-62.xml auth v1275 ap=1 s=10984 n(v0 rc2023-07-21T22:30:29.258203+0000 b10984 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd617f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000fa6:[<0x40000001129/article-62.xml v84>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000fa6:[<0x40000001129/article-62.xml v1275>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.838 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.440+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b84 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-55.xml auth v1229 ap=1 s=4042 n(v0 rc2023-07-21T22:34:43.795030+0000 b4042 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd6647700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b84:[<0x40000001129/comment-55.xml v1123>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b84:[<0x40000001129/comment-55.xml v1229>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.838 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.440+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000106d [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-24.xml auth v1271 ap=1 s=18095 n(v0 rc2023-07-21T22:30:30.114189+0000 b18095 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd66c7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000106d:[<0x40000001129/article-24.xml v386>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000106d:[<0x40000001129/article-24.xml v1271>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.838 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.440+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b9b [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-34.xml auth v1223 ap=1 s=2678 n(v0 rc2023-07-21T22:34:44.602017+0000 b2678 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd4eaf180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b9b:[<0x40000001129/comment-34.xml v1154>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b9b:[<0x40000001129/comment-34.xml v1223>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.838 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.440+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ba5 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-25.xml auth v1221 ap=1 s=9817 n(v0 rc2023-07-21T22:34:44.847013+0000 b9817 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd6232100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ba5:[<0x40000001129/comment-25.xml v1172>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001ba5:[<0x40000001129/comment-25.xml v1221>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.839 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.441+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010f6 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-28.jpg auth v1273 ap=1 s=20206 n(v0 rc2023-07-21T22:30:30.704179+0000 b20206 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd6627600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010f6:[<0x40000001129/picture-28.jpg v787>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010f6:[<0x40000001129/picture-28.jpg v1273>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.839 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.441+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000021ca [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-85.xml auth v1254 ap=1 s=11961 n(v0 rc2023-07-21T22:35:15.147520+0000 b11961 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd7f09180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000021ca:[<0x40000001129/article-85.xml v1250>,<0x4000000033b/blog-21 v174133>,<0x400000001fb/blogtest_in v2617>,<0x400000001f6/src v2246>,<0x40000000000/blogbench-1.0 v1936>,<0x100000005e2/tmp v528>,<0x100000001fb/client.0 v524>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v86>,<0x10000000001/sv_1 v110>,<0x10000000000/_nogroup v95>,<0x1/volumes v93>]//[]","memoryvalue":"(3)0x100000021ca:[<0x40000001129/article-85.xml v1254>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.839 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.441+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b93 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-41.xml auth v1227 ap=1 s=6367 n(v0 rc2023-07-21T22:34:44.320022+0000 b6367 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd38de680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b93:[<0x40000001129/comment-41.xml v1138>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b93:[<0x40000001129/comment-41.xml v1227>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.839 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.441+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000000f8e [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-67.xml auth v1279 ap=1 s=13692 n(v0 rc2023-07-21T22:30:29.171204+0000 b13692 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd6231b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000000f8e:[<0x40000001129/article-67.xml v44>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000000f8e:[<0x40000001129/article-67.xml v1279>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.839 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.441+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b6c [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-75.xml auth v1235 ap=1 s=9635 n(v0 rc2023-07-21T22:34:42.856045+0000 b9635 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd621a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001b6c:[<0x40000001129/comment-75.xml v1081>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001b6c:[<0x40000001129/comment-75.xml v1235>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.839 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.441+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x4000000117b [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-17.jpg auth v1267 ap=1 s=59072 n(v0 rc2023-07-21T22:30:31.007174+0000 b59072 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd88b0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x4000000117b:[<0x40000001129/picture-17.jpg v783>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x4000000117b:[<0x40000001129/picture-17.jpg v1267>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.840 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.442+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001127 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-24.jpg auth v1269 ap=1 s=52137 n(v0 rc2023-07-21T22:30:30.819177+0000 b52137 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd8be6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001127:[<0x40000001129/picture-24.jpg v785>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001127:[<0x40000001129/picture-24.jpg v1269>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.840 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.442+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010c3 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-7.xml auth v1261 ap=1 s=28587 n(v0 rc2023-07-21T22:30:30.483182+0000 b28587 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd866ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010c3:[<0x40000001129/article-7.xml v522>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010c3:[<0x40000001129/article-7.xml v1261>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.840 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.442+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ba7 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-23.xml auth v1217 ap=1 s=9302 n(v0 rc2023-07-21T22:34:44.890012+0000 b9302 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd6628680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ba7:[<0x40000001129/comment-23.xml v1179>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001ba7:[<0x40000001129/comment-23.xml v1217>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.840 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.442+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ba6 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-24.xml auth v1219 ap=1 s=9224 n(v0 rc2023-07-21T22:34:44.886012+0000 b9224 1=1+0) (ifile excl) (iversion lock) cr={15357=0-4194304@1} caps={15357=pAsLsXsFsxcrwb/pFscr@1},l=15357 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x556fd8c78580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001ba6:[<0x40000001129/comment-24.xml v1176>,<0x4000000033b/blog-21 v166658>,<0x400000001fb/blogtest_in v2517>,<0x400000001f6/src v2210>,<0x40000000000/blogbench-1.0 v1906>,<0x100000005e2/tmp v500>,<0x100000001fb/client.0 v451>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v82>,<0x10000000001/sv_1 v106>,<0x10000000000/_nogroup v91>,<0x1/volumes v89>]//[]","memoryvalue":"(3)0x10000001ba6:[<0x40000001129/comment-24.xml v1219>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.840 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.443+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x40000001059 [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-28.xml auth v1261 ap=1 s=1984 n(v0 rc2023-07-21T22:30:30.030190+0000 b1984 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x556fd7062580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x40000001059:[<0x40000001129/article-28.xml v354>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x40000001059:[<0x40000001129/article-28.xml v1261>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:57.840 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:35:57.443+0000 7f0b392fe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x400000010db [2,head] /volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-3.xml auth v1257 ap=1 s=29561 n(v0 rc2023-07-21T22:30:30.575181+0000 b29561 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x556fd4799080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x400000010db:[<0x40000001129/article-3.xml v554>,<0x4000000033b/blog-21 v34451>,<0x400000001fb/blogtest_in v372>,<0x400000001f6/src v1699>,<0x40000000000/blogbench-1.0 v1471>,<0x100000005e2/tmp v72>,<0x100000001fb/client.0 v69>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v28>,<0x10000000001/sv_1 v52>,<0x10000000000/_nogroup v37>,<0x1/volumes v32>]//[]","memoryvalue":"(3)0x400000010db:[<0x40000001129/article-3.xml v1257>,<0x4000000033b/blog-21 v178592>,<0x400000001fb/blogtest_in v2788>,<0x400000001f6/src v2357>,<0x40000000000/blogbench-1.0 v1986>,<0x100000005e2/tmp v576>,<0x100000001fb/client.0 v550>,<0x100000001fa/ae48aa3b-002e-416f-bd30-32c977bb3dd2 v94>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v32>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-21T22:35:58.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 269 op/s 2023-07-21T22:35:58.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:58.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000112a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-72.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000114f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-21.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.142 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x100000005e3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-35.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.143 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b6e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-73.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.143 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b7e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-60.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.143 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bbf (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-1.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.143 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001015 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-40.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.143 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b69 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-78.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.143 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b68 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-79.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.143 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000000038 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-15.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.144 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b95 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-40.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.144 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b9c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-33.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.144 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b96 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-39.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.144 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001084 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-19.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.144 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000115c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-20.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.144 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000000c9d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-82.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.145 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b9d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-32.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.145 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b7b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-63.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.145 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b85 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-54.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.145 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001009 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-42.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.145 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x100000007d8 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-38.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.145 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000f9e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-64.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.145 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bb3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-13.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.146 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001ba9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-21.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.146 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001ba3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-26.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.146 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001025 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-37.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.146 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001101 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-27.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.146 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000101b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-39.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.146 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000102a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-36.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.146 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010aa (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-12.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.146 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010e5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-1.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.147 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b9f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-30.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.147 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000000ee4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-90.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.147 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bbd (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-3.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.147 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bad (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-18.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.147 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001185 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-16.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.147 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000f94 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-66.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.147 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bb4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-12.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.148 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b8b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-49.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.148 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b92 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-42.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.148 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b77 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-65.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.148 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001072 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-23.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.148 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000108f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-17.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.148 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001829 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-63.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.148 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b89 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-51.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.149 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000000c9f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-81.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.149 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001225 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-3.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.149 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bb6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-10.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.149 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b86 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-53.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.149 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bba (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-6.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.149 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000011c3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-11.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.149 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010cf (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-5.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.150 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001038 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-34.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.150 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b9a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-35.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.150 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001ba1 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-28.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.150 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b82 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-57.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.150 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x1000000197b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-78.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.150 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001054 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-29.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.150 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fac (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-60.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.151 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b7d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-61.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.151 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001077 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-22.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.151 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b97 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-38.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.151 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b74 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-68.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.151 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fdb (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-49.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.151 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bb8 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-8.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.151 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000000ca0 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-80.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.151 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000113b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-23.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.152 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000109e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-14.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.152 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fee (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-46.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.152 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b7c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-62.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.152 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000000545 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-91.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.152 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000000544 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-66.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.152 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fe8 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-47.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.153 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000f77 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-71.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.153 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010b1 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-11.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.153 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b7f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-59.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.153 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001148 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-22.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.153 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001094 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-16.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.153 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b6a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-77.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.153 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fbc (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-54.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.153 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001064 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-26.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.154 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000118f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-15.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.154 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010b4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-10.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.154 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000011b6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-12.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.154 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b8a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-50.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.154 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fe3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-48.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.154 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b8c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-48.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.154 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010d5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-4.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.155 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001068 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-25.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.155 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000101f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-38.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.155 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b9e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-31.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.155 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000011a9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-13.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.155 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fad (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-59.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.156 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b83 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-56.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.156 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000116f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-18.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.156 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000108a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-18.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.156 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bbb (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-5.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.156 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001167 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-19.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.156 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001ba0 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-29.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.157 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fce (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-51.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.157 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000105d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-27.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.157 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000121f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-4.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.157 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000122e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-2.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.157 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000f7c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-70.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.157 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001080 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-20.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.157 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000119c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-14.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x1000000150f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-83.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000faa (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-61.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000000ee5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-41.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000ff7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-45.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000f84 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-69.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001828 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-61.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.158 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000011f3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-7.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.159 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000104c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-31.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.159 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000011dc (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-9.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.159 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bb5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-11.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.159 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001baa (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-20.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.159 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fb7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-55.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.159 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000111a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-25.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.159 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010bf (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-8.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.159 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b8d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-47.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.160 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b8e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-46.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.160 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000f88 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-68.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.160 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b72 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-70.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.160 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000011e9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-8.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.160 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000000ef6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-26.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.160 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001044 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-32.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.160 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b73 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-69.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.161 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000107b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-21.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.161 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001050 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-30.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.161 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b6d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-74.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.161 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b99 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-36.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.161 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010a4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-13.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.161 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b7a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-64.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.161 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b6f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-72.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.161 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bbc (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-4.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.162 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fd3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-50.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.162 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010c9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-6.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.162 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010df (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-2.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.162 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bb2 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-14.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.162 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b75 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-67.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.162 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010eb (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-29.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.162 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bac (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-19.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.163 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fb4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-56.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.163 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001ba8 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-22.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.164 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bbe (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-2.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.164 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b90 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-44.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.164 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001003 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-43.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.164 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000103f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-33.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.164 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001212 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-5.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.164 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001ba2 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-27.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.165 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b6b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-76.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.165 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fb2 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-57.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.165 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b87 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-52.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.165 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b76 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-66.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.165 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bb9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-7.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.165 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bb1 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-15.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.166 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000000caa (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-58.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.166 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fc1 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-53.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.166 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001baf (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-17.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.166 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b91 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-43.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.166 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bb7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-9.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.166 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fc6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-52.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.166 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b8f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-45.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.166 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x100000021c9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-46.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.167 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b98 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-37.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.167 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b70 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-71.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.167 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000011d0 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-10.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.167 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001235 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-1.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.167 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001bb0 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-16.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.167 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010b9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-9.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.167 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000000ef7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-44.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.168 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001201 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-6.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.168 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x100000007d9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-99.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.168 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000f98 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-65.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.168 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b80 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-58.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.168 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000fa6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-62.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.168 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b84 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-55.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.169 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000106d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-24.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.169 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b9b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-34.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.169 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001ba5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-25.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.169 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010f6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-28.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.169 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x100000021ca (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-85.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.169 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b93 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-41.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.169 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000000f8e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-67.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.170 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001b6c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-75.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.170 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x4000000117b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-17.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.170 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001127 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-24.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.170 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010c3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-7.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.170 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001ba7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-23.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.170 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x10000001ba6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-24.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.170 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x40000001059 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-28.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.171 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: Scrub error on inode 0x400000010db (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-3.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.171 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:57 smithi190 ceph-mon[98334]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:58.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 269 op/s 2023-07-21T22:35:58.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:58.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000112a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-72.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000114f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-21.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x100000005e3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-35.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b6e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-73.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b7e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-60.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bbf (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-1.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001015 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-40.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.184 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b69 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-78.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.184 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b68 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-79.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.184 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000000038 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-15.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.184 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b95 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-40.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.184 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b9c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-33.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.184 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b96 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-39.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.185 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001084 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-19.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.185 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000115c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-20.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.185 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000000c9d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-82.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.185 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b9d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-32.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.185 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b7b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-63.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.185 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b85 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-54.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.185 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001009 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-42.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.186 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x100000007d8 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-38.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.186 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000f9e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-64.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.186 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bb3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-13.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.186 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001ba9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-21.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.186 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001ba3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-26.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.186 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001025 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-37.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.186 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001101 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-27.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.187 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000101b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-39.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.187 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000102a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-36.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.187 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010aa (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-12.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.187 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010e5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-1.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.187 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b9f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-30.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.187 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000000ee4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-90.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.187 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bbd (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-3.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bad (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-18.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001185 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-16.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000f94 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-66.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bb4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-12.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.188 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b8b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-49.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b92 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-42.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b77 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-65.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001072 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-23.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000108f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-17.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001829 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-63.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b89 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-51.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.189 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000000c9f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-81.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001225 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-3.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bb6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-10.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b86 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-53.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bba (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-6.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000011c3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-11.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.190 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010cf (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-5.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001038 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-34.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b9a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-35.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001ba1 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-28.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b82 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-57.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x1000000197b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-78.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001054 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-29.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fac (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-60.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.191 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b7d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-61.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001077 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-22.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b97 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-38.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b74 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-68.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fdb (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-49.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.192 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bb8 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-8.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000000ca0 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-80.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000113b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-23.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000109e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-14.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fee (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-46.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b7c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-62.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000000545 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-91.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.193 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000000544 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-66.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fe8 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-47.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000f77 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-71.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010b1 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-11.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b7f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-59.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001148 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-22.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001094 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-16.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b6a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-77.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.194 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fbc (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-54.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001064 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-26.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000118f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-15.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010b4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-10.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000011b6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-12.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b8a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-50.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fe3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-48.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b8c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-48.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.195 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010d5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-4.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001068 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-25.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000101f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-38.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b9e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-31.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000011a9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-13.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fad (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-59.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.196 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b83 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-56.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000116f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-18.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000108a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-18.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bbb (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-5.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001167 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-19.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001ba0 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-29.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fce (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-51.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.197 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000105d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-27.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000121f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-4.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000122e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-2.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000f7c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-70.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001080 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-20.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000119c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-14.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x1000000150f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-83.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000faa (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-61.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.198 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000000ee5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-41.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000ff7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-45.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.199 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000f84 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-69.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001828 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-61.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000011f3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-7.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000104c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-31.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000011dc (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-9.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bb5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-11.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.201 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001baa (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-20.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fb7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-55.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000111a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-25.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010bf (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-8.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b8d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-47.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b8e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-46.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000f88 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-68.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.202 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b72 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-70.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000011e9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-8.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000000ef6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-26.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001044 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-32.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b73 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-69.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000107b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-21.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001050 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-30.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.203 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b6d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-74.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b99 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-36.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010a4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-13.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b7a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-64.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b6f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-72.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bbc (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-4.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fd3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-50.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.204 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010c9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-6.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010df (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-2.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bb2 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-14.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b75 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-67.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010eb (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-29.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bac (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-19.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fb4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-56.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.205 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001ba8 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-22.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bbe (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-2.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b90 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-44.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001003 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-43.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000103f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-33.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001212 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-5.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001ba2 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-27.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.206 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b6b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-76.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fb2 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-57.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b87 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-52.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b76 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-66.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bb9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-7.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bb1 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-15.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.207 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000000caa (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-58.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.208 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fc1 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-53.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.208 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001baf (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-17.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.208 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b91 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-43.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.208 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bb7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-9.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.208 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fc6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-52.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.208 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b8f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-45.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.208 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x100000021c9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-46.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.209 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b98 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-37.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.209 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b70 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-71.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.209 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000011d0 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-10.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.209 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001235 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-1.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.209 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001bb0 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-16.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.209 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010b9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-9.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.209 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000000ef7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-44.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.209 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001201 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-6.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.210 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x100000007d9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-99.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.210 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000f98 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-65.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.210 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b80 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-58.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.210 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000fa6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-62.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.210 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b84 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-55.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.210 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000106d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-24.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.211 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b9b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-34.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.211 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001ba5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-25.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.211 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010f6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-28.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.211 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x100000021ca (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-85.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.211 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b93 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-41.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.211 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000000f8e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-67.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.211 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001b6c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-75.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.212 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x4000000117b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-17.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.212 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001127 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-24.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.212 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010c3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-7.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.212 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001ba7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-23.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.212 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x10000001ba6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-24.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.212 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x40000001059 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-28.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.212 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: Scrub error on inode 0x400000010db (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-3.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.213 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:57 smithi132 ceph-mon[97863]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:58.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 269 op/s 2023-07-21T22:35:58.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:58.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000112a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-72.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000114f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-21.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x100000005e3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-35.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b6e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-73.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b7e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-60.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bbf (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-1.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001015 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-40.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b69 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-78.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b68 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-79.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000000038 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-15.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b95 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-40.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b9c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-33.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b96 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-39.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001084 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-19.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000115c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-20.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000000c9d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-82.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b9d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-32.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b7b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-63.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b85 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-54.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001009 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-42.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x100000007d8 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-38.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000f9e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-64.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bb3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-13.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001ba9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-21.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001ba3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-26.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001025 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-37.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001101 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-27.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000101b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-39.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000102a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-36.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010aa (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-12.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010e5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-1.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b9f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-30.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000000ee4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-90.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bbd (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-3.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bad (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-18.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001185 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-16.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000f94 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-66.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bb4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-12.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b8b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-49.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b92 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-42.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b77 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-65.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001072 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-23.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000108f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-17.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001829 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-63.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b89 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-51.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000000c9f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-81.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001225 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-3.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bb6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-10.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b86 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-53.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bba (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-6.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000011c3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-11.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010cf (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-5.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001038 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-34.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b9a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-35.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001ba1 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-28.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b82 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-57.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x1000000197b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-78.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001054 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-29.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fac (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-60.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b7d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-61.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001077 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-22.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b97 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-38.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b74 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-68.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fdb (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-49.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bb8 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-8.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000000ca0 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-80.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000113b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-23.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000109e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-14.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fee (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-46.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b7c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-62.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000000545 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-91.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000000544 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-66.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fe8 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-47.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000f77 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-71.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010b1 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-11.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b7f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-59.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001148 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-22.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001094 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-16.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b6a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-77.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fbc (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-54.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001064 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-26.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000118f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-15.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010b4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-10.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000011b6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-12.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b8a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-50.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fe3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-48.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b8c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-48.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010d5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-4.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001068 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-25.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000101f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-38.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b9e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-31.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000011a9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-13.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fad (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-59.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b83 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-56.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000116f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-18.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000108a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-18.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bbb (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-5.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001167 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-19.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001ba0 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-29.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fce (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-51.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000105d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-27.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000121f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-4.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000122e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-2.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000f7c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-70.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001080 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-20.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000119c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-14.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x1000000150f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-83.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000faa (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-61.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000000ee5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-41.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000ff7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-45.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000f84 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-69.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001828 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-61.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000011f3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-7.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000104c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-31.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000011dc (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-9.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bb5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-11.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001baa (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-20.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fb7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-55.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000111a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-25.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010bf (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-8.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b8d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-47.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b8e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-46.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000f88 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-68.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b72 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-70.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000011e9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-8.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000000ef6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-26.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001044 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-32.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b73 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-69.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000107b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-21.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001050 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-30.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b6d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-74.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b99 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-36.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010a4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-13.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b7a (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-64.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b6f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-72.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bbc (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-4.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fd3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-50.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010c9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-6.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010df (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-2.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bb2 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-14.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b75 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-67.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010eb (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-29.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bac (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-19.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fb4 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-56.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001ba8 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-22.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bbe (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-2.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b90 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-44.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001003 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-43.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000103f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-33.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001212 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-5.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001ba2 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-27.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b6b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-76.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fb2 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-57.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b87 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-52.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b76 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-66.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bb9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-7.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bb1 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-15.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000000caa (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-58.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fc1 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-53.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001baf (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-17.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b91 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-43.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bb7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-9.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fc6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-52.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b8f (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-45.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x100000021c9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-46.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b98 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-37.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b70 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-71.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000011d0 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-10.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001235 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-1.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001bb0 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-16.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010b9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-9.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000000ef7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-44.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001201 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-6.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x100000007d9 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-99.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000f98 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-65.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b80 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-58.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000fa6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-62.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b84 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-55.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000106d (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-24.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b9b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-34.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001ba5 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-25.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010f6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-28.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x100000021ca (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-85.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b93 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-41.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000000f8e (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-67.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001b6c (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-75.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x4000000117b (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-17.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001127 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-24.jpg) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010c3 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-7.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001ba7 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-23.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x10000001ba6 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-24.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x40000001059 (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-28.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: Scrub error on inode 0x400000010db (/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-3.xml) see mds.f log and `damage ls` output for details 2023-07-21T22:35:58.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:57 smithi107 ceph-mon[96543]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:59.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:58 smithi190 ceph-mon[98334]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:35:59.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:58 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] up:active 2023-07-21T22:35:59.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:58 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:59.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:58 smithi132 ceph-mon[97863]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:35:59.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:58 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] up:active 2023-07-21T22:35:59.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:58 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:35:59.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:58 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-a[96519]: 2023-07-21T22:35:58.789+0000 7f3b434df700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:35:59.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:58 smithi107 ceph-mon[96543]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:35:59.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:58 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] up:active 2023-07-21T22:35:59.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:58 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:00.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:35:59 smithi132 ceph-mon[97863]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 399 op/s 2023-07-21T22:36:00.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:35:59 smithi107 ceph-mon[96543]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 399 op/s 2023-07-21T22:36:00.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:35:59 smithi190 ceph-mon[98334]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 399 op/s 2023-07-21T22:36:01.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:00 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:01.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:00 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:01.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:00 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:02.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:01 smithi132 ceph-mon[97863]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 353 op/s 2023-07-21T22:36:02.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:01 smithi107 ceph-mon[96543]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 353 op/s 2023-07-21T22:36:02.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:01 smithi190 ceph-mon[98334]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 353 op/s 2023-07-21T22:36:04.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:03 smithi132 ceph-mon[97863]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 452 op/s 2023-07-21T22:36:04.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:03 smithi107 ceph-mon[96543]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 452 op/s 2023-07-21T22:36:04.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:03 smithi190 ceph-mon[98334]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 452 op/s 2023-07-21T22:36:06.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:05 smithi132 ceph-mon[97863]: pgmap v504: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 444 op/s 2023-07-21T22:36:06.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:05 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:06.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:05 smithi107 ceph-mon[96543]: pgmap v504: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 444 op/s 2023-07-21T22:36:06.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:05 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:06.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:05 smithi190 ceph-mon[98334]: pgmap v504: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 444 op/s 2023-07-21T22:36:06.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:05 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:07.735 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:36:07.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.820+0000 7f590cdab700 1 -- 172.21.15.107:0/2995799933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59080ffbd0 msgr2=0x7f5908100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:07.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.820+0000 7f590cdab700 1 --2- 172.21.15.107:0/2995799933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59080ffbd0 0x7f5908100020 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009990 tx=0x7f58fc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.820+0000 7f590cdab700 1 -- 172.21.15.107:0/2995799933 shutdown_connections 2023-07-21T22:36:07.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.820+0000 7f590cdab700 1 --2- 172.21.15.107:0/2995799933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5908100560 0x7f5908106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.820+0000 7f590cdab700 1 --2- 172.21.15.107:0/2995799933 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59080fe9e0 0x7f59080fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.820+0000 7f590cdab700 1 --2- 172.21.15.107:0/2995799933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59080ffbd0 0x7f5908100020 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.820+0000 7f590cdab700 1 -- 172.21.15.107:0/2995799933 >> 172.21.15.107:0/2995799933 conn(0x7f59080fa000 msgr2=0x7f59080fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:07.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.820+0000 7f590cdab700 1 -- 172.21.15.107:0/2995799933 shutdown_connections 2023-07-21T22:36:07.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.820+0000 7f590cdab700 1 -- 172.21.15.107:0/2995799933 wait complete. 2023-07-21T22:36:07.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.821+0000 7f590cdab700 1 Processor -- start 2023-07-21T22:36:07.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.821+0000 7f590cdab700 1 -- start start 2023-07-21T22:36:07.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.821+0000 7f590cdab700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59080fe9e0 0x7f59081947b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.821+0000 7f590cdab700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59080ffbd0 0x7f5908194cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.821+0000 7f590cdab700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5908100560 0x7f590819bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.821+0000 7f590cdab700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590819c330 con 0x7f5908100560 2023-07-21T22:36:07.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.821+0000 7f590cdab700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590819c470 con 0x7f59080fe9e0 2023-07-21T22:36:07.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.821+0000 7f590cdab700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590819c600 con 0x7f59080ffbd0 2023-07-21T22:36:07.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.821+0000 7f5906d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5908100560 0x7f590819bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.821+0000 7f5906d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5908100560 0x7f590819bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50894/0 (socket says 172.21.15.107:50894) 2023-07-21T22:36:07.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.821+0000 7f590659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59080fe9e0 0x7f59081947b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.821+0000 7f5906d9d700 1 -- 172.21.15.107:0/3619004567 learned_addr learned my addr 172.21.15.107:0/3619004567 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:07.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.821+0000 7f5905d9b700 1 --2- 172.21.15.107:0/3619004567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59080ffbd0 0x7f5908194cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.822+0000 7f5906d9d700 1 -- 172.21.15.107:0/3619004567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59080ffbd0 msgr2=0x7f5908194cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:07.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.822+0000 7f5906d9d700 1 --2- 172.21.15.107:0/3619004567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59080ffbd0 0x7f5908194cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.822+0000 7f5906d9d700 1 -- 172.21.15.107:0/3619004567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59080fe9e0 msgr2=0x7f59081947b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:07.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.822+0000 7f5906d9d700 1 --2- 172.21.15.107:0/3619004567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59080fe9e0 0x7f59081947b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.822+0000 7f5906d9d700 1 -- 172.21.15.107:0/3619004567 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc0095f0 con 0x7f5908100560 2023-07-21T22:36:07.872 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.822+0000 7f5906d9d700 1 --2- 172.21.15.107:0/3619004567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5908100560 0x7f590819bd60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5900007ee0 tx=0x7f590000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:07.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.822+0000 7f58f77fe700 1 -- 172.21.15.107:0/3619004567 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900017070 con 0x7f5908100560 2023-07-21T22:36:07.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.822+0000 7f58f77fe700 1 -- 172.21.15.107:0/3619004567 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f590000f040 con 0x7f5908100560 2023-07-21T22:36:07.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.822+0000 7f58f77fe700 1 -- 172.21.15.107:0/3619004567 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900005580 con 0x7f5908100560 2023-07-21T22:36:07.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.822+0000 7f590cdab700 1 -- 172.21.15.107:0/3619004567 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590819c8e0 con 0x7f5908100560 2023-07-21T22:36:07.873 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.822+0000 7f590cdab700 1 -- 172.21.15.107:0/3619004567 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590819cda0 con 0x7f5908100560 2023-07-21T22:36:07.877 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.824+0000 7f58f77fe700 1 -- 172.21.15.107:0/3619004567 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f590000db20 con 0x7f5908100560 2023-07-21T22:36:07.877 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.825+0000 7f58f77fe700 1 --2- 172.21.15.107:0/3619004567 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f58f00a1150 0x7f58f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.877 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.825+0000 7f58f77fe700 1 -- 172.21.15.107:0/3619004567 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f59000cc040 con 0x7f5908100560 2023-07-21T22:36:07.877 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.825+0000 7f590659c700 1 --2- 172.21.15.107:0/3619004567 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f58f00a1150 0x7f58f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.825+0000 7f590659c700 1 --2- 172.21.15.107:0/3619004567 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f58f00a1150 0x7f58f00a3600 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f59080ffa30 tx=0x7f58f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:07.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f5918f30b80 1 Processor -- start 2023-07-21T22:36:07.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f5918f30b80 1 -- start start 2023-07-21T22:36:07.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f5918f30b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515afa60 0x55ae515afe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f5918f30b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515b03f0 0x55ae515ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f5918f30b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 0x55ae515af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f5918f30b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae514b1cb0 con 0x55ae515aeb90 2023-07-21T22:36:07.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f5918f30b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae51504a40 con 0x55ae515afa60 2023-07-21T22:36:07.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f5918f30b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae514fded0 con 0x55ae515b03f0 2023-07-21T22:36:07.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f590659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515afa60 0x55ae515afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f5906d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 0x55ae515af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f590659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515afa60 0x55ae515afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41726/0 (socket says 172.21.15.107:41726) 2023-07-21T22:36:07.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f590659c700 1 -- 172.21.15.107:0/749023849 learned_addr learned my addr 172.21.15.107:0/749023849 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:07.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f5906d9d700 1 --2- 172.21.15.107:0/749023849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 0x55ae515af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50902/0 (socket says 172.21.15.107:50902) 2023-07-21T22:36:07.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.826+0000 7f5905d9b700 1 --2- 172.21.15.107:0/749023849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515b03f0 0x55ae515ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.827+0000 7f5906d9d700 1 -- 172.21.15.107:0/749023849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515b03f0 msgr2=0x55ae515ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.827+0000 7f5906d9d700 1 --2- 172.21.15.107:0/749023849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515b03f0 0x55ae515ae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.827+0000 7f5906d9d700 1 -- 172.21.15.107:0/749023849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515afa60 msgr2=0x55ae515afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.827+0000 7f5906d9d700 1 --2- 172.21.15.107:0/749023849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515afa60 0x55ae515afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.827+0000 7f5906d9d700 1 -- 172.21.15.107:0/749023849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae5132bde0 con 0x55ae515aeb90 2023-07-21T22:36:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.827+0000 7f5906d9d700 1 --2- 172.21.15.107:0/749023849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 0x55ae515af030 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f59000c9e90 tx=0x7f5900095040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f2fb3806cca770 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:07.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.827+0000 7f58f4ff9700 1 -- 172.21.15.107:0/749023849 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59000a1880 con 0x55ae515aeb90 2023-07-21T22:36:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.827+0000 7f58f4ff9700 1 -- 172.21.15.107:0/749023849 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f590000ddd0 con 0x55ae515aeb90 2023-07-21T22:36:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.827+0000 7f58f4ff9700 1 -- 172.21.15.107:0/749023849 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59000978c0 con 0x55ae515aeb90 2023-07-21T22:36:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.828+0000 7f5918f30b80 1 -- 172.21.15.107:0/749023849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 msgr2=0x55ae515af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.828+0000 7f5918f30b80 1 --2- 172.21.15.107:0/749023849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 0x55ae515af030 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f59000c9e90 tx=0x7f5900095040 comp rx=0 tx=0).stop 2023-07-21T22:36:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.828+0000 7f5918f30b80 1 -- 172.21.15.107:0/749023849 shutdown_connections 2023-07-21T22:36:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.828+0000 7f5918f30b80 1 --2- 172.21.15.107:0/749023849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 0x55ae515af030 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.828+0000 7f5918f30b80 1 --2- 172.21.15.107:0/749023849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515afa60 0x55ae515afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.828+0000 7f5918f30b80 1 --2- 172.21.15.107:0/749023849 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515b03f0 0x55ae515ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.828+0000 7f5918f30b80 1 -- 172.21.15.107:0/749023849 >> 172.21.15.107:0/749023849 conn(0x55ae515a1ef0 msgr2=0x55ae515a7360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.828+0000 7f5918f30b80 1 -- 172.21.15.107:0/749023849 shutdown_connections 2023-07-21T22:36:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.828+0000 7f5918f30b80 1 -- 172.21.15.107:0/749023849 wait complete. 2023-07-21T22:36:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.828+0000 7f5918f30b80 1 -- start start 2023-07-21T22:36:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.828+0000 7f5918f30b80 1 Processor -- start 2023-07-21T22:36:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f5918f30b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 0x55ae515b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f5918f30b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515afa60 0x55ae515b8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f5918f30b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515b03f0 0x55ae515bf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f5918f30b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5149dac0 con 0x55ae515aeb90 2023-07-21T22:36:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f5918f30b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae5149d7c0 con 0x55ae515afa60 2023-07-21T22:36:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f5918f30b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae51381920 con 0x55ae515b03f0 2023-07-21T22:36:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f5918f30b80 10 client.0 _ll_register_callbacks cb 0x55ae5149b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f590659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 0x55ae515b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f590659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 0x55ae515b8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50904/0 (socket says 172.21.15.107:50904) 2023-07-21T22:36:07.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f590659c700 1 -- 172.21.15.107:0/1049245376 learned_addr learned my addr 172.21.15.107:0/1049245376 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f5905d9b700 1 --2- 172.21.15.107:0/1049245376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515afa60 0x55ae515b8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f5906d9d700 1 --2- 172.21.15.107:0/1049245376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515b03f0 0x55ae515bf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f590659c700 1 -- 172.21.15.107:0/1049245376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515b03f0 msgr2=0x55ae515bf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f590659c700 1 --2- 172.21.15.107:0/1049245376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515b03f0 0x55ae515bf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f590659c700 1 -- 172.21.15.107:0/1049245376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515afa60 msgr2=0x55ae515b8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f590659c700 1 --2- 172.21.15.107:0/1049245376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515afa60 0x55ae515b8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f590659c700 1 -- 172.21.15.107:0/1049245376 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590819cda0 con 0x55ae515aeb90 2023-07-21T22:36:07.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f590659c700 1 --2- 172.21.15.107:0/1049245376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 0x55ae515b8310 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009b20 tx=0x7f58f80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e963f2389bdaaf5e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:07.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f58edffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:07.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f58edffb700 1 -- 172.21.15.107:0/1049245376 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8022070 con 0x55ae515aeb90 2023-07-21T22:36:07.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f58edffb700 1 -- 172.21.15.107:0/1049245376 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f58f801c810 con 0x55ae515aeb90 2023-07-21T22:36:07.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.829+0000 7f5918f30b80 1 -- 172.21.15.107:0/1049245376 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae514574f0 con 0x55ae515aeb90 2023-07-21T22:36:07.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.830+0000 7f58edffb700 1 -- 172.21.15.107:0/1049245376 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58f801c9d0 con 0x55ae515aeb90 2023-07-21T22:36:07.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.830+0000 7f5918f30b80 10 client.16494 fetch_fsmap learned FSMap version 45 2023-07-21T22:36:07.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.830+0000 7f5918f30b80 1 -- 172.21.15.107:0/1049245376 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ae51314c60 con 0x55ae515aeb90 2023-07-21T22:36:07.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.830+0000 7f58edffb700 1 -- 172.21.15.107:0/1049245376 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f8035080 con 0x55ae515aeb90 2023-07-21T22:36:07.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.830+0000 7f58edffb700 1 -- 172.21.15.107:0/1049245376 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 45) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f58f802f030 con 0x55ae515aeb90 2023-07-21T22:36:07.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.830+0000 7f5918f30b80 10 client.16494 fetch_fsmap finished waiting for FSMap version 45 2023-07-21T22:36:07.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.830+0000 7f5918f30b80 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:07.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.830+0000 7f5918f30b80 20 client.16494 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:07.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.830+0000 7f5918f30b80 1 --2- 172.21.15.107:0/1049245376 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ae515c41f0 0x55ae515c6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.830+0000 7f5918f30b80 4 client.16494 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:07.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.830+0000 7f5918f30b80 1 -- 172.21.15.107:0/1049245376 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae514f6eb0 con 0x55ae515c41f0 2023-07-21T22:36:07.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.830+0000 7f5905d9b700 1 --2- 172.21.15.107:0/1049245376 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ae515c41f0 0x55ae515c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.831+0000 7f58edffb700 10 client.16494 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:07.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.831+0000 7f5905d9b700 1 --2- 172.21.15.107:0/1049245376 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ae515c41f0 0x55ae515c6600 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c411269573ca3d0 server_cookie=ae574db41202520d in_seq=0 out_seq=0 2023-07-21T22:36:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.831+0000 7f58edffb700 1 -- 172.21.15.107:0/1049245376 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ae514f6eb0 con 0x55ae515c41f0 2023-07-21T22:36:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.831+0000 7f58edffb700 10 client.16494 handle_command_reply: tid=0 2023-07-21T22:36:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.831+0000 7f5918f30b80 1 client.16494 shutdown 2023-07-21T22:36:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.832+0000 7f5918f30b80 1 -- 172.21.15.107:0/1049245376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 msgr2=0x55ae515b8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.832+0000 7f5918f30b80 1 --2- 172.21.15.107:0/1049245376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 0x55ae515b8310 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009b20 tx=0x7f58f80047c0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.832+0000 7f5918f30b80 1 -- 172.21.15.107:0/1049245376 shutdown_connections 2023-07-21T22:36:07.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.832+0000 7f5918f30b80 1 --2- 172.21.15.107:0/1049245376 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ae515c41f0 0x55ae515c6600 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.832+0000 7f5918f30b80 1 --2- 172.21.15.107:0/1049245376 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515b03f0 0x55ae515bf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.832+0000 7f58edffb700 0 client.16494 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:07.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.832+0000 7f5918f30b80 1 --2- 172.21.15.107:0/1049245376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515aeb90 0x55ae515b8310 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.832+0000 7f5918f30b80 1 --2- 172.21.15.107:0/1049245376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515afa60 0x55ae515b8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.832+0000 7f5918f30b80 1 -- 172.21.15.107:0/1049245376 >> 172.21.15.107:0/1049245376 conn(0x55ae515a1ef0 msgr2=0x55ae515a6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:07.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.832+0000 7f5918f30b80 1 -- 172.21.15.107:0/1049245376 shutdown_connections 2023-07-21T22:36:07.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.832+0000 7f5918f30b80 1 -- 172.21.15.107:0/1049245376 wait complete. 2023-07-21T22:36:07.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.832+0000 7f5918f30b80 20 client.16494 trim_cache size 0 max 16384 2023-07-21T22:36:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.848+0000 7f5918f30b80 1 Processor -- start 2023-07-21T22:36:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.848+0000 7f5918f30b80 1 -- start start 2023-07-21T22:36:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5918f30b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515aeb90 0x55ae515ae560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5918f30b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515afa60 0x55ae515bf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5918f30b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515b03f0 0x55ae515b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5918f30b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae515b7ee0 con 0x55ae515afa60 2023-07-21T22:36:07.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5918f30b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae515c1500 con 0x55ae515b03f0 2023-07-21T22:36:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5918f30b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae515a19b0 con 0x55ae515aeb90 2023-07-21T22:36:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5905d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515afa60 0x55ae515bf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5905d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515afa60 0x55ae515bf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50914/0 (socket says 172.21.15.107:50914) 2023-07-21T22:36:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5905d9b700 1 -- 172.21.15.107:0/3995386578 learned_addr learned my addr 172.21.15.107:0/3995386578 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f590659c700 1 --2- 172.21.15.107:0/3995386578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515aeb90 0x55ae515ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5906d9d700 1 --2- 172.21.15.107:0/3995386578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515b03f0 0x55ae515b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5905d9b700 1 -- 172.21.15.107:0/3995386578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515aeb90 msgr2=0x55ae515ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5905d9b700 1 --2- 172.21.15.107:0/3995386578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515aeb90 0x55ae515ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5905d9b700 1 -- 172.21.15.107:0/3995386578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515b03f0 msgr2=0x55ae515b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5905d9b700 1 --2- 172.21.15.107:0/3995386578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515b03f0 0x55ae515b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5905d9b700 1 -- 172.21.15.107:0/3995386578 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae51352250 con 0x55ae515afa60 2023-07-21T22:36:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5906d9d700 1 --2- 172.21.15.107:0/3995386578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515b03f0 0x55ae515b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:36:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f590659c700 1 --2- 172.21.15.107:0/3995386578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515aeb90 0x55ae515ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:36:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f5905d9b700 1 --2- 172.21.15.107:0/3995386578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515afa60 0x55ae515bf850 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f58fc005f50 tx=0x7f58fc005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d23b8e4f3f269cd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:07.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f58f57fa700 1 -- 172.21.15.107:0/3995386578 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc02a070 con 0x55ae515afa60 2023-07-21T22:36:07.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f58f57fa700 1 -- 172.21.15.107:0/3995386578 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58fc02e070 con 0x55ae515afa60 2023-07-21T22:36:07.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.849+0000 7f58f57fa700 1 -- 172.21.15.107:0/3995386578 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc01d040 con 0x55ae515afa60 2023-07-21T22:36:07.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.850+0000 7f5918f30b80 1 -- 172.21.15.107:0/3995386578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515afa60 msgr2=0x55ae515bf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:07.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.850+0000 7f5918f30b80 1 --2- 172.21.15.107:0/3995386578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515afa60 0x55ae515bf850 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f58fc005f50 tx=0x7f58fc005960 comp rx=0 tx=0).stop 2023-07-21T22:36:07.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.850+0000 7f5918f30b80 1 -- 172.21.15.107:0/3995386578 shutdown_connections 2023-07-21T22:36:07.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.850+0000 7f5918f30b80 1 --2- 172.21.15.107:0/3995386578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515b03f0 0x55ae515b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.850+0000 7f5918f30b80 1 --2- 172.21.15.107:0/3995386578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515aeb90 0x55ae515ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.850+0000 7f5918f30b80 1 --2- 172.21.15.107:0/3995386578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515afa60 0x55ae515bf850 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.850+0000 7f5918f30b80 1 -- 172.21.15.107:0/3995386578 >> 172.21.15.107:0/3995386578 conn(0x55ae515a1ef0 msgr2=0x55ae515b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:07.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.850+0000 7f5918f30b80 1 -- 172.21.15.107:0/3995386578 shutdown_connections 2023-07-21T22:36:07.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.850+0000 7f5918f30b80 1 -- 172.21.15.107:0/3995386578 wait complete. 2023-07-21T22:36:07.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.850+0000 7f5918f30b80 1 -- start start 2023-07-21T22:36:07.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.850+0000 7f5918f30b80 1 Processor -- start 2023-07-21T22:36:07.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5918f30b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515aeb90 0x55ae515bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5918f30b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515afa60 0x55ae515bc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5918f30b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515b03f0 0x55ae515bc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5918f30b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae515bcb30 con 0x55ae515b03f0 2023-07-21T22:36:07.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5918f30b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae515bcca0 con 0x55ae515aeb90 2023-07-21T22:36:07.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5918f30b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae51618060 con 0x55ae515afa60 2023-07-21T22:36:07.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5918f30b80 10 client.0 _ll_register_callbacks cb 0x55ae5149b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:07.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5906d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515b03f0 0x55ae515bc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f590659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515aeb90 0x55ae515bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5906d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515b03f0 0x55ae515bc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50922/0 (socket says 172.21.15.107:50922) 2023-07-21T22:36:07.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5906d9d700 1 -- 172.21.15.107:0/3164781233 learned_addr learned my addr 172.21.15.107:0/3164781233 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:07.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5905d9b700 1 --2- 172.21.15.107:0/3164781233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515afa60 0x55ae515bc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5906d9d700 1 -- 172.21.15.107:0/3164781233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515afa60 msgr2=0x55ae515bc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:07.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5906d9d700 1 --2- 172.21.15.107:0/3164781233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515afa60 0x55ae515bc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5906d9d700 1 -- 172.21.15.107:0/3164781233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515aeb90 msgr2=0x55ae515bf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:07.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5906d9d700 1 --2- 172.21.15.107:0/3164781233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515aeb90 0x55ae515bf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5906d9d700 1 -- 172.21.15.107:0/3164781233 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5908107750 con 0x55ae515b03f0 2023-07-21T22:36:07.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5906d9d700 1 --2- 172.21.15.107:0/3164781233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515b03f0 0x55ae515bc5f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5900099130 tx=0x7f5900007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6006473f78477ff3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:07.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f58edffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:07.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f58edffb700 1 -- 172.21.15.107:0/3164781233 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59000ae070 con 0x55ae515b03f0 2023-07-21T22:36:07.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f58edffb700 1 -- 172.21.15.107:0/3164781233 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f59000a1880 con 0x55ae515b03f0 2023-07-21T22:36:07.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5918f30b80 1 -- 172.21.15.107:0/3164781233 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f590000ddd0 con 0x55ae515b03f0 2023-07-21T22:36:07.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f58edffb700 1 -- 172.21.15.107:0/3164781233 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5900084080 con 0x55ae515b03f0 2023-07-21T22:36:07.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5918f30b80 10 client.16500 fetch_fsmap learned FSMap version 45 2023-07-21T22:36:07.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.851+0000 7f5918f30b80 1 -- 172.21.15.107:0/3164781233 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55ae5132a2b0 con 0x55ae515b03f0 2023-07-21T22:36:07.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.852+0000 7f58edffb700 1 -- 172.21.15.107:0/3164781233 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f590000ddd0 con 0x55ae515b03f0 2023-07-21T22:36:07.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.852+0000 7f58edffb700 1 -- 172.21.15.107:0/3164781233 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 45) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f5900098920 con 0x55ae515b03f0 2023-07-21T22:36:07.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.852+0000 7f5918f30b80 10 client.16500 fetch_fsmap finished waiting for FSMap version 45 2023-07-21T22:36:07.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.852+0000 7f5918f30b80 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:07.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.852+0000 7f5918f30b80 20 client.16500 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:07.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.852+0000 7f5918f30b80 1 --2- 172.21.15.107:0/3164781233 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ae51619710 0x55ae5161bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:07.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.852+0000 7f5918f30b80 4 client.16500 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:36:07.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.852+0000 7f5918f30b80 1 -- 172.21.15.107:0/3164781233 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ae514b9050 con 0x55ae51619710 2023-07-21T22:36:07.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.852+0000 7f590659c700 1 --2- 172.21.15.107:0/3164781233 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ae51619710 0x55ae5161bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:07.907 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.852+0000 7f58edffb700 10 client.16500 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:07.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.852+0000 7f590659c700 1 --2- 172.21.15.107:0/3164781233 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ae51619710 0x55ae5161bb20 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aec690742158ff0 server_cookie=8dfe181391c74d54 in_seq=0 out_seq=0 2023-07-21T22:36:07.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f58edffb700 1 -- 172.21.15.107:0/3164781233 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ae514b9050 con 0x55ae51619710 2023-07-21T22:36:07.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f58edffb700 10 client.16500 handle_command_reply: tid=0 2023-07-21T22:36:07.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f5918f30b80 1 client.16500 shutdown 2023-07-21T22:36:07.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f5918f30b80 1 -- 172.21.15.107:0/3164781233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515b03f0 msgr2=0x55ae515bc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:07.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f5918f30b80 1 --2- 172.21.15.107:0/3164781233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515b03f0 0x55ae515bc5f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5900099130 tx=0x7f5900007f10 comp rx=0 tx=0).stop 2023-07-21T22:36:07.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f5918f30b80 1 -- 172.21.15.107:0/3164781233 shutdown_connections 2023-07-21T22:36:07.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f5918f30b80 1 --2- 172.21.15.107:0/3164781233 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55ae51619710 0x55ae5161bb20 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f5918f30b80 1 --2- 172.21.15.107:0/3164781233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae515b03f0 0x55ae515bc5f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f5918f30b80 1 --2- 172.21.15.107:0/3164781233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ae515aeb90 0x55ae515bf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f58edffb700 0 client.16500 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:07.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f5918f30b80 1 --2- 172.21.15.107:0/3164781233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ae515afa60 0x55ae515bc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f5918f30b80 1 -- 172.21.15.107:0/3164781233 >> 172.21.15.107:0/3164781233 conn(0x55ae515a1ef0 msgr2=0x55ae515b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:07.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f5918f30b80 1 -- 172.21.15.107:0/3164781233 shutdown_connections 2023-07-21T22:36:07.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f5918f30b80 1 -- 172.21.15.107:0/3164781233 wait complete. 2023-07-21T22:36:07.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f5918f30b80 20 client.16500 trim_cache size 0 max 16384 2023-07-21T22:36:07.910 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:36:07.910 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:36:07.910 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2023-07-21T22:36:07.910 INFO:teuthology.orchestra.run.smithi107.stdout: "d037412e-e4e0-4d2c-8d6e-c7193371fe0f": { 2023-07-21T22:36:07.910 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2023-07-21T22:36:07.911 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d037412e-e4e0-4d2c-8d6e-c7193371fe0f", 2023-07-21T22:36:07.911 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2023-07-21T22:36:07.911 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:36:07.911 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:36:07.911 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:36:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f590cdab700 1 -- 172.21.15.107:0/3619004567 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f58f00a1150 msgr2=0x7f58f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f590cdab700 1 --2- 172.21.15.107:0/3619004567 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f58f00a1150 0x7f58f00a3600 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f59080ffa30 tx=0x7f58f8009260 comp rx=0 tx=0).stop 2023-07-21T22:36:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f590cdab700 1 -- 172.21.15.107:0/3619004567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5908100560 msgr2=0x7f590819bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:07.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f590cdab700 1 --2- 172.21.15.107:0/3619004567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5908100560 0x7f590819bd60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5900007ee0 tx=0x7f590000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f590cdab700 1 -- 172.21.15.107:0/3619004567 shutdown_connections 2023-07-21T22:36:07.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f590cdab700 1 --2- 172.21.15.107:0/3619004567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5908100560 0x7f590819bd60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f590cdab700 1 --2- 172.21.15.107:0/3619004567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59080fe9e0 0x7f59081947b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f590cdab700 1 --2- 172.21.15.107:0/3619004567 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f58f00a1150 0x7f58f00a3600 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f590cdab700 1 --2- 172.21.15.107:0/3619004567 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59080ffbd0 0x7f5908194cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:07.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.853+0000 7f590cdab700 1 -- 172.21.15.107:0/3619004567 >> 172.21.15.107:0/3619004567 conn(0x7f59080fa000 msgr2=0x7f5908103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:07.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.854+0000 7f590cdab700 1 -- 172.21.15.107:0/3619004567 shutdown_connections 2023-07-21T22:36:07.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:07.854+0000 7f590cdab700 1 -- 172.21.15.107:0/3619004567 wait complete. 2023-07-21T22:36:07.914 INFO:tasks.cephfs.filesystem:scrub status for tag:d037412e-e4e0-4d2c-8d6e-c7193371fe0f - {'path': '/', 'tag': 'd037412e-e4e0-4d2c-8d6e-c7193371fe0f', 'options': 'recursive,force'} 2023-07-21T22:36:08.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:07 smithi132 ceph-mon[97863]: pgmap v505: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 439 op/s 2023-07-21T22:36:08.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:07 smithi107 ceph-mon[96543]: pgmap v505: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 439 op/s 2023-07-21T22:36:08.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:07 smithi190 ceph-mon[98334]: pgmap v505: 97 pgs: 97 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 439 op/s 2023-07-21T22:36:09.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:08 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:36:09.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:08 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:36:09.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:08 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:36:10.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:09 smithi107 ceph-mon[96543]: pgmap v506: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 520 op/s 2023-07-21T22:36:10.314 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:09 smithi132 ceph-mon[97863]: pgmap v506: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 520 op/s 2023-07-21T22:36:10.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:09 smithi190 ceph-mon[98334]: pgmap v506: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 520 op/s 2023-07-21T22:36:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.407+0000 7f9a499a9700 1 -- 172.21.15.107:0/3034710629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a440ff370 msgr2=0x7f9a440ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.407+0000 7f9a499a9700 1 --2- 172.21.15.107:0/3034710629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a440ff370 0x7f9a440ff7c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009990 tx=0x7f9a34009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.407+0000 7f9a499a9700 1 -- 172.21.15.107:0/3034710629 shutdown_connections 2023-07-21T22:36:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.407+0000 7f9a499a9700 1 --2- 172.21.15.107:0/3034710629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a440ffd00 0x7f9a441065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.407+0000 7f9a499a9700 1 --2- 172.21.15.107:0/3034710629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a440fe180 0x7f9a440fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.407+0000 7f9a499a9700 1 --2- 172.21.15.107:0/3034710629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a440ff370 0x7f9a440ff7c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.407+0000 7f9a499a9700 1 -- 172.21.15.107:0/3034710629 >> 172.21.15.107:0/3034710629 conn(0x7f9a440f9780 msgr2=0x7f9a440fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.407+0000 7f9a499a9700 1 -- 172.21.15.107:0/3034710629 shutdown_connections 2023-07-21T22:36:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.407+0000 7f9a499a9700 1 -- 172.21.15.107:0/3034710629 wait complete. 2023-07-21T22:36:10.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a499a9700 1 Processor -- start 2023-07-21T22:36:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a499a9700 1 -- start start 2023-07-21T22:36:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a499a9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a440fe180 0x7f9a441958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a499a9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a440ff370 0x7f9a44195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a499a9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a440ffd00 0x7f9a44196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a499a9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a44197310 con 0x7f9a440fe180 2023-07-21T22:36:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a499a9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a44197480 con 0x7f9a440ff370 2023-07-21T22:36:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a499a9700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a44197640 con 0x7f9a440ffd00 2023-07-21T22:36:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a42ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a440fe180 0x7f9a441958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a42ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a440fe180 0x7f9a441958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50932/0 (socket says 172.21.15.107:50932) 2023-07-21T22:36:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a42ffd700 1 -- 172.21.15.107:0/2754316158 learned_addr learned my addr 172.21.15.107:0/2754316158 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a437fe700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a440ffd00 0x7f9a44196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a427fc700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a440ff370 0x7f9a44195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a42ffd700 1 -- 172.21.15.107:0/2754316158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a440ffd00 msgr2=0x7f9a44196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a42ffd700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a440ffd00 0x7f9a44196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a42ffd700 1 -- 172.21.15.107:0/2754316158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a440ff370 msgr2=0x7f9a44195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a42ffd700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a440ff370 0x7f9a44195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.408+0000 7f9a42ffd700 1 -- 172.21.15.107:0/2754316158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a340095f0 con 0x7f9a440fe180 2023-07-21T22:36:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.409+0000 7f9a42ffd700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a440fe180 0x7f9a441958f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00ea60 tx=0x7f9a2c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.409+0000 7f9a489a7700 1 -- 172.21.15.107:0/2754316158 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c00ccd0 con 0x7f9a440fe180 2023-07-21T22:36:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.409+0000 7f9a489a7700 1 -- 172.21.15.107:0/2754316158 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a2c00ce30 con 0x7f9a440fe180 2023-07-21T22:36:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.409+0000 7f9a499a9700 1 -- 172.21.15.107:0/2754316158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a44197920 con 0x7f9a440fe180 2023-07-21T22:36:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.409+0000 7f9a489a7700 1 -- 172.21.15.107:0/2754316158 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c017480 con 0x7f9a440fe180 2023-07-21T22:36:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.409+0000 7f9a499a9700 1 -- 172.21.15.107:0/2754316158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4419e0e0 con 0x7f9a440fe180 2023-07-21T22:36:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.411+0000 7f9a489a7700 1 -- 172.21.15.107:0/2754316158 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9a2c004000 con 0x7f9a440fe180 2023-07-21T22:36:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.411+0000 7f9a489a7700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a300a1150 0x7f9a300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.411+0000 7f9a489a7700 1 -- 172.21.15.107:0/2754316158 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9a2c0ccb70 con 0x7f9a440fe180 2023-07-21T22:36:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.411+0000 7f9a427fc700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a300a1150 0x7f9a300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a427fc700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a300a1150 0x7f9a300a3600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400b2e0 tx=0x7f9a34005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a55b2eb80 1 Processor -- start 2023-07-21T22:36:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a55b2eb80 1 -- start start 2023-07-21T22:36:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a55b2eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15a1f6b0 0x559b15bc32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a55b2eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15bc3830 0x559b15bc4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a55b2eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc4580 0x559b15bc49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a55b2eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15ac8c50 con 0x559b15bc4580 2023-07-21T22:36:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a55b2eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15b1b9e0 con 0x559b15a1f6b0 2023-07-21T22:36:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a55b2eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15b14e70 con 0x559b15bc3830 2023-07-21T22:36:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a437fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc4580 0x559b15bc49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a437fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc4580 0x559b15bc49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50944/0 (socket says 172.21.15.107:50944) 2023-07-21T22:36:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a437fe700 1 -- 172.21.15.107:0/3998675948 learned_addr learned my addr 172.21.15.107:0/3998675948 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a427fc700 1 --2- 172.21.15.107:0/3998675948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15bc3830 0x559b15bc4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.412+0000 7f9a42ffd700 1 --2- 172.21.15.107:0/3998675948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15a1f6b0 0x559b15bc32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.413+0000 7f9a437fe700 1 -- 172.21.15.107:0/3998675948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15bc3830 msgr2=0x559b15bc4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.413+0000 7f9a437fe700 1 --2- 172.21.15.107:0/3998675948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15bc3830 0x559b15bc4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.413+0000 7f9a437fe700 1 -- 172.21.15.107:0/3998675948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15a1f6b0 msgr2=0x559b15bc32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.413+0000 7f9a437fe700 1 --2- 172.21.15.107:0/3998675948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15a1f6b0 0x559b15bc32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.413+0000 7f9a437fe700 1 -- 172.21.15.107:0/3998675948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b15942d80 con 0x559b15bc4580 2023-07-21T22:36:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.413+0000 7f9a42ffd700 1 --2- 172.21.15.107:0/3998675948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15a1f6b0 0x559b15bc32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:36:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.413+0000 7f9a437fe700 1 --2- 172.21.15.107:0/3998675948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc4580 0x559b15bc49f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00f5a0 tx=0x7f9a3c00f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee15451db4a0faf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.413+0000 7f9a39ffb700 1 -- 172.21.15.107:0/3998675948 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c004bb0 con 0x559b15bc4580 2023-07-21T22:36:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.413+0000 7f9a39ffb700 1 -- 172.21.15.107:0/3998675948 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a3c004d10 con 0x559b15bc4580 2023-07-21T22:36:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.413+0000 7f9a39ffb700 1 -- 172.21.15.107:0/3998675948 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c005040 con 0x559b15bc4580 2023-07-21T22:36:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3998675948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc4580 msgr2=0x559b15bc49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3998675948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc4580 0x559b15bc49f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00f5a0 tx=0x7f9a3c00f8b0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3998675948 shutdown_connections 2023-07-21T22:36:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3998675948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc4580 0x559b15bc49f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3998675948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15a1f6b0 0x559b15bc32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3998675948 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15bc3830 0x559b15bc4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3998675948 >> 172.21.15.107:0/3998675948 conn(0x559b15bb8ae0 msgr2=0x559b15bbdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3998675948 shutdown_connections 2023-07-21T22:36:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3998675948 wait complete. 2023-07-21T22:36:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 -- start start 2023-07-21T22:36:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 Processor -- start 2023-07-21T22:36:10.469 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15a1f6b0 0x559b15bcc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 0x559b15bccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 0x559b15bd3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15ab4a60 con 0x559b15bc3830 2023-07-21T22:36:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.414+0000 7f9a55b2eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15ab4760 con 0x559b15bc4580 2023-07-21T22:36:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a55b2eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b159988c0 con 0x559b15a1f6b0 2023-07-21T22:36:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a55b2eb80 10 client.0 _ll_register_callbacks cb 0x559b15ab2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a427fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 0x559b15bccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.470 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a427fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 0x559b15bccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50948/0 (socket says 172.21.15.107:50948) 2023-07-21T22:36:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a427fc700 1 -- 172.21.15.107:0/3716030158 learned_addr learned my addr 172.21.15.107:0/3716030158 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a42ffd700 1 --2- 172.21.15.107:0/3716030158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15a1f6b0 0x559b15bcc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a437fe700 1 --2- 172.21.15.107:0/3716030158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 0x559b15bd3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a427fc700 1 -- 172.21.15.107:0/3716030158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15a1f6b0 msgr2=0x559b15bcc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a427fc700 1 --2- 172.21.15.107:0/3716030158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15a1f6b0 0x559b15bcc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a427fc700 1 -- 172.21.15.107:0/3716030158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 msgr2=0x559b15bd3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a427fc700 1 --2- 172.21.15.107:0/3716030158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 0x559b15bd3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a427fc700 1 -- 172.21.15.107:0/3716030158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c00f1d0 con 0x559b15bc3830 2023-07-21T22:36:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a427fc700 1 --2- 172.21.15.107:0/3716030158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 0x559b15bccc60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009990 tx=0x7f9a34009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693a5436c3f5ef6f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a22ffd700 1 -- 172.21.15.107:0/3716030158 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34024620 con 0x559b15bc3830 2023-07-21T22:36:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a22ffd700 1 -- 172.21.15.107:0/3716030158 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9a34024780 con 0x559b15bc3830 2023-07-21T22:36:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3716030158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b15a6e490 con 0x559b15bc3830 2023-07-21T22:36:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a22ffd700 1 -- 172.21.15.107:0/3716030158 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a34024940 con 0x559b15bc3830 2023-07-21T22:36:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a55b2eb80 10 client.16512 fetch_fsmap learned FSMap version 45 2023-07-21T22:36:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.415+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3716030158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x559b1592bc00 con 0x559b15bc3830 2023-07-21T22:36:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.416+0000 7f9a22ffd700 1 -- 172.21.15.107:0/3716030158 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a3403f020 con 0x559b15bc3830 2023-07-21T22:36:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.416+0000 7f9a22ffd700 1 -- 172.21.15.107:0/3716030158 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 45) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f9a3402f070 con 0x559b15bc3830 2023-07-21T22:36:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.416+0000 7f9a55b2eb80 10 client.16512 fetch_fsmap finished waiting for FSMap version 45 2023-07-21T22:36:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.416+0000 7f9a55b2eb80 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.416+0000 7f9a55b2eb80 20 client.16512 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.416+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3716030158 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559b15bd86b0 0x559b15bdaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.416+0000 7f9a55b2eb80 4 client.16512 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.416+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3716030158 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b15b0de50 con 0x559b15bd86b0 2023-07-21T22:36:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.416+0000 7f9a42ffd700 1 --2- 172.21.15.107:0/3716030158 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559b15bd86b0 0x559b15bdaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.416+0000 7f9a22ffd700 10 client.16512 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.416+0000 7f9a42ffd700 1 --2- 172.21.15.107:0/3716030158 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559b15bd86b0 0x559b15bdaac0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473aa362988ff1a2 server_cookie=984d3291073338ae in_seq=0 out_seq=0 2023-07-21T22:36:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a22ffd700 1 -- 172.21.15.107:0/3716030158 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559b15b0de50 con 0x559b15bd86b0 2023-07-21T22:36:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a22ffd700 10 client.16512 handle_command_reply: tid=0 2023-07-21T22:36:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a55b2eb80 1 client.16512 shutdown 2023-07-21T22:36:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3716030158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 msgr2=0x559b15bccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3716030158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 0x559b15bccc60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009990 tx=0x7f9a34009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3716030158 shutdown_connections 2023-07-21T22:36:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3716030158 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559b15bd86b0 0x559b15bdaac0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3716030158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 0x559b15bd3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a22ffd700 0 client.16512 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3716030158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15a1f6b0 0x559b15bcc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3716030158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 0x559b15bccc60 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3716030158 >> 172.21.15.107:0/3716030158 conn(0x559b15bb8ae0 msgr2=0x559b15bbd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3716030158 shutdown_connections 2023-07-21T22:36:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3716030158 wait complete. 2023-07-21T22:36:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.417+0000 7f9a55b2eb80 20 client.16512 trim_cache size 0 max 16384 2023-07-21T22:36:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a55b2eb80 1 Processor -- start 2023-07-21T22:36:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a55b2eb80 1 -- start start 2023-07-21T22:36:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a55b2eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15a1f6b0 0x559b15bcb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a55b2eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 0x559b15bd3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a55b2eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 0x559b15bcc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a55b2eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15bcc2f0 con 0x559b15bc3830 2023-07-21T22:36:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a55b2eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15bcbbd0 con 0x559b15bc4580 2023-07-21T22:36:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a55b2eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15bbac00 con 0x559b15a1f6b0 2023-07-21T22:36:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a427fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 0x559b15bd3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a42ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15a1f6b0 0x559b15bcb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a42ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15a1f6b0 0x559b15bcb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39512/0 (socket says 172.21.15.107:39512) 2023-07-21T22:36:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a42ffd700 1 -- 172.21.15.107:0/3339840156 learned_addr learned my addr 172.21.15.107:0/3339840156 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a427fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 0x559b15bd3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50956/0 (socket says 172.21.15.107:50956) 2023-07-21T22:36:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.432+0000 7f9a437fe700 1 --2- 172.21.15.107:0/3339840156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 0x559b15bcc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.433+0000 7f9a427fc700 1 -- 172.21.15.107:0/3339840156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15a1f6b0 msgr2=0x559b15bcb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.433+0000 7f9a427fc700 1 --2- 172.21.15.107:0/3339840156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15a1f6b0 0x559b15bcb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.433+0000 7f9a427fc700 1 -- 172.21.15.107:0/3339840156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 msgr2=0x559b15bcc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.433+0000 7f9a427fc700 1 --2- 172.21.15.107:0/3339840156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 0x559b15bcc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.433+0000 7f9a427fc700 1 -- 172.21.15.107:0/3339840156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b159691f0 con 0x559b15bc3830 2023-07-21T22:36:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.433+0000 7f9a42ffd700 1 --2- 172.21.15.107:0/3339840156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15a1f6b0 0x559b15bcb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.433+0000 7f9a427fc700 1 --2- 172.21.15.107:0/3339840156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 0x559b15bd3c60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9a34039b20 tx=0x7f9a34005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adaabb871153095a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.433+0000 7f9a3a7fc700 1 -- 172.21.15.107:0/3339840156 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34039ec0 con 0x559b15bc3830 2023-07-21T22:36:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.433+0000 7f9a3a7fc700 1 -- 172.21.15.107:0/3339840156 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a3403f070 con 0x559b15bc3830 2023-07-21T22:36:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.433+0000 7f9a3a7fc700 1 -- 172.21.15.107:0/3339840156 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34024e50 con 0x559b15bc3830 2023-07-21T22:36:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3339840156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 msgr2=0x559b15bd3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3339840156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 0x559b15bd3c60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9a34039b20 tx=0x7f9a34005a50 comp rx=0 tx=0).stop 2023-07-21T22:36:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3339840156 shutdown_connections 2023-07-21T22:36:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3339840156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 0x559b15bcc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.486 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3339840156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15a1f6b0 0x559b15bcb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.486 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/3339840156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15bc3830 0x559b15bd3c60 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.486 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3339840156 >> 172.21.15.107:0/3339840156 conn(0x559b15bb8ae0 msgr2=0x559b15bdf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:10.486 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3339840156 shutdown_connections 2023-07-21T22:36:10.486 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/3339840156 wait complete. 2023-07-21T22:36:10.486 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 -- start start 2023-07-21T22:36:10.486 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 Processor -- start 2023-07-21T22:36:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15a1f6b0 0x559b15bd1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15bc3830 0x559b15c28500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 0x559b15c2ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15c2b280 con 0x559b15a1f6b0 2023-07-21T22:36:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15c2b3f0 con 0x559b15bc4580 2023-07-21T22:36:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b15c2b5b0 con 0x559b15bc3830 2023-07-21T22:36:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a55b2eb80 10 client.0 _ll_register_callbacks cb 0x7f9a3400be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a42ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15a1f6b0 0x559b15bd1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.488 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a42ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15a1f6b0 0x559b15bd1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50960/0 (socket says 172.21.15.107:50960) 2023-07-21T22:36:10.488 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.434+0000 7f9a42ffd700 1 -- 172.21.15.107:0/2382520276 learned_addr learned my addr 172.21.15.107:0/2382520276 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:10.488 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a427fc700 1 --2- 172.21.15.107:0/2382520276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15bc3830 0x559b15c28500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.488 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a437fe700 1 --2- 172.21.15.107:0/2382520276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 0x559b15c2ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.488 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a42ffd700 1 -- 172.21.15.107:0/2382520276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15bc3830 msgr2=0x559b15c28500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:10.488 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a42ffd700 1 --2- 172.21.15.107:0/2382520276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15bc3830 0x559b15c28500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.488 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a42ffd700 1 -- 172.21.15.107:0/2382520276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 msgr2=0x559b15c2ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a42ffd700 1 --2- 172.21.15.107:0/2382520276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 0x559b15c2ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a42ffd700 1 -- 172.21.15.107:0/2382520276 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4419e0e0 con 0x559b15a1f6b0 2023-07-21T22:36:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a42ffd700 1 --2- 172.21.15.107:0/2382520276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15a1f6b0 0x559b15bd1c20 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c6b0 tx=0x7f9a2c09e110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4741462fffca2990 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a22ffd700 1 -- 172.21.15.107:0/2382520276 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0b2070 con 0x559b15a1f6b0 2023-07-21T22:36:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a22ffd700 1 -- 172.21.15.107:0/2382520276 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9a2c003680 con 0x559b15a1f6b0 2023-07-21T22:36:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/2382520276 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a3c004d10 con 0x559b15a1f6b0 2023-07-21T22:36:10.489 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a22ffd700 1 -- 172.21.15.107:0/2382520276 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a2c098160 con 0x559b15a1f6b0 2023-07-21T22:36:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a55b2eb80 10 client.16518 fetch_fsmap learned FSMap version 45 2023-07-21T22:36:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/2382520276 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x559b1592bc00 con 0x559b15a1f6b0 2023-07-21T22:36:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a22ffd700 1 -- 172.21.15.107:0/2382520276 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a3c004d10 con 0x559b15a1f6b0 2023-07-21T22:36:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.435+0000 7f9a22ffd700 1 -- 172.21.15.107:0/2382520276 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 45) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f9a2c0b8020 con 0x559b15a1f6b0 2023-07-21T22:36:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.436+0000 7f9a55b2eb80 10 client.16518 fetch_fsmap finished waiting for FSMap version 45 2023-07-21T22:36:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.436+0000 7f9a55b2eb80 10 client.16518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.436+0000 7f9a55b2eb80 20 client.16518 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.436+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/2382520276 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559b15c2cfe0 0x559b15c2f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.436+0000 7f9a55b2eb80 4 client.16518 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:36:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.436+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/2382520276 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559b15acfff0 con 0x559b15c2cfe0 2023-07-21T22:36:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.436+0000 7f9a427fc700 1 --2- 172.21.15.107:0/2382520276 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559b15c2cfe0 0x559b15c2f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.436+0000 7f9a22ffd700 10 client.16518 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:10.491 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.436+0000 7f9a427fc700 1 --2- 172.21.15.107:0/2382520276 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559b15c2cfe0 0x559b15c2f3c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb3a6de70524cd9b server_cookie=2a797d88725f275 in_seq=0 out_seq=0 2023-07-21T22:36:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.719+0000 7f9a22ffd700 1 -- 172.21.15.107:0/2382520276 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559b15acfff0 con 0x559b15c2cfe0 2023-07-21T22:36:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.719+0000 7f9a22ffd700 10 client.16518 handle_command_reply: tid=0 2023-07-21T22:36:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.719+0000 7f9a55b2eb80 1 client.16518 shutdown 2023-07-21T22:36:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/2382520276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15a1f6b0 msgr2=0x559b15bd1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/2382520276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15a1f6b0 0x559b15bd1c20 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c6b0 tx=0x7f9a2c09e110 comp rx=0 tx=0).stop 2023-07-21T22:36:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/2382520276 shutdown_connections 2023-07-21T22:36:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/2382520276 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559b15c2cfe0 0x559b15c2f3c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/2382520276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559b15bc4580 0x559b15c2ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/2382520276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b15a1f6b0 0x559b15bd1c20 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a22ffd700 0 client.16518 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a55b2eb80 1 --2- 172.21.15.107:0/2382520276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559b15bc3830 0x559b15c28500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/2382520276 >> 172.21.15.107:0/2382520276 conn(0x559b15bb8ae0 msgr2=0x559b15bca840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/2382520276 shutdown_connections 2023-07-21T22:36:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a55b2eb80 1 -- 172.21.15.107:0/2382520276 wait complete. 2023-07-21T22:36:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a55b2eb80 20 client.16518 trim_cache size 0 max 16384 2023-07-21T22:36:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:36:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:36:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:36:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:36:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a499a9700 1 -- 172.21.15.107:0/2754316158 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a300a1150 msgr2=0x7f9a300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a499a9700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a300a1150 0x7f9a300a3600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400b2e0 tx=0x7f9a34005960 comp rx=0 tx=0).stop 2023-07-21T22:36:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a499a9700 1 -- 172.21.15.107:0/2754316158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a440fe180 msgr2=0x7f9a441958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a499a9700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a440fe180 0x7f9a441958f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00ea60 tx=0x7f9a2c00ed70 comp rx=0 tx=0).stop 2023-07-21T22:36:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.720+0000 7f9a499a9700 1 -- 172.21.15.107:0/2754316158 shutdown_connections 2023-07-21T22:36:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.721+0000 7f9a499a9700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9a300a1150 0x7f9a300a3600 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.721+0000 7f9a499a9700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a440ffd00 0x7f9a44196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.721+0000 7f9a499a9700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a440fe180 0x7f9a441958f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.721+0000 7f9a499a9700 1 --2- 172.21.15.107:0/2754316158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a440ff370 0x7f9a44195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.721+0000 7f9a499a9700 1 -- 172.21.15.107:0/2754316158 >> 172.21.15.107:0/2754316158 conn(0x7f9a440f9780 msgr2=0x7f9a440fb970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.721+0000 7f9a499a9700 1 -- 172.21.15.107:0/2754316158 shutdown_connections 2023-07-21T22:36:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:10.721+0000 7f9a499a9700 1 -- 172.21.15.107:0/2754316158 wait complete. 2023-07-21T22:36:12.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:11 smithi132 ceph-mon[97863]: pgmap v507: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 399 op/s 2023-07-21T22:36:12.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:11 smithi132 ceph-mon[97863]: scrub summary: idle 2023-07-21T22:36:12.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:11 smithi132 ceph-mon[97863]: scrub summary: idle 2023-07-21T22:36:12.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:11 smithi132 ceph-mon[97863]: scrub summary: idle 2023-07-21T22:36:12.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:11 smithi132 ceph-mon[97863]: scrub summary: idle 2023-07-21T22:36:12.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:36:12.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:36:12.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:11 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:36:12.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:11 smithi107 ceph-mon[96543]: pgmap v507: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 399 op/s 2023-07-21T22:36:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:11 smithi107 ceph-mon[96543]: scrub summary: idle 2023-07-21T22:36:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:11 smithi107 ceph-mon[96543]: scrub summary: idle 2023-07-21T22:36:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:11 smithi107 ceph-mon[96543]: scrub summary: idle 2023-07-21T22:36:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:11 smithi107 ceph-mon[96543]: scrub summary: idle 2023-07-21T22:36:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:36:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:36:12.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:11 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:36:12.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:11 smithi190 ceph-mon[98334]: pgmap v507: 97 pgs: 97 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 399 op/s 2023-07-21T22:36:12.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:11 smithi190 ceph-mon[98334]: scrub summary: idle 2023-07-21T22:36:12.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:11 smithi190 ceph-mon[98334]: scrub summary: idle 2023-07-21T22:36:12.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:11 smithi190 ceph-mon[98334]: scrub summary: idle 2023-07-21T22:36:12.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:11 smithi190 ceph-mon[98334]: scrub summary: idle 2023-07-21T22:36:12.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:36:12.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:36:12.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:11 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:36:14.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:13 smithi132 ceph-mon[97863]: pgmap v508: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 492 op/s 2023-07-21T22:36:14.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:13 smithi107 ceph-mon[96543]: pgmap v508: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 492 op/s 2023-07-21T22:36:14.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:13 smithi190 ceph-mon[98334]: pgmap v508: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 492 op/s 2023-07-21T22:36:16.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:15 smithi132 ceph-mon[97863]: pgmap v509: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 460 op/s 2023-07-21T22:36:16.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:15 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:16.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:15 smithi107 ceph-mon[96543]: pgmap v509: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 460 op/s 2023-07-21T22:36:16.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:15 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:16.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:15 smithi190 ceph-mon[98334]: pgmap v509: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 460 op/s 2023-07-21T22:36:16.391 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:15 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:18.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:17 smithi132 ceph-mon[97863]: pgmap v510: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 459 op/s 2023-07-21T22:36:18.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:17 smithi107 ceph-mon[96543]: pgmap v510: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 459 op/s 2023-07-21T22:36:18.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:17 smithi190 ceph-mon[98334]: pgmap v510: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 459 op/s 2023-07-21T22:36:20.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:19 smithi107 ceph-mon[96543]: pgmap v511: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 523 op/s 2023-07-21T22:36:20.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:19 smithi190 ceph-mon[98334]: pgmap v511: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 523 op/s 2023-07-21T22:36:20.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:19 smithi132 ceph-mon[97863]: pgmap v511: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 523 op/s 2023-07-21T22:36:22.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:21 smithi190 ceph-mon[98334]: pgmap v512: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 417 op/s 2023-07-21T22:36:22.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:21 smithi132 ceph-mon[97863]: pgmap v512: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 417 op/s 2023-07-21T22:36:22.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:21 smithi107 ceph-mon[96543]: pgmap v512: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 417 op/s 2023-07-21T22:36:23.390 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:22 smithi190 ceph-mon[98334]: pgmap v513: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 451 op/s 2023-07-21T22:36:23.431 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:22 smithi132 ceph-mon[97863]: pgmap v513: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 451 op/s 2023-07-21T22:36:23.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:22 smithi107 ceph-mon[96543]: pgmap v513: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 451 op/s 2023-07-21T22:36:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.833+0000 7f55e7d2c700 1 -- 172.21.15.107:0/2889301786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00fea00 msgr2=0x7f55e00fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.833+0000 7f55e7d2c700 1 --2- 172.21.15.107:0/2889301786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00fea00 0x7f55e00fee10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009990 tx=0x7f55d0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.833+0000 7f55e7d2c700 1 -- 172.21.15.107:0/2889301786 shutdown_connections 2023-07-21T22:36:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.833+0000 7f55e7d2c700 1 --2- 172.21.15.107:0/2889301786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55e0100580 0x7f55e0106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.833+0000 7f55e7d2c700 1 --2- 172.21.15.107:0/2889301786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00fea00 0x7f55e00fee10 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.833+0000 7f55e7d2c700 1 --2- 172.21.15.107:0/2889301786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55e00ffbf0 0x7f55e0100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.833+0000 7f55e7d2c700 1 -- 172.21.15.107:0/2889301786 >> 172.21.15.107:0/2889301786 conn(0x7f55e00f9fe0 msgr2=0x7f55e00fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.833+0000 7f55e7d2c700 1 -- 172.21.15.107:0/2889301786 shutdown_connections 2023-07-21T22:36:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.833+0000 7f55e7d2c700 1 -- 172.21.15.107:0/2889301786 wait complete. 2023-07-21T22:36:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.833+0000 7f55e7d2c700 1 Processor -- start 2023-07-21T22:36:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e7d2c700 1 -- start start 2023-07-21T22:36:23.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e7d2c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00fea00 0x7f55e01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e7d2c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55e00ffbf0 0x7f55e019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e7d2c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55e0100580 0x7f55e019d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e7d2c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e019d640 con 0x7f55e00fea00 2023-07-21T22:36:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e7d2c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e019d780 con 0x7f55e0100580 2023-07-21T22:36:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e7d2c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e019d940 con 0x7f55e00ffbf0 2023-07-21T22:36:23.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e5ac8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00fea00 0x7f55e01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e5ac8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00fea00 0x7f55e01985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50974/0 (socket says 172.21.15.107:50974) 2023-07-21T22:36:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e5ac8700 1 -- 172.21.15.107:0/3521825275 learned_addr learned my addr 172.21.15.107:0/3521825275 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e52c7700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55e00ffbf0 0x7f55e019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e62c9700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55e0100580 0x7f55e019d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e5ac8700 1 -- 172.21.15.107:0/3521825275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55e00ffbf0 msgr2=0x7f55e019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e5ac8700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55e00ffbf0 0x7f55e019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e5ac8700 1 -- 172.21.15.107:0/3521825275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55e0100580 msgr2=0x7f55e019d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e5ac8700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55e0100580 0x7f55e019d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e5ac8700 1 -- 172.21.15.107:0/3521825275 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d00095f0 con 0x7f55e00fea00 2023-07-21T22:36:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e5ac8700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00fea00 0x7f55e01985d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f55d00098e0 tx=0x7f55d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55d6ffd700 1 -- 172.21.15.107:0/3521825275 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d001d070 con 0x7f55e00fea00 2023-07-21T22:36:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55e7d2c700 1 -- 172.21.15.107:0/3521825275 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e019dbc0 con 0x7f55e00fea00 2023-07-21T22:36:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55d6ffd700 1 -- 172.21.15.107:0/3521825275 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d0005030 con 0x7f55e00fea00 2023-07-21T22:36:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.834+0000 7f55d6ffd700 1 -- 172.21.15.107:0/3521825275 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d0004b50 con 0x7f55e00fea00 2023-07-21T22:36:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.835+0000 7f55e7d2c700 1 -- 172.21.15.107:0/3521825275 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e019df80 con 0x7f55e00fea00 2023-07-21T22:36:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.837+0000 7f55d6ffd700 1 -- 172.21.15.107:0/3521825275 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f55d0004d30 con 0x7f55e00fea00 2023-07-21T22:36:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.837+0000 7f55d6ffd700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f55cc0a1150 0x7f55cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.837+0000 7f55e52c7700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f55cc0a1150 0x7f55cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.837+0000 7f55d6ffd700 1 -- 172.21.15.107:0/3521825275 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f55d00cdc10 con 0x7f55e00fea00 2023-07-21T22:36:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55e52c7700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f55cc0a1150 0x7f55cc0a3600 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f55e0068950 tx=0x7f55dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55f3eb1b80 1 Processor -- start 2023-07-21T22:36:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55f3eb1b80 1 -- start start 2023-07-21T22:36:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55f3eb1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 0x55b05ad9b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55f3eb1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 0x55b05ad9c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55f3eb1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05ad9c580 0x55b05ad9c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55f3eb1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05aca0c50 con 0x55b05abf76b0 2023-07-21T22:36:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55f3eb1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05acf39e0 con 0x55b05ad9c580 2023-07-21T22:36:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55f3eb1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05acece70 con 0x55b05ad9b830 2023-07-21T22:36:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55e5ac8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 0x55b05ad9b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55e52c7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 0x55b05ad9c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55e5ac8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 0x55b05ad9b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50982/0 (socket says 172.21.15.107:50982) 2023-07-21T22:36:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55e5ac8700 1 -- 172.21.15.107:0/484753739 learned_addr learned my addr 172.21.15.107:0/484753739 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.838+0000 7f55e62c9700 1 --2- 172.21.15.107:0/484753739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05ad9c580 0x55b05ad9c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.839+0000 7f55e5ac8700 1 -- 172.21.15.107:0/484753739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 msgr2=0x55b05ad9c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.839+0000 7f55e5ac8700 1 --2- 172.21.15.107:0/484753739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 0x55b05ad9c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.839+0000 7f55e5ac8700 1 -- 172.21.15.107:0/484753739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05ad9c580 msgr2=0x55b05ad9c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.839+0000 7f55e5ac8700 1 --2- 172.21.15.107:0/484753739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05ad9c580 0x55b05ad9c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.839+0000 7f55e5ac8700 1 -- 172.21.15.107:0/484753739 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b05ab1ad80 con 0x55b05abf76b0 2023-07-21T22:36:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.839+0000 7f55e5ac8700 1 --2- 172.21.15.107:0/484753739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 0x55b05ad9b2f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f55d00a2ec0 tx=0x7f55d009aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1612db7e07ed044 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.839+0000 7f55cbfff700 1 -- 172.21.15.107:0/484753739 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d00a33f0 con 0x55b05abf76b0 2023-07-21T22:36:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.839+0000 7f55cbfff700 1 -- 172.21.15.107:0/484753739 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d00051a0 con 0x55b05abf76b0 2023-07-21T22:36:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.839+0000 7f55cbfff700 1 -- 172.21.15.107:0/484753739 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d00a2f60 con 0x55b05abf76b0 2023-07-21T22:36:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.840+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/484753739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 msgr2=0x55b05ad9b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.840+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/484753739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 0x55b05ad9b2f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f55d00a2ec0 tx=0x7f55d009aca0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.840+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/484753739 shutdown_connections 2023-07-21T22:36:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.840+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/484753739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05ad9c580 0x55b05ad9c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.840+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/484753739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 0x55b05ad9b2f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.840+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/484753739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 0x55b05ad9c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.840+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/484753739 >> 172.21.15.107:0/484753739 conn(0x55b05ad90ae0 msgr2=0x55b05ad95c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.840+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/484753739 shutdown_connections 2023-07-21T22:36:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.840+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/484753739 wait complete. 2023-07-21T22:36:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.840+0000 7f55f3eb1b80 1 -- start start 2023-07-21T22:36:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.840+0000 7f55f3eb1b80 1 Processor -- start 2023-07-21T22:36:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55f3eb1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 0x55b05ada4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55f3eb1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05ad9b830 0x55b05ada4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55f3eb1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9c580 0x55b05adab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55f3eb1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05ac8ca60 con 0x55b05abf76b0 2023-07-21T22:36:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55f3eb1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05ac8c760 con 0x55b05ad9b830 2023-07-21T22:36:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55f3eb1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05ab708c0 con 0x55b05ad9c580 2023-07-21T22:36:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55f3eb1b80 10 client.0 _ll_register_callbacks cb 0x55b05ac8a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55e5ac8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 0x55b05ada4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55e5ac8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 0x55b05ada4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50988/0 (socket says 172.21.15.107:50988) 2023-07-21T22:36:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55e5ac8700 1 -- 172.21.15.107:0/782361444 learned_addr learned my addr 172.21.15.107:0/782361444 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55e52c7700 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05ad9b830 0x55b05ada4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55e62c9700 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9c580 0x55b05adab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.899 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55e5ac8700 1 -- 172.21.15.107:0/782361444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9c580 msgr2=0x55b05adab9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55e5ac8700 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9c580 0x55b05adab9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55e5ac8700 1 -- 172.21.15.107:0/782361444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05ad9b830 msgr2=0x55b05ada4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55e5ac8700 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05ad9b830 0x55b05ada4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55e5ac8700 1 -- 172.21.15.107:0/782361444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e019df80 con 0x55b05abf76b0 2023-07-21T22:36:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55e52c7700 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05ad9b830 0x55b05ada4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55e5ac8700 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 0x55b05ada4440 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f55d00095c0 tx=0x7f55d009a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4e86b3dda377e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55c97fa700 1 -- 172.21.15.107:0/782361444 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d00a33f0 con 0x55b05abf76b0 2023-07-21T22:36:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55c97fa700 1 -- 172.21.15.107:0/782361444 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f55d00a2f60 con 0x55b05abf76b0 2023-07-21T22:36:23.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/782361444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b05ac46490 con 0x55b05abf76b0 2023-07-21T22:36:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.841+0000 7f55c97fa700 1 -- 172.21.15.107:0/782361444 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55d00a3120 con 0x55b05abf76b0 2023-07-21T22:36:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.842+0000 7f55f3eb1b80 10 client.16530 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.842+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/782361444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55b05ab03c00 con 0x55b05abf76b0 2023-07-21T22:36:23.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.842+0000 7f55c97fa700 1 -- 172.21.15.107:0/782361444 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b05ac46490 con 0x55b05abf76b0 2023-07-21T22:36:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.842+0000 7f55c97fa700 1 -- 172.21.15.107:0/782361444 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f55d00cbb50 con 0x55b05abf76b0 2023-07-21T22:36:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.842+0000 7f55f3eb1b80 10 client.16530 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.842+0000 7f55f3eb1b80 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.842+0000 7f55f3eb1b80 20 client.16530 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.842+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55b05adafe10 0x55b05adb2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.842+0000 7f55f3eb1b80 4 client.16530 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.842+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/782361444 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b05ace5e50 con 0x55b05adafe10 2023-07-21T22:36:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.842+0000 7f55e52c7700 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55b05adafe10 0x55b05adb2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55e52c7700 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55b05adafe10 0x55b05adb2220 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e68215f0b2d82799 server_cookie=5c9a4bd0b9674300 in_seq=0 out_seq=0 2023-07-21T22:36:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55c97fa700 10 client.16530 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55c97fa700 1 -- 172.21.15.107:0/782361444 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b05ace5e50 con 0x55b05adafe10 2023-07-21T22:36:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55c97fa700 10 client.16530 handle_command_reply: tid=0 2023-07-21T22:36:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55f3eb1b80 1 client.16530 shutdown 2023-07-21T22:36:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/782361444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 msgr2=0x55b05ada4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 0x55b05ada4440 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f55d00095c0 tx=0x7f55d009a8e0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/782361444 shutdown_connections 2023-07-21T22:36:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55b05adafe10 0x55b05adb2220 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9c580 0x55b05adab9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55c97fa700 0 client.16530 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05abf76b0 0x55b05ada4440 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/782361444 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05ad9b830 0x55b05ada4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/782361444 >> 172.21.15.107:0/782361444 conn(0x55b05ad90ae0 msgr2=0x55b05ad953e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/782361444 shutdown_connections 2023-07-21T22:36:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.843+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/782361444 wait complete. 2023-07-21T22:36:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.844+0000 7f55f3eb1b80 20 client.16530 trim_cache size 0 max 16384 2023-07-21T22:36:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.858+0000 7f55f3eb1b80 1 Processor -- start 2023-07-21T22:36:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.858+0000 7f55f3eb1b80 1 -- start start 2023-07-21T22:36:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.858+0000 7f55f3eb1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05abf76b0 0x55b05ada4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.858+0000 7f55f3eb1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 0x55b05adab980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.858+0000 7f55f3eb1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 0x55b05ada2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.858+0000 7f55f3eb1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05ad60fc0 con 0x55b05ad9c580 2023-07-21T22:36:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.858+0000 7f55f3eb1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05ace5e80 con 0x55b05abf76b0 2023-07-21T22:36:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.858+0000 7f55f3eb1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05ad9bed0 con 0x55b05ad9b830 2023-07-21T22:36:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.858+0000 7f55e62c9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 0x55b05ada2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.858+0000 7f55e52c7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 0x55b05adab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.858+0000 7f55e62c9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 0x55b05ada2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51000/0 (socket says 172.21.15.107:51000) 2023-07-21T22:36:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.858+0000 7f55e62c9700 1 -- 172.21.15.107:0/356790550 learned_addr learned my addr 172.21.15.107:0/356790550 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.858+0000 7f55e5ac8700 1 --2- 172.21.15.107:0/356790550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05abf76b0 0x55b05ada4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55e62c9700 1 -- 172.21.15.107:0/356790550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 msgr2=0x55b05adab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55e62c9700 1 --2- 172.21.15.107:0/356790550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 0x55b05adab980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55e62c9700 1 -- 172.21.15.107:0/356790550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05abf76b0 msgr2=0x55b05ada4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55e62c9700 1 --2- 172.21.15.107:0/356790550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05abf76b0 0x55b05ada4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55e62c9700 1 -- 172.21.15.107:0/356790550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b05ab411f0 con 0x55b05ad9c580 2023-07-21T22:36:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55e52c7700 1 --2- 172.21.15.107:0/356790550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 0x55b05adab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55e62c9700 1 --2- 172.21.15.107:0/356790550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 0x55b05ada2800 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009d80 tx=0x7f55d8015920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7b75c346f68abd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55d4ff9700 1 -- 172.21.15.107:0/356790550 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d801a050 con 0x55b05ad9c580 2023-07-21T22:36:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55d4ff9700 1 -- 172.21.15.107:0/356790550 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d801b040 con 0x55b05ad9c580 2023-07-21T22:36:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55d4ff9700 1 -- 172.21.15.107:0/356790550 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d801fb00 con 0x55b05ad9c580 2023-07-21T22:36:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/356790550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 msgr2=0x55b05ada2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/356790550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 0x55b05ada2800 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009d80 tx=0x7f55d8015920 comp rx=0 tx=0).stop 2023-07-21T22:36:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/356790550 shutdown_connections 2023-07-21T22:36:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/356790550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 0x55b05ada2800 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/356790550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05abf76b0 0x55b05ada4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/356790550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 0x55b05adab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.859+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/356790550 >> 172.21.15.107:0/356790550 conn(0x55b05ad90ae0 msgr2=0x55b05ada41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/356790550 shutdown_connections 2023-07-21T22:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/356790550 wait complete. 2023-07-21T22:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55f3eb1b80 1 -- start start 2023-07-21T22:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55f3eb1b80 1 Processor -- start 2023-07-21T22:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55f3eb1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05abf76b0 0x55b05ada78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.915 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55f3eb1b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 0x55b05ada7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55f3eb1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 0x55b05adaae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55f3eb1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05adab350 con 0x55b05ad9c580 2023-07-21T22:36:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55f3eb1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05adab4c0 con 0x55b05abf76b0 2023-07-21T22:36:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55f3eb1b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b05adab680 con 0x55b05ad9b830 2023-07-21T22:36:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55f3eb1b80 10 client.0 _ll_register_callbacks cb 0x7f55d80094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:23.916 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55e62c9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 0x55b05adaae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55e62c9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 0x55b05adaae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51006/0 (socket says 172.21.15.107:51006) 2023-07-21T22:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55e5ac8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05abf76b0 0x55b05ada78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55e52c7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 0x55b05ada7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.860+0000 7f55e62c9700 1 -- 172.21.15.107:0/2624390701 learned_addr learned my addr 172.21.15.107:0/2624390701 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55e62c9700 1 -- 172.21.15.107:0/2624390701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 msgr2=0x55b05ada7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55e62c9700 1 --2- 172.21.15.107:0/2624390701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 0x55b05ada7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55e62c9700 1 -- 172.21.15.107:0/2624390701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05abf76b0 msgr2=0x55b05ada78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:23.917 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55e62c9700 1 --2- 172.21.15.107:0/2624390701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05abf76b0 0x55b05ada78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55e62c9700 1 -- 172.21.15.107:0/2624390701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55dc017070 con 0x55b05ad9c580 2023-07-21T22:36:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55e5ac8700 1 --2- 172.21.15.107:0/2624390701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05abf76b0 0x55b05ada78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55e62c9700 1 --2- 172.21.15.107:0/2624390701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 0x55b05adaae10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009d50 tx=0x7f55d801ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b440e2c5505676 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55c97fa700 1 -- 172.21.15.107:0/2624390701 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8007910 con 0x55b05ad9c580 2023-07-21T22:36:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/2624390701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55d00051a0 con 0x55b05ad9c580 2023-07-21T22:36:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55c97fa700 1 -- 172.21.15.107:0/2624390701 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f55d8007a70 con 0x55b05ad9c580 2023-07-21T22:36:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55c97fa700 1 -- 172.21.15.107:0/2624390701 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55d801a020 con 0x55b05ad9c580 2023-07-21T22:36:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55f3eb1b80 10 client.16536 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/2624390701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f55d8009a30 con 0x55b05ad9c580 2023-07-21T22:36:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55c97fa700 1 -- 172.21.15.107:0/2624390701 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d00051a0 con 0x55b05ad9c580 2023-07-21T22:36:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55c97fa700 1 -- 172.21.15.107:0/2624390701 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f55d8027380 con 0x55b05ad9c580 2023-07-21T22:36:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55f3eb1b80 10 client.16536 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55f3eb1b80 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:23.919 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55f3eb1b80 20 client.16536 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/2624390701 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55b05ae04d40 0x55b05ae07150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55f3eb1b80 4 client.16536 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:36:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.861+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/2624390701 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b05aca7ff0 con 0x55b05ae04d40 2023-07-21T22:36:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.862+0000 7f55e5ac8700 1 --2- 172.21.15.107:0/2624390701 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55b05ae04d40 0x55b05ae07150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:23.920 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.862+0000 7f55c97fa700 10 client.16536 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.862+0000 7f55e5ac8700 1 --2- 172.21.15.107:0/2624390701 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55b05ae04d40 0x55b05ae07150 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a52bcec92cad56 server_cookie=389a45df4e5e274 in_seq=0 out_seq=0 2023-07-21T22:36:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55c97fa700 1 -- 172.21.15.107:0/2624390701 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b05aca7ff0 con 0x55b05ae04d40 2023-07-21T22:36:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55c97fa700 10 client.16536 handle_command_reply: tid=0 2023-07-21T22:36:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55f3eb1b80 1 client.16536 shutdown 2023-07-21T22:36:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/2624390701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 msgr2=0x55b05adaae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/2624390701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 0x55b05adaae10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009d50 tx=0x7f55d801ff00 comp rx=0 tx=0).stop 2023-07-21T22:36:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/2624390701 shutdown_connections 2023-07-21T22:36:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/2624390701 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55b05ae04d40 0x55b05ae07150 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/2624390701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b05ad9c580 0x55b05adaae10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/2624390701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b05abf76b0 0x55b05ada78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55c97fa700 0 client.16536 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55f3eb1b80 1 --2- 172.21.15.107:0/2624390701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b05ad9b830 0x55b05ada7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/2624390701 >> 172.21.15.107:0/2624390701 conn(0x55b05ad90ae0 msgr2=0x55b05ad94b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/2624390701 shutdown_connections 2023-07-21T22:36:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55f3eb1b80 1 -- 172.21.15.107:0/2624390701 wait complete. 2023-07-21T22:36:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.886+0000 7f55f3eb1b80 20 client.16536 trim_cache size 0 max 16384 2023-07-21T22:36:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:36:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:36:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:36:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:36:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.887+0000 7f55e7d2c700 1 -- 172.21.15.107:0/3521825275 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f55cc0a1150 msgr2=0x7f55cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.887+0000 7f55e7d2c700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f55cc0a1150 0x7f55cc0a3600 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f55e0068950 tx=0x7f55dc009260 comp rx=0 tx=0).stop 2023-07-21T22:36:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.887+0000 7f55e7d2c700 1 -- 172.21.15.107:0/3521825275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00fea00 msgr2=0x7f55e01985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.887+0000 7f55e7d2c700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00fea00 0x7f55e01985d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f55d00098e0 tx=0x7f55d0004730 comp rx=0 tx=0).stop 2023-07-21T22:36:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.887+0000 7f55e7d2c700 1 -- 172.21.15.107:0/3521825275 shutdown_connections 2023-07-21T22:36:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.887+0000 7f55e7d2c700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55e0100580 0x7f55e019d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.887+0000 7f55e7d2c700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e00fea00 0x7f55e01985d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.887+0000 7f55e7d2c700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f55cc0a1150 0x7f55cc0a3600 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.887+0000 7f55e7d2c700 1 --2- 172.21.15.107:0/3521825275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55e00ffbf0 0x7f55e019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.887+0000 7f55e7d2c700 1 -- 172.21.15.107:0/3521825275 >> 172.21.15.107:0/3521825275 conn(0x7f55e00f9fe0 msgr2=0x7f55e00fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.887+0000 7f55e7d2c700 1 -- 172.21.15.107:0/3521825275 shutdown_connections 2023-07-21T22:36:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:23.887+0000 7f55e7d2c700 1 -- 172.21.15.107:0/3521825275 wait complete. 2023-07-21T22:36:25.640 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:25 smithi190 ceph-mon[98334]: pgmap v514: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 359 op/s 2023-07-21T22:36:25.681 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:25 smithi132 ceph-mon[97863]: pgmap v514: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 359 op/s 2023-07-21T22:36:25.687 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:25 smithi107 ceph-mon[96543]: pgmap v514: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 359 op/s 2023-07-21T22:36:27.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:27 smithi190 ceph-mon[98334]: pgmap v515: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 294 op/s 2023-07-21T22:36:27.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:27 smithi132 ceph-mon[97863]: pgmap v515: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 294 op/s 2023-07-21T22:36:27.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:27 smithi107 ceph-mon[96543]: pgmap v515: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 294 op/s 2023-07-21T22:36:29.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:29 smithi190 ceph-mon[98334]: pgmap v516: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 295 op/s 2023-07-21T22:36:29.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:29 smithi132 ceph-mon[97863]: pgmap v516: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 295 op/s 2023-07-21T22:36:29.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:29 smithi107 ceph-mon[96543]: pgmap v516: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.5 MiB/s wr, 295 op/s 2023-07-21T22:36:31.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:31 smithi190 ceph-mon[98334]: pgmap v517: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 197 op/s 2023-07-21T22:36:31.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:31 smithi132 ceph-mon[97863]: pgmap v517: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 197 op/s 2023-07-21T22:36:31.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:31 smithi107 ceph-mon[96543]: pgmap v517: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.8 MiB/s wr, 197 op/s 2023-07-21T22:36:33.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:33 smithi190 ceph-mon[98334]: pgmap v518: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2023-07-21T22:36:33.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:33 smithi132 ceph-mon[97863]: pgmap v518: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2023-07-21T22:36:33.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:33 smithi107 ceph-mon[96543]: pgmap v518: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2023-07-21T22:36:35.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:35 smithi190 ceph-mon[98334]: pgmap v519: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 283 op/s 2023-07-21T22:36:35.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:35 smithi132 ceph-mon[97863]: pgmap v519: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 283 op/s 2023-07-21T22:36:35.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:35 smithi107 ceph-mon[96543]: pgmap v519: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 283 op/s 2023-07-21T22:36:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.997+0000 7f5ca9a89700 1 -- 172.21.15.107:0/608096970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca40ffc30 msgr2=0x7f5ca4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.997+0000 7f5ca9a89700 1 --2- 172.21.15.107:0/608096970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca40ffc30 0x7f5ca4100080 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009990 tx=0x7f5c98009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.997+0000 7f5ca9a89700 1 -- 172.21.15.107:0/608096970 shutdown_connections 2023-07-21T22:36:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.997+0000 7f5ca9a89700 1 --2- 172.21.15.107:0/608096970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ca41005c0 0x7f5ca4106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.997+0000 7f5ca9a89700 1 --2- 172.21.15.107:0/608096970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ca40fea40 0x7f5ca40fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.997+0000 7f5ca9a89700 1 --2- 172.21.15.107:0/608096970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca40ffc30 0x7f5ca4100080 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.997+0000 7f5ca9a89700 1 -- 172.21.15.107:0/608096970 >> 172.21.15.107:0/608096970 conn(0x7f5ca40fa060 msgr2=0x7f5ca40fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:37.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.997+0000 7f5ca9a89700 1 -- 172.21.15.107:0/608096970 shutdown_connections 2023-07-21T22:36:37.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.997+0000 7f5ca9a89700 1 -- 172.21.15.107:0/608096970 wait complete. 2023-07-21T22:36:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.998+0000 7f5ca9a89700 1 Processor -- start 2023-07-21T22:36:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.998+0000 7f5ca9a89700 1 -- start start 2023-07-21T22:36:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.998+0000 7f5ca9a89700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ca40fea40 0x7f5ca4195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.998+0000 7f5ca9a89700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ca40ffc30 0x7f5ca4195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.998+0000 7f5ca9a89700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca41005c0 0x7f5ca4196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.998+0000 7f5ca9a89700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca4197380 con 0x7f5ca41005c0 2023-07-21T22:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.998+0000 7f5ca9a89700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca41974f0 con 0x7f5ca40fea40 2023-07-21T22:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.998+0000 7f5ca9a89700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca41976b0 con 0x7f5ca40ffc30 2023-07-21T22:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.998+0000 7f5ca3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca41005c0 0x7f5ca4196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.998+0000 7f5ca3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca41005c0 0x7f5ca4196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51020/0 (socket says 172.21.15.107:51020) 2023-07-21T22:36:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.998+0000 7f5ca37fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ca40fea40 0x7f5ca4195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.998+0000 7f5ca2ffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ca40ffc30 0x7f5ca4195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.998+0000 7f5ca3fff700 1 -- 172.21.15.107:0/2501060058 learned_addr learned my addr 172.21.15.107:0/2501060058 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.999+0000 7f5ca3fff700 1 -- 172.21.15.107:0/2501060058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ca40ffc30 msgr2=0x7f5ca4195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.999+0000 7f5ca3fff700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ca40ffc30 0x7f5ca4195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.999+0000 7f5ca3fff700 1 -- 172.21.15.107:0/2501060058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ca40fea40 msgr2=0x7f5ca4195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.999+0000 7f5ca3fff700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ca40fea40 0x7f5ca4195960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.999+0000 7f5ca3fff700 1 -- 172.21.15.107:0/2501060058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c980095f0 con 0x7f5ca41005c0 2023-07-21T22:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.999+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ca40fea40 0x7f5ca4195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.999+0000 7f5ca2ffd700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ca40ffc30 0x7f5ca4195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.999+0000 7f5ca3fff700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca41005c0 0x7f5ca4196d20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00ef30 tx=0x7f5c9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.999+0000 7f5ca0ff9700 1 -- 172.21.15.107:0/2501060058 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c018070 con 0x7f5ca41005c0 2023-07-21T22:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.999+0000 7f5ca0ff9700 1 -- 172.21.15.107:0/2501060058 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c9c00f040 con 0x7f5ca41005c0 2023-07-21T22:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.999+0000 7f5ca9a89700 1 -- 172.21.15.107:0/2501060058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca4197930 con 0x7f5ca41005c0 2023-07-21T22:36:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.999+0000 7f5ca0ff9700 1 -- 172.21.15.107:0/2501060058 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00cd00 con 0x7f5ca41005c0 2023-07-21T22:36:37.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:36.999+0000 7f5ca9a89700 1 -- 172.21.15.107:0/2501060058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca419e150 con 0x7f5ca41005c0 2023-07-21T22:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.001+0000 7f5ca0ff9700 1 -- 172.21.15.107:0/2501060058 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5c9c003a00 con 0x7f5ca41005c0 2023-07-21T22:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.002+0000 7f5ca0ff9700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5c8c0a1150 0x7f5c8c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.002+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5c8c0a1150 0x7f5c8c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.002+0000 7f5ca0ff9700 1 -- 172.21.15.107:0/2501060058 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5c9c0cc480 con 0x7f5ca41005c0 2023-07-21T22:36:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.002+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5c8c0a1150 0x7f5c8c0a3600 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40ffa90 tx=0x7f5c94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5cb5c0eb80 1 Processor -- start 2023-07-21T22:36:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5cb5c0eb80 1 -- start start 2023-07-21T22:36:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5cb5c0eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5bd556b0 0x555d5befd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5cb5c0eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befdba0 0x555d5befe520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5cb5c0eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5befea60 0x555d5befeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5cb5c0eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5bdfec50 con 0x555d5bd556b0 2023-07-21T22:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5cb5c0eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5be519e0 con 0x555d5befdba0 2023-07-21T22:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5cb5c0eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5be4ae70 con 0x555d5befea60 2023-07-21T22:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5ca37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5bd556b0 0x555d5befd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5ca37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5bd556b0 0x555d5befd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51024/0 (socket says 172.21.15.107:51024) 2023-07-21T22:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5ca37fe700 1 -- 172.21.15.107:0/2096610891 learned_addr learned my addr 172.21.15.107:0/2096610891 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5ca2ffd700 1 --2- 172.21.15.107:0/2096610891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befdba0 0x555d5befe520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5ca3fff700 1 --2- 172.21.15.107:0/2096610891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5befea60 0x555d5befeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5ca37fe700 1 -- 172.21.15.107:0/2096610891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5befea60 msgr2=0x555d5befeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/2096610891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5befea60 0x555d5befeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5ca37fe700 1 -- 172.21.15.107:0/2096610891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befdba0 msgr2=0x555d5befe520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/2096610891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befdba0 0x555d5befe520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5ca37fe700 1 -- 172.21.15.107:0/2096610891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d5bc78d80 con 0x555d5bd556b0 2023-07-21T22:36:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5ca3fff700 1 --2- 172.21.15.107:0/2096610891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5befea60 0x555d5befeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:36:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.003+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/2096610891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5bd556b0 0x555d5befd660 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5c940043d0 tx=0x7f5c94014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628ad7be888a85e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.004+0000 7f5c89ffb700 1 -- 172.21.15.107:0/2096610891 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94003ce0 con 0x555d5bd556b0 2023-07-21T22:36:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.004+0000 7f5c89ffb700 1 -- 172.21.15.107:0/2096610891 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c94003e40 con 0x555d5bd556b0 2023-07-21T22:36:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.004+0000 7f5c89ffb700 1 -- 172.21.15.107:0/2096610891 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94026800 con 0x555d5bd556b0 2023-07-21T22:36:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.004+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/2096610891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5bd556b0 msgr2=0x555d5befd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.004+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/2096610891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5bd556b0 0x555d5befd660 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5c940043d0 tx=0x7f5c94014000 comp rx=0 tx=0).stop 2023-07-21T22:36:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.004+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/2096610891 shutdown_connections 2023-07-21T22:36:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.004+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/2096610891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5befea60 0x555d5befeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.004+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/2096610891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5bd556b0 0x555d5befd660 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.004+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/2096610891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befdba0 0x555d5befe520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.004+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/2096610891 >> 172.21.15.107:0/2096610891 conn(0x555d5beeeae0 msgr2=0x555d5bef3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.004+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/2096610891 shutdown_connections 2023-07-21T22:36:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.004+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/2096610891 wait complete. 2023-07-21T22:36:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5cb5c0eb80 1 -- start start 2023-07-21T22:36:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5cb5c0eb80 1 Processor -- start 2023-07-21T22:36:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5cb5c0eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5bd556b0 0x555d5bf06be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5cb5c0eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befdba0 0x555d5bf07120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5cb5c0eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 0x555d5bf0e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5cb5c0eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5bdeaa60 con 0x555d5befea60 2023-07-21T22:36:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5cb5c0eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5bdea760 con 0x555d5befdba0 2023-07-21T22:36:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5cb5c0eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5bcce8c0 con 0x555d5bd556b0 2023-07-21T22:36:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5cb5c0eb80 10 client.0 _ll_register_callbacks cb 0x555d5bde8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5ca3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 0x555d5bf0e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5ca2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befdba0 0x555d5bf07120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5ca3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 0x555d5bf0e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51034/0 (socket says 172.21.15.107:51034) 2023-07-21T22:36:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5ca3fff700 1 -- 172.21.15.107:0/1170066620 learned_addr learned my addr 172.21.15.107:0/1170066620 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.005+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/1170066620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5bd556b0 0x555d5bf06be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5ca3fff700 1 -- 172.21.15.107:0/1170066620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5bd556b0 msgr2=0x555d5bf06be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5ca3fff700 1 --2- 172.21.15.107:0/1170066620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5bd556b0 0x555d5bf06be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5ca3fff700 1 -- 172.21.15.107:0/1170066620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befdba0 msgr2=0x555d5bf07120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5ca3fff700 1 --2- 172.21.15.107:0/1170066620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befdba0 0x555d5bf07120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5ca3fff700 1 -- 172.21.15.107:0/1170066620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c94017070 con 0x555d5befea60 2023-07-21T22:36:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5ca2ffd700 1 --2- 172.21.15.107:0/1170066620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befdba0 0x555d5bf07120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5ca3fff700 1 --2- 172.21.15.107:0/1170066620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 0x555d5bf0e160 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c099600 tx=0x7f5c9c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807f66e5d67d7e30 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5c82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5c82ffd700 1 -- 172.21.15.107:0/1170066620 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0b7ea0 con 0x555d5befea60 2023-07-21T22:36:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5c82ffd700 1 -- 172.21.15.107:0/1170066620 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5c9c0cc2c0 con 0x555d5befea60 2023-07-21T22:36:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/1170066620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d5bda4490 con 0x555d5befea60 2023-07-21T22:36:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5c82ffd700 1 -- 172.21.15.107:0/1170066620 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c9c099900 con 0x555d5befea60 2023-07-21T22:36:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5cb5c0eb80 10 client.16548 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/1170066620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x555d5bc61c00 con 0x555d5befea60 2023-07-21T22:36:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5c82ffd700 1 -- 172.21.15.107:0/1170066620 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d5bda4490 con 0x555d5befea60 2023-07-21T22:36:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5c82ffd700 1 -- 172.21.15.107:0/1170066620 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f5c9c0b4070 con 0x555d5befea60 2023-07-21T22:36:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5cb5c0eb80 10 client.16548 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5cb5c0eb80 10 client.16548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5cb5c0eb80 20 client.16548 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:37.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.006+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/1170066620 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d5bf12ab0 0x555d5bf14ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.007+0000 7f5cb5c0eb80 4 client.16548 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.007+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/1170066620 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d5be43e50 con 0x555d5bf12ab0 2023-07-21T22:36:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.007+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/1170066620 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d5bf12ab0 0x555d5bf14ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.007+0000 7f5c82ffd700 10 client.16548 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.007+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/1170066620 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d5bf12ab0 0x555d5bf14ec0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=784c8ee77343890f server_cookie=6c55a0ee221c062a in_seq=0 out_seq=0 2023-07-21T22:36:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5c82ffd700 1 -- 172.21.15.107:0/1170066620 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555d5be43e50 con 0x555d5bf12ab0 2023-07-21T22:36:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5c82ffd700 10 client.16548 handle_command_reply: tid=0 2023-07-21T22:36:37.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5cb5c0eb80 1 client.16548 shutdown 2023-07-21T22:36:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/1170066620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 msgr2=0x555d5bf0e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/1170066620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 0x555d5bf0e160 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c099600 tx=0x7f5c9c004b50 comp rx=0 tx=0).stop 2023-07-21T22:36:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/1170066620 shutdown_connections 2023-07-21T22:36:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/1170066620 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d5bf12ab0 0x555d5bf14ec0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/1170066620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 0x555d5bf0e160 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/1170066620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5bd556b0 0x555d5bf06be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/1170066620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befdba0 0x555d5bf07120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5c82ffd700 0 client.16548 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/1170066620 >> 172.21.15.107:0/1170066620 conn(0x555d5beeeae0 msgr2=0x555d5bef33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/1170066620 shutdown_connections 2023-07-21T22:36:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/1170066620 wait complete. 2023-07-21T22:36:37.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.008+0000 7f5cb5c0eb80 20 client.16548 trim_cache size 0 max 16384 2023-07-21T22:36:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.022+0000 7f5cb5c0eb80 1 Processor -- start 2023-07-21T22:36:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.022+0000 7f5cb5c0eb80 1 -- start start 2023-07-21T22:36:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.022+0000 7f5cb5c0eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5bd556b0 0x555d5bf01930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.022+0000 7f5cb5c0eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befdba0 0x555d5bf01e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.022+0000 7f5cb5c0eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befea60 0x555d5bf0e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.022+0000 7f5cb5c0eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5bef50a0 con 0x555d5befdba0 2023-07-21T22:36:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5cb5c0eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5bf06750 con 0x555d5befea60 2023-07-21T22:36:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5cb5c0eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5befdfa0 con 0x555d5bd556b0 2023-07-21T22:36:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5ca2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befdba0 0x555d5bf01e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5ca2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befdba0 0x555d5bf01e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51040/0 (socket says 172.21.15.107:51040) 2023-07-21T22:36:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5ca2ffd700 1 -- 172.21.15.107:0/4005389383 learned_addr learned my addr 172.21.15.107:0/4005389383 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5ca3fff700 1 --2- 172.21.15.107:0/4005389383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befea60 0x555d5bf0e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/4005389383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5bd556b0 0x555d5bf01930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5ca2ffd700 1 -- 172.21.15.107:0/4005389383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5bd556b0 msgr2=0x555d5bf01930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5ca2ffd700 1 --2- 172.21.15.107:0/4005389383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5bd556b0 0x555d5bf01930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5ca2ffd700 1 -- 172.21.15.107:0/4005389383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befea60 msgr2=0x555d5bf0e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5ca2ffd700 1 --2- 172.21.15.107:0/4005389383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befea60 0x555d5bf0e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5ca2ffd700 1 -- 172.21.15.107:0/4005389383 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d5bc9f1f0 con 0x555d5befdba0 2023-07-21T22:36:37.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5ca2ffd700 1 --2- 172.21.15.107:0/4005389383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befdba0 0x555d5bf01e70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009990 tx=0x7f5c98005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3693bf6f61e23be server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5c8a7fc700 1 -- 172.21.15.107:0/4005389383 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9801d030 con 0x555d5befdba0 2023-07-21T22:36:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5c8a7fc700 1 -- 172.21.15.107:0/4005389383 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c9801e040 con 0x555d5befdba0 2023-07-21T22:36:37.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.023+0000 7f5c8a7fc700 1 -- 172.21.15.107:0/4005389383 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9800f460 con 0x555d5befdba0 2023-07-21T22:36:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.024+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/4005389383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befdba0 msgr2=0x555d5bf01e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.024+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/4005389383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befdba0 0x555d5bf01e70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009990 tx=0x7f5c98005960 comp rx=0 tx=0).stop 2023-07-21T22:36:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.024+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/4005389383 shutdown_connections 2023-07-21T22:36:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.024+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/4005389383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5befea60 0x555d5bf0e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.024+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/4005389383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5bd556b0 0x555d5bf01930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.024+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/4005389383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befdba0 0x555d5bf01e70 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.024+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/4005389383 >> 172.21.15.107:0/4005389383 conn(0x555d5beeeae0 msgr2=0x555d5bf19820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.024+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/4005389383 shutdown_connections 2023-07-21T22:36:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.024+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/4005389383 wait complete. 2023-07-21T22:36:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.024+0000 7f5cb5c0eb80 1 -- start start 2023-07-21T22:36:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.024+0000 7f5cb5c0eb80 1 Processor -- start 2023-07-21T22:36:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5cb5c0eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5bd556b0 0x555d5bf0c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5cb5c0eb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5befdba0 0x555d5bf62940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5cb5c0eb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 0x555d5bf64f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5cb5c0eb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5bf656c0 con 0x555d5befea60 2023-07-21T22:36:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5cb5c0eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5bf65830 con 0x555d5bd556b0 2023-07-21T22:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5cb5c0eb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5bf659f0 con 0x555d5befdba0 2023-07-21T22:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5cb5c0eb80 10 client.0 _ll_register_callbacks cb 0x7f5c9c099440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5ca3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 0x555d5bf64f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5ca3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 0x555d5bf64f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51048/0 (socket says 172.21.15.107:51048) 2023-07-21T22:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5ca3fff700 1 -- 172.21.15.107:0/2653987124 learned_addr learned my addr 172.21.15.107:0/2653987124 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5bd556b0 0x555d5bf0c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5ca2ffd700 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5befdba0 0x555d5bf62940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.076 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5ca3fff700 1 -- 172.21.15.107:0/2653987124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5befdba0 msgr2=0x555d5bf62940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5ca3fff700 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5befdba0 0x555d5bf62940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5ca3fff700 1 -- 172.21.15.107:0/2653987124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5bd556b0 msgr2=0x555d5bf0c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5ca3fff700 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5bd556b0 0x555d5bf0c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5ca3fff700 1 -- 172.21.15.107:0/2653987124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca41077b0 con 0x555d5befea60 2023-07-21T22:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5bd556b0 0x555d5bf0c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5ca3fff700 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 0x555d5bf64f80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0b90d0 tx=0x7f5c9c097eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530ce6d09bd8c58e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:37.077 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5c82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5c82ffd700 1 -- 172.21.15.107:0/2653987124 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0b7ea0 con 0x555d5befea60 2023-07-21T22:36:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5c82ffd700 1 -- 172.21.15.107:0/2653987124 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5c9c099900 con 0x555d5befea60 2023-07-21T22:36:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/2653987124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c94003e40 con 0x555d5befea60 2023-07-21T22:36:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5c82ffd700 1 -- 172.21.15.107:0/2653987124 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c9c0b91a0 con 0x555d5befea60 2023-07-21T22:36:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5cb5c0eb80 10 client.16554 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.025+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/2653987124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x555d5bc77250 con 0x555d5befea60 2023-07-21T22:36:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.026+0000 7f5c82ffd700 1 -- 172.21.15.107:0/2653987124 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c9c065030 con 0x555d5befea60 2023-07-21T22:36:37.078 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.026+0000 7f5c82ffd700 1 -- 172.21.15.107:0/2653987124 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f5c9c0b4070 con 0x555d5befea60 2023-07-21T22:36:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.026+0000 7f5cb5c0eb80 10 client.16554 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.026+0000 7f5cb5c0eb80 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.026+0000 7f5cb5c0eb80 20 client.16554 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.026+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d5bf66eb0 0x555d5bf692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.026+0000 7f5cb5c0eb80 4 client.16554 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:36:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.026+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/2653987124 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555d5be05ff0 con 0x555d5bf66eb0 2023-07-21T22:36:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.026+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d5bf66eb0 0x555d5bf692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:37.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.026+0000 7f5c82ffd700 10 client.16554 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:37.080 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.026+0000 7f5ca37fe700 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d5bf66eb0 0x555d5bf692c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a28e56335343b7ac server_cookie=2a7e2f46daabae69 in_seq=0 out_seq=0 2023-07-21T22:36:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5c82ffd700 1 -- 172.21.15.107:0/2653987124 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555d5be05ff0 con 0x555d5bf66eb0 2023-07-21T22:36:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5c82ffd700 10 client.16554 handle_command_reply: tid=0 2023-07-21T22:36:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5cb5c0eb80 1 client.16554 shutdown 2023-07-21T22:36:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/2653987124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 msgr2=0x555d5bf64f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 0x555d5bf64f80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0b90d0 tx=0x7f5c9c097eb0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/2653987124 shutdown_connections 2023-07-21T22:36:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x555d5bf66eb0 0x555d5bf692c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555d5befea60 0x555d5bf64f80 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555d5bd556b0 0x555d5bf0c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5cb5c0eb80 1 --2- 172.21.15.107:0/2653987124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555d5befdba0 0x555d5bf62940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5c82ffd700 0 client.16554 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/2653987124 >> 172.21.15.107:0/2653987124 conn(0x555d5beeeae0 msgr2=0x555d5bf01930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/2653987124 shutdown_connections 2023-07-21T22:36:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5cb5c0eb80 1 -- 172.21.15.107:0/2653987124 wait complete. 2023-07-21T22:36:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.538+0000 7f5cb5c0eb80 20 client.16554 trim_cache size 0 max 16384 2023-07-21T22:36:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:36:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:36:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:36:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:36:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.539+0000 7f5ca9a89700 1 -- 172.21.15.107:0/2501060058 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5c8c0a1150 msgr2=0x7f5c8c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.539+0000 7f5ca9a89700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5c8c0a1150 0x7f5c8c0a3600 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40ffa90 tx=0x7f5c94009260 comp rx=0 tx=0).stop 2023-07-21T22:36:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.539+0000 7f5ca9a89700 1 -- 172.21.15.107:0/2501060058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca41005c0 msgr2=0x7f5ca4196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.539+0000 7f5ca9a89700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca41005c0 0x7f5ca4196d20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00ef30 tx=0x7f5c9c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.539+0000 7f5ca9a89700 1 -- 172.21.15.107:0/2501060058 shutdown_connections 2023-07-21T22:36:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.539+0000 7f5ca9a89700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ca41005c0 0x7f5ca4196d20 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.539+0000 7f5ca9a89700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ca40fea40 0x7f5ca4195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.539+0000 7f5ca9a89700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f5c8c0a1150 0x7f5c8c0a3600 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.539+0000 7f5ca9a89700 1 --2- 172.21.15.107:0/2501060058 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ca40ffc30 0x7f5ca4195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.539+0000 7f5ca9a89700 1 -- 172.21.15.107:0/2501060058 >> 172.21.15.107:0/2501060058 conn(0x7f5ca40fa060 msgr2=0x7f5ca40fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.539+0000 7f5ca9a89700 1 -- 172.21.15.107:0/2501060058 shutdown_connections 2023-07-21T22:36:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:37.539+0000 7f5ca9a89700 1 -- 172.21.15.107:0/2501060058 wait complete. 2023-07-21T22:36:37.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:37 smithi190 ceph-mon[98334]: pgmap v520: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2023-07-21T22:36:37.916 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:36:37.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:37 smithi132 ceph-mon[97863]: pgmap v520: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2023-07-21T22:36:37.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:37 smithi107 ceph-mon[96543]: pgmap v520: 97 pgs: 97 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 314 op/s 2023-07-21T22:36:38.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.006+0000 7f3e664de700 1 -- 172.21.15.107:0/2110072223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e60105800 msgr2=0x7f3e60107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.006+0000 7f3e664de700 1 --2- 172.21.15.107:0/2110072223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e60105800 0x7f3e60107c00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800b0a0 tx=0x7f3e5800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.006+0000 7f3e664de700 1 -- 172.21.15.107:0/2110072223 shutdown_connections 2023-07-21T22:36:38.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.006+0000 7f3e664de700 1 --2- 172.21.15.107:0/2110072223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e60105800 0x7f3e60107c00 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.006+0000 7f3e664de700 1 --2- 172.21.15.107:0/2110072223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e601005c0 0x7f3e601029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.006+0000 7f3e664de700 1 --2- 172.21.15.107:0/2110072223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e60102ee0 0x7f3e601052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.006+0000 7f3e664de700 1 -- 172.21.15.107:0/2110072223 >> 172.21.15.107:0/2110072223 conn(0x7f3e600fa000 msgr2=0x7f3e600fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.006+0000 7f3e664de700 1 -- 172.21.15.107:0/2110072223 shutdown_connections 2023-07-21T22:36:38.052 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.006+0000 7f3e664de700 1 -- 172.21.15.107:0/2110072223 wait complete. 2023-07-21T22:36:38.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.006+0000 7f3e664de700 1 Processor -- start 2023-07-21T22:36:38.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e664de700 1 -- start start 2023-07-21T22:36:38.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e664de700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e601005c0 0x7f3e601958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.054 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e664de700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e60102ee0 0x7f3e60195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e664de700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60105800 0x7f3e6019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e664de700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6019d460 con 0x7f3e601005c0 2023-07-21T22:36:38.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e664de700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6019d5a0 con 0x7f3e60102ee0 2023-07-21T22:36:38.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e664de700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6019d730 con 0x7f3e60105800 2023-07-21T22:36:38.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e5ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e601005c0 0x7f3e601958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e5ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e601005c0 0x7f3e601958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51058/0 (socket says 172.21.15.107:51058) 2023-07-21T22:36:38.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e5ffff700 1 -- 172.21.15.107:0/2869533142 learned_addr learned my addr 172.21.15.107:0/2869533142 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.055 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e5f7fe700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e60102ee0 0x7f3e60195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e64a7b700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60105800 0x7f3e6019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e5ffff700 1 -- 172.21.15.107:0/2869533142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60105800 msgr2=0x7f3e6019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60105800 0x7f3e6019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e5ffff700 1 -- 172.21.15.107:0/2869533142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e60102ee0 msgr2=0x7f3e60195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e60102ee0 0x7f3e60195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.007+0000 7f3e5ffff700 1 -- 172.21.15.107:0/2869533142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50009520 con 0x7f3e601005c0 2023-07-21T22:36:38.056 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.008+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e601005c0 0x7f3e601958e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800e9a0 tx=0x7f3e4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.008+0000 7f3e5d7fa700 1 -- 172.21.15.107:0/2869533142 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48009790 con 0x7f3e601005c0 2023-07-21T22:36:38.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.008+0000 7f3e5d7fa700 1 -- 172.21.15.107:0/2869533142 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e48010430 con 0x7f3e601005c0 2023-07-21T22:36:38.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.008+0000 7f3e664de700 1 -- 172.21.15.107:0/2869533142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e58009d00 con 0x7f3e601005c0 2023-07-21T22:36:38.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.008+0000 7f3e5d7fa700 1 -- 172.21.15.107:0/2869533142 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48018680 con 0x7f3e601005c0 2023-07-21T22:36:38.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.008+0000 7f3e664de700 1 -- 172.21.15.107:0/2869533142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6019dde0 con 0x7f3e601005c0 2023-07-21T22:36:38.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.010+0000 7f3e5d7fa700 1 -- 172.21.15.107:0/2869533142 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3e480105a0 con 0x7f3e601005c0 2023-07-21T22:36:38.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.011+0000 7f3e5d7fa700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3e4c0a11d0 0x7f3e4c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.011+0000 7f3e5d7fa700 1 -- 172.21.15.107:0/2869533142 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3e48014070 con 0x7f3e601005c0 2023-07-21T22:36:38.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.011+0000 7f3e5f7fe700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3e4c0a11d0 0x7f3e4c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.011+0000 7f3e5f7fe700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3e4c0a11d0 0x7f3e4c0a3680 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009df0 tx=0x7f3e50009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e72663b80 1 Processor -- start 2023-07-21T22:36:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e72663b80 1 -- start start 2023-07-21T22:36:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e72663b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299bac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e72663b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299bb1a0 0x562e299b9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e72663b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299b9a10 0x562e299b9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e72663b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e298becb0 con 0x562e299ba830 2023-07-21T22:36:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e72663b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e29911a40 con 0x562e299b9a10 2023-07-21T22:36:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e72663b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2990aed0 con 0x562e299bb1a0 2023-07-21T22:36:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e5ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299bac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e5ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299bac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51066/0 (socket says 172.21.15.107:51066) 2023-07-21T22:36:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e5ffff700 1 -- 172.21.15.107:0/1899242173 learned_addr learned my addr 172.21.15.107:0/1899242173 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e64a7b700 1 --2- 172.21.15.107:0/1899242173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299b9a10 0x562e299b9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e5f7fe700 1 --2- 172.21.15.107:0/1899242173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299bb1a0 0x562e299b9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e5ffff700 1 -- 172.21.15.107:0/1899242173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299bb1a0 msgr2=0x562e299b9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/1899242173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299bb1a0 0x562e299b9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e5ffff700 1 -- 172.21.15.107:0/1899242173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299b9a10 msgr2=0x562e299b9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/1899242173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299b9a10 0x562e299b9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e5ffff700 1 -- 172.21.15.107:0/1899242173 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e29738de0 con 0x562e299ba830 2023-07-21T22:36:38.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e64a7b700 1 --2- 172.21.15.107:0/1899242173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299b9a10 0x562e299b9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:38.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.012+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/1899242173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299bac40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3e48099e20 tx=0x7f3e48099870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f403cfb67ff614 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.013+0000 7f3e567fc700 1 -- 172.21.15.107:0/1899242173 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e480b1070 con 0x562e299ba830 2023-07-21T22:36:38.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.013+0000 7f3e567fc700 1 -- 172.21.15.107:0/1899242173 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e4809dec0 con 0x562e299ba830 2023-07-21T22:36:38.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.013+0000 7f3e567fc700 1 -- 172.21.15.107:0/1899242173 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e480995e0 con 0x562e299ba830 2023-07-21T22:36:38.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.013+0000 7f3e72663b80 1 -- 172.21.15.107:0/1899242173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 msgr2=0x562e299bac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.013+0000 7f3e72663b80 1 --2- 172.21.15.107:0/1899242173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299bac40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3e48099e20 tx=0x7f3e48099870 comp rx=0 tx=0).stop 2023-07-21T22:36:38.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.013+0000 7f3e72663b80 1 -- 172.21.15.107:0/1899242173 shutdown_connections 2023-07-21T22:36:38.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.013+0000 7f3e72663b80 1 --2- 172.21.15.107:0/1899242173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299b9a10 0x562e299b9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.013+0000 7f3e72663b80 1 --2- 172.21.15.107:0/1899242173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299bac40 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.066 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.013+0000 7f3e72663b80 1 --2- 172.21.15.107:0/1899242173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299bb1a0 0x562e299b9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.013+0000 7f3e72663b80 1 -- 172.21.15.107:0/1899242173 >> 172.21.15.107:0/1899242173 conn(0x562e299aeef0 msgr2=0x562e299b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.013+0000 7f3e72663b80 1 -- 172.21.15.107:0/1899242173 shutdown_connections 2023-07-21T22:36:38.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.013+0000 7f3e72663b80 1 -- 172.21.15.107:0/1899242173 wait complete. 2023-07-21T22:36:38.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e72663b80 1 -- start start 2023-07-21T22:36:38.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e72663b80 1 Processor -- start 2023-07-21T22:36:38.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e72663b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 0x562e299c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.067 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e72663b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299ba830 0x562e299c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e72663b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299bb1a0 0x562e299ca670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e72663b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e298aaac0 con 0x562e299bb1a0 2023-07-21T22:36:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e72663b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e298aa7c0 con 0x562e299ba830 2023-07-21T22:36:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e72663b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e2978e920 con 0x562e299b9a10 2023-07-21T22:36:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e72663b80 10 client.0 _ll_register_callbacks cb 0x562e298a8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e5f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299ba830 0x562e299c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e5f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299ba830 0x562e299c3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41902/0 (socket says 172.21.15.107:41902) 2023-07-21T22:36:38.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e5f7fe700 1 -- 172.21.15.107:0/1598590041 learned_addr learned my addr 172.21.15.107:0/1598590041 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e64a7b700 1 --2- 172.21.15.107:0/1598590041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299bb1a0 0x562e299ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/1598590041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 0x562e299c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e5f7fe700 1 -- 172.21.15.107:0/1598590041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 msgr2=0x562e299c30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e5f7fe700 1 --2- 172.21.15.107:0/1598590041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 0x562e299c30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e5f7fe700 1 -- 172.21.15.107:0/1598590041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299bb1a0 msgr2=0x562e299ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.014+0000 7f3e5f7fe700 1 --2- 172.21.15.107:0/1598590041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299bb1a0 0x562e299ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e5f7fe700 1 -- 172.21.15.107:0/1598590041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e601087f0 con 0x562e299ba830 2023-07-21T22:36:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e5f7fe700 1 --2- 172.21.15.107:0/1598590041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299ba830 0x562e299c3630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3e50005d90 tx=0x7f3e500047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=616bc88dd0ca0d93 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e3f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:36:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e3f7fe700 1 -- 172.21.15.107:0/1598590041 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000e9c0 con 0x562e299ba830 2023-07-21T22:36:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e3f7fe700 1 -- 172.21.15.107:0/1598590041 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3e5000eb20 con 0x562e299ba830 2023-07-21T22:36:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e72663b80 1 -- 172.21.15.107:0/1598590041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e298644f0 con 0x562e299ba830 2023-07-21T22:36:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e3f7fe700 1 -- 172.21.15.107:0/1598590041 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e5000ece0 con 0x562e299ba830 2023-07-21T22:36:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e72663b80 10 client.26105 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:38.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e72663b80 1 -- 172.21.15.107:0/1598590041 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f3e6019dde0 con 0x562e299ba830 2023-07-21T22:36:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e3f7fe700 1 -- 172.21.15.107:0/1598590041 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e298644f0 con 0x562e299ba830 2023-07-21T22:36:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e3f7fe700 1 -- 172.21.15.107:0/1598590041 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f3e50032050 con 0x562e299ba830 2023-07-21T22:36:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e72663b80 10 client.26105 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e72663b80 10 client.26105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e72663b80 20 client.26105 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e72663b80 1 --2- 172.21.15.107:0/1598590041 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562e299cefd0 0x562e299d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e72663b80 4 client.26105 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:38.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.015+0000 7f3e72663b80 1 -- 172.21.15.107:0/1598590041 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e29903eb0 con 0x562e299cefd0 2023-07-21T22:36:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.016+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/1598590041 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562e299cefd0 0x562e299d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.016+0000 7f3e3f7fe700 10 client.26105 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.016+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/1598590041 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562e299cefd0 0x562e299d13e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d57623392841351 server_cookie=b94e114203688e58 in_seq=0 out_seq=0 2023-07-21T22:36:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e3f7fe700 1 -- 172.21.15.107:0/1598590041 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562e29903eb0 con 0x562e299cefd0 2023-07-21T22:36:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e3f7fe700 10 client.26105 handle_command_reply: tid=0 2023-07-21T22:36:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e72663b80 1 client.26105 shutdown 2023-07-21T22:36:38.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e72663b80 1 -- 172.21.15.107:0/1598590041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299ba830 msgr2=0x562e299c3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e72663b80 1 --2- 172.21.15.107:0/1598590041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299ba830 0x562e299c3630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3e50005d90 tx=0x7f3e500047c0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e72663b80 1 -- 172.21.15.107:0/1598590041 shutdown_connections 2023-07-21T22:36:38.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e72663b80 1 --2- 172.21.15.107:0/1598590041 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562e299cefd0 0x562e299d13e0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e72663b80 1 --2- 172.21.15.107:0/1598590041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299bb1a0 0x562e299ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e72663b80 1 --2- 172.21.15.107:0/1598590041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 0x562e299c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e72663b80 1 --2- 172.21.15.107:0/1598590041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299ba830 0x562e299c3630 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e3f7fe700 0 client.26105 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:38.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e72663b80 1 -- 172.21.15.107:0/1598590041 >> 172.21.15.107:0/1598590041 conn(0x562e299aeef0 msgr2=0x562e299b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e72663b80 1 -- 172.21.15.107:0/1598590041 shutdown_connections 2023-07-21T22:36:38.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e72663b80 1 -- 172.21.15.107:0/1598590041 wait complete. 2023-07-21T22:36:38.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.017+0000 7f3e72663b80 20 client.26105 trim_cache size 0 max 16384 2023-07-21T22:36:38.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.034+0000 7f3e72663b80 1 Processor -- start 2023-07-21T22:36:38.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.034+0000 7f3e72663b80 1 -- start start 2023-07-21T22:36:38.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.034+0000 7f3e72663b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 0x562e299b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.034+0000 7f3e72663b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.034+0000 7f3e72663b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299bb1a0 0x562e299caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.034+0000 7f3e72663b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e299ae9b0 con 0x562e299ba830 2023-07-21T22:36:38.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.034+0000 7f3e72663b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e299c25a0 con 0x562e299bb1a0 2023-07-21T22:36:38.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.034+0000 7f3e72663b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e299c2c90 con 0x562e299b9a10 2023-07-21T22:36:38.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.034+0000 7f3e5f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.034+0000 7f3e5f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51082/0 (socket says 172.21.15.107:51082) 2023-07-21T22:36:38.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.034+0000 7f3e5f7fe700 1 -- 172.21.15.107:0/926688931 learned_addr learned my addr 172.21.15.107:0/926688931 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.034+0000 7f3e64a7b700 1 --2- 172.21.15.107:0/926688931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299bb1a0 0x562e299caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.035+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/926688931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 0x562e299b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.035+0000 7f3e5f7fe700 1 -- 172.21.15.107:0/926688931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 msgr2=0x562e299b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.035+0000 7f3e5f7fe700 1 --2- 172.21.15.107:0/926688931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 0x562e299b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.035+0000 7f3e5f7fe700 1 -- 172.21.15.107:0/926688931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299bb1a0 msgr2=0x562e299caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.035+0000 7f3e5f7fe700 1 --2- 172.21.15.107:0/926688931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299bb1a0 0x562e299caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.035+0000 7f3e5f7fe700 1 -- 172.21.15.107:0/926688931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e29721c60 con 0x562e299ba830 2023-07-21T22:36:38.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.035+0000 7f3e64a7b700 1 --2- 172.21.15.107:0/926688931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299bb1a0 0x562e299caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:36:38.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.035+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/926688931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 0x562e299b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:36:38.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.035+0000 7f3e5f7fe700 1 --2- 172.21.15.107:0/926688931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299ca590 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009470 tx=0x7f3e5001c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb41beb605c53f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.035+0000 7f3e56ffd700 1 -- 172.21.15.107:0/926688931 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5001c830 con 0x562e299ba830 2023-07-21T22:36:38.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.035+0000 7f3e56ffd700 1 -- 172.21.15.107:0/926688931 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5001c9f0 con 0x562e299ba830 2023-07-21T22:36:38.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.035+0000 7f3e56ffd700 1 -- 172.21.15.107:0/926688931 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50004450 con 0x562e299ba830 2023-07-21T22:36:38.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.036+0000 7f3e72663b80 1 -- 172.21.15.107:0/926688931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 msgr2=0x562e299ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.036+0000 7f3e72663b80 1 --2- 172.21.15.107:0/926688931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299ca590 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009470 tx=0x7f3e5001c660 comp rx=0 tx=0).stop 2023-07-21T22:36:38.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.036+0000 7f3e72663b80 1 -- 172.21.15.107:0/926688931 shutdown_connections 2023-07-21T22:36:38.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.036+0000 7f3e72663b80 1 --2- 172.21.15.107:0/926688931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299bb1a0 0x562e299caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.036+0000 7f3e72663b80 1 --2- 172.21.15.107:0/926688931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 0x562e299b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.036+0000 7f3e72663b80 1 --2- 172.21.15.107:0/926688931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299ca590 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.036+0000 7f3e72663b80 1 -- 172.21.15.107:0/926688931 >> 172.21.15.107:0/926688931 conn(0x562e299aeef0 msgr2=0x562e299c1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.036+0000 7f3e72663b80 1 -- 172.21.15.107:0/926688931 shutdown_connections 2023-07-21T22:36:38.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.036+0000 7f3e72663b80 1 -- 172.21.15.107:0/926688931 wait complete. 2023-07-21T22:36:38.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.036+0000 7f3e72663b80 1 -- start start 2023-07-21T22:36:38.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.036+0000 7f3e72663b80 1 Processor -- start 2023-07-21T22:36:38.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e72663b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 0x562e299c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e72663b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e72663b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299bb1a0 0x562e299c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e72663b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e299c7220 con 0x562e299ba830 2023-07-21T22:36:38.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e72663b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e299c7390 con 0x562e299bb1a0 2023-07-21T22:36:38.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e72663b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e299c7550 con 0x562e299b9a10 2023-07-21T22:36:38.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e72663b80 10 client.0 _ll_register_callbacks cb 0x7f3e5001e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:38.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e5f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e5f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299ca4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51088/0 (socket says 172.21.15.107:51088) 2023-07-21T22:36:38.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e5f7fe700 1 -- 172.21.15.107:0/3215915627 learned_addr learned my addr 172.21.15.107:0/3215915627 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e64a7b700 1 --2- 172.21.15.107:0/3215915627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299bb1a0 0x562e299c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/3215915627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 0x562e299c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e5f7fe700 1 -- 172.21.15.107:0/3215915627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 msgr2=0x562e299c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e5f7fe700 1 --2- 172.21.15.107:0/3215915627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 0x562e299c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e5f7fe700 1 -- 172.21.15.107:0/3215915627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299bb1a0 msgr2=0x562e299c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e5f7fe700 1 --2- 172.21.15.107:0/3215915627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299bb1a0 0x562e299c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e5f7fe700 1 -- 172.21.15.107:0/3215915627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6019dde0 con 0x562e299ba830 2023-07-21T22:36:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e5f7fe700 1 --2- 172.21.15.107:0/3215915627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299ca4e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009ab0 tx=0x7f3e5001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d877a995d1bf56 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e3f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e3f7fe700 1 -- 172.21.15.107:0/3215915627 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000ec50 con 0x562e299ba830 2023-07-21T22:36:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e3f7fe700 1 -- 172.21.15.107:0/3215915627 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3e50005240 con 0x562e299ba830 2023-07-21T22:36:38.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e72663b80 1 -- 172.21.15.107:0/3215915627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e4809dec0 con 0x562e299ba830 2023-07-21T22:36:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e3f7fe700 1 -- 172.21.15.107:0/3215915627 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e50005400 con 0x562e299ba830 2023-07-21T22:36:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e72663b80 10 client.16572 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.037+0000 7f3e72663b80 1 -- 172.21.15.107:0/3215915627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x562e2975f250 con 0x562e299ba830 2023-07-21T22:36:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.038+0000 7f3e3f7fe700 1 -- 172.21.15.107:0/3215915627 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5004c020 con 0x562e299ba830 2023-07-21T22:36:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.038+0000 7f3e3f7fe700 1 -- 172.21.15.107:0/3215915627 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f3e50032070 con 0x562e299ba830 2023-07-21T22:36:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.038+0000 7f3e72663b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:38.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.038+0000 7f3e72663b80 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:38.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.038+0000 7f3e72663b80 20 client.16572 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:38.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.038+0000 7f3e72663b80 1 --2- 172.21.15.107:0/3215915627 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562e29a24010 0x562e29a26420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.038+0000 7f3e72663b80 4 client.16572 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:36:38.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.038+0000 7f3e72663b80 1 -- 172.21.15.107:0/3215915627 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562e298c6050 con 0x562e29a24010 2023-07-21T22:36:38.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.038+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/3215915627 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562e29a24010 0x562e29a26420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.038+0000 7f3e3f7fe700 10 client.16572 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:38.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.038+0000 7f3e5ffff700 1 --2- 172.21.15.107:0/3215915627 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562e29a24010 0x562e29a26420 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43bd365ee7855ee5 server_cookie=2bd621223edaaa17 in_seq=0 out_seq=0 2023-07-21T22:36:38.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e3f7fe700 1 -- 172.21.15.107:0/3215915627 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562e298c6050 con 0x562e29a24010 2023-07-21T22:36:38.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e3f7fe700 10 client.16572 handle_command_reply: tid=0 2023-07-21T22:36:38.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e72663b80 1 client.16572 shutdown 2023-07-21T22:36:38.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e72663b80 1 -- 172.21.15.107:0/3215915627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 msgr2=0x562e299ca4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e72663b80 1 --2- 172.21.15.107:0/3215915627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299ca4e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009ab0 tx=0x7f3e5001c930 comp rx=0 tx=0).stop 2023-07-21T22:36:38.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e72663b80 1 -- 172.21.15.107:0/3215915627 shutdown_connections 2023-07-21T22:36:38.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e72663b80 1 --2- 172.21.15.107:0/3215915627 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562e29a24010 0x562e29a26420 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.093 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e72663b80 1 --2- 172.21.15.107:0/3215915627 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e299bb1a0 0x562e299c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e3f7fe700 0 client.16572 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:38.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e72663b80 1 --2- 172.21.15.107:0/3215915627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562e299b9a10 0x562e299c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e72663b80 1 --2- 172.21.15.107:0/3215915627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e299ba830 0x562e299ca4e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e72663b80 1 -- 172.21.15.107:0/3215915627 >> 172.21.15.107:0/3215915627 conn(0x562e299aeef0 msgr2=0x562e299b9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e72663b80 1 -- 172.21.15.107:0/3215915627 shutdown_connections 2023-07-21T22:36:38.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e72663b80 1 -- 172.21.15.107:0/3215915627 wait complete. 2023-07-21T22:36:38.094 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e72663b80 20 client.16572 trim_cache size 0 max 16384 2023-07-21T22:36:38.095 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:36:38.095 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2023-07-21T22:36:38.095 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2023-07-21T22:36:38.095 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:36:38.096 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e664de700 1 -- 172.21.15.107:0/2869533142 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3e4c0a11d0 msgr2=0x7f3e4c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e664de700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3e4c0a11d0 0x7f3e4c0a3680 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009df0 tx=0x7f3e50009260 comp rx=0 tx=0).stop 2023-07-21T22:36:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e664de700 1 -- 172.21.15.107:0/2869533142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e601005c0 msgr2=0x7f3e601958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e664de700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e601005c0 0x7f3e601958e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800e9a0 tx=0x7f3e4800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e664de700 1 -- 172.21.15.107:0/2869533142 shutdown_connections 2023-07-21T22:36:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e664de700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3e4c0a11d0 0x7f3e4c0a3680 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e664de700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e60105800 0x7f3e6019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.097 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e664de700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e601005c0 0x7f3e601958e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e664de700 1 --2- 172.21.15.107:0/2869533142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e60102ee0 0x7f3e60195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.039+0000 7f3e664de700 1 -- 172.21.15.107:0/2869533142 >> 172.21.15.107:0/2869533142 conn(0x7f3e600fa000 msgr2=0x7f3e60101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.040+0000 7f3e664de700 1 -- 172.21.15.107:0/2869533142 shutdown_connections 2023-07-21T22:36:38.098 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.040+0000 7f3e664de700 1 -- 172.21.15.107:0/2869533142 wait complete. 2023-07-21T22:36:38.098 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:36:38.099 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:36:38.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.218+0000 7fbe2088c700 1 -- 172.21.15.107:0/2837046112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe1c0ffc10 msgr2=0x7fbe1c100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.218+0000 7fbe2088c700 1 --2- 172.21.15.107:0/2837046112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe1c0ffc10 0x7fbe1c100060 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009990 tx=0x7fbe10009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.218+0000 7fbe2088c700 1 -- 172.21.15.107:0/2837046112 shutdown_connections 2023-07-21T22:36:38.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.218+0000 7fbe2088c700 1 --2- 172.21.15.107:0/2837046112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe1c1005a0 0x7fbe1c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.218+0000 7fbe2088c700 1 --2- 172.21.15.107:0/2837046112 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe1c0fea20 0x7fbe1c0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.218+0000 7fbe2088c700 1 --2- 172.21.15.107:0/2837046112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe1c0ffc10 0x7fbe1c100060 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.218+0000 7fbe2088c700 1 -- 172.21.15.107:0/2837046112 >> 172.21.15.107:0/2837046112 conn(0x7fbe1c0fa020 msgr2=0x7fbe1c0fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.218+0000 7fbe2088c700 1 -- 172.21.15.107:0/2837046112 shutdown_connections 2023-07-21T22:36:38.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.218+0000 7fbe2088c700 1 -- 172.21.15.107:0/2837046112 wait complete. 2023-07-21T22:36:38.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.218+0000 7fbe2088c700 1 Processor -- start 2023-07-21T22:36:38.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.218+0000 7fbe2088c700 1 -- start start 2023-07-21T22:36:38.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe2088c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe1c0fea20 0x7fbe1c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe2088c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe1c0ffc10 0x7fbe1c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe2088c700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe1c1005a0 0x7fbe1c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe2088c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c19d2b0 con 0x7fbe1c0ffc10 2023-07-21T22:36:38.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe2088c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c19d420 con 0x7fbe1c0fea20 2023-07-21T22:36:38.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe2088c700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c19d5e0 con 0x7fbe1c1005a0 2023-07-21T22:36:38.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe19d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe1c0ffc10 0x7fbe1c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe1a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe1c0fea20 0x7fbe1c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe19d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe1c0ffc10 0x7fbe1c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51102/0 (socket says 172.21.15.107:51102) 2023-07-21T22:36:38.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe19d9b700 1 -- 172.21.15.107:0/1752713407 learned_addr learned my addr 172.21.15.107:0/1752713407 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe1ad9d700 1 --2- 172.21.15.107:0/1752713407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe1c1005a0 0x7fbe1c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe19d9b700 1 -- 172.21.15.107:0/1752713407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe1c1005a0 msgr2=0x7fbe1c19cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe19d9b700 1 --2- 172.21.15.107:0/1752713407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe1c1005a0 0x7fbe1c19cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe19d9b700 1 -- 172.21.15.107:0/1752713407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe1c0fea20 msgr2=0x7fbe1c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe19d9b700 1 --2- 172.21.15.107:0/1752713407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe1c0fea20 0x7fbe1c069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe19d9b700 1 -- 172.21.15.107:0/1752713407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe100095f0 con 0x7fbe1c0ffc10 2023-07-21T22:36:38.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe19d9b700 1 --2- 172.21.15.107:0/1752713407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe1c0ffc10 0x7fbe1c19a4d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbe10005f50 tx=0x7fbe100057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe0b7fe700 1 -- 172.21.15.107:0/1752713407 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1001d070 con 0x7fbe1c0ffc10 2023-07-21T22:36:38.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe2088c700 1 -- 172.21.15.107:0/1752713407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c19d860 con 0x7fbe1c0ffc10 2023-07-21T22:36:38.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe0b7fe700 1 -- 172.21.15.107:0/1752713407 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe10021410 con 0x7fbe1c0ffc10 2023-07-21T22:36:38.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe2088c700 1 -- 172.21.15.107:0/1752713407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1c19dd60 con 0x7fbe1c0ffc10 2023-07-21T22:36:38.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.219+0000 7fbe0b7fe700 1 -- 172.21.15.107:0/1752713407 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe10017650 con 0x7fbe1c0ffc10 2023-07-21T22:36:38.269 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.220+0000 7fbe2088c700 1 -- 172.21.15.107:0/1752713407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1c105880 con 0x7fbe1c0ffc10 2023-07-21T22:36:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.221+0000 7fbe0b7fe700 1 -- 172.21.15.107:0/1752713407 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fbe10021580 con 0x7fbe1c0ffc10 2023-07-21T22:36:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.222+0000 7fbe0b7fe700 1 --2- 172.21.15.107:0/1752713407 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fbe040a11a0 0x7fbe040a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.222+0000 7fbe1a59c700 1 --2- 172.21.15.107:0/1752713407 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fbe040a11a0 0x7fbe040a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.222+0000 7fbe0b7fe700 1 -- 172.21.15.107:0/1752713407 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbe100ce380 con 0x7fbe1c0ffc10 2023-07-21T22:36:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.222+0000 7fbe1a59c700 1 --2- 172.21.15.107:0/1752713407 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fbe040a11a0 0x7fbe040a3650 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0ffa70 tx=0x7fbe0c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.225+0000 7fbe0b7fe700 1 -- 172.21.15.107:0/1752713407 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbe10097b40 con 0x7fbe1c0ffc10 2023-07-21T22:36:38.522 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.477+0000 7fbe2088c700 1 -- 172.21.15.107:0/1752713407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe1c02c850 con 0x7fbe1c0ffc10 2023-07-21T22:36:38.523 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.478+0000 7fbe0b7fe700 1 -- 172.21.15.107:0/1752713407 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+10897 (secure 0 0 0) 0x7fbe1009b160 con 0x7fbe1c0ffc10 2023-07-21T22:36:38.523 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:36:38.523 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:36:14.878084+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":130,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25553":{"gid":25553,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2828468744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2828468744},{"type":"v1","addr":"172.21.15.132:6837","nonce":2828468744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:36:38.525 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.480+0000 7fbe2088c700 1 -- 172.21.15.107:0/1752713407 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fbe040a11a0 msgr2=0x7fbe040a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.525 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.480+0000 7fbe2088c700 1 --2- 172.21.15.107:0/1752713407 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fbe040a11a0 0x7fbe040a3650 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0ffa70 tx=0x7fbe0c00b410 comp rx=0 tx=0).stop 2023-07-21T22:36:38.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.480+0000 7fbe2088c700 1 -- 172.21.15.107:0/1752713407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe1c0ffc10 msgr2=0x7fbe1c19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.480+0000 7fbe2088c700 1 --2- 172.21.15.107:0/1752713407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe1c0ffc10 0x7fbe1c19a4d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbe10005f50 tx=0x7fbe100057b0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.480+0000 7fbe2088c700 1 -- 172.21.15.107:0/1752713407 shutdown_connections 2023-07-21T22:36:38.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.480+0000 7fbe2088c700 1 --2- 172.21.15.107:0/1752713407 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fbe040a11a0 0x7fbe040a3650 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.480+0000 7fbe2088c700 1 --2- 172.21.15.107:0/1752713407 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe1c1005a0 0x7fbe1c19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.480+0000 7fbe2088c700 1 --2- 172.21.15.107:0/1752713407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe1c0fea20 0x7fbe1c069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.526 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.480+0000 7fbe2088c700 1 --2- 172.21.15.107:0/1752713407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe1c0ffc10 0x7fbe1c19a4d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.480+0000 7fbe2088c700 1 -- 172.21.15.107:0/1752713407 >> 172.21.15.107:0/1752713407 conn(0x7fbe1c0fa020 msgr2=0x7fbe1c0fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.480+0000 7fbe2088c700 1 -- 172.21.15.107:0/1752713407 shutdown_connections 2023-07-21T22:36:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.480+0000 7fbe2088c700 1 -- 172.21.15.107:0/1752713407 wait complete. 2023-07-21T22:36:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 46 2023-07-21T22:36:38.538 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-21T22:36:38.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.630+0000 7f56c553f700 1 -- 172.21.15.107:0/3416406303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c00ffbd0 msgr2=0x7f56c0100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.630+0000 7f56c553f700 1 --2- 172.21.15.107:0/3416406303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c00ffbd0 0x7f56c0100020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009990 tx=0x7f56a8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.630+0000 7f56c553f700 1 -- 172.21.15.107:0/3416406303 shutdown_connections 2023-07-21T22:36:38.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.630+0000 7f56c553f700 1 --2- 172.21.15.107:0/3416406303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56c0100560 0x7f56c0106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.630+0000 7f56c553f700 1 --2- 172.21.15.107:0/3416406303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56c00fe9e0 0x7f56c00fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.630+0000 7f56c553f700 1 --2- 172.21.15.107:0/3416406303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c00ffbd0 0x7f56c0100020 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.630+0000 7f56c553f700 1 -- 172.21.15.107:0/3416406303 >> 172.21.15.107:0/3416406303 conn(0x7f56c00fa000 msgr2=0x7f56c00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.630+0000 7f56c553f700 1 -- 172.21.15.107:0/3416406303 shutdown_connections 2023-07-21T22:36:38.677 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.630+0000 7f56c553f700 1 -- 172.21.15.107:0/3416406303 wait complete. 2023-07-21T22:36:38.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.631+0000 7f56c553f700 1 Processor -- start 2023-07-21T22:36:38.678 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.631+0000 7f56c553f700 1 -- start start 2023-07-21T22:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.631+0000 7f56c553f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56c00fe9e0 0x7f56c01977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.631+0000 7f56c553f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56c00ffbd0 0x7f56c0197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.631+0000 7f56c553f700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0100560 0x7f56c019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.631+0000 7f56c553f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c019f360 con 0x7f56c00fe9e0 2023-07-21T22:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.631+0000 7f56c553f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c019f4a0 con 0x7f56c00ffbd0 2023-07-21T22:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.631+0000 7f56c553f700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c019f630 con 0x7f56c0100560 2023-07-21T22:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.631+0000 7f56beffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56c00fe9e0 0x7f56c01977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56beffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56c00fe9e0 0x7f56c01977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51114/0 (socket says 172.21.15.107:51114) 2023-07-21T22:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56bf7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0100560 0x7f56c019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56beffd700 1 -- 172.21.15.107:0/315368745 learned_addr learned my addr 172.21.15.107:0/315368745 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56be7fc700 1 --2- 172.21.15.107:0/315368745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56c00ffbd0 0x7f56c0197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56beffd700 1 -- 172.21.15.107:0/315368745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0100560 msgr2=0x7f56c019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56beffd700 1 --2- 172.21.15.107:0/315368745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0100560 0x7f56c019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56beffd700 1 -- 172.21.15.107:0/315368745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56c00ffbd0 msgr2=0x7f56c0197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56beffd700 1 --2- 172.21.15.107:0/315368745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56c00ffbd0 0x7f56c0197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56beffd700 1 -- 172.21.15.107:0/315368745 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a80095f0 con 0x7f56c00fe9e0 2023-07-21T22:36:38.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56beffd700 1 --2- 172.21.15.107:0/315368745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56c00fe9e0 0x7f56c01977e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f56b000d980 tx=0x7f56b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56b7fff700 1 -- 172.21.15.107:0/315368745 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b000f890 con 0x7f56c00fe9e0 2023-07-21T22:36:38.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56b7fff700 1 -- 172.21.15.107:0/315368745 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b0010460 con 0x7f56c00fe9e0 2023-07-21T22:36:38.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56b7fff700 1 -- 172.21.15.107:0/315368745 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b00195c0 con 0x7f56c00fe9e0 2023-07-21T22:36:38.681 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56c553f700 1 -- 172.21.15.107:0/315368745 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c019f910 con 0x7f56c00fe9e0 2023-07-21T22:36:38.682 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.632+0000 7f56c553f700 1 -- 172.21.15.107:0/315368745 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c019fdd0 con 0x7f56c00fe9e0 2023-07-21T22:36:38.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.635+0000 7f56b7fff700 1 -- 172.21.15.107:0/315368745 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f56b00105d0 con 0x7f56c00fe9e0 2023-07-21T22:36:38.683 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.636+0000 7f56b7fff700 1 --2- 172.21.15.107:0/315368745 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56ac0a1150 0x7f56ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.636+0000 7f56b7fff700 1 -- 172.21.15.107:0/315368745 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f56b00cc690 con 0x7f56c00fe9e0 2023-07-21T22:36:38.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.636+0000 7f56be7fc700 1 --2- 172.21.15.107:0/315368745 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56ac0a1150 0x7f56ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.637+0000 7f56d16c4b80 1 Processor -- start 2023-07-21T22:36:38.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.637+0000 7f56d16c4b80 1 -- start start 2023-07-21T22:36:38.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.637+0000 7f56be7fc700 1 --2- 172.21.15.107:0/315368745 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56ac0a1150 0x7f56ac0a3600 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f56a800b2e0 tx=0x7f56a8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.684 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.637+0000 7f56d16c4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea7830 0x5581d5ea7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.637+0000 7f56d16c4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea81a0 0x5581d5ea6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.637+0000 7f56d16c4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea6a10 0x5581d5ea6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.637+0000 7f56d16c4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d5dabcb0 con 0x5581d5ea6a10 2023-07-21T22:36:38.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.637+0000 7f56d16c4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d5dfea40 con 0x5581d5ea81a0 2023-07-21T22:36:38.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.637+0000 7f56d16c4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d5df7ed0 con 0x5581d5ea7830 2023-07-21T22:36:38.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.637+0000 7f56bf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea6a10 0x5581d5ea6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.685 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.637+0000 7f56bf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea6a10 0x5581d5ea6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51126/0 (socket says 172.21.15.107:51126) 2023-07-21T22:36:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.637+0000 7f56bf7fe700 1 -- 172.21.15.107:0/2576388911 learned_addr learned my addr 172.21.15.107:0/2576388911 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.638+0000 7f56beffd700 1 --2- 172.21.15.107:0/2576388911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea7830 0x5581d5ea7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.638+0000 7f56be7fc700 1 --2- 172.21.15.107:0/2576388911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea81a0 0x5581d5ea6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.638+0000 7f56bf7fe700 1 -- 172.21.15.107:0/2576388911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea7830 msgr2=0x5581d5ea7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.638+0000 7f56bf7fe700 1 --2- 172.21.15.107:0/2576388911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea7830 0x5581d5ea7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.638+0000 7f56bf7fe700 1 -- 172.21.15.107:0/2576388911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea81a0 msgr2=0x5581d5ea6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.638+0000 7f56bf7fe700 1 --2- 172.21.15.107:0/2576388911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea81a0 0x5581d5ea6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.686 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.638+0000 7f56bf7fe700 1 -- 172.21.15.107:0/2576388911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581d5c25de0 con 0x5581d5ea6a10 2023-07-21T22:36:38.687 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.638+0000 7f56bf7fe700 1 --2- 172.21.15.107:0/2576388911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea6a10 0x5581d5ea6e90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f56b80094d0 tx=0x7f56b8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91bed984caa82a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.687 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.638+0000 7f56b57fa700 1 -- 172.21.15.107:0/2576388911 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8003c00 con 0x5581d5ea6a10 2023-07-21T22:36:38.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.638+0000 7f56b57fa700 1 -- 172.21.15.107:0/2576388911 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b8003d60 con 0x5581d5ea6a10 2023-07-21T22:36:38.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.638+0000 7f56b57fa700 1 -- 172.21.15.107:0/2576388911 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8012890 con 0x5581d5ea6a10 2023-07-21T22:36:38.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.639+0000 7f56d16c4b80 1 -- 172.21.15.107:0/2576388911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea6a10 msgr2=0x5581d5ea6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.639+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/2576388911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea6a10 0x5581d5ea6e90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f56b80094d0 tx=0x7f56b8009920 comp rx=0 tx=0).stop 2023-07-21T22:36:38.689 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.639+0000 7f56d16c4b80 1 -- 172.21.15.107:0/2576388911 shutdown_connections 2023-07-21T22:36:38.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.639+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/2576388911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea6a10 0x5581d5ea6e90 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.639+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/2576388911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea7830 0x5581d5ea7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.639+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/2576388911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea81a0 0x5581d5ea6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.639+0000 7f56d16c4b80 1 -- 172.21.15.107:0/2576388911 >> 172.21.15.107:0/2576388911 conn(0x5581d5e9bef0 msgr2=0x5581d5ea1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.639+0000 7f56d16c4b80 1 -- 172.21.15.107:0/2576388911 shutdown_connections 2023-07-21T22:36:38.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.639+0000 7f56d16c4b80 1 -- 172.21.15.107:0/2576388911 wait complete. 2023-07-21T22:36:38.690 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.640+0000 7f56d16c4b80 1 -- start start 2023-07-21T22:36:38.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.640+0000 7f56d16c4b80 1 Processor -- start 2023-07-21T22:36:38.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.640+0000 7f56d16c4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 0x5581d5eb00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.640+0000 7f56d16c4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea7830 0x5581d5eb0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.640+0000 7f56d16c4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea81a0 0x5581d5ec8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.640+0000 7f56d16c4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d5d97ac0 con 0x5581d5ea7830 2023-07-21T22:36:38.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.640+0000 7f56d16c4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d5d977c0 con 0x5581d5ea81a0 2023-07-21T22:36:38.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.640+0000 7f56d16c4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d5c7b920 con 0x5581d5ea6a10 2023-07-21T22:36:38.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.640+0000 7f56d16c4b80 10 client.0 _ll_register_callbacks cb 0x5581d5d95760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:38.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.640+0000 7f56beffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 0x5581d5eb00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.640+0000 7f56beffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 0x5581d5eb00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39686/0 (socket says 172.21.15.107:39686) 2023-07-21T22:36:38.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.640+0000 7f56beffd700 1 -- 172.21.15.107:0/1898820602 learned_addr learned my addr 172.21.15.107:0/1898820602 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.641+0000 7f56beffd700 1 -- 172.21.15.107:0/1898820602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea81a0 msgr2=0x5581d5ec8680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:36:38.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.641+0000 7f56beffd700 1 --2- 172.21.15.107:0/1898820602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea81a0 0x5581d5ec8680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.641+0000 7f56beffd700 1 -- 172.21.15.107:0/1898820602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea7830 msgr2=0x5581d5eb0610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:36:38.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.641+0000 7f56beffd700 1 --2- 172.21.15.107:0/1898820602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea7830 0x5581d5eb0610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.641+0000 7f56beffd700 1 -- 172.21.15.107:0/1898820602 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b8009180 con 0x5581d5ea6a10 2023-07-21T22:36:38.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.641+0000 7f56beffd700 1 --2- 172.21.15.107:0/1898820602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 0x5581d5eb00d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f56b00ca170 tx=0x7f56b009dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e45cc9857e97e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.641+0000 7f56a67fc700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:36:38.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.641+0000 7f56a67fc700 1 -- 172.21.15.107:0/1898820602 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b00108b0 con 0x5581d5ea6a10 2023-07-21T22:36:38.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.641+0000 7f56a67fc700 1 -- 172.21.15.107:0/1898820602 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f56b0098f70 con 0x5581d5ea6a10 2023-07-21T22:36:38.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.641+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1898820602 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581d5d514f0 con 0x5581d5ea6a10 2023-07-21T22:36:38.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.642+0000 7f56a67fc700 1 -- 172.21.15.107:0/1898820602 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56b0010a10 con 0x5581d5ea6a10 2023-07-21T22:36:38.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.642+0000 7f56d16c4b80 10 client.26194 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:38.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.642+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1898820602 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5581d5c0ec60 con 0x5581d5ea6a10 2023-07-21T22:36:38.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.642+0000 7f56a67fc700 1 -- 172.21.15.107:0/1898820602 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5581d5d514f0 con 0x5581d5ea6a10 2023-07-21T22:36:38.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.643+0000 7f56a67fc700 1 -- 172.21.15.107:0/1898820602 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f56b00c5020 con 0x5581d5ea6a10 2023-07-21T22:36:38.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.643+0000 7f56d16c4b80 10 client.26194 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:38.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.643+0000 7f56d16c4b80 10 client.26194 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-21T22:36:38.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.643+0000 7f56d16c4b80 20 client.26194 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:38.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.643+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/1898820602 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x5581d5eccdf0 0x5581d5ecf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.643+0000 7f56d16c4b80 4 client.26194 mds_command: new command op to 24437 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:38.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.643+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1898820602 --> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581d5df0eb0 con 0x5581d5eccdf0 2023-07-21T22:36:38.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.643+0000 7f56be7fc700 1 --2- 172.21.15.107:0/1898820602 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x5581d5eccdf0 0x5581d5ecf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.644+0000 7f56a67fc700 10 client.26194 ms_handle_connect on v2:172.21.15.132:6832/3293513462 2023-07-21T22:36:38.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.644+0000 7f56be7fc700 1 --2- 172.21.15.107:0/1898820602 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x5581d5eccdf0 0x5581d5ecf240 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=621a3fadca2ddd86 server_cookie=6a6dba766a243f2b in_seq=0 out_seq=0 2023-07-21T22:36:38.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.645+0000 7f56a67fc700 1 -- 172.21.15.107:0/1898820602 <== mds.4 v2:172.21.15.132:6832/3293513462 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5581d5df0eb0 con 0x5581d5eccdf0 2023-07-21T22:36:38.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.645+0000 7f56a67fc700 10 client.26194 handle_command_reply: tid=0 2023-07-21T22:36:38.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.645+0000 7f56d16c4b80 1 client.26194 shutdown 2023-07-21T22:36:38.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.646+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1898820602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 msgr2=0x5581d5eb00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.646+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/1898820602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 0x5581d5eb00d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f56b00ca170 tx=0x7f56b009dab0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.646+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1898820602 shutdown_connections 2023-07-21T22:36:38.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.646+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/1898820602 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x5581d5eccdf0 0x5581d5ecf240 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.646+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/1898820602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea81a0 0x5581d5ec8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.646+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/1898820602 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 0x5581d5eb00d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.646+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/1898820602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea7830 0x5581d5eb0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.646+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1898820602 >> 172.21.15.107:0/1898820602 conn(0x5581d5e9bef0 msgr2=0x5581d5ea0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.646+0000 7f56a67fc700 0 client.26194 ms_handle_reset on v2:172.21.15.132:6832/3293513462 2023-07-21T22:36:38.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.646+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1898820602 shutdown_connections 2023-07-21T22:36:38.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.646+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1898820602 wait complete. 2023-07-21T22:36:38.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.646+0000 7f56d16c4b80 20 client.26194 trim_cache size 0 max 16384 2023-07-21T22:36:38.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.666+0000 7f56d16c4b80 1 Processor -- start 2023-07-21T22:36:38.711 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.666+0000 7f56d16c4b80 1 -- start start 2023-07-21T22:36:38.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.666+0000 7f56d16c4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 0x5581d5ea6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.666+0000 7f56d16c4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea7830 0x5581d5ea0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.666+0000 7f56d16c4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea81a0 0x5581d5ecee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.712 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.666+0000 7f56d16c4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d5ea6890 con 0x5581d5ea81a0 2023-07-21T22:36:38.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.667+0000 7f56d16c4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d5e9e030 con 0x5581d5ea7830 2023-07-21T22:36:38.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.667+0000 7f56d16c4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d5ecc930 con 0x5581d5ea6a10 2023-07-21T22:36:38.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.667+0000 7f56beffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 0x5581d5ea6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.667+0000 7f56beffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 0x5581d5ea6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39690/0 (socket says 172.21.15.107:39690) 2023-07-21T22:36:38.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.667+0000 7f56beffd700 1 -- 172.21.15.107:0/2450664767 learned_addr learned my addr 172.21.15.107:0/2450664767 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.667+0000 7f56beffd700 1 -- 172.21.15.107:0/2450664767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea7830 msgr2=0x5581d5ea0a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:36:38.713 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.667+0000 7f56beffd700 1 --2- 172.21.15.107:0/2450664767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea7830 0x5581d5ea0a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.667+0000 7f56beffd700 1 -- 172.21.15.107:0/2450664767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea81a0 msgr2=0x5581d5ecee00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:36:38.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.667+0000 7f56beffd700 1 --2- 172.21.15.107:0/2450664767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea81a0 0x5581d5ecee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.714 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.667+0000 7f56beffd700 1 -- 172.21.15.107:0/2450664767 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581d5c4c250 con 0x5581d5ea6a10 2023-07-21T22:36:38.715 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.668+0000 7f56beffd700 1 --2- 172.21.15.107:0/2450664767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 0x5581d5ea6350 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f56b0099a10 tx=0x7f56b00a1db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8307212e060c733e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.715 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.668+0000 7f56b5ffb700 1 -- 172.21.15.107:0/2450664767 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0014030 con 0x5581d5ea6a10 2023-07-21T22:36:38.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.668+0000 7f56b5ffb700 1 -- 172.21.15.107:0/2450664767 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b00a1b60 con 0x5581d5ea6a10 2023-07-21T22:36:38.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.668+0000 7f56b5ffb700 1 -- 172.21.15.107:0/2450664767 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0098c40 con 0x5581d5ea6a10 2023-07-21T22:36:38.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.669+0000 7f56d16c4b80 1 -- 172.21.15.107:0/2450664767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 msgr2=0x5581d5ea6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.669+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/2450664767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 0x5581d5ea6350 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f56b0099a10 tx=0x7f56b00a1db0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.669+0000 7f56d16c4b80 1 -- 172.21.15.107:0/2450664767 shutdown_connections 2023-07-21T22:36:38.716 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.669+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/2450664767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea81a0 0x5581d5ecee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.669+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/2450664767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea6a10 0x5581d5ea6350 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.669+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/2450664767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea7830 0x5581d5ea0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.669+0000 7f56d16c4b80 1 -- 172.21.15.107:0/2450664767 >> 172.21.15.107:0/2450664767 conn(0x5581d5e9bef0 msgr2=0x5581d5eaede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.669+0000 7f56d16c4b80 1 -- 172.21.15.107:0/2450664767 shutdown_connections 2023-07-21T22:36:38.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.669+0000 7f56d16c4b80 1 -- 172.21.15.107:0/2450664767 wait complete. 2023-07-21T22:36:38.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.670+0000 7f56d16c4b80 1 -- start start 2023-07-21T22:36:38.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.670+0000 7f56d16c4b80 1 Processor -- start 2023-07-21T22:36:38.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.670+0000 7f56d16c4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea6a10 0x5581d5eb5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.670+0000 7f56d16c4b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea7830 0x5581d5eb5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.670+0000 7f56d16c4b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea81a0 0x5581d5eb5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.670+0000 7f56d16c4b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d5eb66c0 con 0x5581d5ea7830 2023-07-21T22:36:38.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.670+0000 7f56d16c4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d5eb6830 con 0x5581d5ea6a10 2023-07-21T22:36:38.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.670+0000 7f56d16c4b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581d5eb69f0 con 0x5581d5ea81a0 2023-07-21T22:36:38.718 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.670+0000 7f56d16c4b80 10 client.0 _ll_register_callbacks cb 0x7f56b00993e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:38.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.671+0000 7f56bf7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea81a0 0x5581d5eb5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.671+0000 7f56bf7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea81a0 0x5581d5eb5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39692/0 (socket says 172.21.15.107:39692) 2023-07-21T22:36:38.719 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.671+0000 7f56bf7fe700 1 -- 172.21.15.107:0/1227960737 learned_addr learned my addr 172.21.15.107:0/1227960737 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.720 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.671+0000 7f56be7fc700 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea7830 0x5581d5eb5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.671+0000 7f56beffd700 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea6a10 0x5581d5eb5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.672+0000 7f56be7fc700 1 -- 172.21.15.107:0/1227960737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea81a0 msgr2=0x5581d5eb5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.672+0000 7f56be7fc700 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea81a0 0x5581d5eb5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.672+0000 7f56be7fc700 1 -- 172.21.15.107:0/1227960737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea6a10 msgr2=0x5581d5eb5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.672+0000 7f56be7fc700 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea6a10 0x5581d5eb5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.721 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.672+0000 7f56be7fc700 1 -- 172.21.15.107:0/1227960737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581d5c0ec60 con 0x5581d5ea7830 2023-07-21T22:36:38.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.672+0000 7f56beffd700 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea6a10 0x5581d5eb5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:38.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.672+0000 7f56be7fc700 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea7830 0x5581d5eb5aa0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x5581d5ecfb90 tx=0x7f56a801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63560410baf34c68 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.672+0000 7f56a67fc700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:38.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.672+0000 7f56a67fc700 1 -- 172.21.15.107:0/1227960737 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8005690 con 0x5581d5ea7830 2023-07-21T22:36:38.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.672+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1227960737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56b8003d60 con 0x5581d5ea7830 2023-07-21T22:36:38.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.672+0000 7f56a67fc700 1 -- 172.21.15.107:0/1227960737 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f56a8033a60 con 0x5581d5ea7830 2023-07-21T22:36:38.722 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.673+0000 7f56a67fc700 1 -- 172.21.15.107:0/1227960737 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56a8033c60 con 0x5581d5ea7830 2023-07-21T22:36:38.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.673+0000 7f56d16c4b80 10 client.16590 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:38.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.673+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1227960737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5581d5c242b0 con 0x5581d5ea7830 2023-07-21T22:36:38.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.673+0000 7f56a67fc700 1 -- 172.21.15.107:0/1227960737 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b8003d60 con 0x5581d5ea7830 2023-07-21T22:36:38.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.674+0000 7f56a67fc700 1 -- 172.21.15.107:0/1227960737 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f56a802f070 con 0x5581d5ea7830 2023-07-21T22:36:38.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.674+0000 7f56d16c4b80 10 client.16590 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:38.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.674+0000 7f56d16c4b80 10 client.16590 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-21T22:36:38.723 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.674+0000 7f56d16c4b80 20 client.16590 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:38.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.674+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x5581d5f21a70 0x5581d5f23e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.674+0000 7f56d16c4b80 4 client.16590 mds_command: new command op to 24437 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:36:38.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.674+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1227960737 --> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5581d5db3050 con 0x5581d5f21a70 2023-07-21T22:36:38.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.674+0000 7f56beffd700 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x5581d5f21a70 0x5581d5f23e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56a67fc700 10 client.16590 ms_handle_connect on v2:172.21.15.132:6832/3293513462 2023-07-21T22:36:38.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56beffd700 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x5581d5f21a70 0x5581d5f23e80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fdafa931b85f54cf server_cookie=2d366103655d8730 in_seq=0 out_seq=0 2023-07-21T22:36:38.724 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56a67fc700 1 -- 172.21.15.107:0/1227960737 <== mds.4 v2:172.21.15.132:6832/3293513462 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5581d5db3050 con 0x5581d5f21a70 2023-07-21T22:36:38.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56a67fc700 10 client.16590 handle_command_reply: tid=0 2023-07-21T22:36:38.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56d16c4b80 1 client.16590 shutdown 2023-07-21T22:36:38.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1227960737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea7830 msgr2=0x5581d5eb5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea7830 0x5581d5eb5aa0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x5581d5ecfb90 tx=0x7f56a801a050 comp rx=0 tx=0).stop 2023-07-21T22:36:38.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1227960737 shutdown_connections 2023-07-21T22:36:38.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5581d5ea81a0 0x5581d5eb5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.132:6832/3293513462,v1:172.21.15.132:6833/3293513462] conn(0x5581d5f21a70 0x5581d5f23e80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5581d5ea6a10 0x5581d5eb5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.726 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56d16c4b80 1 --2- 172.21.15.107:0/1227960737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d5ea7830 0x5581d5eb5aa0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1227960737 >> 172.21.15.107:0/1227960737 conn(0x5581d5e9bef0 msgr2=0x5581d5ecee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56a67fc700 0 client.16590 ms_handle_reset on v2:172.21.15.132:6832/3293513462 2023-07-21T22:36:38.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1227960737 shutdown_connections 2023-07-21T22:36:38.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56d16c4b80 1 -- 172.21.15.107:0/1227960737 wait complete. 2023-07-21T22:36:38.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.676+0000 7f56d16c4b80 20 client.16590 trim_cache size 0 max 16384 2023-07-21T22:36:38.727 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:36:38.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.677+0000 7f56c553f700 1 -- 172.21.15.107:0/315368745 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56ac0a1150 msgr2=0x7f56ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.677+0000 7f56c553f700 1 --2- 172.21.15.107:0/315368745 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56ac0a1150 0x7f56ac0a3600 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f56a800b2e0 tx=0x7f56a8005fd0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.677+0000 7f56c553f700 1 -- 172.21.15.107:0/315368745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56c00fe9e0 msgr2=0x7f56c01977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.677+0000 7f56c553f700 1 --2- 172.21.15.107:0/315368745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56c00fe9e0 0x7f56c01977e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f56b000d980 tx=0x7f56b000dc90 comp rx=0 tx=0).stop 2023-07-21T22:36:38.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.677+0000 7f56c553f700 1 -- 172.21.15.107:0/315368745 shutdown_connections 2023-07-21T22:36:38.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.677+0000 7f56c553f700 1 --2- 172.21.15.107:0/315368745 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56ac0a1150 0x7f56ac0a3600 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.677+0000 7f56c553f700 1 --2- 172.21.15.107:0/315368745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56c0100560 0x7f56c019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.677+0000 7f56c553f700 1 --2- 172.21.15.107:0/315368745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56c00fe9e0 0x7f56c01977e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.677+0000 7f56c553f700 1 --2- 172.21.15.107:0/315368745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56c00ffbd0 0x7f56c0197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.677+0000 7f56c553f700 1 -- 172.21.15.107:0/315368745 >> 172.21.15.107:0/315368745 conn(0x7f56c00fa000 msgr2=0x7f56c0102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.677+0000 7f56c553f700 1 -- 172.21.15.107:0/315368745 shutdown_connections 2023-07-21T22:36:38.730 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.677+0000 7f56c553f700 1 -- 172.21.15.107:0/315368745 wait complete. 2023-07-21T22:36:38.737 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-21T22:36:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.819+0000 7f694781d700 1 -- 172.21.15.107:0/3945212064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69400fea00 msgr2=0x7f69400fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.819+0000 7f694781d700 1 --2- 172.21.15.107:0/3945212064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69400fea00 0x7f69400fee10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6934009990 tx=0x7f6934009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.866 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.819+0000 7f694781d700 1 -- 172.21.15.107:0/3945212064 shutdown_connections 2023-07-21T22:36:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.819+0000 7f694781d700 1 --2- 172.21.15.107:0/3945212064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6940100640 0x7f6940106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.819+0000 7f694781d700 1 --2- 172.21.15.107:0/3945212064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69400fea00 0x7f69400fee10 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.819+0000 7f694781d700 1 --2- 172.21.15.107:0/3945212064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69400ffc90 0x7f6940100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.819+0000 7f694781d700 1 -- 172.21.15.107:0/3945212064 >> 172.21.15.107:0/3945212064 conn(0x7f69400fa020 msgr2=0x7f69400fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f694781d700 1 -- 172.21.15.107:0/3945212064 shutdown_connections 2023-07-21T22:36:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f694781d700 1 -- 172.21.15.107:0/3945212064 wait complete. 2023-07-21T22:36:38.867 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f694781d700 1 Processor -- start 2023-07-21T22:36:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f694781d700 1 -- start start 2023-07-21T22:36:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f694781d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69400fea00 0x7f6940102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f694781d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69400ffc90 0x7f6940101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f694781d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6940100640 0x7f69401018c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f694781d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6940102060 con 0x7f6940100640 2023-07-21T22:36:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f694781d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69401021d0 con 0x7f69400fea00 2023-07-21T22:36:38.868 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f694781d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694019f910 con 0x7f69400ffc90 2023-07-21T22:36:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f6945dba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6940100640 0x7f69401018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f69455b9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69400fea00 0x7f6940102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f6945dba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6940100640 0x7f69401018c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51150/0 (socket says 172.21.15.107:51150) 2023-07-21T22:36:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f6945dba700 1 -- 172.21.15.107:0/3182576229 learned_addr learned my addr 172.21.15.107:0/3182576229 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.820+0000 7f6944db8700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69400ffc90 0x7f6940101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.821+0000 7f6945dba700 1 -- 172.21.15.107:0/3182576229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69400ffc90 msgr2=0x7f6940101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.821+0000 7f6945dba700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69400ffc90 0x7f6940101380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.821+0000 7f6945dba700 1 -- 172.21.15.107:0/3182576229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69400fea00 msgr2=0x7f6940102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.821+0000 7f6945dba700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69400fea00 0x7f6940102d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.821+0000 7f6945dba700 1 -- 172.21.15.107:0/3182576229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69340095f0 con 0x7f6940100640 2023-07-21T22:36:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.821+0000 7f69455b9700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69400fea00 0x7f6940102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.821+0000 7f6945dba700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6940100640 0x7f69401018c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f693800ef30 tx=0x7f693800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.821+0000 7f69327fc700 1 -- 172.21.15.107:0/3182576229 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938018070 con 0x7f6940100640 2023-07-21T22:36:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.821+0000 7f69327fc700 1 -- 172.21.15.107:0/3182576229 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f693800f040 con 0x7f6940100640 2023-07-21T22:36:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.821+0000 7f694781d700 1 -- 172.21.15.107:0/3182576229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694019fb90 con 0x7f6940100640 2023-07-21T22:36:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.821+0000 7f69327fc700 1 -- 172.21.15.107:0/3182576229 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693800cd00 con 0x7f6940100640 2023-07-21T22:36:38.871 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.821+0000 7f694781d700 1 -- 172.21.15.107:0/3182576229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69401a00a0 con 0x7f6940100640 2023-07-21T22:36:38.876 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.823+0000 7f69327fc700 1 -- 172.21.15.107:0/3182576229 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6938003a00 con 0x7f6940100640 2023-07-21T22:36:38.876 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.824+0000 7f69327fc700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f692c0a11d0 0x7f692c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.876 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.824+0000 7f69455b9700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f692c0a11d0 0x7f692c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.824+0000 7f69327fc700 1 -- 172.21.15.107:0/3182576229 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f69380cc3e0 con 0x7f6940100640 2023-07-21T22:36:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.824+0000 7f69455b9700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f692c0a11d0 0x7f692c0a3680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f693400b2e0 tx=0x7f6934005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f69539a2b80 1 Processor -- start 2023-07-21T22:36:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f69539a2b80 1 -- start start 2023-07-21T22:36:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f69539a2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881e830 0x5575a881ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f69539a2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 0x5575a881d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f69539a2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a881de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f69539a2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a8722cb0 con 0x5575a881da10 2023-07-21T22:36:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f69539a2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a8775a40 con 0x5575a881e830 2023-07-21T22:36:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f69539a2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a876eed0 con 0x5575a881f1a0 2023-07-21T22:36:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f6945dba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a881de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f6944db8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 0x5575a881d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f69455b9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881e830 0x5575a881ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f6944db8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 0x5575a881d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39714/0 (socket says 172.21.15.107:39714) 2023-07-21T22:36:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f6945dba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a881de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51158/0 (socket says 172.21.15.107:51158) 2023-07-21T22:36:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f6944db8700 1 -- 172.21.15.107:0/4047604021 learned_addr learned my addr 172.21.15.107:0/4047604021 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f69455b9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881e830 0x5575a881ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41982/0 (socket says 172.21.15.107:41982) 2023-07-21T22:36:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.825+0000 7f69455b9700 1 -- 172.21.15.107:0/4047604021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 msgr2=0x5575a881d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.826+0000 7f69455b9700 1 --2- 172.21.15.107:0/4047604021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 0x5575a881d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.826+0000 7f69455b9700 1 -- 172.21.15.107:0/4047604021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 msgr2=0x5575a881de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.826+0000 7f69455b9700 1 --2- 172.21.15.107:0/4047604021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a881de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.826+0000 7f69455b9700 1 -- 172.21.15.107:0/4047604021 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575a859cde0 con 0x5575a881e830 2023-07-21T22:36:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.826+0000 7f6944db8700 1 --2- 172.21.15.107:0/4047604021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 0x5575a881d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.826+0000 7f6945dba700 1 --2- 172.21.15.107:0/4047604021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a881de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.826+0000 7f69455b9700 1 --2- 172.21.15.107:0/4047604021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881e830 0x5575a881ec40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69340098e0 tx=0x7f6934020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee938b26f6def521 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.826+0000 7f692b7fe700 1 -- 172.21.15.107:0/4047604021 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693402b070 con 0x5575a881e830 2023-07-21T22:36:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.826+0000 7f692b7fe700 1 -- 172.21.15.107:0/4047604021 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f693401e040 con 0x5575a881e830 2023-07-21T22:36:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.826+0000 7f692b7fe700 1 -- 172.21.15.107:0/4047604021 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934024d70 con 0x5575a881e830 2023-07-21T22:36:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 -- 172.21.15.107:0/4047604021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881e830 msgr2=0x5575a881ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 --2- 172.21.15.107:0/4047604021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881e830 0x5575a881ec40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69340098e0 tx=0x7f6934020000 comp rx=0 tx=0).stop 2023-07-21T22:36:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 -- 172.21.15.107:0/4047604021 shutdown_connections 2023-07-21T22:36:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 --2- 172.21.15.107:0/4047604021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a881de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 --2- 172.21.15.107:0/4047604021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881e830 0x5575a881ec40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 --2- 172.21.15.107:0/4047604021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 0x5575a881d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 -- 172.21.15.107:0/4047604021 >> 172.21.15.107:0/4047604021 conn(0x5575a8812ef0 msgr2=0x5575a8818360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 -- 172.21.15.107:0/4047604021 shutdown_connections 2023-07-21T22:36:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 -- 172.21.15.107:0/4047604021 wait complete. 2023-07-21T22:36:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 -- start start 2023-07-21T22:36:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 Processor -- start 2023-07-21T22:36:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881da10 0x5575a8826fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881e830 0x5575a8827520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881f1a0 0x5575a882e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a870eac0 con 0x5575a881f1a0 2023-07-21T22:36:38.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a870e7c0 con 0x5575a881da10 2023-07-21T22:36:38.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a85f2920 con 0x5575a881e830 2023-07-21T22:36:38.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.827+0000 7f69539a2b80 10 client.0 _ll_register_callbacks cb 0x5575a870c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:38.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6945dba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881f1a0 0x5575a882e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6944db8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881e830 0x5575a8827520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.883 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f69455b9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881da10 0x5575a8826fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6945dba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881f1a0 0x5575a882e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51164/0 (socket says 172.21.15.107:51164) 2023-07-21T22:36:38.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6944db8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881e830 0x5575a8827520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39720/0 (socket says 172.21.15.107:39720) 2023-07-21T22:36:38.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f69455b9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881da10 0x5575a8826fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41988/0 (socket says 172.21.15.107:41988) 2023-07-21T22:36:38.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6945dba700 1 -- 172.21.15.107:0/3883200327 learned_addr learned my addr 172.21.15.107:0/3883200327 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6944db8700 1 -- 172.21.15.107:0/3883200327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881da10 msgr2=0x5575a8826fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6944db8700 1 --2- 172.21.15.107:0/3883200327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881da10 0x5575a8826fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.884 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6944db8700 1 -- 172.21.15.107:0/3883200327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881f1a0 msgr2=0x5575a882e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6944db8700 1 --2- 172.21.15.107:0/3883200327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881f1a0 0x5575a882e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6944db8700 1 -- 172.21.15.107:0/3883200327 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69401a00a0 con 0x5575a881e830 2023-07-21T22:36:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6945dba700 1 --2- 172.21.15.107:0/3883200327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881f1a0 0x5575a882e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6944db8700 1 --2- 172.21.15.107:0/3883200327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881e830 0x5575a8827520 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f693c012950 tx=0x7f693c012c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a112ae429e951922 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6928ff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:36:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6928ff9700 1 -- 172.21.15.107:0/3883200327 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c017070 con 0x5575a881e830 2023-07-21T22:36:38.885 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f69539a2b80 1 -- 172.21.15.107:0/3883200327 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575a86c84f0 con 0x5575a881e830 2023-07-21T22:36:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6928ff9700 1 -- 172.21.15.107:0/3883200327 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f693c004050 con 0x5575a881e830 2023-07-21T22:36:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f6928ff9700 1 -- 172.21.15.107:0/3883200327 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f693c004210 con 0x5575a881e830 2023-07-21T22:36:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f69539a2b80 10 client.26212 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.828+0000 7f69539a2b80 1 -- 172.21.15.107:0/3883200327 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f693c0099e0 con 0x5575a881e830 2023-07-21T22:36:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.829+0000 7f6928ff9700 1 -- 172.21.15.107:0/3883200327 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5575a86c84f0 con 0x5575a881e830 2023-07-21T22:36:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.829+0000 7f6928ff9700 1 -- 172.21.15.107:0/3883200327 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f693c025050 con 0x5575a881e830 2023-07-21T22:36:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.829+0000 7f69539a2b80 10 client.26212 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.829+0000 7f69539a2b80 10 client.26212 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T22:36:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.829+0000 7f69539a2b80 20 client.26212 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.829+0000 7f69539a2b80 1 --2- 172.21.15.107:0/3883200327 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5575a8832e40 0x5575a8835250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.829+0000 7f69539a2b80 4 client.26212 mds_command: new command op to 24455 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.829+0000 7f69539a2b80 1 -- 172.21.15.107:0/3883200327 --> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575a8767eb0 con 0x5575a8832e40 2023-07-21T22:36:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.829+0000 7f69455b9700 1 --2- 172.21.15.107:0/3883200327 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5575a8832e40 0x5575a8835250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.829+0000 7f6928ff9700 10 client.26212 ms_handle_connect on v2:172.21.15.132:6834/1022724305 2023-07-21T22:36:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.829+0000 7f69455b9700 1 --2- 172.21.15.107:0/3883200327 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5575a8832e40 0x5575a8835250 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5d10ace5ef4d128b server_cookie=e7bb70a966d90d55 in_seq=0 out_seq=0 2023-07-21T22:36:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f6928ff9700 1 -- 172.21.15.107:0/3883200327 <== mds.3 v2:172.21.15.132:6834/1022724305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5575a8767eb0 con 0x5575a8832e40 2023-07-21T22:36:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f6928ff9700 10 client.26212 handle_command_reply: tid=0 2023-07-21T22:36:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f69539a2b80 1 client.26212 shutdown 2023-07-21T22:36:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f69539a2b80 1 -- 172.21.15.107:0/3883200327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881e830 msgr2=0x5575a8827520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f69539a2b80 1 --2- 172.21.15.107:0/3883200327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881e830 0x5575a8827520 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f693c012950 tx=0x7f693c012c60 comp rx=0 tx=0).stop 2023-07-21T22:36:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f69539a2b80 1 -- 172.21.15.107:0/3883200327 shutdown_connections 2023-07-21T22:36:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f69539a2b80 1 --2- 172.21.15.107:0/3883200327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881f1a0 0x5575a882e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f69539a2b80 1 --2- 172.21.15.107:0/3883200327 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5575a8832e40 0x5575a8835250 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f69539a2b80 1 --2- 172.21.15.107:0/3883200327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881da10 0x5575a8826fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f6928ff9700 0 client.26212 ms_handle_reset on v2:172.21.15.132:6834/1022724305 2023-07-21T22:36:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f69539a2b80 1 --2- 172.21.15.107:0/3883200327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881e830 0x5575a8827520 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f69539a2b80 1 -- 172.21.15.107:0/3883200327 >> 172.21.15.107:0/3883200327 conn(0x5575a8812ef0 msgr2=0x5575a8817a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f69539a2b80 1 -- 172.21.15.107:0/3883200327 shutdown_connections 2023-07-21T22:36:38.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f69539a2b80 1 -- 172.21.15.107:0/3883200327 wait complete. 2023-07-21T22:36:38.890 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.830+0000 7f69539a2b80 20 client.26212 trim_cache size 0 max 16384 2023-07-21T22:36:38.894 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f69539a2b80 1 Processor -- start 2023-07-21T22:36:38.894 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f69539a2b80 1 -- start start 2023-07-21T22:36:38.894 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f69539a2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a8839bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f69539a2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881e830 0x5575a882e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f69539a2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 0x5575a8824fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f69539a2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a8826490 con 0x5575a881da10 2023-07-21T22:36:38.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f69539a2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a8826b50 con 0x5575a881e830 2023-07-21T22:36:38.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f69539a2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a88301d0 con 0x5575a881f1a0 2023-07-21T22:36:38.895 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f69455b9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a8839bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f6944db8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881e830 0x5575a882e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f6945dba700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 0x5575a8824fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f6945dba700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 0x5575a8824fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39730/0 (socket says 172.21.15.107:39730) 2023-07-21T22:36:38.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f69455b9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a8839bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51168/0 (socket says 172.21.15.107:51168) 2023-07-21T22:36:38.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f6944db8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881e830 0x5575a882e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:41996/0 (socket says 172.21.15.107:41996) 2023-07-21T22:36:38.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.844+0000 7f6945dba700 1 -- 172.21.15.107:0/1282554255 learned_addr learned my addr 172.21.15.107:0/1282554255 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.896 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.845+0000 7f6945dba700 1 -- 172.21.15.107:0/1282554255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881e830 msgr2=0x5575a882e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.845+0000 7f6945dba700 1 --2- 172.21.15.107:0/1282554255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881e830 0x5575a882e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.845+0000 7f6945dba700 1 -- 172.21.15.107:0/1282554255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 msgr2=0x5575a8839bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.845+0000 7f6945dba700 1 --2- 172.21.15.107:0/1282554255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a8839bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.845+0000 7f6945dba700 1 -- 172.21.15.107:0/1282554255 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6940107750 con 0x5575a881f1a0 2023-07-21T22:36:38.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.845+0000 7f6945dba700 1 --2- 172.21.15.107:0/1282554255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 0x5575a8824fa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f69380c9f10 tx=0x7f693809f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d9c4ef46b974662 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.845+0000 7f692bfff700 1 -- 172.21.15.107:0/1282554255 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938004ad0 con 0x5575a881f1a0 2023-07-21T22:36:38.897 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.845+0000 7f692bfff700 1 -- 172.21.15.107:0/1282554255 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f693800ce60 con 0x5575a881f1a0 2023-07-21T22:36:38.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.845+0000 7f692bfff700 1 -- 172.21.15.107:0/1282554255 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938099ac0 con 0x5575a881f1a0 2023-07-21T22:36:38.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.846+0000 7f69539a2b80 1 -- 172.21.15.107:0/1282554255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 msgr2=0x5575a8824fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.846+0000 7f69539a2b80 1 --2- 172.21.15.107:0/1282554255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 0x5575a8824fa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f69380c9f10 tx=0x7f693809f040 comp rx=0 tx=0).stop 2023-07-21T22:36:38.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.846+0000 7f69539a2b80 1 -- 172.21.15.107:0/1282554255 shutdown_connections 2023-07-21T22:36:38.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.846+0000 7f69539a2b80 1 --2- 172.21.15.107:0/1282554255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881f1a0 0x5575a8824fa0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.846+0000 7f69539a2b80 1 --2- 172.21.15.107:0/1282554255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a8839bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.898 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.846+0000 7f69539a2b80 1 --2- 172.21.15.107:0/1282554255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881e830 0x5575a882e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.846+0000 7f69539a2b80 1 -- 172.21.15.107:0/1282554255 >> 172.21.15.107:0/1282554255 conn(0x5575a8812ef0 msgr2=0x5575a8825d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.846+0000 7f69539a2b80 1 -- 172.21.15.107:0/1282554255 shutdown_connections 2023-07-21T22:36:38.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.846+0000 7f69539a2b80 1 -- 172.21.15.107:0/1282554255 wait complete. 2023-07-21T22:36:38.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.846+0000 7f69539a2b80 1 -- start start 2023-07-21T22:36:38.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.846+0000 7f69539a2b80 1 Processor -- start 2023-07-21T22:36:38.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.846+0000 7f69539a2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a882a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69539a2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881e830 0x5575a882a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69539a2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881f1a0 0x5575a882d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69539a2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a882dc40 con 0x5575a881da10 2023-07-21T22:36:38.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69539a2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a882ddb0 con 0x5575a881f1a0 2023-07-21T22:36:38.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69539a2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575a882df70 con 0x5575a881e830 2023-07-21T22:36:38.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69539a2b80 10 client.0 _ll_register_callbacks cb 0x7f693c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:38.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69455b9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a882a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69455b9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a882a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51174/0 (socket says 172.21.15.107:51174) 2023-07-21T22:36:38.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69455b9700 1 -- 172.21.15.107:0/1750349160 learned_addr learned my addr 172.21.15.107:0/1750349160 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:38.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f6944db8700 1 --2- 172.21.15.107:0/1750349160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881e830 0x5575a882a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f6945dba700 1 --2- 172.21.15.107:0/1750349160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881f1a0 0x5575a882d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69455b9700 1 -- 172.21.15.107:0/1750349160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881e830 msgr2=0x5575a882a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69455b9700 1 --2- 172.21.15.107:0/1750349160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881e830 0x5575a882a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69455b9700 1 -- 172.21.15.107:0/1750349160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881f1a0 msgr2=0x5575a882d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:38.901 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69455b9700 1 --2- 172.21.15.107:0/1750349160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881f1a0 0x5575a882d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69455b9700 1 -- 172.21.15.107:0/1750349160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694019fb90 con 0x5575a881da10 2023-07-21T22:36:38.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69455b9700 1 --2- 172.21.15.107:0/1750349160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a882a1f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6934009990 tx=0x7f6934009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee459a687e429ae server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:38.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f6928ff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:38.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f6928ff9700 1 -- 172.21.15.107:0/1750349160 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693402b040 con 0x5575a881da10 2023-07-21T22:36:38.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f6928ff9700 1 -- 172.21.15.107:0/1750349160 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f693400fe60 con 0x5575a881da10 2023-07-21T22:36:38.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69539a2b80 1 -- 172.21.15.107:0/1750349160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f693401e040 con 0x5575a881da10 2023-07-21T22:36:38.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f6928ff9700 1 -- 172.21.15.107:0/1750349160 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6934024410 con 0x5575a881da10 2023-07-21T22:36:38.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69539a2b80 10 client.16608 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:38.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.847+0000 7f69539a2b80 1 -- 172.21.15.107:0/1750349160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5575a8585c60 con 0x5575a881da10 2023-07-21T22:36:38.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.848+0000 7f6928ff9700 1 -- 172.21.15.107:0/1750349160 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f693404b020 con 0x5575a881da10 2023-07-21T22:36:38.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.848+0000 7f6928ff9700 1 -- 172.21.15.107:0/1750349160 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f693403a3f0 con 0x5575a881da10 2023-07-21T22:36:38.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.848+0000 7f69539a2b80 10 client.16608 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:38.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.848+0000 7f69539a2b80 10 client.16608 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-21T22:36:38.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.848+0000 7f69539a2b80 20 client.16608 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:38.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.848+0000 7f69539a2b80 1 --2- 172.21.15.107:0/1750349160 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5575a8832e40 0x5575a8889760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:38.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.848+0000 7f69539a2b80 4 client.16608 mds_command: new command op to 24455 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:36:38.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.848+0000 7f69539a2b80 1 -- 172.21.15.107:0/1750349160 --> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5575a872a050 con 0x5575a8832e40 2023-07-21T22:36:38.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.848+0000 7f6944db8700 1 --2- 172.21.15.107:0/1750349160 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5575a8832e40 0x5575a8889760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:38.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.848+0000 7f6928ff9700 10 client.16608 ms_handle_connect on v2:172.21.15.132:6834/1022724305 2023-07-21T22:36:38.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.848+0000 7f6944db8700 1 --2- 172.21.15.107:0/1750349160 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5575a8832e40 0x5575a8889760 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ced5ac972120c21 server_cookie=b9ca33bdaceffab0 in_seq=0 out_seq=0 2023-07-21T22:36:38.904 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.848+0000 7f6928ff9700 1 -- 172.21.15.107:0/1750349160 <== mds.3 v2:172.21.15.132:6834/1022724305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5575a872a050 con 0x5575a8832e40 2023-07-21T22:36:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f6928ff9700 10 client.16608 handle_command_reply: tid=0 2023-07-21T22:36:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f69539a2b80 1 client.16608 shutdown 2023-07-21T22:36:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f69539a2b80 1 -- 172.21.15.107:0/1750349160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 msgr2=0x5575a882a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f69539a2b80 1 --2- 172.21.15.107:0/1750349160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a882a1f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6934009990 tx=0x7f6934009310 comp rx=0 tx=0).stop 2023-07-21T22:36:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f69539a2b80 1 -- 172.21.15.107:0/1750349160 shutdown_connections 2023-07-21T22:36:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f69539a2b80 1 --2- 172.21.15.107:0/1750349160 >> [v2:172.21.15.132:6834/1022724305,v1:172.21.15.132:6835/1022724305] conn(0x5575a8832e40 0x5575a8889760 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.905 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f69539a2b80 1 --2- 172.21.15.107:0/1750349160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5575a881f1a0 0x5575a882d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f69539a2b80 1 --2- 172.21.15.107:0/1750349160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5575a881da10 0x5575a882a1f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f69539a2b80 1 --2- 172.21.15.107:0/1750349160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5575a881e830 0x5575a882a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f6928ff9700 0 client.16608 ms_handle_reset on v2:172.21.15.132:6834/1022724305 2023-07-21T22:36:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f69539a2b80 1 -- 172.21.15.107:0/1750349160 >> 172.21.15.107:0/1750349160 conn(0x5575a8812ef0 msgr2=0x5575a8839bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f69539a2b80 1 -- 172.21.15.107:0/1750349160 shutdown_connections 2023-07-21T22:36:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f69539a2b80 1 -- 172.21.15.107:0/1750349160 wait complete. 2023-07-21T22:36:38.906 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f69539a2b80 20 client.16608 trim_cache size 0 max 16384 2023-07-21T22:36:38.907 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:36:38.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f694781d700 1 -- 172.21.15.107:0/3182576229 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f692c0a11d0 msgr2=0x7f692c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f694781d700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f692c0a11d0 0x7f692c0a3680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f693400b2e0 tx=0x7f6934005fb0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f694781d700 1 -- 172.21.15.107:0/3182576229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6940100640 msgr2=0x7f69401018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:38.908 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f694781d700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6940100640 0x7f69401018c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f693800ef30 tx=0x7f693800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f694781d700 1 -- 172.21.15.107:0/3182576229 shutdown_connections 2023-07-21T22:36:38.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f694781d700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6940100640 0x7f69401018c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f694781d700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69400fea00 0x7f6940102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f694781d700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f692c0a11d0 0x7f692c0a3680 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f694781d700 1 --2- 172.21.15.107:0/3182576229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69400ffc90 0x7f6940101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:38.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.849+0000 7f694781d700 1 -- 172.21.15.107:0/3182576229 >> 172.21.15.107:0/3182576229 conn(0x7f69400fa020 msgr2=0x7f69400fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:38.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.850+0000 7f694781d700 1 -- 172.21.15.107:0/3182576229 shutdown_connections 2023-07-21T22:36:38.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:38.850+0000 7f694781d700 1 -- 172.21.15.107:0/3182576229 wait complete. 2023-07-21T22:36:38.910 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:36:39.068 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.023+0000 7f623292d700 1 -- 172.21.15.107:0/4256998397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c10a760 msgr2=0x7f622c10cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.023+0000 7f623292d700 1 --2- 172.21.15.107:0/4256998397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c10a760 0x7f622c10cb40 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f62200098f0 tx=0x7f6220009c00 comp rx=0 tx=0).stop 2023-07-21T22:36:39.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.023+0000 7f623292d700 1 -- 172.21.15.107:0/4256998397 shutdown_connections 2023-07-21T22:36:39.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.023+0000 7f623292d700 1 --2- 172.21.15.107:0/4256998397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622c10d080 0x7f622c10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.023+0000 7f623292d700 1 --2- 172.21.15.107:0/4256998397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c071310 0x7f622c071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.023+0000 7f623292d700 1 --2- 172.21.15.107:0/4256998397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c10a760 0x7f622c10cb40 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.069 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.023+0000 7f623292d700 1 -- 172.21.15.107:0/4256998397 >> 172.21.15.107:0/4256998397 conn(0x7f622c06ca90 msgr2=0x7f622c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.023+0000 7f623292d700 1 -- 172.21.15.107:0/4256998397 shutdown_connections 2023-07-21T22:36:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.023+0000 7f623292d700 1 -- 172.21.15.107:0/4256998397 wait complete. 2023-07-21T22:36:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.024+0000 7f623292d700 1 Processor -- start 2023-07-21T22:36:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.024+0000 7f623292d700 1 -- start start 2023-07-21T22:36:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.024+0000 7f623292d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c071310 0x7f622c117350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.024+0000 7f623292d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622c10a760 0x7f622c117890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.024+0000 7f623292d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c10d080 0x7f622c11e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.024+0000 7f623292d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c117fe0 con 0x7f622c071310 2023-07-21T22:36:39.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.024+0000 7f623292d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1b4d20 con 0x7f622c10d080 2023-07-21T22:36:39.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.024+0000 7f623292d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c1b4ee0 con 0x7f622c10a760 2023-07-21T22:36:39.072 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.025+0000 7f622bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c071310 0x7f622c117350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.025+0000 7f622bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c071310 0x7f622c117350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51190/0 (socket says 172.21.15.107:51190) 2023-07-21T22:36:39.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.025+0000 7f622bfff700 1 -- 172.21.15.107:0/2752346274 learned_addr learned my addr 172.21.15.107:0/2752346274 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.025+0000 7f622b7fe700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622c10a760 0x7f622c117890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.025+0000 7f6230eca700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c10d080 0x7f622c11e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.025+0000 7f622bfff700 1 -- 172.21.15.107:0/2752346274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622c10a760 msgr2=0x7f622c117890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.025+0000 7f622bfff700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622c10a760 0x7f622c117890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.073 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.025+0000 7f622bfff700 1 -- 172.21.15.107:0/2752346274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c10d080 msgr2=0x7f622c11e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.025+0000 7f622bfff700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c10d080 0x7f622c11e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.025+0000 7f622bfff700 1 -- 172.21.15.107:0/2752346274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6220009520 con 0x7f622c071310 2023-07-21T22:36:39.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.025+0000 7f6230eca700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c10d080 0x7f622c11e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:36:39.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.025+0000 7f622bfff700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c071310 0x7f622c117350 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f621c00e9c0 tx=0x7f621c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.026+0000 7f62297fa700 1 -- 172.21.15.107:0/2752346274 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c00cbd0 con 0x7f622c071310 2023-07-21T22:36:39.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.026+0000 7f623292d700 1 -- 172.21.15.107:0/2752346274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c1b51c0 con 0x7f622c071310 2023-07-21T22:36:39.074 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.026+0000 7f62297fa700 1 -- 172.21.15.107:0/2752346274 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f621c00eeb0 con 0x7f622c071310 2023-07-21T22:36:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.026+0000 7f62297fa700 1 -- 172.21.15.107:0/2752346274 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c005580 con 0x7f622c071310 2023-07-21T22:36:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.027+0000 7f623292d700 1 -- 172.21.15.107:0/2752346274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c1b56d0 con 0x7f622c071310 2023-07-21T22:36:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.029+0000 7f62297fa700 1 -- 172.21.15.107:0/2752346274 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f621c004750 con 0x7f622c071310 2023-07-21T22:36:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.029+0000 7f62297fa700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62140a1150 0x7f62140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.030+0000 7f62297fa700 1 -- 172.21.15.107:0/2752346274 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f621c014070 con 0x7f622c071310 2023-07-21T22:36:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.030+0000 7f622b7fe700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62140a1150 0x7f62140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.031+0000 7f622b7fe700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62140a1150 0x7f62140a3600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f622000fa40 tx=0x7f6220019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.031+0000 7f623eab2b80 1 Processor -- start 2023-07-21T22:36:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.031+0000 7f623eab2b80 1 -- start start 2023-07-21T22:36:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f623eab2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e9a830 0x563445e9ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f623eab2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9b1a0 0x563445e99430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f623eab2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e99a10 0x563445e99e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f623eab2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563445d9ecb0 con 0x563445e99a10 2023-07-21T22:36:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f623eab2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563445df1a40 con 0x563445e9b1a0 2023-07-21T22:36:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f623eab2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563445deaed0 con 0x563445e9a830 2023-07-21T22:36:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f622bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e9a830 0x563445e9ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f622bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e9a830 0x563445e9ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39754/0 (socket says 172.21.15.107:39754) 2023-07-21T22:36:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f622bfff700 1 -- 172.21.15.107:0/1944343267 learned_addr learned my addr 172.21.15.107:0/1944343267 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f622b7fe700 1 --2- 172.21.15.107:0/1944343267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9b1a0 0x563445e99430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f6230eca700 1 --2- 172.21.15.107:0/1944343267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e99a10 0x563445e99e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f622bfff700 1 -- 172.21.15.107:0/1944343267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9b1a0 msgr2=0x563445e99430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f622bfff700 1 --2- 172.21.15.107:0/1944343267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9b1a0 0x563445e99430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f622bfff700 1 -- 172.21.15.107:0/1944343267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e99a10 msgr2=0x563445e99e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f622bfff700 1 --2- 172.21.15.107:0/1944343267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e99a10 0x563445e99e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f622bfff700 1 -- 172.21.15.107:0/1944343267 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563445c18de0 con 0x563445e9a830 2023-07-21T22:36:39.080 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.032+0000 7f622bfff700 1 --2- 172.21.15.107:0/1944343267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e9a830 0x563445e9ac40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f621c0a1e80 tx=0x7f621c009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76666c63a31400f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.033+0000 7f62127fc700 1 -- 172.21.15.107:0/1944343267 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c09df90 con 0x563445e9a830 2023-07-21T22:36:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.033+0000 7f62127fc700 1 -- 172.21.15.107:0/1944343267 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f621c0a23b0 con 0x563445e9a830 2023-07-21T22:36:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.034+0000 7f62127fc700 1 -- 172.21.15.107:0/1944343267 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0a1f20 con 0x563445e9a830 2023-07-21T22:36:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.034+0000 7f623eab2b80 1 -- 172.21.15.107:0/1944343267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e9a830 msgr2=0x563445e9ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.034+0000 7f623eab2b80 1 --2- 172.21.15.107:0/1944343267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e9a830 0x563445e9ac40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f621c0a1e80 tx=0x7f621c009550 comp rx=0 tx=0).stop 2023-07-21T22:36:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.034+0000 7f623eab2b80 1 -- 172.21.15.107:0/1944343267 shutdown_connections 2023-07-21T22:36:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.034+0000 7f623eab2b80 1 --2- 172.21.15.107:0/1944343267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e99a10 0x563445e99e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.034+0000 7f623eab2b80 1 --2- 172.21.15.107:0/1944343267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e9a830 0x563445e9ac40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.034+0000 7f623eab2b80 1 --2- 172.21.15.107:0/1944343267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9b1a0 0x563445e99430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.034+0000 7f623eab2b80 1 -- 172.21.15.107:0/1944343267 >> 172.21.15.107:0/1944343267 conn(0x563445e8eef0 msgr2=0x563445e94360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.034+0000 7f623eab2b80 1 -- 172.21.15.107:0/1944343267 shutdown_connections 2023-07-21T22:36:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.034+0000 7f623eab2b80 1 -- 172.21.15.107:0/1944343267 wait complete. 2023-07-21T22:36:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.035+0000 7f623eab2b80 1 -- start start 2023-07-21T22:36:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.035+0000 7f623eab2b80 1 Processor -- start 2023-07-21T22:36:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.036+0000 7f623eab2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e99a10 0x563445ea3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.036+0000 7f623eab2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9a830 0x563445ea36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.036+0000 7f623eab2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9b1a0 0x563445eaa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.036+0000 7f623eab2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563445d8aac0 con 0x563445e9b1a0 2023-07-21T22:36:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.036+0000 7f623eab2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563445d8a7c0 con 0x563445e9a830 2023-07-21T22:36:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.036+0000 7f623eab2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563445c6e920 con 0x563445e99a10 2023-07-21T22:36:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.036+0000 7f623eab2b80 10 client.0 _ll_register_callbacks cb 0x563445d88760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.037+0000 7f622bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e99a10 0x563445ea3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.037+0000 7f622bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e99a10 0x563445ea3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39762/0 (socket says 172.21.15.107:39762) 2023-07-21T22:36:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.037+0000 7f622b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9a830 0x563445ea36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.037+0000 7f622bfff700 1 -- 172.21.15.107:0/2449857949 learned_addr learned my addr 172.21.15.107:0/2449857949 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.037+0000 7f622bfff700 1 -- 172.21.15.107:0/2449857949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9a830 msgr2=0x563445ea36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.037+0000 7f622bfff700 1 --2- 172.21.15.107:0/2449857949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9a830 0x563445ea36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.037+0000 7f622bfff700 1 -- 172.21.15.107:0/2449857949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9b1a0 msgr2=0x563445eaa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.037+0000 7f622bfff700 1 --2- 172.21.15.107:0/2449857949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9b1a0 0x563445eaa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.037+0000 7f622bfff700 1 -- 172.21.15.107:0/2449857949 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c1b56d0 con 0x563445e99a10 2023-07-21T22:36:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.037+0000 7f622bfff700 1 --2- 172.21.15.107:0/2449857949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e99a10 0x563445ea3190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f621c0969d0 tx=0x7f621c09df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8322aac0a586f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.037+0000 7f620b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:36:39.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f620b7fe700 1 -- 172.21.15.107:0/2449857949 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0ca270 con 0x563445e99a10 2023-07-21T22:36:39.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f620b7fe700 1 -- 172.21.15.107:0/2449857949 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f621c0ca3d0 con 0x563445e99a10 2023-07-21T22:36:39.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f623eab2b80 1 -- 172.21.15.107:0/2449857949 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563445d444f0 con 0x563445e99a10 2023-07-21T22:36:39.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f620b7fe700 1 -- 172.21.15.107:0/2449857949 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f621c0a1f20 con 0x563445e99a10 2023-07-21T22:36:39.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f623eab2b80 10 client.26230 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:39.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f623eab2b80 1 -- 172.21.15.107:0/2449857949 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x563445c01c60 con 0x563445e99a10 2023-07-21T22:36:39.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f620b7fe700 1 -- 172.21.15.107:0/2449857949 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f621c07c080 con 0x563445e99a10 2023-07-21T22:36:39.088 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f620b7fe700 1 -- 172.21.15.107:0/2449857949 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f621c095d50 con 0x563445e99a10 2023-07-21T22:36:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f623eab2b80 10 client.26230 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f623eab2b80 10 client.26230 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-07-21T22:36:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f623eab2b80 20 client.26230 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f623eab2b80 1 --2- 172.21.15.107:0/2449857949 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x563445eaebe0 0x563445eb0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f623eab2b80 4 client.26230 mds_command: new command op to 24466 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.038+0000 7f623eab2b80 1 -- 172.21.15.107:0/2449857949 --> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563445de3eb0 con 0x563445eaebe0 2023-07-21T22:36:39.089 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.039+0000 7f622b7fe700 1 --2- 172.21.15.107:0/2449857949 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x563445eaebe0 0x563445eb0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.039+0000 7f620b7fe700 10 client.26230 ms_handle_connect on v2:172.21.15.190:6834/2286851849 2023-07-21T22:36:39.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.039+0000 7f622b7fe700 1 --2- 172.21.15.107:0/2449857949 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x563445eaebe0 0x563445eb0ff0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb30ea602417ba3 server_cookie=bd3d7d3baadeb6a6 in_seq=0 out_seq=0 2023-07-21T22:36:39.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f620b7fe700 1 -- 172.21.15.107:0/2449857949 <== mds.2 v2:172.21.15.190:6834/2286851849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563445de3eb0 con 0x563445eaebe0 2023-07-21T22:36:39.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f620b7fe700 10 client.26230 handle_command_reply: tid=0 2023-07-21T22:36:39.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f623eab2b80 1 client.26230 shutdown 2023-07-21T22:36:39.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f623eab2b80 1 -- 172.21.15.107:0/2449857949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e99a10 msgr2=0x563445ea3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.090 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f623eab2b80 1 --2- 172.21.15.107:0/2449857949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e99a10 0x563445ea3190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f621c0969d0 tx=0x7f621c09df90 comp rx=0 tx=0).stop 2023-07-21T22:36:39.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f623eab2b80 1 -- 172.21.15.107:0/2449857949 shutdown_connections 2023-07-21T22:36:39.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f623eab2b80 1 --2- 172.21.15.107:0/2449857949 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x563445eaebe0 0x563445eb0ff0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f623eab2b80 1 --2- 172.21.15.107:0/2449857949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9b1a0 0x563445eaa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f620b7fe700 0 client.26230 ms_handle_reset on v2:172.21.15.190:6834/2286851849 2023-07-21T22:36:39.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f623eab2b80 1 --2- 172.21.15.107:0/2449857949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e99a10 0x563445ea3190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f623eab2b80 1 --2- 172.21.15.107:0/2449857949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9a830 0x563445ea36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.091 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f623eab2b80 1 -- 172.21.15.107:0/2449857949 >> 172.21.15.107:0/2449857949 conn(0x563445e8eef0 msgr2=0x563445e93a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f623eab2b80 1 -- 172.21.15.107:0/2449857949 shutdown_connections 2023-07-21T22:36:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f623eab2b80 1 -- 172.21.15.107:0/2449857949 wait complete. 2023-07-21T22:36:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.040+0000 7f623eab2b80 20 client.26230 trim_cache size 0 max 16384 2023-07-21T22:36:39.100 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.054+0000 7f623eab2b80 1 Processor -- start 2023-07-21T22:36:39.100 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f623eab2b80 1 -- start start 2023-07-21T22:36:39.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f623eab2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e99a10 0x563445eab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f623eab2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9a830 0x563445eaa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f623eab2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9b1a0 0x563445e99de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f623eab2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563445e95430 con 0x563445e9b1a0 2023-07-21T22:36:39.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f623eab2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563445eae900 con 0x563445e9a830 2023-07-21T22:36:39.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f623eab2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563445ea1b10 con 0x563445e99a10 2023-07-21T22:36:39.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f6230eca700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9b1a0 0x563445e99de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f6230eca700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9b1a0 0x563445e99de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51214/0 (socket says 172.21.15.107:51214) 2023-07-21T22:36:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f6230eca700 1 -- 172.21.15.107:0/1330134870 learned_addr learned my addr 172.21.15.107:0/1330134870 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f622b7fe700 1 --2- 172.21.15.107:0/1330134870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9a830 0x563445eaa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f622bfff700 1 --2- 172.21.15.107:0/1330134870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e99a10 0x563445eab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f6230eca700 1 -- 172.21.15.107:0/1330134870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e99a10 msgr2=0x563445eab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f6230eca700 1 --2- 172.21.15.107:0/1330134870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e99a10 0x563445eab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f6230eca700 1 -- 172.21.15.107:0/1330134870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9a830 msgr2=0x563445eaa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f6230eca700 1 --2- 172.21.15.107:0/1330134870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9a830 0x563445eaa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f6230eca700 1 -- 172.21.15.107:0/1330134870 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563445c3f250 con 0x563445e9b1a0 2023-07-21T22:36:39.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f6230eca700 1 --2- 172.21.15.107:0/1330134870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9b1a0 0x563445e99de0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6224009400 tx=0x7f6224009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc592f91b52296a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.055+0000 7f6212ffd700 1 -- 172.21.15.107:0/1330134870 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622400e030 con 0x563445e9b1a0 2023-07-21T22:36:39.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.056+0000 7f6212ffd700 1 -- 172.21.15.107:0/1330134870 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f622400f040 con 0x563445e9b1a0 2023-07-21T22:36:39.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.056+0000 7f6212ffd700 1 -- 172.21.15.107:0/1330134870 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224004840 con 0x563445e9b1a0 2023-07-21T22:36:39.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.056+0000 7f623eab2b80 1 -- 172.21.15.107:0/1330134870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9b1a0 msgr2=0x563445e99de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.056+0000 7f623eab2b80 1 --2- 172.21.15.107:0/1330134870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9b1a0 0x563445e99de0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6224009400 tx=0x7f6224009920 comp rx=0 tx=0).stop 2023-07-21T22:36:39.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.056+0000 7f623eab2b80 1 -- 172.21.15.107:0/1330134870 shutdown_connections 2023-07-21T22:36:39.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.056+0000 7f623eab2b80 1 --2- 172.21.15.107:0/1330134870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9b1a0 0x563445e99de0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.107 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.056+0000 7f623eab2b80 1 --2- 172.21.15.107:0/1330134870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e99a10 0x563445eab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.056+0000 7f623eab2b80 1 --2- 172.21.15.107:0/1330134870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e9a830 0x563445eaa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.056+0000 7f623eab2b80 1 -- 172.21.15.107:0/1330134870 >> 172.21.15.107:0/1330134870 conn(0x563445e8eef0 msgr2=0x563445ea1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.056+0000 7f623eab2b80 1 -- 172.21.15.107:0/1330134870 shutdown_connections 2023-07-21T22:36:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.056+0000 7f623eab2b80 1 -- 172.21.15.107:0/1330134870 wait complete. 2023-07-21T22:36:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f623eab2b80 1 -- start start 2023-07-21T22:36:39.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f623eab2b80 1 Processor -- start 2023-07-21T22:36:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f623eab2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e99a10 0x563445eaa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f623eab2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9a830 0x563445eaa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f623eab2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e9b1a0 0x563445ea8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f623eab2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563445ea8ee0 con 0x563445e9a830 2023-07-21T22:36:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f623eab2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563445ea9050 con 0x563445e99a10 2023-07-21T22:36:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f623eab2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563445ea9210 con 0x563445e9b1a0 2023-07-21T22:36:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f623eab2b80 10 client.0 _ll_register_callbacks cb 0x7f621c099b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:39.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f622bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e99a10 0x563445eaa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f622b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9a830 0x563445eaa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f6230eca700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e9b1a0 0x563445ea8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f622bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e99a10 0x563445eaa100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:42044/0 (socket says 172.21.15.107:42044) 2023-07-21T22:36:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f622bfff700 1 -- 172.21.15.107:0/3168755377 learned_addr learned my addr 172.21.15.107:0/3168755377 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f622bfff700 1 -- 172.21.15.107:0/3168755377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e9b1a0 msgr2=0x563445ea8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f622bfff700 1 --2- 172.21.15.107:0/3168755377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e9b1a0 0x563445ea8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.110 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f622bfff700 1 -- 172.21.15.107:0/3168755377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9a830 msgr2=0x563445eaa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f622bfff700 1 --2- 172.21.15.107:0/3168755377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9a830 0x563445eaa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f622bfff700 1 -- 172.21.15.107:0/3168755377 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62240090b0 con 0x563445e99a10 2023-07-21T22:36:39.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f622b7fe700 1 --2- 172.21.15.107:0/3168755377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9a830 0x563445eaa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:39.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.057+0000 7f6230eca700 1 --2- 172.21.15.107:0/3168755377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e9b1a0 0x563445ea8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:39.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f622bfff700 1 --2- 172.21.15.107:0/3168755377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e99a10 0x563445eaa100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c5b0 tx=0x7f621c021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56176a305e1ce26d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f620b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:36:39.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f620b7fe700 1 -- 172.21.15.107:0/3168755377 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c098f10 con 0x563445e99a10 2023-07-21T22:36:39.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f623eab2b80 1 -- 172.21.15.107:0/3168755377 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f621c0a23b0 con 0x563445e99a10 2023-07-21T22:36:39.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f620b7fe700 1 -- 172.21.15.107:0/3168755377 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f621c0ca270 con 0x563445e99a10 2023-07-21T22:36:39.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f620b7fe700 1 -- 172.21.15.107:0/3168755377 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f621c0ca430 con 0x563445e99a10 2023-07-21T22:36:39.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f623eab2b80 10 client.26165 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:39.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f623eab2b80 1 -- 172.21.15.107:0/3168755377 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x563445c172b0 con 0x563445e99a10 2023-07-21T22:36:39.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f620b7fe700 1 -- 172.21.15.107:0/3168755377 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f621c0a23b0 con 0x563445e99a10 2023-07-21T22:36:39.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f620b7fe700 1 -- 172.21.15.107:0/3168755377 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f621c094b70 con 0x563445e99a10 2023-07-21T22:36:39.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f623eab2b80 10 client.26165 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:39.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f623eab2b80 10 client.26165 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2023-07-21T22:36:39.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f623eab2b80 20 client.26165 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:39.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f623eab2b80 1 --2- 172.21.15.107:0/3168755377 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x563445f03a20 0x563445f05e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f623eab2b80 4 client.26165 mds_command: new command op to 24466 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:36:39.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.058+0000 7f623eab2b80 1 -- 172.21.15.107:0/3168755377 --> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563445da6050 con 0x563445f03a20 2023-07-21T22:36:39.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.059+0000 7f622b7fe700 1 --2- 172.21.15.107:0/3168755377 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x563445f03a20 0x563445f05e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.059+0000 7f620b7fe700 10 client.26165 ms_handle_connect on v2:172.21.15.190:6834/2286851849 2023-07-21T22:36:39.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.059+0000 7f622b7fe700 1 --2- 172.21.15.107:0/3168755377 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x563445f03a20 0x563445f05e30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a84f2b2ba9fc76e8 server_cookie=9103d6a0e1f2cdd5 in_seq=0 out_seq=0 2023-07-21T22:36:39.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.060+0000 7f620b7fe700 1 -- 172.21.15.107:0/3168755377 <== mds.2 v2:172.21.15.190:6834/2286851849 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+47974 (crc 0 0 0) 0x563445da6050 con 0x563445f03a20 2023-07-21T22:36:39.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.060+0000 7f620b7fe700 10 client.26165 handle_command_reply: tid=0 2023-07-21T22:36:39.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.060+0000 7f623eab2b80 1 client.26165 shutdown 2023-07-21T22:36:39.114 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623eab2b80 1 -- 172.21.15.107:0/3168755377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e99a10 msgr2=0x563445eaa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623eab2b80 1 --2- 172.21.15.107:0/3168755377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e99a10 0x563445eaa100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c5b0 tx=0x7f621c021000 comp rx=0 tx=0).stop 2023-07-21T22:36:39.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623eab2b80 1 -- 172.21.15.107:0/3168755377 shutdown_connections 2023-07-21T22:36:39.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623eab2b80 1 --2- 172.21.15.107:0/3168755377 >> [v2:172.21.15.190:6834/2286851849,v1:172.21.15.190:6835/2286851849] conn(0x563445f03a20 0x563445f05e30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623eab2b80 1 --2- 172.21.15.107:0/3168755377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563445e9b1a0 0x563445ea8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f620b7fe700 0 client.26165 ms_handle_reset on v2:172.21.15.190:6834/2286851849 2023-07-21T22:36:39.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623eab2b80 1 --2- 172.21.15.107:0/3168755377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563445e99a10 0x563445eaa100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623eab2b80 1 --2- 172.21.15.107:0/3168755377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563445e9a830 0x563445eaa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.115 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623eab2b80 1 -- 172.21.15.107:0/3168755377 >> 172.21.15.107:0/3168755377 conn(0x563445e8eef0 msgr2=0x563445e99de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623eab2b80 1 -- 172.21.15.107:0/3168755377 shutdown_connections 2023-07-21T22:36:39.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623eab2b80 1 -- 172.21.15.107:0/3168755377 wait complete. 2023-07-21T22:36:39.116 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623eab2b80 20 client.26165 trim_cache size 0 max 16384 2023-07-21T22:36:39.117 INFO:teuthology.orchestra.run.smithi107.stdout:[ 2023-07-21T22:36:39.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.117 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.117 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 21158943, 2023-07-21T22:36:39.117 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634863, 2023-07-21T22:36:39.117 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-17.xml" 2023-07-21T22:36:39.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.118 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.118 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.118 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 25974584, 2023-07-21T22:36:39.118 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511629283, 2023-07-21T22:36:39.118 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-35.xml" 2023-07-21T22:36:39.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.119 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.119 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 31908056, 2023-07-21T22:36:39.119 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515446, 2023-07-21T22:36:39.119 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-28.jpg" 2023-07-21T22:36:39.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.119 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.120 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.120 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 61627180, 2023-07-21T22:36:39.120 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634799, 2023-07-21T22:36:39.120 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-72.xml" 2023-07-21T22:36:39.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.120 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.121 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 138774222, 2023-07-21T22:36:39.121 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515730, 2023-07-21T22:36:39.121 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-5.jpg" 2023-07-21T22:36:39.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.121 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.121 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.121 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 141611757, 2023-07-21T22:36:39.121 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634877, 2023-07-21T22:36:39.122 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-3.xml" 2023-07-21T22:36:39.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.122 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.122 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.122 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 160011544, 2023-07-21T22:36:39.122 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634879, 2023-07-21T22:36:39.122 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-1.xml" 2023-07-21T22:36:39.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.123 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.123 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 168016027, 2023-07-21T22:36:39.123 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634876, 2023-07-21T22:36:39.123 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-4.xml" 2023-07-21T22:36:39.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.124 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.124 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 174210707, 2023-07-21T22:36:39.124 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634835, 2023-07-21T22:36:39.124 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-41.xml" 2023-07-21T22:36:39.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.124 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.125 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.125 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 194437862, 2023-07-21T22:36:39.125 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634811, 2023-07-21T22:36:39.125 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-63.xml" 2023-07-21T22:36:39.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.125 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.125 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.126 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 211376050, 2023-07-21T22:36:39.126 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515309, 2023-07-21T22:36:39.126 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-24.xml" 2023-07-21T22:36:39.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.126 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.126 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.126 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 222531143, 2023-07-21T22:36:39.127 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634847, 2023-07-21T22:36:39.127 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-30.xml" 2023-07-21T22:36:39.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.127 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.127 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.127 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 243221646, 2023-07-21T22:36:39.127 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515713, 2023-07-21T22:36:39.127 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-6.jpg" 2023-07-21T22:36:39.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.128 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.128 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.128 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 267834194, 2023-07-21T22:36:39.128 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515743, 2023-07-21T22:36:39.128 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-4.jpg" 2023-07-21T22:36:39.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.129 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.129 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 295341970, 2023-07-21T22:36:39.129 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634812, 2023-07-21T22:36:39.129 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-62.xml" 2023-07-21T22:36:39.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.129 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.130 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.130 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 302570097, 2023-07-21T22:36:39.130 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634875, 2023-07-21T22:36:39.130 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-5.xml" 2023-07-21T22:36:39.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.130 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.130 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.131 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 323006527, 2023-07-21T22:36:39.131 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634805, 2023-07-21T22:36:39.131 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-67.xml" 2023-07-21T22:36:39.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.131 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.131 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.131 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 324385409, 2023-07-21T22:36:39.132 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634845, 2023-07-21T22:36:39.132 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-32.xml" 2023-07-21T22:36:39.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.132 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.132 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.132 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 344649510, 2023-07-21T22:36:39.132 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634800, 2023-07-21T22:36:39.133 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-71.xml" 2023-07-21T22:36:39.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.133 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.133 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 347466352, 2023-07-21T22:36:39.133 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634871, 2023-07-21T22:36:39.133 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-9.xml" 2023-07-21T22:36:39.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.134 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.134 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 361854996, 2023-07-21T22:36:39.134 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515171, 2023-07-21T22:36:39.134 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-48.xml" 2023-07-21T22:36:39.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.135 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.135 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 367296624, 2023-07-21T22:36:39.135 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515548, 2023-07-21T22:36:39.135 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-20.jpg" 2023-07-21T22:36:39.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.135 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.136 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.136 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 375341505, 2023-07-21T22:36:39.136 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634815, 2023-07-21T22:36:39.136 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-59.xml" 2023-07-21T22:36:39.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.136 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.136 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.137 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 380832038, 2023-07-21T22:36:39.137 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515124, 2023-07-21T22:36:39.137 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-56.xml" 2023-07-21T22:36:39.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.137 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.137 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 395763333, 2023-07-21T22:36:39.137 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634868, 2023-07-21T22:36:39.138 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-12.xml" 2023-07-21T22:36:39.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.138 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.138 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.138 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 428677332, 2023-07-21T22:36:39.138 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515132, 2023-07-21T22:36:39.138 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-54.xml" 2023-07-21T22:36:39.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.139 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.139 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.139 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 443836555, 2023-07-21T22:36:39.139 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634796, 2023-07-21T22:36:39.139 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-75.xml" 2023-07-21T22:36:39.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.140 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.140 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 461659812, 2023-07-21T22:36:39.140 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634843, 2023-07-21T22:36:39.140 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-34.xml" 2023-07-21T22:36:39.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.141 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.141 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 480016157, 2023-07-21T22:36:39.141 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634793, 2023-07-21T22:36:39.141 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-78.xml" 2023-07-21T22:36:39.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.141 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.141 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.142 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 496024734, 2023-07-21T22:36:39.142 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634849, 2023-07-21T22:36:39.142 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-28.xml" 2023-07-21T22:36:39.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.142 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.142 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.142 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 568675816, 2023-07-21T22:36:39.143 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515377, 2023-07-21T22:36:39.143 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-11.xml" 2023-07-21T22:36:39.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.143 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.143 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 578955250, 2023-07-21T22:36:39.143 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634866, 2023-07-21T22:36:39.143 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-14.xml" 2023-07-21T22:36:39.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.144 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.144 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.144 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 585523635, 2023-07-21T22:36:39.144 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515391, 2023-07-21T22:36:39.144 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-8.xml" 2023-07-21T22:36:39.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.145 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.145 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 588094203, 2023-07-21T22:36:39.145 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515209, 2023-07-21T22:36:39.145 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-42.xml" 2023-07-21T22:36:39.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.146 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.146 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 613016252, 2023-07-21T22:36:39.146 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634299, 2023-07-21T22:36:39.146 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-78.xml" 2023-07-21T22:36:39.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.146 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.147 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 623542076, 2023-07-21T22:36:39.147 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634857, 2023-07-21T22:36:39.147 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-21.xml" 2023-07-21T22:36:39.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.147 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.147 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.147 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 657496010, 2023-07-21T22:36:39.148 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515092, 2023-07-21T22:36:39.148 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-66.xml" 2023-07-21T22:36:39.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.148 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.148 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.148 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 708926443, 2023-07-21T22:36:39.148 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515203, 2023-07-21T22:36:39.149 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-43.xml" 2023-07-21T22:36:39.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.149 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.149 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.149 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 715203108, 2023-07-21T22:36:39.149 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515300, 2023-07-21T22:36:39.149 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-26.xml" 2023-07-21T22:36:39.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.150 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.150 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.150 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 738762119, 2023-07-21T22:36:39.150 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634825, 2023-07-21T22:36:39.150 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-51.xml" 2023-07-21T22:36:39.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.151 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.151 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 740251588, 2023-07-21T22:36:39.151 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634850, 2023-07-21T22:36:39.151 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-27.xml" 2023-07-21T22:36:39.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.151 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.152 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.152 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 752934324, 2023-07-21T22:36:39.152 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511636425, 2023-07-21T22:36:39.152 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-46.jpg" 2023-07-21T22:36:39.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.152 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.152 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.153 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 765643061, 2023-07-21T22:36:39.153 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515612, 2023-07-21T22:36:39.153 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-14.jpg" 2023-07-21T22:36:39.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.153 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.153 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 795631500, 2023-07-21T22:36:39.154 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634870, 2023-07-21T22:36:39.154 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-10.xml" 2023-07-21T22:36:39.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.154 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.154 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.154 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 858908003, 2023-07-21T22:36:39.154 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515263, 2023-07-21T22:36:39.154 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-33.xml" 2023-07-21T22:36:39.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.155 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.155 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 876257260, 2023-07-21T22:36:39.155 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515328, 2023-07-21T22:36:39.155 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-20.xml" 2023-07-21T22:36:39.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.156 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.156 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 878022718, 2023-07-21T22:36:39.156 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515419, 2023-07-21T22:36:39.156 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-3.xml" 2023-07-21T22:36:39.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.157 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.157 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 907065582, 2023-07-21T22:36:39.157 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634861, 2023-07-21T22:36:39.157 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-18.xml" 2023-07-21T22:36:39.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.157 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.157 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.158 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 943144400, 2023-07-21T22:36:39.158 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515227, 2023-07-21T22:36:39.158 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-39.xml" 2023-07-21T22:36:39.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.158 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.158 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 964608954, 2023-07-21T22:36:39.159 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515559, 2023-07-21T22:36:39.159 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-19.jpg" 2023-07-21T22:36:39.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.159 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.159 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 976186631, 2023-07-21T22:36:39.159 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515268, 2023-07-21T22:36:39.160 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-32.xml" 2023-07-21T22:36:39.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.160 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.160 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.160 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 977969131, 2023-07-21T22:36:39.160 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515289, 2023-07-21T22:36:39.160 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-28.xml" 2023-07-21T22:36:39.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.161 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.161 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.161 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 985344231, 2023-07-21T22:36:39.161 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634798, 2023-07-21T22:36:39.161 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-73.xml" 2023-07-21T22:36:39.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.162 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.162 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 992645308, 2023-07-21T22:36:39.162 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634838, 2023-07-21T22:36:39.162 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-39.xml" 2023-07-21T22:36:39.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.162 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.162 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.163 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 999203861, 2023-07-21T22:36:39.163 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515150, 2023-07-21T22:36:39.163 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-51.xml" 2023-07-21T22:36:39.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.163 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.163 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1049285366, 2023-07-21T22:36:39.164 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515589, 2023-07-21T22:36:39.164 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-16.jpg" 2023-07-21T22:36:39.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.164 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.164 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.164 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1054292580, 2023-07-21T22:36:39.164 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634840, 2023-07-21T22:36:39.165 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-37.xml" 2023-07-21T22:36:39.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.165 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.165 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1087056602, 2023-07-21T22:36:39.165 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634818, 2023-07-21T22:36:39.165 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-57.xml" 2023-07-21T22:36:39.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.166 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.166 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1088393615, 2023-07-21T22:36:39.166 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511631589, 2023-07-21T22:36:39.166 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-41.xml" 2023-07-21T22:36:39.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.167 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.167 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1107808556, 2023-07-21T22:36:39.167 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634821, 2023-07-21T22:36:39.167 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-54.xml" 2023-07-21T22:36:39.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.167 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.168 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.168 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1119225795, 2023-07-21T22:36:39.168 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515765, 2023-07-21T22:36:39.168 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-1.jpg" 2023-07-21T22:36:39.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.168 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.168 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.169 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1131511747, 2023-07-21T22:36:39.169 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515096, 2023-07-21T22:36:39.169 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-65.xml" 2023-07-21T22:36:39.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.169 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.169 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1211283891, 2023-07-21T22:36:39.170 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515076, 2023-07-21T22:36:39.170 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-69.xml" 2023-07-21T22:36:39.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.170 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.170 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.170 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1235275062, 2023-07-21T22:36:39.170 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634865, 2023-07-21T22:36:39.171 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-15.xml" 2023-07-21T22:36:39.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.171 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.171 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.171 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1269446848, 2023-07-21T22:36:39.171 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634807, 2023-07-21T22:36:39.171 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-65.xml" 2023-07-21T22:36:39.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.172 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.172 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.172 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1272929557, 2023-07-21T22:36:39.172 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515127, 2023-07-21T22:36:39.172 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-55.xml" 2023-07-21T22:36:39.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.172 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.173 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.173 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1287213455, 2023-07-21T22:36:39.173 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515758, 2023-07-21T22:36:39.173 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-2.jpg" 2023-07-21T22:36:39.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.173 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.173 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.174 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1300335612, 2023-07-21T22:36:39.174 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634823, 2023-07-21T22:36:39.174 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-52.xml" 2023-07-21T22:36:39.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.174 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.174 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.174 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1317805414, 2023-07-21T22:36:39.175 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515237, 2023-07-21T22:36:39.175 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-37.xml" 2023-07-21T22:36:39.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.175 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.175 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.175 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1337400957, 2023-07-21T22:36:39.175 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634851, 2023-07-21T22:36:39.176 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-26.xml" 2023-07-21T22:36:39.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.176 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.176 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.176 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1359043961, 2023-07-21T22:36:39.176 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515370, 2023-07-21T22:36:39.176 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-12.xml" 2023-07-21T22:36:39.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.177 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.177 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.177 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1389591493, 2023-07-21T22:36:39.177 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515323, 2023-07-21T22:36:39.177 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-21.xml" 2023-07-21T22:36:39.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.178 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.178 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.178 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1410364704, 2023-07-21T22:36:39.178 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515117, 2023-07-21T22:36:39.178 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-59.xml" 2023-07-21T22:36:39.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.178 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1423255978, 2023-07-21T22:36:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515110, 2023-07-21T22:36:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-62.xml" 2023-07-21T22:36:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1453004066, 2023-07-21T22:36:39.180 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634853, 2023-07-21T22:36:39.180 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-25.xml" 2023-07-21T22:36:39.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.180 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.180 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.180 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1466241590, 2023-07-21T22:36:39.180 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515314, 2023-07-21T22:36:39.181 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-23.xml" 2023-07-21T22:36:39.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.181 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.181 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.181 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1471048035, 2023-07-21T22:36:39.181 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515498, 2023-07-21T22:36:39.181 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-72.xml" 2023-07-21T22:36:39.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.182 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.182 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.182 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1517563796, 2023-07-21T22:36:39.182 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515221, 2023-07-21T22:36:39.182 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-40.xml" 2023-07-21T22:36:39.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.183 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.183 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.183 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1521826208, 2023-07-21T22:36:39.183 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634855, 2023-07-21T22:36:39.183 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-23.xml" 2023-07-21T22:36:39.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.183 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.184 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.184 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1528736858, 2023-07-21T22:36:39.184 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515304, 2023-07-21T22:36:39.184 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-25.xml" 2023-07-21T22:36:39.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.184 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.184 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.185 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1561203631, 2023-07-21T22:36:39.185 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515276, 2023-07-21T22:36:39.185 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-31.xml" 2023-07-21T22:36:39.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.185 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.185 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.185 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1571307189, 2023-07-21T22:36:39.185 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634804, 2023-07-21T22:36:39.186 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-68.xml" 2023-07-21T22:36:39.186 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.186 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.186 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.186 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1608293541, 2023-07-21T22:36:39.186 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634794, 2023-07-21T22:36:39.186 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-77.xml" 2023-07-21T22:36:39.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.187 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.187 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.187 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1624357881, 2023-07-21T22:36:39.187 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634846, 2023-07-21T22:36:39.187 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-31.xml" 2023-07-21T22:36:39.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.188 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.188 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.188 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1655471853, 2023-07-21T22:36:39.188 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634819, 2023-07-21T22:36:39.188 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-56.xml" 2023-07-21T22:36:39.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.188 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.189 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.189 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1658642625, 2023-07-21T22:36:39.189 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511633961, 2023-07-21T22:36:39.189 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-63.xml" 2023-07-21T22:36:39.189 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.189 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.189 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.190 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1666272466, 2023-07-21T22:36:39.190 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634806, 2023-07-21T22:36:39.190 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-66.xml" 2023-07-21T22:36:39.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.190 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.190 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.190 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1668876062, 2023-07-21T22:36:39.191 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515535, 2023-07-21T22:36:39.191 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-21.jpg" 2023-07-21T22:36:39.191 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.191 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.191 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.191 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1677093771, 2023-07-21T22:36:39.191 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515155, 2023-07-21T22:36:39.191 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-50.xml" 2023-07-21T22:36:39.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.192 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.192 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.192 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1739378643, 2023-07-21T22:36:39.192 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634873, 2023-07-21T22:36:39.192 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-7.xml" 2023-07-21T22:36:39.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.193 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.193 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.193 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1760843571, 2023-07-21T22:36:39.193 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515699, 2023-07-21T22:36:39.193 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-7.jpg" 2023-07-21T22:36:39.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.194 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.194 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.194 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1829749078, 2023-07-21T22:36:39.194 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515395, 2023-07-21T22:36:39.194 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-7.xml" 2023-07-21T22:36:39.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.194 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.195 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.195 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1869085673, 2023-07-21T22:36:39.195 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515068, 2023-07-21T22:36:39.195 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-70.xml" 2023-07-21T22:36:39.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.195 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.195 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.195 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1885023847, 2023-07-21T22:36:39.196 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515191, 2023-07-21T22:36:39.196 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-45.xml" 2023-07-21T22:36:39.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.196 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.196 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.196 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1895457319, 2023-07-21T22:36:39.196 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511633167, 2023-07-21T22:36:39.197 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-83.xml" 2023-07-21T22:36:39.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.197 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.197 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.197 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1928418528, 2023-07-21T22:36:39.197 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511631018, 2023-07-21T22:36:39.197 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-58.xml" 2023-07-21T22:36:39.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.198 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.198 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.198 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 1957623582, 2023-07-21T22:36:39.198 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515293, 2023-07-21T22:36:39.198 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-27.xml" 2023-07-21T22:36:39.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.199 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.199 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.199 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2005603817, 2023-07-21T22:36:39.199 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634828, 2023-07-21T22:36:39.199 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-48.xml" 2023-07-21T22:36:39.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.199 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.200 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.200 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2051232131, 2023-07-21T22:36:39.200 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634858, 2023-07-21T22:36:39.200 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-20.xml" 2023-07-21T22:36:39.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.200 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.200 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.201 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2065195366, 2023-07-21T22:36:39.201 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511629124, 2023-07-21T22:36:39.201 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-66.jpg" 2023-07-21T22:36:39.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.201 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.201 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.201 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2120699241, 2023-07-21T22:36:39.202 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515385, 2023-07-21T22:36:39.202 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-9.xml" 2023-07-21T22:36:39.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.202 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.202 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.202 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2156555037, 2023-07-21T22:36:39.202 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515182, 2023-07-21T22:36:39.202 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-46.xml" 2023-07-21T22:36:39.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.203 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.203 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.203 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2165579709, 2023-07-21T22:36:39.203 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515567, 2023-07-21T22:36:39.203 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-18.jpg" 2023-07-21T22:36:39.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.204 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.204 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.204 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2208735844, 2023-07-21T22:36:39.204 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634813, 2023-07-21T22:36:39.204 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-61.xml" 2023-07-21T22:36:39.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.204 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.205 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.205 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2214003741, 2023-07-21T22:36:39.205 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515599, 2023-07-21T22:36:39.205 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-15.jpg" 2023-07-21T22:36:39.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.205 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.206 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2239402363, 2023-07-21T22:36:39.206 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634797, 2023-07-21T22:36:39.206 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-74.xml" 2023-07-21T22:36:39.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.206 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.206 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.206 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2245802303, 2023-07-21T22:36:39.207 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634837, 2023-07-21T22:36:39.207 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-40.xml" 2023-07-21T22:36:39.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.207 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.207 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.207 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2285163168, 2023-07-21T22:36:39.207 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634856, 2023-07-21T22:36:39.208 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-22.xml" 2023-07-21T22:36:39.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.208 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.208 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2331719281, 2023-07-21T22:36:39.208 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515231, 2023-07-21T22:36:39.208 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-38.xml" 2023-07-21T22:36:39.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.209 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.209 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2354112475, 2023-07-21T22:36:39.209 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515063, 2023-07-21T22:36:39.209 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-71.xml" 2023-07-21T22:36:39.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.209 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.210 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.210 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2386303897, 2023-07-21T22:36:39.210 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515413, 2023-07-21T22:36:39.210 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-4.xml" 2023-07-21T22:36:39.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.210 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.211 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2441547885, 2023-07-21T22:36:39.211 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515364, 2023-07-21T22:36:39.211 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-13.xml" 2023-07-21T22:36:39.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.211 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.211 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.211 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2441889471, 2023-07-21T22:36:39.212 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634826, 2023-07-21T22:36:39.212 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-50.xml" 2023-07-21T22:36:39.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.212 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.212 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2466041789, 2023-07-21T22:36:39.212 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515625, 2023-07-21T22:36:39.213 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-13.jpg" 2023-07-21T22:36:39.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.213 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.213 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.213 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2473796724, 2023-07-21T22:36:39.213 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634839, 2023-07-21T22:36:39.213 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-38.xml" 2023-07-21T22:36:39.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.214 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.214 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.214 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2505816910, 2023-07-21T22:36:39.214 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515176, 2023-07-21T22:36:39.214 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-47.xml" 2023-07-21T22:36:39.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.215 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.215 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2559160667, 2023-07-21T22:36:39.215 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515407, 2023-07-21T22:36:39.215 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-5.xml" 2023-07-21T22:36:39.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.215 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.216 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.216 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2572750535, 2023-07-21T22:36:39.216 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511636426, 2023-07-21T22:36:39.216 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-85.xml" 2023-07-21T22:36:39.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.216 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.217 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2579503660, 2023-07-21T22:36:39.217 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515086, 2023-07-21T22:36:39.217 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-67.xml" 2023-07-21T22:36:39.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.217 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.217 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.217 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2647111129, 2023-07-21T22:36:39.218 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515338, 2023-07-21T22:36:39.218 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-18.xml" 2023-07-21T22:36:39.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.218 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.218 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.218 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2669850736, 2023-07-21T22:36:39.218 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515348, 2023-07-21T22:36:39.219 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-16.xml" 2023-07-21T22:36:39.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.219 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.219 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2680093729, 2023-07-21T22:36:39.219 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515102, 2023-07-21T22:36:39.219 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-64.xml" 2023-07-21T22:36:39.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.220 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.220 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.220 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2718829971, 2023-07-21T22:36:39.220 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515163, 2023-07-21T22:36:39.220 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-49.xml" 2023-07-21T22:36:39.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.221 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.221 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2743538238, 2023-07-21T22:36:39.221 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515401, 2023-07-21T22:36:39.221 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-6.xml" 2023-07-21T22:36:39.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.221 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.222 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.222 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2774113112, 2023-07-21T22:36:39.222 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634814, 2023-07-21T22:36:39.222 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-60.xml" 2023-07-21T22:36:39.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.222 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.223 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2780800367, 2023-07-21T22:36:39.223 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515380, 2023-07-21T22:36:39.223 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-10.xml" 2023-07-21T22:36:39.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.223 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.223 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.223 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2812491824, 2023-07-21T22:36:39.224 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515284, 2023-07-21T22:36:39.224 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-29.xml" 2023-07-21T22:36:39.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.224 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.224 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.224 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2860300081, 2023-07-21T22:36:39.224 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515515, 2023-07-21T22:36:39.225 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-23.jpg" 2023-07-21T22:36:39.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.225 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.225 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2898166882, 2023-07-21T22:36:39.225 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634834, 2023-07-21T22:36:39.225 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-42.xml" 2023-07-21T22:36:39.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.226 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.226 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2923936086, 2023-07-21T22:36:39.226 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515664, 2023-07-21T22:36:39.226 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-10.jpg" 2023-07-21T22:36:39.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.226 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.227 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.227 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2928718824, 2023-07-21T22:36:39.227 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634848, 2023-07-21T22:36:39.227 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-29.xml" 2023-07-21T22:36:39.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.227 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.227 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.228 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2963561912, 2023-07-21T22:36:39.228 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634792, 2023-07-21T22:36:39.228 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-79.xml" 2023-07-21T22:36:39.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.228 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.228 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2978790822, 2023-07-21T22:36:39.229 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634832, 2023-07-21T22:36:39.229 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-44.xml" 2023-07-21T22:36:39.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.229 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.229 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2989524817, 2023-07-21T22:36:39.229 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634867, 2023-07-21T22:36:39.230 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-13.xml" 2023-07-21T22:36:39.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.230 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.230 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.230 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 2990529164, 2023-07-21T22:36:39.230 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515114, 2023-07-21T22:36:39.230 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-61.xml" 2023-07-21T22:36:39.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.231 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.231 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.231 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3021834678, 2023-07-21T22:36:39.231 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515242, 2023-07-21T22:36:39.231 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-36.xml" 2023-07-21T22:36:39.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.232 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.232 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3064113566, 2023-07-21T22:36:39.232 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511631007, 2023-07-21T22:36:39.232 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-81.xml" 2023-07-21T22:36:39.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.233 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.233 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3135603824, 2023-07-21T22:36:39.233 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515689, 2023-07-21T22:36:39.233 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-8.jpg" 2023-07-21T22:36:39.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.233 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.233 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.233 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3157144195, 2023-07-21T22:36:39.234 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634854, 2023-07-21T22:36:39.234 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-24.xml" 2023-07-21T22:36:39.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.234 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.234 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.234 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3166955163, 2023-07-21T22:36:39.234 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511631588, 2023-07-21T22:36:39.235 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-90.jpg" 2023-07-21T22:36:39.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.235 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.235 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.235 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3167362209, 2023-07-21T22:36:39.235 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511629785, 2023-07-21T22:36:39.235 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-99.xml" 2023-07-21T22:36:39.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.236 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.236 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3204447757, 2023-07-21T22:36:39.236 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515457, 2023-07-21T22:36:39.236 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-27.jpg" 2023-07-21T22:36:39.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.237 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.237 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3215996262, 2023-07-21T22:36:39.237 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515528, 2023-07-21T22:36:39.237 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-22.jpg" 2023-07-21T22:36:39.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.237 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.238 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.238 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3253450795, 2023-07-21T22:36:39.238 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515423, 2023-07-21T22:36:39.238 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-2.xml" 2023-07-21T22:36:39.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.238 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.238 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.239 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3267134901, 2023-07-21T22:36:39.239 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515482, 2023-07-21T22:36:39.239 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-25.jpg" 2023-07-21T22:36:39.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.239 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.239 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.239 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3280909657, 2023-07-21T22:36:39.239 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515142, 2023-07-21T22:36:39.240 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-52.xml" 2023-07-21T22:36:39.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.240 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.240 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3283906153, 2023-07-21T22:36:39.240 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634820, 2023-07-21T22:36:39.240 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-55.xml" 2023-07-21T22:36:39.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.241 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.241 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.241 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3314249874, 2023-07-21T22:36:39.241 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515280, 2023-07-21T22:36:39.241 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-30.xml" 2023-07-21T22:36:39.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.242 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.242 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3325824332, 2023-07-21T22:36:39.242 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634803, 2023-07-21T22:36:39.242 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-69.xml" 2023-07-21T22:36:39.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.243 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.243 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3327615099, 2023-07-21T22:36:39.243 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634860, 2023-07-21T22:36:39.243 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-19.xml" 2023-07-21T22:36:39.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.243 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.244 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3398446860, 2023-07-21T22:36:39.244 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634827, 2023-07-21T22:36:39.244 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-49.xml" 2023-07-21T22:36:39.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.244 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.244 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.244 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3414343753, 2023-07-21T22:36:39.245 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634816, 2023-07-21T22:36:39.245 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-58.xml" 2023-07-21T22:36:39.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.245 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.245 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.245 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3423857029, 2023-07-21T22:36:39.245 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515676, 2023-07-21T22:36:39.245 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-9.jpg" 2023-07-21T22:36:39.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.246 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.246 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.246 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3456523614, 2023-07-21T22:36:39.246 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511631607, 2023-07-21T22:36:39.246 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-44.xml" 2023-07-21T22:36:39.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.247 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.247 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3459989461, 2023-07-21T22:36:39.247 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511629125, 2023-07-21T22:36:39.247 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-91.xml" 2023-07-21T22:36:39.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.247 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.248 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.248 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3518898054, 2023-07-21T22:36:39.248 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634864, 2023-07-21T22:36:39.248 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-16.xml" 2023-07-21T22:36:39.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.248 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.249 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3527048183, 2023-07-21T22:36:39.249 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634831, 2023-07-21T22:36:39.249 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-45.xml" 2023-07-21T22:36:39.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.249 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.249 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.249 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3532503175, 2023-07-21T22:36:39.250 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511631005, 2023-07-21T22:36:39.250 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-82.xml" 2023-07-21T22:36:39.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.250 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.250 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.250 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3560271532, 2023-07-21T22:36:39.250 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634874, 2023-07-21T22:36:39.251 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-6.xml" 2023-07-21T22:36:39.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.251 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.251 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.251 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3606345127, 2023-07-21T22:36:39.251 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515435, 2023-07-21T22:36:39.251 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-29.jpg" 2023-07-21T22:36:39.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.252 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.252 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.252 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3621783074, 2023-07-21T22:36:39.252 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634795, 2023-07-21T22:36:39.252 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-76.xml" 2023-07-21T22:36:39.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.253 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.253 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3636130301, 2023-07-21T22:36:39.253 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634833, 2023-07-21T22:36:39.253 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-43.xml" 2023-07-21T22:36:39.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.253 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.253 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.254 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3643424922, 2023-07-21T22:36:39.254 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511627832, 2023-07-21T22:36:39.254 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-15.xml" 2023-07-21T22:36:39.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.254 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.254 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.254 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3654160732, 2023-07-21T22:36:39.255 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634810, 2023-07-21T22:36:39.255 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-64.xml" 2023-07-21T22:36:39.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.255 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.255 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3759825691, 2023-07-21T22:36:39.255 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515429, 2023-07-21T22:36:39.256 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-1.xml" 2023-07-21T22:36:39.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.256 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.256 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.256 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3763252841, 2023-07-21T22:36:39.256 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515651, 2023-07-21T22:36:39.256 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-11.jpg" 2023-07-21T22:36:39.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.257 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.257 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3787424579, 2023-07-21T22:36:39.257 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515343, 2023-07-21T22:36:39.257 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-17.xml" 2023-07-21T22:36:39.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.258 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.258 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3790930342, 2023-07-21T22:36:39.258 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511631606, 2023-07-21T22:36:39.258 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-26.jpg" 2023-07-21T22:36:39.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.259 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.259 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3812567430, 2023-07-21T22:36:39.259 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515319, 2023-07-21T22:36:39.259 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-22.xml" 2023-07-21T22:36:39.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.259 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.259 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.260 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3814292104, 2023-07-21T22:36:39.260 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634869, 2023-07-21T22:36:39.260 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-11.xml" 2023-07-21T22:36:39.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.260 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.260 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.260 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3878270663, 2023-07-21T22:36:39.261 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515638, 2023-07-21T22:36:39.261 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-12.jpg" 2023-07-21T22:36:39.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.261 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.261 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3879481641, 2023-07-21T22:36:39.261 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634844, 2023-07-21T22:36:39.262 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-33.xml" 2023-07-21T22:36:39.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.262 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.262 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.262 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3895483771, 2023-07-21T22:36:39.262 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511629784, 2023-07-21T22:36:39.262 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-38.jpg" 2023-07-21T22:36:39.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.263 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.263 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3925276455, 2023-07-21T22:36:39.263 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515358, 2023-07-21T22:36:39.263 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-14.xml" 2023-07-21T22:36:39.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.264 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.264 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3937935806, 2023-07-21T22:36:39.264 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511631008, 2023-07-21T22:36:39.264 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-80.xml" 2023-07-21T22:36:39.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.265 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.265 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3982885179, 2023-07-21T22:36:39.265 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634841, 2023-07-21T22:36:39.265 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-36.xml" 2023-07-21T22:36:39.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.265 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.265 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.266 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 3999793729, 2023-07-21T22:36:39.266 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511633960, 2023-07-21T22:36:39.266 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-61.jpg" 2023-07-21T22:36:39.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.266 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.266 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.266 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4022925223, 2023-07-21T22:36:39.267 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515579, 2023-07-21T22:36:39.267 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-17.jpg" 2023-07-21T22:36:39.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.267 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.267 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4049028454, 2023-07-21T22:36:39.267 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634842, 2023-07-21T22:36:39.267 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-35.xml" 2023-07-21T22:36:39.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.268 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.268 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4070979809, 2023-07-21T22:36:39.268 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634830, 2023-07-21T22:36:39.268 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-46.xml" 2023-07-21T22:36:39.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.269 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.269 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4071847979, 2023-07-21T22:36:39.269 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634822, 2023-07-21T22:36:39.269 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-53.xml" 2023-07-21T22:36:39.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.270 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.270 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4120931537, 2023-07-21T22:36:39.270 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515137, 2023-07-21T22:36:39.270 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-53.xml" 2023-07-21T22:36:39.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.270 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.270 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.271 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4133368694, 2023-07-21T22:36:39.271 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515495, 2023-07-21T22:36:39.271 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-24.jpg" 2023-07-21T22:36:39.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.271 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.271 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4136146020, 2023-07-21T22:36:39.271 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515332, 2023-07-21T22:36:39.272 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-19.xml" 2023-07-21T22:36:39.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.272 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.272 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.272 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4140404699, 2023-07-21T22:36:39.272 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634878, 2023-07-21T22:36:39.272 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-2.xml" 2023-07-21T22:36:39.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.273 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.273 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.273 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4169533317, 2023-07-21T22:36:39.273 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515749, 2023-07-21T22:36:39.273 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-3.jpg" 2023-07-21T22:36:39.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.274 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.274 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4190584471, 2023-07-21T22:36:39.274 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515122, 2023-07-21T22:36:39.274 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-57.xml" 2023-07-21T22:36:39.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.274 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.275 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.275 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4215026728, 2023-07-21T22:36:39.275 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515116, 2023-07-21T22:36:39.275 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-60.xml" 2023-07-21T22:36:39.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.275 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.275 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.276 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4240513615, 2023-07-21T22:36:39.276 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515080, 2023-07-21T22:36:39.276 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-68.xml" 2023-07-21T22:36:39.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.276 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.276 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.276 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4254070846, 2023-07-21T22:36:39.277 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634802, 2023-07-21T22:36:39.277 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-70.xml" 2023-07-21T22:36:39.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.277 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.277 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.277 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4254643373, 2023-07-21T22:36:39.277 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 4398046515256, 2023-07-21T22:36:39.278 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-34.xml" 2023-07-21T22:36:39.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.278 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.278 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.278 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4262721508, 2023-07-21T22:36:39.278 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634829, 2023-07-21T22:36:39.278 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-47.xml" 2023-07-21T22:36:39.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:36:39.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:36:39.279 INFO:teuthology.orchestra.run.smithi107.stdout: "damage_type": "backtrace", 2023-07-21T22:36:39.279 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4264396562, 2023-07-21T22:36:39.279 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1099511634872, 2023-07-21T22:36:39.279 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-8.xml" 2023-07-21T22:36:39.279 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:36:39.280 INFO:teuthology.orchestra.run.smithi107.stdout:] 2023-07-21T22:36:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623292d700 1 -- 172.21.15.107:0/2752346274 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62140a1150 msgr2=0x7f62140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623292d700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62140a1150 0x7f62140a3600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f622000fa40 tx=0x7f6220019040 comp rx=0 tx=0).stop 2023-07-21T22:36:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623292d700 1 -- 172.21.15.107:0/2752346274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c071310 msgr2=0x7f622c117350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623292d700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c071310 0x7f622c117350 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f621c00e9c0 tx=0x7f621c00ecd0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623292d700 1 -- 172.21.15.107:0/2752346274 shutdown_connections 2023-07-21T22:36:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623292d700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f622c10d080 0x7f622c11e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623292d700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c071310 0x7f622c117350 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623292d700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f62140a1150 0x7f62140a3600 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623292d700 1 --2- 172.21.15.107:0/2752346274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f622c10a760 0x7f622c117890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623292d700 1 -- 172.21.15.107:0/2752346274 >> 172.21.15.107:0/2752346274 conn(0x7f622c06ca90 msgr2=0x7f622c10c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.061+0000 7f623292d700 1 -- 172.21.15.107:0/2752346274 shutdown_connections 2023-07-21T22:36:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.062+0000 7f623292d700 1 -- 172.21.15.107:0/2752346274 wait complete. 2023-07-21T22:36:39.283 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:36:39.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.364+0000 7f3ec742d700 1 -- 172.21.15.107:0/2200462612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ec00ffbd0 msgr2=0x7f3ec0100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.415 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.364+0000 7f3ec742d700 1 --2- 172.21.15.107:0/2200462612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ec00ffbd0 0x7f3ec0100020 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009990 tx=0x7f3eb4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.364+0000 7f3ec742d700 1 -- 172.21.15.107:0/2200462612 shutdown_connections 2023-07-21T22:36:39.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.364+0000 7f3ec742d700 1 --2- 172.21.15.107:0/2200462612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ec0100560 0x7f3ec0106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.364+0000 7f3ec742d700 1 --2- 172.21.15.107:0/2200462612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ec00fe9e0 0x7f3ec00fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.364+0000 7f3ec742d700 1 --2- 172.21.15.107:0/2200462612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ec00ffbd0 0x7f3ec0100020 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.364+0000 7f3ec742d700 1 -- 172.21.15.107:0/2200462612 >> 172.21.15.107:0/2200462612 conn(0x7f3ec00fa000 msgr2=0x7f3ec00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.364+0000 7f3ec742d700 1 -- 172.21.15.107:0/2200462612 shutdown_connections 2023-07-21T22:36:39.416 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.364+0000 7f3ec742d700 1 -- 172.21.15.107:0/2200462612 wait complete. 2023-07-21T22:36:39.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.364+0000 7f3ec742d700 1 Processor -- start 2023-07-21T22:36:39.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec742d700 1 -- start start 2023-07-21T22:36:39.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec742d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ec00fe9e0 0x7f3ec01947b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec742d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ec00ffbd0 0x7f3ec0194cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec742d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ec0100560 0x7f3ec019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec742d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec019c330 con 0x7f3ec0100560 2023-07-21T22:36:39.417 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec742d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec019c470 con 0x7f3ec00ffbd0 2023-07-21T22:36:39.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec742d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec019c600 con 0x7f3ec00fe9e0 2023-07-21T22:36:39.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec59ca700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ec0100560 0x7f3ec019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec59ca700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ec0100560 0x7f3ec019bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51234/0 (socket says 172.21.15.107:51234) 2023-07-21T22:36:39.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec59ca700 1 -- 172.21.15.107:0/2877434561 learned_addr learned my addr 172.21.15.107:0/2877434561 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec49c8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ec00ffbd0 0x7f3ec0194cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.418 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec51c9700 1 --2- 172.21.15.107:0/2877434561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ec00fe9e0 0x7f3ec01947b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.419 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec59ca700 1 -- 172.21.15.107:0/2877434561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ec00fe9e0 msgr2=0x7f3ec01947b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.419 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec59ca700 1 --2- 172.21.15.107:0/2877434561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ec00fe9e0 0x7f3ec01947b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.419 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec59ca700 1 -- 172.21.15.107:0/2877434561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ec00ffbd0 msgr2=0x7f3ec0194cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.419 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec59ca700 1 --2- 172.21.15.107:0/2877434561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ec00ffbd0 0x7f3ec0194cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.419 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec59ca700 1 -- 172.21.15.107:0/2877434561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb40095f0 con 0x7f3ec0100560 2023-07-21T22:36:39.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec59ca700 1 --2- 172.21.15.107:0/2877434561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ec0100560 0x7f3ec019bd60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000e580 tx=0x7f3eb000e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ebe7fc700 1 -- 172.21.15.107:0/2877434561 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0004db0 con 0x7f3ec0100560 2023-07-21T22:36:39.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ec742d700 1 -- 172.21.15.107:0/2877434561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec00676d0 con 0x7f3ec0100560 2023-07-21T22:36:39.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ebe7fc700 1 -- 172.21.15.107:0/2877434561 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb00090d0 con 0x7f3ec0100560 2023-07-21T22:36:39.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.365+0000 7f3ebe7fc700 1 -- 172.21.15.107:0/2877434561 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00045f0 con 0x7f3ec0100560 2023-07-21T22:36:39.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.366+0000 7f3ec742d700 1 -- 172.21.15.107:0/2877434561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec0067b90 con 0x7f3ec0100560 2023-07-21T22:36:39.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.367+0000 7f3ebe7fc700 1 -- 172.21.15.107:0/2877434561 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3eb0009240 con 0x7f3ec0100560 2023-07-21T22:36:39.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.368+0000 7f3ebe7fc700 1 --2- 172.21.15.107:0/2877434561 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3ea80a1150 0x7f3ea80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.368+0000 7f3ec51c9700 1 --2- 172.21.15.107:0/2877434561 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3ea80a1150 0x7f3ea80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.368+0000 7f3ebe7fc700 1 -- 172.21.15.107:0/2877434561 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3eb0016070 con 0x7f3ec0100560 2023-07-21T22:36:39.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.368+0000 7f3ec51c9700 1 --2- 172.21.15.107:0/2877434561 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3ea80a1150 0x7f3ea80a3600 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00ffa30 tx=0x7f3eb8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.369+0000 7f3ed35b2b80 1 Processor -- start 2023-07-21T22:36:39.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.369+0000 7f3ed35b2b80 1 -- start start 2023-07-21T22:36:39.421 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.369+0000 7f3ed35b2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941b830 0x55954941bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.369+0000 7f3ed35b2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 0x55954941a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.369+0000 7f3ed35b2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x55954941ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.369+0000 7f3ed35b2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954931fcb0 con 0x55954941c1a0 2023-07-21T22:36:39.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.369+0000 7f3ed35b2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559549372a40 con 0x55954941b830 2023-07-21T22:36:39.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.369+0000 7f3ed35b2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954936bed0 con 0x55954941aa10 2023-07-21T22:36:39.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec51c9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941b830 0x55954941bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec49c8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 0x55954941a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec59ca700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x55954941ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec51c9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941b830 0x55954941bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:42066/0 (socket says 172.21.15.107:42066) 2023-07-21T22:36:39.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec49c8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 0x55954941a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51240/0 (socket says 172.21.15.107:51240) 2023-07-21T22:36:39.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec51c9700 1 -- 172.21.15.107:0/657006555 learned_addr learned my addr 172.21.15.107:0/657006555 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec59ca700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x55954941ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39800/0 (socket says 172.21.15.107:39800) 2023-07-21T22:36:39.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec59ca700 1 -- 172.21.15.107:0/657006555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941b830 msgr2=0x55954941bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec59ca700 1 --2- 172.21.15.107:0/657006555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941b830 0x55954941bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec59ca700 1 -- 172.21.15.107:0/657006555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 msgr2=0x55954941a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.423 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec59ca700 1 --2- 172.21.15.107:0/657006555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 0x55954941a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec59ca700 1 -- 172.21.15.107:0/657006555 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559549199de0 con 0x55954941aa10 2023-07-21T22:36:39.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec49c8700 1 --2- 172.21.15.107:0/657006555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 0x55954941a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:39.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec51c9700 1 --2- 172.21.15.107:0/657006555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941b830 0x55954941bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:39.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ec59ca700 1 --2- 172.21.15.107:0/657006555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x55954941ae90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00cc240 tx=0x7f3eb0010fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdcc3f88019058e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ea77fe700 1 -- 172.21.15.107:0/657006555 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00bb050 con 0x55954941aa10 2023-07-21T22:36:39.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ea77fe700 1 -- 172.21.15.107:0/657006555 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb009fb30 con 0x55954941aa10 2023-07-21T22:36:39.424 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.370+0000 7f3ea77fe700 1 -- 172.21.15.107:0/657006555 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb009aff0 con 0x55954941aa10 2023-07-21T22:36:39.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.371+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/657006555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 msgr2=0x55954941ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.371+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/657006555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x55954941ae90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00cc240 tx=0x7f3eb0010fb0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.371+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/657006555 shutdown_connections 2023-07-21T22:36:39.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.371+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/657006555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x55954941ae90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.371+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/657006555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941b830 0x55954941bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.371+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/657006555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 0x55954941a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.371+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/657006555 >> 172.21.15.107:0/657006555 conn(0x55954940fef0 msgr2=0x559549415360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.371+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/657006555 shutdown_connections 2023-07-21T22:36:39.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.371+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/657006555 wait complete. 2023-07-21T22:36:39.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.371+0000 7f3ed35b2b80 1 -- start start 2023-07-21T22:36:39.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.371+0000 7f3ed35b2b80 1 Processor -- start 2023-07-21T22:36:39.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ed35b2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x5595494240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ed35b2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 0x5595494245f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ed35b2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941c1a0 0x55954942b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ed35b2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954930bac0 con 0x55954941b830 2023-07-21T22:36:39.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ed35b2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954930b7c0 con 0x55954941c1a0 2023-07-21T22:36:39.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ed35b2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595491ef920 con 0x55954941aa10 2023-07-21T22:36:39.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ed35b2b80 10 client.0 _ll_register_callbacks cb 0x559549309760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:39.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ec49c8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 0x5595494245f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ec49c8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 0x5595494245f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51246/0 (socket says 172.21.15.107:51246) 2023-07-21T22:36:39.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ec49c8700 1 -- 172.21.15.107:0/2879655288 learned_addr learned my addr 172.21.15.107:0/2879655288 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ec51c9700 1 --2- 172.21.15.107:0/2879655288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x5595494240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.430 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ec59ca700 1 --2- 172.21.15.107:0/2879655288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941c1a0 0x55954942b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ec49c8700 1 -- 172.21.15.107:0/2879655288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 msgr2=0x5595494240b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ec49c8700 1 --2- 172.21.15.107:0/2879655288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x5595494240b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ec49c8700 1 -- 172.21.15.107:0/2879655288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941c1a0 msgr2=0x55954942b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ec49c8700 1 --2- 172.21.15.107:0/2879655288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941c1a0 0x55954942b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.372+0000 7f3ec49c8700 1 -- 172.21.15.107:0/2879655288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb8017070 con 0x55954941b830 2023-07-21T22:36:39.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ec49c8700 1 --2- 172.21.15.107:0/2879655288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 0x5595494245f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009990 tx=0x7f3eb4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d514c72f3a04826 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ea4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:39.431 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ea4ff9700 1 -- 172.21.15.107:0/2879655288 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb401d070 con 0x55954941b830 2023-07-21T22:36:39.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ea4ff9700 1 -- 172.21.15.107:0/2879655288 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3eb4004700 con 0x55954941b830 2023-07-21T22:36:39.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2879655288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595492c54f0 con 0x55954941b830 2023-07-21T22:36:39.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ea4ff9700 1 -- 172.21.15.107:0/2879655288 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3eb400be90 con 0x55954941b830 2023-07-21T22:36:39.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ed35b2b80 10 client.16635 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:39.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2879655288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f3ec0107750 con 0x55954941b830 2023-07-21T22:36:39.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ea4ff9700 1 -- 172.21.15.107:0/2879655288 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5595492c54f0 con 0x55954941b830 2023-07-21T22:36:39.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ea4ff9700 1 -- 172.21.15.107:0/2879655288 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f3eb400f9f0 con 0x55954941b830 2023-07-21T22:36:39.432 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ed35b2b80 10 client.16635 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:39.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ed35b2b80 10 client.16635 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-21T22:36:39.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ed35b2b80 20 client.16635 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:39.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/2879655288 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55954942ff50 0x559549432360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ed35b2b80 4 client.16635 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:39.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.373+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2879655288 --> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559549364eb0 con 0x55954942ff50 2023-07-21T22:36:39.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.374+0000 7f3ec51c9700 1 --2- 172.21.15.107:0/2879655288 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55954942ff50 0x559549432360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.433 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.374+0000 7f3ea4ff9700 10 client.16635 ms_handle_connect on v2:172.21.15.190:6838/2960500982 2023-07-21T22:36:39.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.374+0000 7f3ec51c9700 1 --2- 172.21.15.107:0/2879655288 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55954942ff50 0x559549432360 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36f168ec6a7a1585 server_cookie=ded720667fc74632 in_seq=0 out_seq=0 2023-07-21T22:36:39.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ea4ff9700 1 -- 172.21.15.107:0/2879655288 <== mds.1 v2:172.21.15.190:6838/2960500982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559549364eb0 con 0x55954942ff50 2023-07-21T22:36:39.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ea4ff9700 10 client.16635 handle_command_reply: tid=0 2023-07-21T22:36:39.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ed35b2b80 1 client.16635 shutdown 2023-07-21T22:36:39.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2879655288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 msgr2=0x5595494245f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/2879655288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 0x5595494245f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009990 tx=0x7f3eb4005800 comp rx=0 tx=0).stop 2023-07-21T22:36:39.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2879655288 shutdown_connections 2023-07-21T22:36:39.434 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/2879655288 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x55954942ff50 0x559549432360 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/2879655288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941c1a0 0x55954942b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ea4ff9700 0 client.16635 ms_handle_reset on v2:172.21.15.190:6838/2960500982 2023-07-21T22:36:39.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/2879655288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x5595494240b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/2879655288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 0x5595494245f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2879655288 >> 172.21.15.107:0/2879655288 conn(0x55954940fef0 msgr2=0x559549414a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2879655288 shutdown_connections 2023-07-21T22:36:39.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2879655288 wait complete. 2023-07-21T22:36:39.435 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.375+0000 7f3ed35b2b80 20 client.16635 trim_cache size 0 max 16384 2023-07-21T22:36:39.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ed35b2b80 1 Processor -- start 2023-07-21T22:36:39.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ed35b2b80 1 -- start start 2023-07-21T22:36:39.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ed35b2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x559549422d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ed35b2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 0x55954942b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ed35b2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941c1a0 0x559549422020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ed35b2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559549424da0 con 0x55954941b830 2023-07-21T22:36:39.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ed35b2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559549421dd0 con 0x55954941c1a0 2023-07-21T22:36:39.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ed35b2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954942fc50 con 0x55954941aa10 2023-07-21T22:36:39.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ec49c8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 0x55954942b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ec49c8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 0x55954942b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51254/0 (socket says 172.21.15.107:51254) 2023-07-21T22:36:39.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ec49c8700 1 -- 172.21.15.107:0/3029313338 learned_addr learned my addr 172.21.15.107:0/3029313338 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ec59ca700 1 --2- 172.21.15.107:0/3029313338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941c1a0 0x559549422020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ec51c9700 1 --2- 172.21.15.107:0/3029313338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x559549422d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ec49c8700 1 -- 172.21.15.107:0/3029313338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 msgr2=0x559549422d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ec49c8700 1 --2- 172.21.15.107:0/3029313338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x559549422d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ec49c8700 1 -- 172.21.15.107:0/3029313338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941c1a0 msgr2=0x559549422020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ec49c8700 1 --2- 172.21.15.107:0/3029313338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941c1a0 0x559549422020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ec49c8700 1 -- 172.21.15.107:0/3029313338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec0067b90 con 0x55954941b830 2023-07-21T22:36:39.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ec49c8700 1 --2- 172.21.15.107:0/3029313338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 0x55954942b5f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400b2e0 tx=0x7f3eb4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797ff1ed97ca3c06 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.388+0000 7f3ea7fff700 1 -- 172.21.15.107:0/3029313338 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb401d070 con 0x55954941b830 2023-07-21T22:36:39.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.389+0000 7f3ea7fff700 1 -- 172.21.15.107:0/3029313338 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb401e040 con 0x55954941b830 2023-07-21T22:36:39.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.389+0000 7f3ea7fff700 1 -- 172.21.15.107:0/3029313338 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400f840 con 0x55954941b830 2023-07-21T22:36:39.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.389+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/3029313338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 msgr2=0x55954942b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.389+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/3029313338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 0x55954942b5f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400b2e0 tx=0x7f3eb4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.389+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/3029313338 shutdown_connections 2023-07-21T22:36:39.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.389+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/3029313338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941c1a0 0x559549422020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.389+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/3029313338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x559549422d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.389+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/3029313338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941b830 0x55954942b5f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.389+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/3029313338 >> 172.21.15.107:0/3029313338 conn(0x55954940fef0 msgr2=0x559549436cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.389+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/3029313338 shutdown_connections 2023-07-21T22:36:39.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.389+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/3029313338 wait complete. 2023-07-21T22:36:39.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.389+0000 7f3ed35b2b80 1 -- start start 2023-07-21T22:36:39.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ed35b2b80 1 Processor -- start 2023-07-21T22:36:39.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ed35b2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x559549426ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ed35b2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941b830 0x5595494273e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ed35b2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 0x55954942a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.444 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ed35b2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954942a760 con 0x55954941c1a0 2023-07-21T22:36:39.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ed35b2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954942a8d0 con 0x55954941b830 2023-07-21T22:36:39.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ed35b2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55954942aa90 con 0x55954941aa10 2023-07-21T22:36:39.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ed35b2b80 10 client.0 _ll_register_callbacks cb 0x7f3eb4005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:39.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ec59ca700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 0x55954942a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ec59ca700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 0x55954942a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51262/0 (socket says 172.21.15.107:51262) 2023-07-21T22:36:39.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ec49c8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941b830 0x5595494273e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.445 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ec51c9700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x559549426ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ec59ca700 1 -- 172.21.15.107:0/2178459919 learned_addr learned my addr 172.21.15.107:0/2178459919 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ec59ca700 1 -- 172.21.15.107:0/2178459919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 msgr2=0x559549426ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ec59ca700 1 --2- 172.21.15.107:0/2178459919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x559549426ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ec59ca700 1 -- 172.21.15.107:0/2178459919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941b830 msgr2=0x5595494273e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ec59ca700 1 --2- 172.21.15.107:0/2178459919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941b830 0x5595494273e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ec59ca700 1 -- 172.21.15.107:0/2178459919 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec0107750 con 0x55954941c1a0 2023-07-21T22:36:39.446 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.390+0000 7f3ec51c9700 1 --2- 172.21.15.107:0/2178459919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x559549426ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:39.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ec59ca700 1 --2- 172.21.15.107:0/2178459919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 0x55954942a220 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00cc560 tx=0x7f3eb0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c2bd534f2edce0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ea4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:39.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ea4ff9700 1 -- 172.21.15.107:0/2178459919 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00a3f50 con 0x55954941c1a0 2023-07-21T22:36:39.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ea4ff9700 1 -- 172.21.15.107:0/2178459919 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3eb0099ea0 con 0x55954941c1a0 2023-07-21T22:36:39.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2178459919 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3eb009fb30 con 0x55954941c1a0 2023-07-21T22:36:39.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ea4ff9700 1 -- 172.21.15.107:0/2178459919 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3eb00bb020 con 0x55954941c1a0 2023-07-21T22:36:39.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ed35b2b80 10 client.16641 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:39.447 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2178459919 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x559549182c60 con 0x55954941c1a0 2023-07-21T22:36:39.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ea4ff9700 1 -- 172.21.15.107:0/2178459919 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb009fb30 con 0x55954941c1a0 2023-07-21T22:36:39.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ea4ff9700 1 -- 172.21.15.107:0/2178459919 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f3eb009aa60 con 0x55954941c1a0 2023-07-21T22:36:39.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ed35b2b80 10 client.16641 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:39.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ed35b2b80 10 client.16641 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-21T22:36:39.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ed35b2b80 20 client.16641 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:39.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/2178459919 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x559549484450 0x559549486860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.448 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ed35b2b80 4 client.16641 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:36:39.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2178459919 --> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559549327050 con 0x559549484450 2023-07-21T22:36:39.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.391+0000 7f3ec51c9700 1 --2- 172.21.15.107:0/2178459919 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x559549484450 0x559549486860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ea4ff9700 10 client.16641 ms_handle_connect on v2:172.21.15.190:6838/2960500982 2023-07-21T22:36:39.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ec51c9700 1 --2- 172.21.15.107:0/2178459919 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x559549484450 0x559549486860 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b3525bd4fada14f server_cookie=6ab7a622531b2189 in_seq=0 out_seq=0 2023-07-21T22:36:39.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ea4ff9700 1 -- 172.21.15.107:0/2178459919 <== mds.1 v2:172.21.15.190:6838/2960500982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559549327050 con 0x559549484450 2023-07-21T22:36:39.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ea4ff9700 10 client.16641 handle_command_reply: tid=0 2023-07-21T22:36:39.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ed35b2b80 1 client.16641 shutdown 2023-07-21T22:36:39.449 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2178459919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 msgr2=0x55954942a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/2178459919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 0x55954942a220 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00cc560 tx=0x7f3eb0009560 comp rx=0 tx=0).stop 2023-07-21T22:36:39.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2178459919 shutdown_connections 2023-07-21T22:36:39.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/2178459919 >> [v2:172.21.15.190:6838/2960500982,v1:172.21.15.190:6839/2960500982] conn(0x559549484450 0x559549486860 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/2178459919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55954941c1a0 0x55954942a220 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/2178459919 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55954941aa10 0x559549426ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ea4ff9700 0 client.16641 ms_handle_reset on v2:172.21.15.190:6838/2960500982 2023-07-21T22:36:39.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ed35b2b80 1 --2- 172.21.15.107:0/2178459919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55954941b830 0x5595494273e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2178459919 >> 172.21.15.107:0/2178459919 conn(0x55954940fef0 msgr2=0x559549436cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2178459919 shutdown_connections 2023-07-21T22:36:39.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ed35b2b80 1 -- 172.21.15.107:0/2178459919 wait complete. 2023-07-21T22:36:39.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.392+0000 7f3ed35b2b80 20 client.16641 trim_cache size 0 max 16384 2023-07-21T22:36:39.451 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:36:39.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.393+0000 7f3ec742d700 1 -- 172.21.15.107:0/2877434561 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3ea80a1150 msgr2=0x7f3ea80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.393+0000 7f3ec742d700 1 --2- 172.21.15.107:0/2877434561 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3ea80a1150 0x7f3ea80a3600 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00ffa30 tx=0x7f3eb8009260 comp rx=0 tx=0).stop 2023-07-21T22:36:39.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.393+0000 7f3ec742d700 1 -- 172.21.15.107:0/2877434561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ec0100560 msgr2=0x7f3ec019bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.393+0000 7f3ec742d700 1 --2- 172.21.15.107:0/2877434561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ec0100560 0x7f3ec019bd60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000e580 tx=0x7f3eb000e890 comp rx=0 tx=0).stop 2023-07-21T22:36:39.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.393+0000 7f3ec742d700 1 -- 172.21.15.107:0/2877434561 shutdown_connections 2023-07-21T22:36:39.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.393+0000 7f3ec742d700 1 --2- 172.21.15.107:0/2877434561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ec0100560 0x7f3ec019bd60 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.393+0000 7f3ec742d700 1 --2- 172.21.15.107:0/2877434561 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3ea80a1150 0x7f3ea80a3600 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.393+0000 7f3ec742d700 1 --2- 172.21.15.107:0/2877434561 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ec00fe9e0 0x7f3ec01947b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.393+0000 7f3ec742d700 1 --2- 172.21.15.107:0/2877434561 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ec00ffbd0 0x7f3ec0194cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.393+0000 7f3ec742d700 1 -- 172.21.15.107:0/2877434561 >> 172.21.15.107:0/2877434561 conn(0x7f3ec00fa000 msgr2=0x7f3ec0103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.393+0000 7f3ec742d700 1 -- 172.21.15.107:0/2877434561 shutdown_connections 2023-07-21T22:36:39.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.393+0000 7f3ec742d700 1 -- 172.21.15.107:0/2877434561 wait complete. 2023-07-21T22:36:39.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:36:39.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.536+0000 7f56f1b4d700 1 -- 172.21.15.107:0/75798700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ec0ffaf0 msgr2=0x7f56ec0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.536+0000 7f56f1b4d700 1 --2- 172.21.15.107:0/75798700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ec0ffaf0 0x7f56ec0fff00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009990 tx=0x7f56d4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.581 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.536+0000 7f56f1b4d700 1 -- 172.21.15.107:0/75798700 shutdown_connections 2023-07-21T22:36:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.536+0000 7f56f1b4d700 1 --2- 172.21.15.107:0/75798700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ec101670 0x7f56ec107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.536+0000 7f56f1b4d700 1 --2- 172.21.15.107:0/75798700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ec0ffaf0 0x7f56ec0fff00 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.536+0000 7f56f1b4d700 1 --2- 172.21.15.107:0/75798700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ec100ce0 0x7f56ec101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.536+0000 7f56f1b4d700 1 -- 172.21.15.107:0/75798700 >> 172.21.15.107:0/75798700 conn(0x7f56ec0fb130 msgr2=0x7f56ec0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.536+0000 7f56f1b4d700 1 -- 172.21.15.107:0/75798700 shutdown_connections 2023-07-21T22:36:39.582 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.536+0000 7f56f1b4d700 1 -- 172.21.15.107:0/75798700 wait complete. 2023-07-21T22:36:39.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56f1b4d700 1 Processor -- start 2023-07-21T22:36:39.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56f1b4d700 1 -- start start 2023-07-21T22:36:39.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56f1b4d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ec0ffaf0 0x7f56ec195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56f1b4d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ec100ce0 0x7f56ec195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56f1b4d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ec101670 0x7f56ec19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56f1b4d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec19d3a0 con 0x7f56ec101670 2023-07-21T22:36:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56f1b4d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec19d4e0 con 0x7f56ec0ffaf0 2023-07-21T22:36:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56f1b4d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec19d670 con 0x7f56ec100ce0 2023-07-21T22:36:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56ebfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ec101670 0x7f56ec19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ec0ffaf0 0x7f56ec195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56ebfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ec101670 0x7f56ec19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51276/0 (socket says 172.21.15.107:51276) 2023-07-21T22:36:39.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ec0ffaf0 0x7f56ec195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:42100/0 (socket says 172.21.15.107:42100) 2023-07-21T22:36:39.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56ebfff700 1 -- 172.21.15.107:0/1292425955 learned_addr learned my addr 172.21.15.107:0/1292425955 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.537+0000 7f56eaffd700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ec100ce0 0x7f56ec195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.538+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1292425955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ec100ce0 msgr2=0x7f56ec195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.538+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ec100ce0 0x7f56ec195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.538+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1292425955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ec101670 msgr2=0x7f56ec19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.538+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ec101670 0x7f56ec19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.538+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1292425955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d40095f0 con 0x7f56ec0ffaf0 2023-07-21T22:36:39.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.538+0000 7f56ebfff700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ec101670 0x7f56ec19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:39.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.538+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ec0ffaf0 0x7f56ec195820 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f56d400b620 tx=0x7f56d4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.538+0000 7f56e8ff9700 1 -- 172.21.15.107:0/1292425955 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d401d070 con 0x7f56ec0ffaf0 2023-07-21T22:36:39.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.538+0000 7f56e8ff9700 1 -- 172.21.15.107:0/1292425955 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d4022470 con 0x7f56ec0ffaf0 2023-07-21T22:36:39.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.538+0000 7f56f1b4d700 1 -- 172.21.15.107:0/1292425955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec19d8f0 con 0x7f56ec0ffaf0 2023-07-21T22:36:39.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.538+0000 7f56e8ff9700 1 -- 172.21.15.107:0/1292425955 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4017650 con 0x7f56ec0ffaf0 2023-07-21T22:36:39.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.538+0000 7f56f1b4d700 1 -- 172.21.15.107:0/1292425955 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec19dd10 con 0x7f56ec0ffaf0 2023-07-21T22:36:39.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.541+0000 7f56e8ff9700 1 -- 172.21.15.107:0/1292425955 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f56d40225e0 con 0x7f56ec0ffaf0 2023-07-21T22:36:39.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.541+0000 7f56e8ff9700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56d80a1150 0x7f56d80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.541+0000 7f56eaffd700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56d80a1150 0x7f56d80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.541+0000 7f56e8ff9700 1 -- 172.21.15.107:0/1292425955 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f56d40cddb0 con 0x7f56ec0ffaf0 2023-07-21T22:36:39.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56eaffd700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56d80a1150 0x7f56d80a3600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009bf0 tx=0x7f56dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56fdcd2b80 1 Processor -- start 2023-07-21T22:36:39.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56fdcd2b80 1 -- start start 2023-07-21T22:36:39.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56fdcd2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebba80 0x561e73ebbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56fdcd2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebc3d0 0x561e73ebca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56fdcd2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebcff0 0x561e73ebd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56fdcd2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e73dbccb0 con 0x561e73ebcff0 2023-07-21T22:36:39.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56fdcd2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e73e0fa40 con 0x561e73ebba80 2023-07-21T22:36:39.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56fdcd2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e73e08ed0 con 0x561e73ebc3d0 2023-07-21T22:36:39.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebba80 0x561e73ebbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56ebfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebcff0 0x561e73ebd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebba80 0x561e73ebbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:42108/0 (socket says 172.21.15.107:42108) 2023-07-21T22:36:39.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1248569097 learned_addr learned my addr 172.21.15.107:0/1248569097 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.542+0000 7f56eaffd700 1 --2- 172.21.15.107:0/1248569097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebc3d0 0x561e73ebca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.543+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1248569097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebc3d0 msgr2=0x561e73ebca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.543+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/1248569097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebc3d0 0x561e73ebca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.543+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1248569097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebcff0 msgr2=0x561e73ebd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.543+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/1248569097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebcff0 0x561e73ebd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.543+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1248569097 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e73c36de0 con 0x561e73ebba80 2023-07-21T22:36:39.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.543+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/1248569097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebba80 0x561e73ebbe90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f56d4097850 tx=0x7f56d409acb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85dcc250a941b19f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.543+0000 7f56e1ffb700 1 -- 172.21.15.107:0/1248569097 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d409f380 con 0x561e73ebba80 2023-07-21T22:36:39.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.543+0000 7f56e1ffb700 1 -- 172.21.15.107:0/1248569097 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d40a3710 con 0x561e73ebba80 2023-07-21T22:36:39.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.543+0000 7f56e1ffb700 1 -- 172.21.15.107:0/1248569097 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d409a840 con 0x561e73ebba80 2023-07-21T22:36:39.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.543+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1248569097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebba80 msgr2=0x561e73ebbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.543+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1248569097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebba80 0x561e73ebbe90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f56d4097850 tx=0x7f56d409acb0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1248569097 shutdown_connections 2023-07-21T22:36:39.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1248569097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebcff0 0x561e73ebd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1248569097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebba80 0x561e73ebbe90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1248569097 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebc3d0 0x561e73ebca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1248569097 >> 172.21.15.107:0/1248569097 conn(0x561e73eacef0 msgr2=0x561e73eb2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1248569097 shutdown_connections 2023-07-21T22:36:39.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1248569097 wait complete. 2023-07-21T22:36:39.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 -- start start 2023-07-21T22:36:39.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 Processor -- start 2023-07-21T22:36:39.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 0x561e73ec50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebc3d0 0x561e73ec5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebcff0 0x561e73ecc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e73da8ac0 con 0x561e73ebba80 2023-07-21T22:36:39.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e73da87c0 con 0x561e73ebc3d0 2023-07-21T22:36:39.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e73c8c920 con 0x561e73ebcff0 2023-07-21T22:36:39.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56fdcd2b80 10 client.0 _ll_register_callbacks cb 0x561e73da6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:39.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 0x561e73ec50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56eaffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebc3d0 0x561e73ec5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.544+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 0x561e73ec50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51286/0 (socket says 172.21.15.107:51286) 2023-07-21T22:36:39.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1994142644 learned_addr learned my addr 172.21.15.107:0/1994142644 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56ebfff700 1 --2- 172.21.15.107:0/1994142644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebcff0 0x561e73ecc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1994142644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebcff0 msgr2=0x561e73ecc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/1994142644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebcff0 0x561e73ecc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1994142644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebc3d0 msgr2=0x561e73ec5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/1994142644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebc3d0 0x561e73ec5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1994142644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec19dd10 con 0x561e73ebba80 2023-07-21T22:36:39.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56eaffd700 1 --2- 172.21.15.107:0/1994142644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebc3d0 0x561e73ec5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:39.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/1994142644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 0x561e73ec50f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f56d40ce090 tx=0x7f56d409b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3acf45ab1232e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56caffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:39.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56caffd700 1 -- 172.21.15.107:0/1994142644 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d409aa40 con 0x561e73ebba80 2023-07-21T22:36:39.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56caffd700 1 -- 172.21.15.107:0/1994142644 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f56d40a3280 con 0x561e73ebba80 2023-07-21T22:36:39.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1994142644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e73d624f0 con 0x561e73ebba80 2023-07-21T22:36:39.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56caffd700 1 -- 172.21.15.107:0/1994142644 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56d40a3440 con 0x561e73ebba80 2023-07-21T22:36:39.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56fdcd2b80 10 client.16653 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:39.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1994142644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x561e73c1fc60 con 0x561e73ebba80 2023-07-21T22:36:39.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.545+0000 7f56caffd700 1 -- 172.21.15.107:0/1994142644 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d40cb020 con 0x561e73ebba80 2023-07-21T22:36:39.600 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.546+0000 7f56caffd700 1 -- 172.21.15.107:0/1994142644 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f56d409a3d0 con 0x561e73ebba80 2023-07-21T22:36:39.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.546+0000 7f56fdcd2b80 10 client.16653 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:39.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.546+0000 7f56fdcd2b80 10 client.16653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:39.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.546+0000 7f56fdcd2b80 20 client.16653 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:39.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.546+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1994142644 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561e73ed0a00 0x561e73ed2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.546+0000 7f56fdcd2b80 4 client.16653 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:39.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.546+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1994142644 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e73e01eb0 con 0x561e73ed0a00 2023-07-21T22:36:39.601 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.546+0000 7f56eaffd700 1 --2- 172.21.15.107:0/1994142644 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561e73ed0a00 0x561e73ed2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.602 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.546+0000 7f56caffd700 10 client.16653 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:39.602 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.546+0000 7f56eaffd700 1 --2- 172.21.15.107:0/1994142644 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561e73ed0a00 0x561e73ed2e10 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e78adf64fe936ba server_cookie=f7ea1ad37569cc12 in_seq=0 out_seq=0 2023-07-21T22:36:39.602 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56caffd700 1 -- 172.21.15.107:0/1994142644 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561e73e01eb0 con 0x561e73ed0a00 2023-07-21T22:36:39.602 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56caffd700 10 client.16653 handle_command_reply: tid=0 2023-07-21T22:36:39.602 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56fdcd2b80 1 client.16653 shutdown 2023-07-21T22:36:39.602 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1994142644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 msgr2=0x561e73ec50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.602 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1994142644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 0x561e73ec50f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f56d40ce090 tx=0x7f56d409b4b0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.602 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1994142644 shutdown_connections 2023-07-21T22:36:39.603 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1994142644 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561e73ed0a00 0x561e73ed2e10 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.603 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1994142644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebcff0 0x561e73ecc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.603 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56caffd700 0 client.16653 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:39.603 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1994142644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 0x561e73ec50f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.603 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1994142644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebc3d0 0x561e73ec5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.603 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1994142644 >> 172.21.15.107:0/1994142644 conn(0x561e73eacef0 msgr2=0x561e73eb1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.603 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1994142644 shutdown_connections 2023-07-21T22:36:39.603 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1994142644 wait complete. 2023-07-21T22:36:39.604 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.547+0000 7f56fdcd2b80 20 client.16653 trim_cache size 0 max 16384 2023-07-21T22:36:39.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56fdcd2b80 1 Processor -- start 2023-07-21T22:36:39.605 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56fdcd2b80 1 -- start start 2023-07-21T22:36:39.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56fdcd2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 0x561e73ebd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56fdcd2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebc3d0 0x561e73ecc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56fdcd2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebcff0 0x561e73eccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56fdcd2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e73ec4cc0 con 0x561e73ebba80 2023-07-21T22:36:39.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56fdcd2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e73ea9e50 con 0x561e73ebcff0 2023-07-21T22:36:39.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56fdcd2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e73eaa030 con 0x561e73ebc3d0 2023-07-21T22:36:39.606 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 0x561e73ebd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56eb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 0x561e73ebd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51294/0 (socket says 172.21.15.107:51294) 2023-07-21T22:36:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1049783450 learned_addr learned my addr 172.21.15.107:0/1049783450 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56eaffd700 1 --2- 172.21.15.107:0/1049783450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebc3d0 0x561e73ecc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56ebfff700 1 --2- 172.21.15.107:0/1049783450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebcff0 0x561e73eccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1049783450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebc3d0 msgr2=0x561e73ecc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/1049783450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebc3d0 0x561e73ecc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1049783450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebcff0 msgr2=0x561e73eccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.607 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/1049783450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebcff0 0x561e73eccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.560+0000 7f56eb7fe700 1 -- 172.21.15.107:0/1049783450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e73c5d250 con 0x561e73ebba80 2023-07-21T22:36:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.561+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/1049783450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 0x561e73ebd3c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f56d4097dc0 tx=0x7f56d4099900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb80a3cba70ee37 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.561+0000 7f56e27fc700 1 -- 172.21.15.107:0/1049783450 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d40b9340 con 0x561e73ebba80 2023-07-21T22:36:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.561+0000 7f56e27fc700 1 -- 172.21.15.107:0/1049783450 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d40cb030 con 0x561e73ebba80 2023-07-21T22:36:39.608 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.561+0000 7f56e27fc700 1 -- 172.21.15.107:0/1049783450 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d40a34e0 con 0x561e73ebba80 2023-07-21T22:36:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.561+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1049783450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 msgr2=0x561e73ebd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.561+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1049783450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 0x561e73ebd3c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f56d4097dc0 tx=0x7f56d4099900 comp rx=0 tx=0).stop 2023-07-21T22:36:39.611 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.561+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1049783450 shutdown_connections 2023-07-21T22:36:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.561+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1049783450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebcff0 0x561e73eccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.561+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1049783450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebba80 0x561e73ebd3c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.561+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/1049783450 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebc3d0 0x561e73ecc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.561+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1049783450 >> 172.21.15.107:0/1049783450 conn(0x561e73eacef0 msgr2=0x561e73ed7770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.561+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1049783450 shutdown_connections 2023-07-21T22:36:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.561+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/1049783450 wait complete. 2023-07-21T22:36:39.612 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56fdcd2b80 1 -- start start 2023-07-21T22:36:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56fdcd2b80 1 Processor -- start 2023-07-21T22:36:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56fdcd2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebba80 0x561e73ec8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56fdcd2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebc3d0 0x561e73ec8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56fdcd2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebcff0 0x561e73ecb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56fdcd2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e73ecbda0 con 0x561e73ebc3d0 2023-07-21T22:36:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56fdcd2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e73ecbf10 con 0x561e73ebba80 2023-07-21T22:36:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56fdcd2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e73ecc0d0 con 0x561e73ebcff0 2023-07-21T22:36:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56fdcd2b80 10 client.0 _ll_register_callbacks cb 0x561e73da6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56eaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebc3d0 0x561e73ec8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56eaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebc3d0 0x561e73ec8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51300/0 (socket says 172.21.15.107:51300) 2023-07-21T22:36:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56eaffd700 1 -- 172.21.15.107:0/941954242 learned_addr learned my addr 172.21.15.107:0/941954242 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebba80 0x561e73ec8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56ebfff700 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebcff0 0x561e73ecb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56eaffd700 1 -- 172.21.15.107:0/941954242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebcff0 msgr2=0x561e73ecb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56eaffd700 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebcff0 0x561e73ecb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56eaffd700 1 -- 172.21.15.107:0/941954242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebba80 msgr2=0x561e73ec8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56eaffd700 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebba80 0x561e73ec8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56eaffd700 1 -- 172.21.15.107:0/941954242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e73c1fc60 con 0x561e73ebc3d0 2023-07-21T22:36:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebba80 0x561e73ec8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.562+0000 7f56ebfff700 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebcff0 0x561e73ecb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:36:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56eaffd700 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebc3d0 0x561e73ec8840 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x561e73ed36f0 tx=0x7f56dc014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd9a9f2cee38bff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56caffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56caffd700 1 -- 172.21.15.107:0/941954242 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc028070 con 0x561e73ebc3d0 2023-07-21T22:36:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56caffd700 1 -- 172.21.15.107:0/941954242 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f56dc018070 con 0x561e73ebc3d0 2023-07-21T22:36:39.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/941954242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56d40a3710 con 0x561e73ebc3d0 2023-07-21T22:36:39.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56caffd700 1 -- 172.21.15.107:0/941954242 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56dc0285e0 con 0x561e73ebc3d0 2023-07-21T22:36:39.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56fdcd2b80 10 client.16659 fetch_fsmap learned FSMap version 46 2023-07-21T22:36:39.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/941954242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x561e73c352b0 con 0x561e73ebc3d0 2023-07-21T22:36:39.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56caffd700 1 -- 172.21.15.107:0/941954242 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d40a3710 con 0x561e73ebc3d0 2023-07-21T22:36:39.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56caffd700 1 -- 172.21.15.107:0/941954242 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 46) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f56dc003680 con 0x561e73ebc3d0 2023-07-21T22:36:39.616 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56fdcd2b80 10 client.16659 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:36:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56fdcd2b80 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56fdcd2b80 20 client.16659 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561e73f258d0 0x561e73f27ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56fdcd2b80 4 client.16659 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:36:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/941954242 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561e73dc4050 con 0x561e73f258d0 2023-07-21T22:36:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.563+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561e73f258d0 0x561e73f27ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56caffd700 10 client.16659 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56eb7fe700 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561e73f258d0 0x561e73f27ce0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb404f0f090f5b31 server_cookie=7bece18b09e51a8a in_seq=0 out_seq=0 2023-07-21T22:36:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56caffd700 1 -- 172.21.15.107:0/941954242 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561e73dc4050 con 0x561e73f258d0 2023-07-21T22:36:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56caffd700 10 client.16659 handle_command_reply: tid=0 2023-07-21T22:36:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56fdcd2b80 1 client.16659 shutdown 2023-07-21T22:36:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/941954242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebc3d0 msgr2=0x561e73ec8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebc3d0 0x561e73ec8840 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x561e73ed36f0 tx=0x7f56dc014070 comp rx=0 tx=0).stop 2023-07-21T22:36:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/941954242 shutdown_connections 2023-07-21T22:36:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x561e73f258d0 0x561e73f27ce0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561e73ebcff0 0x561e73ecb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56caffd700 0 client.16659 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:39.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561e73ebba80 0x561e73ec8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56fdcd2b80 1 --2- 172.21.15.107:0/941954242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e73ebc3d0 0x561e73ec8840 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/941954242 >> 172.21.15.107:0/941954242 conn(0x561e73eacef0 msgr2=0x561e73eccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/941954242 shutdown_connections 2023-07-21T22:36:39.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56fdcd2b80 1 -- 172.21.15.107:0/941954242 wait complete. 2023-07-21T22:36:39.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.564+0000 7f56fdcd2b80 20 client.16659 trim_cache size 0 max 16384 2023-07-21T22:36:39.620 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2023-07-21T22:36:39.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.565+0000 7f56f1b4d700 1 -- 172.21.15.107:0/1292425955 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56d80a1150 msgr2=0x7f56d80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.565+0000 7f56f1b4d700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56d80a1150 0x7f56d80a3600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009bf0 tx=0x7f56dc009260 comp rx=0 tx=0).stop 2023-07-21T22:36:39.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.565+0000 7f56f1b4d700 1 -- 172.21.15.107:0/1292425955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ec0ffaf0 msgr2=0x7f56ec195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:39.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.565+0000 7f56f1b4d700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ec0ffaf0 0x7f56ec195820 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f56d400b620 tx=0x7f56d4004730 comp rx=0 tx=0).stop 2023-07-21T22:36:39.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.565+0000 7f56f1b4d700 1 -- 172.21.15.107:0/1292425955 shutdown_connections 2023-07-21T22:36:39.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.565+0000 7f56f1b4d700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56ec101670 0x7f56ec19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.565+0000 7f56f1b4d700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56ec0ffaf0 0x7f56ec195820 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.565+0000 7f56f1b4d700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f56d80a1150 0x7f56d80a3600 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.565+0000 7f56f1b4d700 1 --2- 172.21.15.107:0/1292425955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56ec100ce0 0x7f56ec195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.565+0000 7f56f1b4d700 1 -- 172.21.15.107:0/1292425955 >> 172.21.15.107:0/1292425955 conn(0x7f56ec0fb130 msgr2=0x7f56ec0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.565+0000 7f56f1b4d700 1 -- 172.21.15.107:0/1292425955 shutdown_connections 2023-07-21T22:36:39.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:36:39.565+0000 7f56f1b4d700 1 -- 172.21.15.107:0/1292425955 wait complete. 2023-07-21T22:36:39.622 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:39 smithi107 ceph-mon[96543]: pgmap v521: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 539 op/s 2023-07-21T22:36:39.623 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:39 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1752713407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:36:39.624 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 21158943, 'ino': 1099511634863, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-17.xml'}, {'damage_type': 'backtrace', 'id': 25974584, 'ino': 1099511629283, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-35.xml'}, {'damage_type': 'backtrace', 'id': 31908056, 'ino': 4398046515446, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-28.jpg'}, {'damage_type': 'backtrace', 'id': 61627180, 'ino': 1099511634799, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-72.xml'}, {'damage_type': 'backtrace', 'id': 138774222, 'ino': 4398046515730, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-5.jpg'}, {'damage_type': 'backtrace', 'id': 141611757, 'ino': 1099511634877, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-3.xml'}, {'damage_type': 'backtrace', 'id': 160011544, 'ino': 1099511634879, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-1.xml'}, {'damage_type': 'backtrace', 'id': 168016027, 'ino': 1099511634876, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-4.xml'}, {'damage_type': 'backtrace', 'id': 174210707, 'ino': 1099511634835, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-41.xml'}, {'damage_type': 'backtrace', 'id': 194437862, 'ino': 1099511634811, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-63.xml'}, {'damage_type': 'backtrace', 'id': 211376050, 'ino': 4398046515309, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-24.xml'}, {'damage_type': 'backtrace', 'id': 222531143, 'ino': 1099511634847, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-30.xml'}, {'damage_type': 'backtrace', 'id': 243221646, 'ino': 4398046515713, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-6.jpg'}, {'damage_type': 'backtrace', 'id': 267834194, 'ino': 4398046515743, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-4.jpg'}, {'damage_type': 'backtrace', 'id': 295341970, 'ino': 1099511634812, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-62.xml'}, {'damage_type': 'backtrace', 'id': 302570097, 'ino': 1099511634875, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-5.xml'}, {'damage_type': 'backtrace', 'id': 323006527, 'ino': 1099511634805, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-67.xml'}, {'damage_type': 'backtrace', 'id': 324385409, 'ino': 1099511634845, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-32.xml'}, {'damage_type': 'backtrace', 'id': 344649510, 'ino': 1099511634800, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-71.xml'}, {'damage_type': 'backtrace', 'id': 347466352, 'ino': 1099511634871, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-9.xml'}, {'damage_type': 'backtrace', 'id': 361854996, 'ino': 4398046515171, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-48.xml'}, {'damage_type': 'backtrace', 'id': 367296624, 'ino': 4398046515548, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-20.jpg'}, {'damage_type': 'backtrace', 'id': 375341505, 'ino': 1099511634815, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-59.xml'}, {'damage_type': 'backtrace', 'id': 380832038, 'ino': 4398046515124, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-56.xml'}, {'damage_type': 'backtrace', 'id': 395763333, 'ino': 1099511634868, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-12.xml'}, {'damage_type': 'backtrace', 'id': 428677332, 'ino': 4398046515132, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-54.xml'}, {'damage_type': 'backtrace', 'id': 443836555, 'ino': 1099511634796, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-75.xml'}, {'damage_type': 'backtrace', 'id': 461659812, 'ino': 1099511634843, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-34.xml'}, {'damage_type': 'backtrace', 'id': 480016157, 'ino': 1099511634793, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-78.xml'}, {'damage_type': 'backtrace', 'id': 496024734, 'ino': 1099511634849, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-28.xml'}, {'damage_type': 'backtrace', 'id': 568675816, 'ino': 4398046515377, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-11.xml'}, {'damage_type': 'backtrace', 'id': 578955250, 'ino': 1099511634866, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-14.xml'}, {'damage_type': 'backtrace', 'id': 585523635, 'ino': 4398046515391, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-8.xml'}, {'damage_type': 'backtrace', 'id': 588094203, 'ino': 4398046515209, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-42.xml'}, {'damage_type': 'backtrace', 'id': 613016252, 'ino': 1099511634299, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-78.xml'}, {'damage_type': 'backtrace', 'id': 623542076, 'ino': 1099511634857, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-21.xml'}, {'damage_type': 'backtrace', 'id': 657496010, 'ino': 4398046515092, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-66.xml'}, {'damage_type': 'backtrace', 'id': 708926443, 'ino': 4398046515203, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-43.xml'}, {'damage_type': 'backtrace', 'id': 715203108, 'ino': 4398046515300, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-26.xml'}, {'damage_type': 'backtrace', 'id': 738762119, 'ino': 1099511634825, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-51.xml'}, {'damage_type': 'backtrace', 'id': 740251588, 'ino': 1099511634850, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-27.xml'}, {'damage_type': 'backtrace', 'id': 752934324, 'ino': 1099511636425, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-46.jpg'}, {'damage_type': 'backtrace', 'id': 765643061, 'ino': 4398046515612, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-14.jpg'}, {'damage_type': 'backtrace', 'id': 795631500, 'ino': 1099511634870, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-10.xml'}, {'damage_type': 'backtrace', 'id': 858908003, 'ino': 4398046515263, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-33.xml'}, {'damage_type': 'backtrace', 'id': 876257260, 'ino': 4398046515328, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-20.xml'}, {'damage_type': 'backtrace', 'id': 878022718, 'ino': 4398046515419, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-3.xml'}, {'damage_type': 'backtrace', 'id': 907065582, 'ino': 1099511634861, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-18.xml'}, {'damage_type': 'backtrace', 'id': 943144400, 'ino': 4398046515227, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-39.xml'}, {'damage_type': 'backtrace', 'id': 964608954, 'ino': 4398046515559, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-19.jpg'}, {'damage_type': 'backtrace', 'id': 976186631, 'ino': 4398046515268, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-32.xml'}, {'damage_type': 'backtrace', 'id': 977969131, 'ino': 4398046515289, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-28.xml'}, {'damage_type': 'backtrace', 'id': 985344231, 'ino': 1099511634798, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-73.xml'}, {'damage_type': 'backtrace', 'id': 992645308, 'ino': 1099511634838, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-39.xml'}, {'damage_type': 'backtrace', 'id': 999203861, 'ino': 4398046515150, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-51.xml'}, {'damage_type': 'backtrace', 'id': 1049285366, 'ino': 4398046515589, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-16.jpg'}, {'damage_type': 'backtrace', 'id': 1054292580, 'ino': 1099511634840, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-37.xml'}, {'damage_type': 'backtrace', 'id': 1087056602, 'ino': 1099511634818, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-57.xml'}, {'damage_type': 'backtrace', 'id': 1088393615, 'ino': 1099511631589, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-41.xml'}, {'damage_type': 'backtrace', 'id': 1107808556, 'ino': 1099511634821, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-54.xml'}, {'damage_type': 'backtrace', 'id': 1119225795, 'ino': 4398046515765, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-1.jpg'}, {'damage_type': 'backtrace', 'id': 1131511747, 'ino': 4398046515096, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-65.xml'}, {'damage_type': 'backtrace', 'id': 1211283891, 'ino': 4398046515076, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-69.xml'}, {'damage_type': 'backtrace', 'id': 1235275062, 'ino': 1099511634865, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-15.xml'}, {'damage_type': 'backtrace', 'id': 1269446848, 'ino': 1099511634807, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-65.xml'}, {'damage_type': 'backtrace', 'id': 1272929557, 'ino': 4398046515127, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-55.xml'}, {'damage_type': 'backtrace', 'id': 1287213455, 'ino': 4398046515758, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-2.jpg'}, {'damage_type': 'backtrace', 'id': 1300335612, 'ino': 1099511634823, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-52.xml'}, {'damage_type': 'backtrace', 'id': 1317805414, 'ino': 4398046515237, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-37.xml'}, {'damage_type': 'backtrace', 'id': 1337400957, 'ino': 1099511634851, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-26.xml'}, {'damage_type': 'backtrace', 'id': 1359043961, 'ino': 4398046515370, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-12.xml'}, {'damage_type': 'backtrace', 'id': 1389591493, 'ino': 4398046515323, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-21.xml'}, {'damage_type': 'backtrace', 'id': 1410364704, 'ino': 4398046515117, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-59.xml'}, {'damage_type': 'backtrace', 'id': 1423255978, 'ino': 4398046515110, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-62.xml'}, {'damage_type': 'backtrace', 'id': 1453004066, 'ino': 1099511634853, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-25.xml'}, {'damage_type': 'backtrace', 'id': 1466241590, 'ino': 4398046515314, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-23.xml'}, {'damage_type': 'backtrace', 'id': 1471048035, 'ino': 4398046515498, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-72.xml'}, {'damage_type': 'backtrace', 'id': 1517563796, 'ino': 4398046515221, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-40.xml'}, {'damage_type': 'backtrace', 'id': 1521826208, 'ino': 1099511634855, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-23.xml'}, {'damage_type': 'backtrace', 'id': 1528736858, 'ino': 4398046515304, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-25.xml'}, {'damage_type': 'backtrace', 'id': 1561203631, 'ino': 4398046515276, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-31.xml'}, {'damage_type': 'backtrace', 'id': 1571307189, 'ino': 1099511634804, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-68.xml'}, {'damage_type': 'backtrace', 'id': 1608293541, 'ino': 1099511634794, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-77.xml'}, {'damage_type': 'backtrace', 'id': 1624357881, 'ino': 1099511634846, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-31.xml'}, {'damage_type': 'backtrace', 'id': 1655471853, 'ino': 1099511634819, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-56.xml'}, {'damage_type': 'backtrace', 'id': 1658642625, 'ino': 1099511633961, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-63.xml'}, {'damage_type': 'backtrace', 'id': 1666272466, 'ino': 1099511634806, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-66.xml'}, {'damage_type': 'backtrace', 'id': 1668876062, 'ino': 4398046515535, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-21.jpg'}, {'damage_type': 'backtrace', 'id': 1677093771, 'ino': 4398046515155, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-50.xml'}, {'damage_type': 'backtrace', 'id': 1739378643, 'ino': 1099511634873, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-7.xml'}, {'damage_type': 'backtrace', 'id': 1760843571, 'ino': 4398046515699, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-7.jpg'}, {'damage_type': 'backtrace', 'id': 1829749078, 'ino': 4398046515395, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-7.xml'}, {'damage_type': 'backtrace', 'id': 1869085673, 'ino': 4398046515068, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-70.xml'}, {'damage_type': 'backtrace', 'id': 1885023847, 'ino': 4398046515191, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-45.xml'}, {'damage_type': 'backtrace', 'id': 1895457319, 'ino': 1099511633167, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-83.xml'}, {'damage_type': 'backtrace', 'id': 1928418528, 'ino': 1099511631018, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-58.xml'}, {'damage_type': 'backtrace', 'id': 1957623582, 'ino': 4398046515293, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-27.xml'}, {'damage_type': 'backtrace', 'id': 2005603817, 'ino': 1099511634828, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-48.xml'}, {'damage_type': 'backtrace', 'id': 2051232131, 'ino': 1099511634858, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-20.xml'}, {'damage_type': 'backtrace', 'id': 2065195366, 'ino': 1099511629124, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-66.jpg'}, {'damage_type': 'backtrace', 'id': 2120699241, 'ino': 4398046515385, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-9.xml'}, {'damage_type': 'backtrace', 'id': 2156555037, 'ino': 4398046515182, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-46.xml'}, {'damage_type': 'backtrace', 'id': 2165579709, 'ino': 4398046515567, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-18.jpg'}, {'damage_type': 'backtrace', 'id': 2208735844, 'ino': 1099511634813, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-61.xml'}, {'damage_type': 'backtrace', 'id': 2214003741, 'ino': 4398046515599, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-15.jpg'}, {'damage_type': 'backtrace', 'id': 2239402363, 'ino': 1099511634797, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-74.xml'}, {'damage_type': 'backtrace', 'id': 2245802303, 'ino': 1099511634837, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-40.xml'}, {'damage_type': 'backtrace', 'id': 2285163168, 'ino': 1099511634856, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-22.xml'}, {'damage_type': 'backtrace', 'id': 2331719281, 'ino': 4398046515231, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-38.xml'}, {'damage_type': 'backtrace', 'id': 2354112475, 'ino': 4398046515063, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-71.xml'}, {'damage_type': 'backtrace', 'id': 2386303897, 'ino': 4398046515413, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-4.xml'}, {'damage_type': 'backtrace', 'id': 2441547885, 'ino': 4398046515364, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-13.xml'}, {'damage_type': 'backtrace', 'id': 2441889471, 'ino': 1099511634826, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-50.xml'}, {'damage_type': 'backtrace', 'id': 2466041789, 'ino': 4398046515625, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-13.jpg'}, {'damage_type': 'backtrace', 'id': 2473796724, 'ino': 1099511634839, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-38.xml'}, {'damage_type': 'backtrace', 'id': 2505816910, 'ino': 4398046515176, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-47.xml'}, {'damage_type': 'backtrace', 'id': 2559160667, 'ino': 4398046515407, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-5.xml'}, {'damage_type': 'backtrace', 'id': 2572750535, 'ino': 1099511636426, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-85.xml'}, {'damage_type': 'backtrace', 'id': 2579503660, 'ino': 4398046515086, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-67.xml'}, {'damage_type': 'backtrace', 'id': 2647111129, 'ino': 4398046515338, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-18.xml'}, {'damage_type': 'backtrace', 'id': 2669850736, 'ino': 4398046515348, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-16.xml'}, {'damage_type': 'backtrace', 'id': 2680093729, 'ino': 4398046515102, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-64.xml'}, {'damage_type': 'backtrace', 'id': 2718829971, 'ino': 4398046515163, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-49.xml'}, {'damage_type': 'backtrace', 'id': 2743538238, 'ino': 4398046515401, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-6.xml'}, {'damage_type': 'backtrace', 'id': 2774113112, 'ino': 1099511634814, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-60.xml'}, {'damage_type': 'backtrace', 'id': 2780800367, 'ino': 4398046515380, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-10.xml'}, {'damage_type': 'backtrace', 'id': 2812491824, 'ino': 4398046515284, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-29.xml'}, {'damage_type': 'backtrace', 'id': 2860300081, 'ino': 4398046515515, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-23.jpg'}, {'damage_type': 'backtrace', 'id': 2898166882, 'ino': 1099511634834, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-42.xml'}, {'damage_type': 'backtrace', 'id': 2923936086, 'ino': 4398046515664, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-10.jpg'}, {'damage_type': 'backtrace', 'id': 2928718824, 'ino': 1099511634848, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-29.xml'}, {'damage_type': 'backtrace', 'id': 2963561912, 'ino': 1099511634792, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-79.xml'}, {'damage_type': 'backtrace', 'id': 2978790822, 'ino': 1099511634832, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-44.xml'}, {'damage_type': 'backtrace', 'id': 2989524817, 'ino': 1099511634867, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-13.xml'}, {'damage_type': 'backtrace', 'id': 2990529164, 'ino': 4398046515114, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-61.xml'}, {'damage_type': 'backtrace', 'id': 3021834678, 'ino': 4398046515242, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-36.xml'}, {'damage_type': 'backtrace', 'id': 3064113566, 'ino': 1099511631007, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-81.xml'}, {'damage_type': 'backtrace', 'id': 3135603824, 'ino': 4398046515689, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-8.jpg'}, {'damage_type': 'backtrace', 'id': 3157144195, 'ino': 1099511634854, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-24.xml'}, {'damage_type': 'backtrace', 'id': 3166955163, 'ino': 1099511631588, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-90.jpg'}, {'damage_type': 'backtrace', 'id': 3167362209, 'ino': 1099511629785, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-99.xml'}, {'damage_type': 'backtrace', 'id': 3204447757, 'ino': 4398046515457, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-27.jpg'}, {'damage_type': 'backtrace', 'id': 3215996262, 'ino': 4398046515528, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-22.jpg'}, {'damage_type': 'backtrace', 'id': 3253450795, 'ino': 4398046515423, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-2.xml'}, {'damage_type': 'backtrace', 'id': 3267134901, 'ino': 4398046515482, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-25.jpg'}, {'damage_type': 'backtrace', 'id': 3280909657, 'ino': 4398046515142, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-52.xml'}, {'damage_type': 'backtrace', 'id': 3283906153, 'ino': 1099511634820, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-55.xml'}, {'damage_type': 'backtrace', 'id': 3314249874, 'ino': 4398046515280, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-30.xml'}, {'damage_type': 'backtrace', 'id': 3325824332, 'ino': 1099511634803, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-69.xml'}, {'damage_type': 'backtrace', 'id': 3327615099, 'ino': 1099511634860, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-19.xml'}, {'damage_type': 'backtrace', 'id': 3398446860, 'ino': 1099511634827, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-49.xml'}, {'damage_type': 'backtrace', 'id': 3414343753, 'ino': 1099511634816, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-58.xml'}, {'damage_type': 'backtrace', 'id': 3423857029, 'ino': 4398046515676, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-9.jpg'}, {'damage_type': 'backtrace', 'id': 3456523614, 'ino': 1099511631607, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-44.xml'}, {'damage_type': 'backtrace', 'id': 3459989461, 'ino': 1099511629125, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-91.xml'}, {'damage_type': 'backtrace', 'id': 3518898054, 'ino': 1099511634864, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-16.xml'}, {'damage_type': 'backtrace', 'id': 3527048183, 'ino': 1099511634831, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-45.xml'}, {'damage_type': 'backtrace', 'id': 3532503175, 'ino': 1099511631005, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-82.xml'}, {'damage_type': 'backtrace', 'id': 3560271532, 'ino': 1099511634874, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-6.xml'}, {'damage_type': 'backtrace', 'id': 3606345127, 'ino': 4398046515435, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-29.jpg'}, {'damage_type': 'backtrace', 'id': 3621783074, 'ino': 1099511634795, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-76.xml'}, {'damage_type': 'backtrace', 'id': 3636130301, 'ino': 1099511634833, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-43.xml'}, {'damage_type': 'backtrace', 'id': 3643424922, 'ino': 1099511627832, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-15.xml'}, {'damage_type': 'backtrace', 'id': 3654160732, 'ino': 1099511634810, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-64.xml'}, {'damage_type': 'backtrace', 'id': 3759825691, 'ino': 4398046515429, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-1.xml'}, {'damage_type': 'backtrace', 'id': 3763252841, 'ino': 4398046515651, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-11.jpg'}, {'damage_type': 'backtrace', 'id': 3787424579, 'ino': 4398046515343, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-17.xml'}, {'damage_type': 'backtrace', 'id': 3790930342, 'ino': 1099511631606, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-26.jpg'}, {'damage_type': 'backtrace', 'id': 3812567430, 'ino': 4398046515319, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-22.xml'}, {'damage_type': 'backtrace', 'id': 3814292104, 'ino': 1099511634869, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-11.xml'}, {'damage_type': 'backtrace', 'id': 3878270663, 'ino': 4398046515638, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-12.jpg'}, {'damage_type': 'backtrace', 'id': 3879481641, 'ino': 1099511634844, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-33.xml'}, {'damage_type': 'backtrace', 'id': 3895483771, 'ino': 1099511629784, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-38.jpg'}, {'damage_type': 'backtrace', 'id': 3925276455, 'ino': 4398046515358, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-14.xml'}, {'damage_type': 'backtrace', 'id': 3937935806, 'ino': 1099511631008, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-80.xml'}, {'damage_type': 'backtrace', 'id': 3982885179, 'ino': 1099511634841, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-36.xml'}, {'damage_type': 'backtrace', 'id': 3999793729, 'ino': 1099511633960, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-61.jpg'}, {'damage_type': 'backtrace', 'id': 4022925223, 'ino': 4398046515579, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-17.jpg'}, {'damage_type': 'backtrace', 'id': 4049028454, 'ino': 1099511634842, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-35.xml'}, {'damage_type': 'backtrace', 'id': 4070979809, 'ino': 1099511634830, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-46.xml'}, {'damage_type': 'backtrace', 'id': 4071847979, 'ino': 1099511634822, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-53.xml'}, {'damage_type': 'backtrace', 'id': 4120931537, 'ino': 4398046515137, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-53.xml'}, {'damage_type': 'backtrace', 'id': 4133368694, 'ino': 4398046515495, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-24.jpg'}, {'damage_type': 'backtrace', 'id': 4136146020, 'ino': 4398046515332, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-19.xml'}, {'damage_type': 'backtrace', 'id': 4140404699, 'ino': 1099511634878, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-2.xml'}, {'damage_type': 'backtrace', 'id': 4169533317, 'ino': 4398046515749, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/picture-3.jpg'}, {'damage_type': 'backtrace', 'id': 4190584471, 'ino': 4398046515122, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-57.xml'}, {'damage_type': 'backtrace', 'id': 4215026728, 'ino': 4398046515116, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-60.xml'}, {'damage_type': 'backtrace', 'id': 4240513615, 'ino': 4398046515080, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-68.xml'}, {'damage_type': 'backtrace', 'id': 4254070846, 'ino': 1099511634802, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-70.xml'}, {'damage_type': 'backtrace', 'id': 4254643373, 'ino': 4398046515256, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/article-34.xml'}, {'damage_type': 'backtrace', 'id': 4262721508, 'ino': 1099511634829, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-47.xml'}, {'damage_type': 'backtrace', 'id': 4264396562, 'ino': 1099511634872, 'path': '/volumes/_nogroup/sv_1/ae48aa3b-002e-416f-bd30-32c977bb3dd2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-21/comment-8.xml'}] 2023-07-21T22:36:39.624 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-07-21T22:36:39.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:39 smithi190 ceph-mon[98334]: pgmap v521: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 539 op/s 2023-07-21T22:36:39.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:39 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1752713407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:36:39.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:39 smithi132 ceph-mon[97863]: pgmap v521: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 539 op/s 2023-07-21T22:36:39.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:39 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1752713407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:36:41.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:41 smithi190 ceph-mon[98334]: pgmap v522: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 522 op/s 2023-07-21T22:36:41.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:41 smithi132 ceph-mon[97863]: pgmap v522: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 522 op/s 2023-07-21T22:36:41.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:41 smithi107 ceph-mon[96543]: pgmap v522: 97 pgs: 97 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 522 op/s 2023-07-21T22:36:43.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:43 smithi190 ceph-mon[98334]: pgmap v523: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 758 op/s 2023-07-21T22:36:43.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:43 smithi132 ceph-mon[97863]: pgmap v523: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 758 op/s 2023-07-21T22:36:43.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:43 smithi107 ceph-mon[96543]: pgmap v523: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 758 op/s 2023-07-21T22:36:45.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:45 smithi190 ceph-mon[98334]: pgmap v524: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 731 op/s 2023-07-21T22:36:45.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:45 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:45.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:45 smithi132 ceph-mon[97863]: pgmap v524: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 731 op/s 2023-07-21T22:36:45.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:45 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:45.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:45 smithi107 ceph-mon[96543]: pgmap v524: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 731 op/s 2023-07-21T22:36:45.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:45 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:36:47.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:47 smithi190 ceph-mon[98334]: pgmap v525: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 726 op/s 2023-07-21T22:36:47.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:47 smithi132 ceph-mon[97863]: pgmap v525: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 726 op/s 2023-07-21T22:36:47.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:47 smithi107 ceph-mon[96543]: pgmap v525: 97 pgs: 97 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 726 op/s 2023-07-21T22:36:49.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:49 smithi190 ceph-mon[98334]: pgmap v526: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 777 op/s 2023-07-21T22:36:49.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:49 smithi132 ceph-mon[97863]: pgmap v526: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 777 op/s 2023-07-21T22:36:49.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:49 smithi107 ceph-mon[96543]: pgmap v526: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 777 op/s 2023-07-21T22:36:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.648+0000 7fca11891700 1 -- 172.21.15.107:0/1580199110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca0c0ff5f0 msgr2=0x7fca0c106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.648+0000 7fca11891700 1 --2- 172.21.15.107:0/1580199110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca0c0ff5f0 0x7fca0c106ea0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fca0400b0a0 tx=0x7fca0400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.648+0000 7fca11891700 1 -- 172.21.15.107:0/1580199110 shutdown_connections 2023-07-21T22:36:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.648+0000 7fca11891700 1 --2- 172.21.15.107:0/1580199110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca0c0ff5f0 0x7fca0c106ea0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.648+0000 7fca11891700 1 --2- 172.21.15.107:0/1580199110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca0c0ffff0 0x7fca0c100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.648+0000 7fca11891700 1 --2- 172.21.15.107:0/1580199110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca0c0fec40 0x7fca0c0ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.648+0000 7fca11891700 1 -- 172.21.15.107:0/1580199110 >> 172.21.15.107:0/1580199110 conn(0x7fca0c0f97c0 msgr2=0x7fca0c0fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.648+0000 7fca11891700 1 -- 172.21.15.107:0/1580199110 shutdown_connections 2023-07-21T22:36:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.648+0000 7fca11891700 1 -- 172.21.15.107:0/1580199110 wait complete. 2023-07-21T22:36:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.649+0000 7fca11891700 1 Processor -- start 2023-07-21T22:36:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.649+0000 7fca11891700 1 -- start start 2023-07-21T22:36:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.649+0000 7fca11891700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca0c0fec40 0x7fca0c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.649+0000 7fca11891700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca0c0ff5f0 0x7fca0c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.649+0000 7fca11891700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca0c0ffff0 0x7fca0c19d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.649+0000 7fca11891700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c19d690 con 0x7fca0c0ffff0 2023-07-21T22:36:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.649+0000 7fca11891700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c19d7d0 con 0x7fca0c0ff5f0 2023-07-21T22:36:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.649+0000 7fca11891700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c19d990 con 0x7fca0c0fec40 2023-07-21T22:36:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.649+0000 7fca0b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca0c0ffff0 0x7fca0c19d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.649+0000 7fca0a7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca0c0ff5f0 0x7fca0c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.649+0000 7fca0b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca0c0ffff0 0x7fca0c19d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51328/0 (socket says 172.21.15.107:51328) 2023-07-21T22:36:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.649+0000 7fca0b7fe700 1 -- 172.21.15.107:0/2957168288 learned_addr learned my addr 172.21.15.107:0/2957168288 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.649+0000 7fca0a7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca0c0ff5f0 0x7fca0c19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:42154/0 (socket says 172.21.15.107:42154) 2023-07-21T22:36:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.650+0000 7fca0affd700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca0c0fec40 0x7fca0c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.650+0000 7fca0b7fe700 1 -- 172.21.15.107:0/2957168288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca0c0fec40 msgr2=0x7fca0c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.650+0000 7fca0b7fe700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca0c0fec40 0x7fca0c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.650+0000 7fca0b7fe700 1 -- 172.21.15.107:0/2957168288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca0c0ff5f0 msgr2=0x7fca0c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.650+0000 7fca0b7fe700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca0c0ff5f0 0x7fca0c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.650+0000 7fca0b7fe700 1 -- 172.21.15.107:0/2957168288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca04009d00 con 0x7fca0c0ffff0 2023-07-21T22:36:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.650+0000 7fca0a7fc700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca0c0ff5f0 0x7fca0c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.650+0000 7fca0b7fe700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca0c0ffff0 0x7fca0c19d0c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fca0400b780 tx=0x7fca0400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.650+0000 7fca1088f700 1 -- 172.21.15.107:0/2957168288 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0400e040 con 0x7fca0c0ffff0 2023-07-21T22:36:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.650+0000 7fca11891700 1 -- 172.21.15.107:0/2957168288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c19dc10 con 0x7fca0c0ffff0 2023-07-21T22:36:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.650+0000 7fca1088f700 1 -- 172.21.15.107:0/2957168288 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca040094a0 con 0x7fca0c0ffff0 2023-07-21T22:36:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.650+0000 7fca1088f700 1 -- 172.21.15.107:0/2957168288 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04005390 con 0x7fca0c0ffff0 2023-07-21T22:36:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.650+0000 7fca11891700 1 -- 172.21.15.107:0/2957168288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c19e0e0 con 0x7fca0c0ffff0 2023-07-21T22:36:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.652+0000 7fca1088f700 1 -- 172.21.15.107:0/2957168288 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fca0401d470 con 0x7fca0c0ffff0 2023-07-21T22:36:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.653+0000 7fca1088f700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc9f40a1150 0x7fc9f40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.653+0000 7fca0affd700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc9f40a1150 0x7fc9f40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.653+0000 7fca1088f700 1 -- 172.21.15.107:0/2957168288 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fca040cdf60 con 0x7fca0c0ffff0 2023-07-21T22:36:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.653+0000 7fca0affd700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc9f40a1150 0x7fc9f40a3600 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0fe610 tx=0x7fc9fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca1da16b80 1 Processor -- start 2023-07-21T22:36:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca1da16b80 1 -- start start 2023-07-21T22:36:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca1da16b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad54656b0 0x559ad56092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca1da16b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad5609830 0x559ad560a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca1da16b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 0x559ad560a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca1da16b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad550ec50 con 0x559ad560a580 2023-07-21T22:36:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca1da16b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad55619e0 con 0x559ad5609830 2023-07-21T22:36:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca1da16b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad555ae70 con 0x559ad54656b0 2023-07-21T22:36:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca0b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 0x559ad560a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca0b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 0x559ad560a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51336/0 (socket says 172.21.15.107:51336) 2023-07-21T22:36:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca0b7fe700 1 -- 172.21.15.107:0/4272812509 learned_addr learned my addr 172.21.15.107:0/4272812509 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca0a7fc700 1 --2- 172.21.15.107:0/4272812509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad5609830 0x559ad560a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca0affd700 1 --2- 172.21.15.107:0/4272812509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad54656b0 0x559ad56092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca0b7fe700 1 -- 172.21.15.107:0/4272812509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad54656b0 msgr2=0x559ad56092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca0b7fe700 1 --2- 172.21.15.107:0/4272812509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad54656b0 0x559ad56092f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca0b7fe700 1 -- 172.21.15.107:0/4272812509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad5609830 msgr2=0x559ad560a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca0b7fe700 1 --2- 172.21.15.107:0/4272812509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad5609830 0x559ad560a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca0b7fe700 1 -- 172.21.15.107:0/4272812509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ad5388d80 con 0x559ad560a580 2023-07-21T22:36:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fca0b7fe700 1 --2- 172.21.15.107:0/4272812509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 0x559ad560a9f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fca0409b080 tx=0x7fca04095f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35abb70c627c6d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fc9f1ffb700 1 -- 172.21.15.107:0/4272812509 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca040cbea0 con 0x559ad560a580 2023-07-21T22:36:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fc9f1ffb700 1 -- 172.21.15.107:0/4272812509 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca0409ace0 con 0x559ad560a580 2023-07-21T22:36:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.654+0000 7fc9f1ffb700 1 -- 172.21.15.107:0/4272812509 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0401d720 con 0x559ad560a580 2023-07-21T22:36:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.655+0000 7fca1da16b80 1 -- 172.21.15.107:0/4272812509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 msgr2=0x559ad560a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.655+0000 7fca1da16b80 1 --2- 172.21.15.107:0/4272812509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 0x559ad560a9f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fca0409b080 tx=0x7fca04095f90 comp rx=0 tx=0).stop 2023-07-21T22:36:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.655+0000 7fca1da16b80 1 -- 172.21.15.107:0/4272812509 shutdown_connections 2023-07-21T22:36:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.655+0000 7fca1da16b80 1 --2- 172.21.15.107:0/4272812509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 0x559ad560a9f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.655+0000 7fca1da16b80 1 --2- 172.21.15.107:0/4272812509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad54656b0 0x559ad56092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.655+0000 7fca1da16b80 1 --2- 172.21.15.107:0/4272812509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad5609830 0x559ad560a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.655+0000 7fca1da16b80 1 -- 172.21.15.107:0/4272812509 >> 172.21.15.107:0/4272812509 conn(0x559ad55feae0 msgr2=0x559ad5603c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.655+0000 7fca1da16b80 1 -- 172.21.15.107:0/4272812509 shutdown_connections 2023-07-21T22:36:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.655+0000 7fca1da16b80 1 -- 172.21.15.107:0/4272812509 wait complete. 2023-07-21T22:36:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.655+0000 7fca1da16b80 1 -- start start 2023-07-21T22:36:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca1da16b80 1 Processor -- start 2023-07-21T22:36:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca1da16b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad54656b0 0x559ad56125c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca1da16b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad5609830 0x559ad5612b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca1da16b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad560a580 0x559ad5619b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca1da16b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad54faa60 con 0x559ad5609830 2023-07-21T22:36:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca1da16b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad54fa760 con 0x559ad54656b0 2023-07-21T22:36:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca1da16b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad53de8c0 con 0x559ad560a580 2023-07-21T22:36:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca1da16b80 10 client.0 _ll_register_callbacks cb 0x559ad54f8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca0affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad54656b0 0x559ad56125c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca0a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad5609830 0x559ad5612b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca0affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad54656b0 0x559ad56125c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:42166/0 (socket says 172.21.15.107:42166) 2023-07-21T22:36:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca0affd700 1 -- 172.21.15.107:0/1115600633 learned_addr learned my addr 172.21.15.107:0/1115600633 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca0b7fe700 1 --2- 172.21.15.107:0/1115600633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad560a580 0x559ad5619b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca0affd700 1 -- 172.21.15.107:0/1115600633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad560a580 msgr2=0x559ad5619b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca0affd700 1 --2- 172.21.15.107:0/1115600633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad560a580 0x559ad5619b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca0affd700 1 -- 172.21.15.107:0/1115600633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad5609830 msgr2=0x559ad5612b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca0affd700 1 --2- 172.21.15.107:0/1115600633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad5609830 0x559ad5612b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca0affd700 1 -- 172.21.15.107:0/1115600633 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c19e0e0 con 0x559ad54656b0 2023-07-21T22:36:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.656+0000 7fca0a7fc700 1 --2- 172.21.15.107:0/1115600633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad5609830 0x559ad5612b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fca0affd700 1 --2- 172.21.15.107:0/1115600633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad54656b0 0x559ad56125c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc01e7a0 tx=0x7fc9fc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d60e15e3155f79c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fc9eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-07-21T22:36:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fc9eaffd700 1 -- 172.21.15.107:0/1115600633 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc004220 con 0x559ad54656b0 2023-07-21T22:36:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fca1da16b80 1 -- 172.21.15.107:0/1115600633 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ad54b4490 con 0x559ad54656b0 2023-07-21T22:36:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fc9eaffd700 1 -- 172.21.15.107:0/1115600633 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc9fc004380 con 0x559ad54656b0 2023-07-21T22:36:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fc9eaffd700 1 -- 172.21.15.107:0/1115600633 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9fc004540 con 0x559ad54656b0 2023-07-21T22:36:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fca1da16b80 10 client.26213 fetch_fsmap learned FSMap version 47 2023-07-21T22:36:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fca1da16b80 1 -- 172.21.15.107:0/1115600633 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x559ad5371c00 con 0x559ad54656b0 2023-07-21T22:36:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fc9eaffd700 1 -- 172.21.15.107:0/1115600633 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ad54b4490 con 0x559ad54656b0 2023-07-21T22:36:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fc9eaffd700 1 -- 172.21.15.107:0/1115600633 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 47) v1 ==== 7530+0+0 (secure 0 0 0) 0x7fc9fc030020 con 0x559ad54656b0 2023-07-21T22:36:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fca1da16b80 10 client.26213 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T22:36:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fca1da16b80 10 client.26213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fca1da16b80 20 client.26213 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fca1da16b80 1 --2- 172.21.15.107:0/1115600633 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ad561e500 0x559ad5620910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fca1da16b80 4 client.26213 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:36:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.657+0000 7fca1da16b80 1 -- 172.21.15.107:0/1115600633 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ad5553e50 con 0x559ad561e500 2023-07-21T22:36:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.658+0000 7fca0a7fc700 1 --2- 172.21.15.107:0/1115600633 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ad561e500 0x559ad5620910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.658+0000 7fc9eaffd700 10 client.26213 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.658+0000 7fca0a7fc700 1 --2- 172.21.15.107:0/1115600633 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ad561e500 0x559ad5620910 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d35d64e263a27a45 server_cookie=1d20401e6c2283d8 in_seq=0 out_seq=0 2023-07-21T22:36:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.658+0000 7fc9eaffd700 1 -- 172.21.15.107:0/1115600633 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559ad5553e50 con 0x559ad561e500 2023-07-21T22:36:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.658+0000 7fc9eaffd700 10 client.26213 handle_command_reply: tid=0 2023-07-21T22:36:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.659+0000 7fca1da16b80 1 client.26213 shutdown 2023-07-21T22:36:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.659+0000 7fca1da16b80 1 -- 172.21.15.107:0/1115600633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad54656b0 msgr2=0x559ad56125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.659+0000 7fca1da16b80 1 --2- 172.21.15.107:0/1115600633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad54656b0 0x559ad56125c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc01e7a0 tx=0x7fc9fc0047c0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.659+0000 7fca1da16b80 1 -- 172.21.15.107:0/1115600633 shutdown_connections 2023-07-21T22:36:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.659+0000 7fca1da16b80 1 --2- 172.21.15.107:0/1115600633 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ad561e500 0x559ad5620910 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.659+0000 7fca1da16b80 1 --2- 172.21.15.107:0/1115600633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad560a580 0x559ad5619b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.659+0000 7fca1da16b80 1 --2- 172.21.15.107:0/1115600633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad54656b0 0x559ad56125c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.659+0000 7fc9eaffd700 0 client.26213 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.659+0000 7fca1da16b80 1 --2- 172.21.15.107:0/1115600633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad5609830 0x559ad5612b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.659+0000 7fca1da16b80 1 -- 172.21.15.107:0/1115600633 >> 172.21.15.107:0/1115600633 conn(0x559ad55feae0 msgr2=0x559ad56033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.659+0000 7fca1da16b80 1 -- 172.21.15.107:0/1115600633 shutdown_connections 2023-07-21T22:36:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.659+0000 7fca1da16b80 1 -- 172.21.15.107:0/1115600633 wait complete. 2023-07-21T22:36:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.659+0000 7fca1da16b80 20 client.26213 trim_cache size 0 max 16384 2023-07-21T22:36:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.673+0000 7fca1da16b80 1 Processor -- start 2023-07-21T22:36:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.673+0000 7fca1da16b80 1 -- start start 2023-07-21T22:36:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca1da16b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad54656b0 0x559ad5612370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca1da16b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad5609830 0x559ad5619ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.720 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca1da16b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 0x559ad56128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca1da16b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad561b7d0 con 0x559ad560a580 2023-07-21T22:36:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca1da16b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad5608f40 con 0x559ad54656b0 2023-07-21T22:36:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca1da16b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad561e090 con 0x559ad5609830 2023-07-21T22:36:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca0b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 0x559ad56128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca0affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad54656b0 0x559ad5612370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca0b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 0x559ad56128b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51350/0 (socket says 172.21.15.107:51350) 2023-07-21T22:36:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca0b7fe700 1 -- 172.21.15.107:0/917597159 learned_addr learned my addr 172.21.15.107:0/917597159 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca0a7fc700 1 --2- 172.21.15.107:0/917597159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad5609830 0x559ad5619ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca0b7fe700 1 -- 172.21.15.107:0/917597159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad5609830 msgr2=0x559ad5619ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca0b7fe700 1 --2- 172.21.15.107:0/917597159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad5609830 0x559ad5619ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca0b7fe700 1 -- 172.21.15.107:0/917597159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad54656b0 msgr2=0x559ad5612370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca0b7fe700 1 --2- 172.21.15.107:0/917597159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad54656b0 0x559ad5612370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca0b7fe700 1 -- 172.21.15.107:0/917597159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ad53af1f0 con 0x559ad560a580 2023-07-21T22:36:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca0a7fc700 1 --2- 172.21.15.107:0/917597159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad5609830 0x559ad5619ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:36:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fca0b7fe700 1 --2- 172.21.15.107:0/917597159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 0x559ad56128b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fca04026000 tx=0x7fca04009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a878937c6661186 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fc9f27fc700 1 -- 172.21.15.107:0/917597159 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca040cbea0 con 0x559ad560a580 2023-07-21T22:36:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fc9f27fc700 1 -- 172.21.15.107:0/917597159 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca0409b380 con 0x559ad560a580 2023-07-21T22:36:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.674+0000 7fc9f27fc700 1 -- 172.21.15.107:0/917597159 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca040a10f0 con 0x559ad560a580 2023-07-21T22:36:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.675+0000 7fca1da16b80 1 -- 172.21.15.107:0/917597159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 msgr2=0x559ad56128b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.675+0000 7fca1da16b80 1 --2- 172.21.15.107:0/917597159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 0x559ad56128b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fca04026000 tx=0x7fca04009c90 comp rx=0 tx=0).stop 2023-07-21T22:36:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.675+0000 7fca1da16b80 1 -- 172.21.15.107:0/917597159 shutdown_connections 2023-07-21T22:36:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.675+0000 7fca1da16b80 1 --2- 172.21.15.107:0/917597159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad560a580 0x559ad56128b0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.675+0000 7fca1da16b80 1 --2- 172.21.15.107:0/917597159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad54656b0 0x559ad5612370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.675+0000 7fca1da16b80 1 --2- 172.21.15.107:0/917597159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad5609830 0x559ad5619ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.675+0000 7fca1da16b80 1 -- 172.21.15.107:0/917597159 >> 172.21.15.107:0/917597159 conn(0x559ad55feae0 msgr2=0x559ad560d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.675+0000 7fca1da16b80 1 -- 172.21.15.107:0/917597159 shutdown_connections 2023-07-21T22:36:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.675+0000 7fca1da16b80 1 -- 172.21.15.107:0/917597159 wait complete. 2023-07-21T22:36:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.675+0000 7fca1da16b80 1 -- start start 2023-07-21T22:36:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.675+0000 7fca1da16b80 1 Processor -- start 2023-07-21T22:36:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca1da16b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad54656b0 0x559ad56157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca1da16b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad5609830 0x559ad5615d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca1da16b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad560a580 0x559ad5618d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca1da16b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad5619290 con 0x559ad54656b0 2023-07-21T22:36:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca1da16b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad5619400 con 0x559ad5609830 2023-07-21T22:36:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca1da16b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad56195c0 con 0x559ad560a580 2023-07-21T22:36:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca1da16b80 10 client.0 _ll_register_callbacks cb 0x7fc9fc011f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:36:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca0affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad54656b0 0x559ad56157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca0affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad54656b0 0x559ad56157f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51352/0 (socket says 172.21.15.107:51352) 2023-07-21T22:36:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca0affd700 1 -- 172.21.15.107:0/988339603 learned_addr learned my addr 172.21.15.107:0/988339603 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:36:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca0a7fc700 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad5609830 0x559ad5615d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca0b7fe700 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad560a580 0x559ad5618d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca0affd700 1 -- 172.21.15.107:0/988339603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad560a580 msgr2=0x559ad5618d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca0affd700 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad560a580 0x559ad5618d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca0affd700 1 -- 172.21.15.107:0/988339603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad5609830 msgr2=0x559ad5615d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:36:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca0affd700 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad5609830 0x559ad5615d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca0affd700 1 -- 172.21.15.107:0/988339603 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ad5388d80 con 0x559ad54656b0 2023-07-21T22:36:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca0a7fc700 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad5609830 0x559ad5615d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:36:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fca0affd700 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad54656b0 0x559ad56157f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc003ae0 tx=0x7fc9fc01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7814829b34c516de server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:36:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fc9eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:36:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.676+0000 7fc9eaffd700 1 -- 172.21.15.107:0/988339603 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc0117d0 con 0x559ad54656b0 2023-07-21T22:36:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fc9eaffd700 1 -- 172.21.15.107:0/988339603 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc9fc01cd90 con 0x559ad54656b0 2023-07-21T22:36:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fca1da16b80 1 -- 172.21.15.107:0/988339603 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca0409ace0 con 0x559ad54656b0 2023-07-21T22:36:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fc9eaffd700 1 -- 172.21.15.107:0/988339603 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9fc003d80 con 0x559ad54656b0 2023-07-21T22:36:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fca1da16b80 10 client.16677 fetch_fsmap learned FSMap version 47 2023-07-21T22:36:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fca1da16b80 1 -- 172.21.15.107:0/988339603 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x559ad5387250 con 0x559ad54656b0 2023-07-21T22:36:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fc9eaffd700 1 -- 172.21.15.107:0/988339603 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca0409ace0 con 0x559ad54656b0 2023-07-21T22:36:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fc9eaffd700 1 -- 172.21.15.107:0/988339603 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 47) v1 ==== 7530+0+0 (secure 0 0 0) 0x7fc9fc01e7d0 con 0x559ad54656b0 2023-07-21T22:36:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fca1da16b80 10 client.16677 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T22:36:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fca1da16b80 10 client.16677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:36:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fca1da16b80 20 client.16677 populate_metadata read hostname 'smithi107' 2023-07-21T22:36:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fca1da16b80 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ad56732e0 0x559ad56756c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:36:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fca1da16b80 4 client.16677 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:36:50.732 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fca1da16b80 1 -- 172.21.15.107:0/988339603 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559ad5515ff0 con 0x559ad56732e0 2023-07-21T22:36:50.732 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.677+0000 7fca0a7fc700 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ad56732e0 0x559ad56756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:36:50.732 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.678+0000 7fc9eaffd700 10 client.16677 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:50.732 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.678+0000 7fca0a7fc700 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ad56732e0 0x559ad56756c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4698b7928e1656dc server_cookie=279629cdc8cce9ce in_seq=0 out_seq=0 2023-07-21T22:36:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.690+0000 7fc9eaffd700 1 -- 172.21.15.107:0/988339603 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559ad5515ff0 con 0x559ad56732e0 2023-07-21T22:36:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.690+0000 7fc9eaffd700 10 client.16677 handle_command_reply: tid=0 2023-07-21T22:36:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.690+0000 7fca1da16b80 1 client.16677 shutdown 2023-07-21T22:36:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:36:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:36:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:36:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:36:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.690+0000 7fca1da16b80 1 -- 172.21.15.107:0/988339603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad54656b0 msgr2=0x559ad56157f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.690+0000 7fca1da16b80 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad54656b0 0x559ad56157f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc003ae0 tx=0x7fc9fc01c990 comp rx=0 tx=0).stop 2023-07-21T22:36:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fca1da16b80 1 -- 172.21.15.107:0/988339603 shutdown_connections 2023-07-21T22:36:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fca1da16b80 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x559ad56732e0 0x559ad56756c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fca1da16b80 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ad560a580 0x559ad5618d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fc9eaffd700 0 client.16677 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:36:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fca1da16b80 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ad54656b0 0x559ad56157f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fca1da16b80 1 --2- 172.21.15.107:0/988339603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ad5609830 0x559ad5615d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fca1da16b80 1 -- 172.21.15.107:0/988339603 >> 172.21.15.107:0/988339603 conn(0x559ad55feae0 msgr2=0x559ad5612370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fca1da16b80 1 -- 172.21.15.107:0/988339603 shutdown_connections 2023-07-21T22:36:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fca1da16b80 1 -- 172.21.15.107:0/988339603 wait complete. 2023-07-21T22:36:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fca1da16b80 20 client.16677 trim_cache size 0 max 16384 2023-07-21T22:36:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fca11891700 1 -- 172.21.15.107:0/2957168288 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc9f40a1150 msgr2=0x7fc9f40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fca11891700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc9f40a1150 0x7fc9f40a3600 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0fe610 tx=0x7fc9fc009260 comp rx=0 tx=0).stop 2023-07-21T22:36:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fca11891700 1 -- 172.21.15.107:0/2957168288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca0c0ffff0 msgr2=0x7fca0c19d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:36:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.691+0000 7fca11891700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca0c0ffff0 0x7fca0c19d0c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fca0400b780 tx=0x7fca0400bd60 comp rx=0 tx=0).stop 2023-07-21T22:36:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.692+0000 7fca11891700 1 -- 172.21.15.107:0/2957168288 shutdown_connections 2023-07-21T22:36:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.692+0000 7fca11891700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca0c0ffff0 0x7fca0c19d0c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.692+0000 7fca11891700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc9f40a1150 0x7fc9f40a3600 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.692+0000 7fca11891700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca0c0fec40 0x7fca0c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.692+0000 7fca11891700 1 --2- 172.21.15.107:0/2957168288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca0c0ff5f0 0x7fca0c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:36:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.692+0000 7fca11891700 1 -- 172.21.15.107:0/2957168288 >> 172.21.15.107:0/2957168288 conn(0x7fca0c0f97c0 msgr2=0x7fca0c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:36:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.692+0000 7fca11891700 1 -- 172.21.15.107:0/2957168288 shutdown_connections 2023-07-21T22:36:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:36:50.692+0000 7fca11891700 1 -- 172.21.15.107:0/2957168288 wait complete. 2023-07-21T22:36:51.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:51 smithi190 ceph-mon[98334]: pgmap v527: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 555 op/s 2023-07-21T22:36:51.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:51 smithi132 ceph-mon[97863]: pgmap v527: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 555 op/s 2023-07-21T22:36:51.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:51 smithi107 ceph-mon[96543]: pgmap v527: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 555 op/s 2023-07-21T22:36:53.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:53 smithi190 ceph-mon[98334]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 601 op/s 2023-07-21T22:36:53.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:53 smithi132 ceph-mon[97863]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 601 op/s 2023-07-21T22:36:53.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:53 smithi107 ceph-mon[96543]: pgmap v528: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 601 op/s 2023-07-21T22:36:55.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:55 smithi190 ceph-mon[98334]: pgmap v529: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 413 op/s 2023-07-21T22:36:55.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:55 smithi132 ceph-mon[97863]: pgmap v529: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 413 op/s 2023-07-21T22:36:55.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:55 smithi107 ceph-mon[96543]: pgmap v529: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 413 op/s 2023-07-21T22:36:57.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:57 smithi190 ceph-mon[98334]: pgmap v530: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 407 op/s 2023-07-21T22:36:57.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:57 smithi132 ceph-mon[97863]: pgmap v530: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 407 op/s 2023-07-21T22:36:57.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:57 smithi107 ceph-mon[96543]: pgmap v530: 97 pgs: 97 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 407 op/s 2023-07-21T22:36:59.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:36:59 smithi190 ceph-mon[98334]: pgmap v531: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 467 op/s 2023-07-21T22:36:59.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:36:59 smithi132 ceph-mon[97863]: pgmap v531: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 467 op/s 2023-07-21T22:36:59.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:36:59 smithi107 ceph-mon[96543]: pgmap v531: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 467 op/s 2023-07-21T22:37:00.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:00 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:00.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:00 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:00.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:00 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:01.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:01 smithi190 ceph-mon[98334]: pgmap v532: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 378 op/s 2023-07-21T22:37:01.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:01 smithi132 ceph-mon[97863]: pgmap v532: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 378 op/s 2023-07-21T22:37:01.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:01 smithi107 ceph-mon[96543]: pgmap v532: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 378 op/s 2023-07-21T22:37:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.803+0000 7f87a357e700 1 -- 172.21.15.107:0/226829701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f879c0ffd00 msgr2=0x7f879c1065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.803+0000 7f87a357e700 1 --2- 172.21.15.107:0/226829701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f879c0ffd00 0x7f879c1065c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f879400c210 tx=0x7f879400c520 comp rx=0 tx=0).stop 2023-07-21T22:37:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.803+0000 7f87a357e700 1 -- 172.21.15.107:0/226829701 shutdown_connections 2023-07-21T22:37:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.803+0000 7f87a357e700 1 --2- 172.21.15.107:0/226829701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f879c0ffd00 0x7f879c1065c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.803+0000 7f87a357e700 1 --2- 172.21.15.107:0/226829701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f879c0fe180 0x7f879c0fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.803+0000 7f87a357e700 1 --2- 172.21.15.107:0/226829701 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f879c0ff370 0x7f879c0ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.803+0000 7f87a357e700 1 -- 172.21.15.107:0/226829701 >> 172.21.15.107:0/226829701 conn(0x7f879c0f9780 msgr2=0x7f879c0fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.803+0000 7f87a357e700 1 -- 172.21.15.107:0/226829701 shutdown_connections 2023-07-21T22:37:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.803+0000 7f87a357e700 1 -- 172.21.15.107:0/226829701 wait complete. 2023-07-21T22:37:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a357e700 1 Processor -- start 2023-07-21T22:37:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a357e700 1 -- start start 2023-07-21T22:37:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a357e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f879c0fe180 0x7f879c1959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a357e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f879c0ff370 0x7f879c195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a357e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f879c0ffd00 0x7f879c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a357e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c197410 con 0x7f879c0ffd00 2023-07-21T22:37:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a357e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c197580 con 0x7f879c0ff370 2023-07-21T22:37:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a357e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c197740 con 0x7f879c0fe180 2023-07-21T22:37:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a1b1b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f879c0ffd00 0x7f879c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a1b1b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f879c0ffd00 0x7f879c196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51370/0 (socket says 172.21.15.107:51370) 2023-07-21T22:37:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a1b1b700 1 -- 172.21.15.107:0/2138570536 learned_addr learned my addr 172.21.15.107:0/2138570536 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a0b19700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f879c0ff370 0x7f879c195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a131a700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f879c0fe180 0x7f879c1959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a1b1b700 1 -- 172.21.15.107:0/2138570536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f879c0fe180 msgr2=0x7f879c1959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a1b1b700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f879c0fe180 0x7f879c1959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a1b1b700 1 -- 172.21.15.107:0/2138570536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f879c0ff370 msgr2=0x7f879c195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a1b1b700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f879c0ff370 0x7f879c195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.804+0000 7f87a1b1b700 1 -- 172.21.15.107:0/2138570536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879400ae10 con 0x7f879c0ffd00 2023-07-21T22:37:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.805+0000 7f87a1b1b700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f879c0ffd00 0x7f879c196db0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f879400a040 tx=0x7f8794008360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.805+0000 7f87927fc700 1 -- 172.21.15.107:0/2138570536 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879400f040 con 0x7f879c0ffd00 2023-07-21T22:37:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.805+0000 7f87927fc700 1 -- 172.21.15.107:0/2138570536 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f879400a5e0 con 0x7f879c0ffd00 2023-07-21T22:37:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.805+0000 7f87a357e700 1 -- 172.21.15.107:0/2138570536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c1979c0 con 0x7f879c0ffd00 2023-07-21T22:37:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.805+0000 7f87927fc700 1 -- 172.21.15.107:0/2138570536 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87940074a0 con 0x7f879c0ffd00 2023-07-21T22:37:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.805+0000 7f87a357e700 1 -- 172.21.15.107:0/2138570536 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c19e1e0 con 0x7f879c0ffd00 2023-07-21T22:37:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.807+0000 7f87927fc700 1 -- 172.21.15.107:0/2138570536 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8794005030 con 0x7f879c0ffd00 2023-07-21T22:37:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.808+0000 7f87927fc700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f87880a1150 0x7f87880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.808+0000 7f87927fc700 1 -- 172.21.15.107:0/2138570536 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f87940cdbd0 con 0x7f879c0ffd00 2023-07-21T22:37:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.808+0000 7f87a131a700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f87880a1150 0x7f87880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.808+0000 7f87af703b80 1 Processor -- start 2023-07-21T22:37:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87a131a700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f87880a1150 0x7f87880a3600 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f879c0ff1d0 tx=0x7f879800b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87af703b80 1 -- start start 2023-07-21T22:37:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87af703b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fec72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87af703b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec7830 0x55a81fec8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87af703b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec8580 0x55a81fec89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87af703b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81fdccc50 con 0x55a81fec8580 2023-07-21T22:37:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87af703b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81fe1f9e0 con 0x55a81fec7830 2023-07-21T22:37:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87af703b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81fe18e70 con 0x55a81fd236b0 2023-07-21T22:37:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87a131a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fec72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87a0b19700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec7830 0x55a81fec8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87a131a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fec72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:39932/0 (socket says 172.21.15.107:39932) 2023-07-21T22:37:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87a0b19700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec7830 0x55a81fec8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:42204/0 (socket says 172.21.15.107:42204) 2023-07-21T22:37:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87a1b1b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec8580 0x55a81fec89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87a131a700 1 -- 172.21.15.107:0/3303624203 learned_addr learned my addr 172.21.15.107:0/3303624203 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87a1b1b700 1 -- 172.21.15.107:0/3303624203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 msgr2=0x55a81fec72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87a1b1b700 1 --2- 172.21.15.107:0/3303624203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fec72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87a1b1b700 1 -- 172.21.15.107:0/3303624203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec7830 msgr2=0x55a81fec8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87a1b1b700 1 --2- 172.21.15.107:0/3303624203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec7830 0x55a81fec8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.809+0000 7f87a1b1b700 1 -- 172.21.15.107:0/3303624203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a81fc46d80 con 0x55a81fec8580 2023-07-21T22:37:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.810+0000 7f87a1b1b700 1 --2- 172.21.15.107:0/3303624203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec8580 0x55a81fec89f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f87940062a0 tx=0x7f879400aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf626856cdda135d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.810+0000 7f87877fe700 1 -- 172.21.15.107:0/3303624203 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879409f110 con 0x55a81fec8580 2023-07-21T22:37:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.810+0000 7f87877fe700 1 -- 172.21.15.107:0/3303624203 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87940a34a0 con 0x55a81fec8580 2023-07-21T22:37:03.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.810+0000 7f87877fe700 1 -- 172.21.15.107:0/3303624203 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87940979d0 con 0x55a81fec8580 2023-07-21T22:37:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.810+0000 7f87af703b80 1 -- 172.21.15.107:0/3303624203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec8580 msgr2=0x55a81fec89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.810+0000 7f87af703b80 1 --2- 172.21.15.107:0/3303624203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec8580 0x55a81fec89f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f87940062a0 tx=0x7f879400aaf0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 -- 172.21.15.107:0/3303624203 shutdown_connections 2023-07-21T22:37:03.866 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 --2- 172.21.15.107:0/3303624203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec8580 0x55a81fec89f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 --2- 172.21.15.107:0/3303624203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fec72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 --2- 172.21.15.107:0/3303624203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec7830 0x55a81fec8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 -- 172.21.15.107:0/3303624203 >> 172.21.15.107:0/3303624203 conn(0x55a81febcae0 msgr2=0x55a81fec1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 -- 172.21.15.107:0/3303624203 shutdown_connections 2023-07-21T22:37:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 -- 172.21.15.107:0/3303624203 wait complete. 2023-07-21T22:37:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 -- start start 2023-07-21T22:37:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 Processor -- start 2023-07-21T22:37:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fed0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec7830 0x55a81fed0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec8580 0x55a81fed79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81fdb8a60 con 0x55a81fec8580 2023-07-21T22:37:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81fdb8760 con 0x55a81fec7830 2023-07-21T22:37:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81fc9c8c0 con 0x55a81fd236b0 2023-07-21T22:37:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87af703b80 10 client.0 _ll_register_callbacks cb 0x55a81fdb6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87a1b1b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec8580 0x55a81fed79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87a131a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fed0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.811+0000 7f87a1b1b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec8580 0x55a81fed79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51384/0 (socket says 172.21.15.107:51384) 2023-07-21T22:37:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f87a1b1b700 1 -- 172.21.15.107:0/540867366 learned_addr learned my addr 172.21.15.107:0/540867366 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f87a0b19700 1 --2- 172.21.15.107:0/540867366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec7830 0x55a81fed0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f87a1b1b700 1 -- 172.21.15.107:0/540867366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 msgr2=0x55a81fed0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f87a1b1b700 1 --2- 172.21.15.107:0/540867366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fed0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f87a1b1b700 1 -- 172.21.15.107:0/540867366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec7830 msgr2=0x55a81fed0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f87a1b1b700 1 --2- 172.21.15.107:0/540867366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec7830 0x55a81fed0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f87a1b1b700 1 -- 172.21.15.107:0/540867366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c19e1e0 con 0x55a81fec8580 2023-07-21T22:37:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f87a1b1b700 1 --2- 172.21.15.107:0/540867366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec8580 0x55a81fed79c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f879400c1e0 tx=0x7f87940cb790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15081c893af6075 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f8784ff9700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:37:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f8784ff9700 1 -- 172.21.15.107:0/540867366 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879409f110 con 0x55a81fec8580 2023-07-21T22:37:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f8784ff9700 1 -- 172.21.15.107:0/540867366 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f87940b9310 con 0x55a81fec8580 2023-07-21T22:37:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f87af703b80 1 -- 172.21.15.107:0/540867366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a81fd72490 con 0x55a81fec8580 2023-07-21T22:37:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f8784ff9700 1 -- 172.21.15.107:0/540867366 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f879409b340 con 0x55a81fec8580 2023-07-21T22:37:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f87af703b80 10 client.16689 fetch_fsmap learned FSMap version 48 2023-07-21T22:37:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f87af703b80 1 -- 172.21.15.107:0/540867366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a81fc2fc00 con 0x55a81fec8580 2023-07-21T22:37:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f8784ff9700 1 -- 172.21.15.107:0/540867366 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8794066030 con 0x55a81fec8580 2023-07-21T22:37:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.812+0000 7f8784ff9700 1 -- 172.21.15.107:0/540867366 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 48) v1 ==== 7534+0+0 (secure 0 0 0) 0x7f87940a3010 con 0x55a81fec8580 2023-07-21T22:37:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.813+0000 7f87af703b80 10 client.16689 fetch_fsmap finished waiting for FSMap version 48 2023-07-21T22:37:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.813+0000 7f87af703b80 10 client.16689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:37:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.813+0000 7f87af703b80 20 client.16689 populate_metadata read hostname 'smithi107' 2023-07-21T22:37:03.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.813+0000 7f87af703b80 1 --2- 172.21.15.107:0/540867366 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55a81fedbe70 0x55a81fede280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.813+0000 7f87af703b80 4 client.16689 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:37:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.813+0000 7f87af703b80 1 -- 172.21.15.107:0/540867366 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a81fe11e50 con 0x55a81fedbe70 2023-07-21T22:37:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.813+0000 7f87a131a700 1 --2- 172.21.15.107:0/540867366 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55a81fedbe70 0x55a81fede280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.813+0000 7f87a131a700 1 --2- 172.21.15.107:0/540867366 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55a81fedbe70 0x55a81fede280 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d37f3ebfbfe7ccef server_cookie=e2e71b8359e5b7bf in_seq=0 out_seq=0 2023-07-21T22:37:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.813+0000 7f8784ff9700 10 client.16689 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:03.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f8784ff9700 1 -- 172.21.15.107:0/540867366 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a81fe11e50 con 0x55a81fedbe70 2023-07-21T22:37:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f8784ff9700 10 client.16689 handle_command_reply: tid=0 2023-07-21T22:37:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f87af703b80 1 client.16689 shutdown 2023-07-21T22:37:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f87af703b80 1 -- 172.21.15.107:0/540867366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec8580 msgr2=0x55a81fed79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f87af703b80 1 --2- 172.21.15.107:0/540867366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec8580 0x55a81fed79c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f879400c1e0 tx=0x7f87940cb790 comp rx=0 tx=0).stop 2023-07-21T22:37:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f87af703b80 1 -- 172.21.15.107:0/540867366 shutdown_connections 2023-07-21T22:37:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f87af703b80 1 --2- 172.21.15.107:0/540867366 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55a81fedbe70 0x55a81fede280 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f87af703b80 1 --2- 172.21.15.107:0/540867366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec8580 0x55a81fed79c0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f8784ff9700 0 client.16689 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f87af703b80 1 --2- 172.21.15.107:0/540867366 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fed0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f87af703b80 1 --2- 172.21.15.107:0/540867366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec7830 0x55a81fed0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f87af703b80 1 -- 172.21.15.107:0/540867366 >> 172.21.15.107:0/540867366 conn(0x55a81febcae0 msgr2=0x55a81fec13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f87af703b80 1 -- 172.21.15.107:0/540867366 shutdown_connections 2023-07-21T22:37:03.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f87af703b80 1 -- 172.21.15.107:0/540867366 wait complete. 2023-07-21T22:37:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.814+0000 7f87af703b80 20 client.16689 trim_cache size 0 max 16384 2023-07-21T22:37:03.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87af703b80 1 Processor -- start 2023-07-21T22:37:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87af703b80 1 -- start start 2023-07-21T22:37:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87af703b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fec6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87af703b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 0x55a81fed7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87af703b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec8580 0x55a81fec0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87af703b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81fecf8f0 con 0x55a81fec7830 2023-07-21T22:37:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87af703b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81fecffb0 con 0x55a81fec8580 2023-07-21T22:37:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87af703b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81fed15c0 con 0x55a81fd236b0 2023-07-21T22:37:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87a0b19700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 0x55a81fed7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87a0b19700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 0x55a81fed7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51390/0 (socket says 172.21.15.107:51390) 2023-07-21T22:37:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87a0b19700 1 -- 172.21.15.107:0/3591553465 learned_addr learned my addr 172.21.15.107:0/3591553465 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87a131a700 1 --2- 172.21.15.107:0/3591553465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fec6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87a0b19700 1 -- 172.21.15.107:0/3591553465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 msgr2=0x55a81fec6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87a1b1b700 1 --2- 172.21.15.107:0/3591553465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec8580 0x55a81fec0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87a0b19700 1 --2- 172.21.15.107:0/3591553465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fec6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87a0b19700 1 -- 172.21.15.107:0/3591553465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec8580 msgr2=0x55a81fec0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87a0b19700 1 --2- 172.21.15.107:0/3591553465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec8580 0x55a81fec0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87a0b19700 1 -- 172.21.15.107:0/3591553465 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a81fec83f0 con 0x55a81fec7830 2023-07-21T22:37:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f87a0b19700 1 --2- 172.21.15.107:0/3591553465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 0x55a81fed7980 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f878c009f30 tx=0x7f878c004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ae7d9d24bb009e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f8787fff700 1 -- 172.21.15.107:0/3591553465 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0039c0 con 0x55a81fec7830 2023-07-21T22:37:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f8787fff700 1 -- 172.21.15.107:0/3591553465 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f878c018040 con 0x55a81fec7830 2023-07-21T22:37:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.829+0000 7f8787fff700 1 -- 172.21.15.107:0/3591553465 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c01c3f0 con 0x55a81fec7830 2023-07-21T22:37:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.830+0000 7f87af703b80 1 -- 172.21.15.107:0/3591553465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 msgr2=0x55a81fed7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.830+0000 7f87af703b80 1 --2- 172.21.15.107:0/3591553465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 0x55a81fed7980 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f878c009f30 tx=0x7f878c004320 comp rx=0 tx=0).stop 2023-07-21T22:37:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.830+0000 7f87af703b80 1 -- 172.21.15.107:0/3591553465 shutdown_connections 2023-07-21T22:37:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.830+0000 7f87af703b80 1 --2- 172.21.15.107:0/3591553465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec8580 0x55a81fec0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.830+0000 7f87af703b80 1 --2- 172.21.15.107:0/3591553465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fec6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.830+0000 7f87af703b80 1 --2- 172.21.15.107:0/3591553465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 0x55a81fed7980 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.830+0000 7f87af703b80 1 -- 172.21.15.107:0/3591553465 >> 172.21.15.107:0/3591553465 conn(0x55a81febcae0 msgr2=0x55a81fec5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.830+0000 7f87af703b80 1 -- 172.21.15.107:0/3591553465 shutdown_connections 2023-07-21T22:37:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.830+0000 7f87af703b80 1 -- 172.21.15.107:0/3591553465 wait complete. 2023-07-21T22:37:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.830+0000 7f87af703b80 1 -- start start 2023-07-21T22:37:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.830+0000 7f87af703b80 1 Processor -- start 2023-07-21T22:37:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87af703b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fed7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87af703b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 0x55a81fed7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87af703b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec8580 0x55a81fed3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87af703b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81fed4650 con 0x55a81fec7830 2023-07-21T22:37:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87af703b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81fed47c0 con 0x55a81fec8580 2023-07-21T22:37:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87af703b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81fed4980 con 0x55a81fd236b0 2023-07-21T22:37:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87af703b80 10 client.0 _ll_register_callbacks cb 0x7f878c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87a0b19700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 0x55a81fed7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87a131a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fed7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87a0b19700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 0x55a81fed7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51396/0 (socket says 172.21.15.107:51396) 2023-07-21T22:37:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87a0b19700 1 -- 172.21.15.107:0/89932748 learned_addr learned my addr 172.21.15.107:0/89932748 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87a1b1b700 1 --2- 172.21.15.107:0/89932748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec8580 0x55a81fed3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87a131a700 1 -- 172.21.15.107:0/89932748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec8580 msgr2=0x55a81fed3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87a131a700 1 --2- 172.21.15.107:0/89932748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec8580 0x55a81fed3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87a131a700 1 -- 172.21.15.107:0/89932748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 msgr2=0x55a81fed7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87a131a700 1 --2- 172.21.15.107:0/89932748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 0x55a81fed7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87a131a700 1 -- 172.21.15.107:0/89932748 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878c009be0 con 0x55a81fd236b0 2023-07-21T22:37:03.890 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.831+0000 7f87a0b19700 1 --2- 172.21.15.107:0/89932748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 0x55a81fed7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f87a131a700 1 --2- 172.21.15.107:0/89932748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fed7310 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55a81fedeb20 tx=0x7f87980047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbdc48faa708a982 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f8784ff9700 10 client.0 ms_handle_connect on v2:172.21.15.190:3300/0 2023-07-21T22:37:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f8784ff9700 1 -- 172.21.15.107:0/89932748 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798025030 con 0x55a81fd236b0 2023-07-21T22:37:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f8784ff9700 1 -- 172.21.15.107:0/89932748 <== mon.1 v2:172.21.15.190:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f879802e590 con 0x55a81fd236b0 2023-07-21T22:37:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f87af703b80 1 -- 172.21.15.107:0/89932748 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87940a34a0 con 0x55a81fd236b0 2023-07-21T22:37:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f8784ff9700 1 -- 172.21.15.107:0/89932748 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8798020b60 con 0x55a81fd236b0 2023-07-21T22:37:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f87af703b80 10 client.26302 fetch_fsmap learned FSMap version 48 2023-07-21T22:37:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f87af703b80 1 -- 172.21.15.107:0/89932748 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a81fc6d1f0 con 0x55a81fd236b0 2023-07-21T22:37:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f8784ff9700 1 -- 172.21.15.107:0/89932748 <== mon.1 v2:172.21.15.190:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87940a34a0 con 0x55a81fd236b0 2023-07-21T22:37:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f8784ff9700 1 -- 172.21.15.107:0/89932748 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 48) v1 ==== 7534+0+0 (secure 0 0 0) 0x7f879803f020 con 0x55a81fd236b0 2023-07-21T22:37:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f87af703b80 10 client.26302 fetch_fsmap finished waiting for FSMap version 48 2023-07-21T22:37:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f87af703b80 10 client.26302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:37:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f87af703b80 20 client.26302 populate_metadata read hostname 'smithi107' 2023-07-21T22:37:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f87af703b80 1 --2- 172.21.15.107:0/89932748 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55a81ff31430 0x55a81ff33840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f87af703b80 4 client.26302 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:37:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.832+0000 7f87af703b80 1 -- 172.21.15.107:0/89932748 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a81fdd3ff0 con 0x55a81ff31430 2023-07-21T22:37:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.833+0000 7f87a0b19700 1 --2- 172.21.15.107:0/89932748 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55a81ff31430 0x55a81ff33840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.833+0000 7f8784ff9700 10 client.26302 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.833+0000 7f87a0b19700 1 --2- 172.21.15.107:0/89932748 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55a81ff31430 0x55a81ff33840 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=542ace633bfdf032 server_cookie=a66c7a279de03dd1 in_seq=0 out_seq=0 2023-07-21T22:37:03.894 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:03 smithi190 ceph-mon[98334]: pgmap v533: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 444 op/s 2023-07-21T22:37:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.843+0000 7f8784ff9700 1 -- 172.21.15.107:0/89932748 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a81fdd3ff0 con 0x55a81ff31430 2023-07-21T22:37:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.843+0000 7f8784ff9700 10 client.26302 handle_command_reply: tid=0 2023-07-21T22:37:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.843+0000 7f87af703b80 1 client.26302 shutdown 2023-07-21T22:37:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.844+0000 7f87af703b80 1 -- 172.21.15.107:0/89932748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 msgr2=0x55a81fed7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.844+0000 7f87af703b80 1 --2- 172.21.15.107:0/89932748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fed7310 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55a81fedeb20 tx=0x7f87980047c0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.844+0000 7f87af703b80 1 -- 172.21.15.107:0/89932748 shutdown_connections 2023-07-21T22:37:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.844+0000 7f87af703b80 1 --2- 172.21.15.107:0/89932748 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55a81ff31430 0x55a81ff33840 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.844+0000 7f87af703b80 1 --2- 172.21.15.107:0/89932748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a81fec8580 0x55a81fed3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.844+0000 7f8784ff9700 0 client.26302 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.844+0000 7f87af703b80 1 --2- 172.21.15.107:0/89932748 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a81fd236b0 0x55a81fed7310 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.844+0000 7f87af703b80 1 --2- 172.21.15.107:0/89932748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a81fec7830 0x55a81fed7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.844+0000 7f87af703b80 1 -- 172.21.15.107:0/89932748 >> 172.21.15.107:0/89932748 conn(0x55a81febcae0 msgr2=0x55a81fed08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.844+0000 7f87af703b80 1 -- 172.21.15.107:0/89932748 shutdown_connections 2023-07-21T22:37:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.844+0000 7f87af703b80 1 -- 172.21.15.107:0/89932748 wait complete. 2023-07-21T22:37:03.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.844+0000 7f87af703b80 20 client.26302 trim_cache size 0 max 16384 2023-07-21T22:37:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:37:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:37:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:37:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:37:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.844+0000 7f87a357e700 1 -- 172.21.15.107:0/2138570536 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f87880a1150 msgr2=0x7f87880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.845+0000 7f87a357e700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f87880a1150 0x7f87880a3600 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f879c0ff1d0 tx=0x7f879800b5b0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.845+0000 7f87a357e700 1 -- 172.21.15.107:0/2138570536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f879c0ffd00 msgr2=0x7f879c196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.845+0000 7f87a357e700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f879c0ffd00 0x7f879c196db0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f879400a040 tx=0x7f8794008360 comp rx=0 tx=0).stop 2023-07-21T22:37:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.845+0000 7f87a357e700 1 -- 172.21.15.107:0/2138570536 shutdown_connections 2023-07-21T22:37:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.845+0000 7f87a357e700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f879c0ffd00 0x7f879c196db0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.845+0000 7f87a357e700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f87880a1150 0x7f87880a3600 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.845+0000 7f87a357e700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f879c0fe180 0x7f879c1959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.845+0000 7f87a357e700 1 --2- 172.21.15.107:0/2138570536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f879c0ff370 0x7f879c195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.845+0000 7f87a357e700 1 -- 172.21.15.107:0/2138570536 >> 172.21.15.107:0/2138570536 conn(0x7f879c0f9780 msgr2=0x7f879c1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.845+0000 7f87a357e700 1 -- 172.21.15.107:0/2138570536 shutdown_connections 2023-07-21T22:37:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:03.845+0000 7f87a357e700 1 -- 172.21.15.107:0/2138570536 wait complete. 2023-07-21T22:37:03.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:03 smithi132 ceph-mon[97863]: pgmap v533: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 444 op/s 2023-07-21T22:37:03.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:03 smithi107 ceph-mon[96543]: pgmap v533: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 444 op/s 2023-07-21T22:37:05.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:05 smithi190 ceph-mon[98334]: pgmap v534: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 414 op/s 2023-07-21T22:37:05.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:05 smithi132 ceph-mon[97863]: pgmap v534: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 414 op/s 2023-07-21T22:37:05.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:05 smithi107 ceph-mon[96543]: pgmap v534: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 414 op/s 2023-07-21T22:37:07.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:07 smithi190 ceph-mon[98334]: pgmap v535: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 402 op/s 2023-07-21T22:37:07.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:07 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:07.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:07 smithi132 ceph-mon[97863]: pgmap v535: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 402 op/s 2023-07-21T22:37:07.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:07 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:07.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:07 smithi107 ceph-mon[96543]: pgmap v535: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 402 op/s 2023-07-21T22:37:07.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:07 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:09.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:09 smithi190 ceph-mon[98334]: pgmap v536: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 469 op/s 2023-07-21T22:37:09.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:09 smithi132 ceph-mon[97863]: pgmap v536: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 469 op/s 2023-07-21T22:37:09.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:09 smithi107 ceph-mon[96543]: pgmap v536: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 469 op/s 2023-07-21T22:37:10.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:10 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:10.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:10 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:10.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:10 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:11.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:11 smithi190 ceph-mon[98334]: pgmap v537: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 369 op/s 2023-07-21T22:37:11.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:11 smithi132 ceph-mon[97863]: pgmap v537: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 369 op/s 2023-07-21T22:37:11.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:11 smithi107 ceph-mon[96543]: pgmap v537: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 369 op/s 2023-07-21T22:37:12.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:12 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:37:12.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:12 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:37:12.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:12 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:37:13.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: pgmap v538: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 448 op/s 2023-07-21T22:37:13.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.891 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi190 to 196.2M 2023-07-21T22:37:13.892 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi107 to 196.1M 2023-07-21T22:37:13.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: Unable to set osd_memory_target on smithi190 to 205772492: error parsing value: Value '205772492' is below minimum 939524096 2023-07-21T22:37:13.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: Unable to set osd_memory_target on smithi107 to 205708697: error parsing value: Value '205708697' is below minimum 939524096 2023-07-21T22:37:13.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.893 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.894 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:37:13.894 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:37:13.894 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:37:13.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: pgmap v538: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 448 op/s 2023-07-21T22:37:13.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.931 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi190 to 196.2M 2023-07-21T22:37:13.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi107 to 196.1M 2023-07-21T22:37:13.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: Unable to set osd_memory_target on smithi190 to 205772492: error parsing value: Value '205772492' is below minimum 939524096 2023-07-21T22:37:13.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: Unable to set osd_memory_target on smithi107 to 205708697: error parsing value: Value '205708697' is below minimum 939524096 2023-07-21T22:37:13.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.932 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:37:13.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:37:13.933 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:37:13.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: pgmap v538: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 448 op/s 2023-07-21T22:37:13.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi190 to 196.2M 2023-07-21T22:37:13.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi107 to 196.1M 2023-07-21T22:37:13.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: Unable to set osd_memory_target on smithi190 to 205772492: error parsing value: Value '205772492' is below minimum 939524096 2023-07-21T22:37:13.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: Unable to set osd_memory_target on smithi107 to 205708697: error parsing value: Value '205708697' is below minimum 939524096 2023-07-21T22:37:13.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.938 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:37:13.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:37:13.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:37:13.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:37:14.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:14 smithi190 ceph-mon[98334]: Adjusting osd_memory_target on smithi132 to 196.2M 2023-07-21T22:37:14.890 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:14 smithi190 ceph-mon[98334]: Unable to set osd_memory_target on smithi132 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:37:14.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:14 smithi132 ceph-mon[97863]: Adjusting osd_memory_target on smithi132 to 196.2M 2023-07-21T22:37:14.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:14 smithi132 ceph-mon[97863]: Unable to set osd_memory_target on smithi132 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:37:14.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:14 smithi107 ceph-mon[96543]: Adjusting osd_memory_target on smithi132 to 196.2M 2023-07-21T22:37:14.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:14 smithi107 ceph-mon[96543]: Unable to set osd_memory_target on smithi132 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-21T22:37:15.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:15 smithi190 ceph-mon[98334]: pgmap v539: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 437 op/s 2023-07-21T22:37:15.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:15 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:15.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:15 smithi132 ceph-mon[97863]: pgmap v539: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 437 op/s 2023-07-21T22:37:15.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:15 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:15.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:15 smithi107 ceph-mon[96543]: pgmap v539: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 437 op/s 2023-07-21T22:37:15.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:15 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.955+0000 7f4d1fb5b700 1 -- 172.21.15.107:0/2388625781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d180fe180 msgr2=0x7f4d180fe590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.955+0000 7f4d1fb5b700 1 --2- 172.21.15.107:0/2388625781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d180fe180 0x7f4d180fe590 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0098f0 tx=0x7f4d0c009c00 comp rx=0 tx=0).stop 2023-07-21T22:37:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.955+0000 7f4d1fb5b700 1 -- 172.21.15.107:0/2388625781 shutdown_connections 2023-07-21T22:37:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.955+0000 7f4d1fb5b700 1 --2- 172.21.15.107:0/2388625781 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d180ffd00 0x7f4d181065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.955+0000 7f4d1fb5b700 1 --2- 172.21.15.107:0/2388625781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d180fe180 0x7f4d180fe590 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.955+0000 7f4d1fb5b700 1 --2- 172.21.15.107:0/2388625781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d180ff370 0x7f4d180ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.955+0000 7f4d1fb5b700 1 -- 172.21.15.107:0/2388625781 >> 172.21.15.107:0/2388625781 conn(0x7f4d180f9780 msgr2=0x7f4d180fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.955+0000 7f4d1fb5b700 1 -- 172.21.15.107:0/2388625781 shutdown_connections 2023-07-21T22:37:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.955+0000 7f4d1fb5b700 1 -- 172.21.15.107:0/2388625781 wait complete. 2023-07-21T22:37:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1fb5b700 1 Processor -- start 2023-07-21T22:37:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1fb5b700 1 -- start start 2023-07-21T22:37:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1fb5b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d180fe180 0x7f4d18195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1fb5b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d180ff370 0x7f4d18195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1fb5b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d180ffd00 0x7f4d18196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1fb5b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d181973a0 con 0x7f4d180fe180 2023-07-21T22:37:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1fb5b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d18197510 con 0x7f4d180ffd00 2023-07-21T22:37:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1fb5b700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d181976d0 con 0x7f4d180ff370 2023-07-21T22:37:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1d8f7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d180fe180 0x7f4d18195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1d0f6700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d180ff370 0x7f4d18195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1d8f7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d180fe180 0x7f4d18195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51408/0 (socket says 172.21.15.107:51408) 2023-07-21T22:37:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/643847071 learned_addr learned my addr 172.21.15.107:0/643847071 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1e0f8700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d180ffd00 0x7f4d18196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.956+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/643847071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d180ff370 msgr2=0x7f4d18195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.957+0000 7f4d1d8f7700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d180ff370 0x7f4d18195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.957+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/643847071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d180ffd00 msgr2=0x7f4d18196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.957+0000 7f4d1d8f7700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d180ffd00 0x7f4d18196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.957+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/643847071 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1400a700 con 0x7f4d180fe180 2023-07-21T22:37:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.957+0000 7f4d1d0f6700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d180ff370 0x7f4d18195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.957+0000 7f4d1d8f7700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d180fe180 0x7f4d18195980 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c000c00 tx=0x7f4d0c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.957+0000 7f4d0affd700 1 -- 172.21.15.107:0/643847071 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c01c070 con 0x7f4d180fe180 2023-07-21T22:37:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.957+0000 7f4d0affd700 1 -- 172.21.15.107:0/643847071 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d0c00f7b0 con 0x7f4d180fe180 2023-07-21T22:37:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.957+0000 7f4d1fb5b700 1 -- 172.21.15.107:0/643847071 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0c009520 con 0x7f4d180fe180 2023-07-21T22:37:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.957+0000 7f4d0affd700 1 -- 172.21.15.107:0/643847071 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c003970 con 0x7f4d180fe180 2023-07-21T22:37:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.957+0000 7f4d1fb5b700 1 -- 172.21.15.107:0/643847071 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d18193250 con 0x7f4d180fe180 2023-07-21T22:37:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.960+0000 7f4d0affd700 1 -- 172.21.15.107:0/643847071 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4d0c020030 con 0x7f4d180fe180 2023-07-21T22:37:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.960+0000 7f4d0affd700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4d040a1150 0x7f4d040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.960+0000 7f4d0affd700 1 -- 172.21.15.107:0/643847071 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4d0c0cd9a0 con 0x7f4d180fe180 2023-07-21T22:37:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.960+0000 7f4d1d0f6700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4d040a1150 0x7f4d040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.961+0000 7f4d1d0f6700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4d040a1150 0x7f4d040a3600 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4d14005fd0 tx=0x7f4d14014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.961+0000 7f4d2bce0b80 1 Processor -- start 2023-07-21T22:37:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.961+0000 7f4d2bce0b80 1 -- start start 2023-07-21T22:37:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.961+0000 7f4d2bce0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff91b76b0 0x560ff935b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.961+0000 7f4d2bce0b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff935b830 0x560ff935c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.961+0000 7f4d2bce0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935c580 0x560ff935c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.961+0000 7f4d2bce0b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff9260c50 con 0x560ff935b830 2023-07-21T22:37:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.961+0000 7f4d2bce0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff92b39e0 con 0x560ff935c580 2023-07-21T22:37:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.961+0000 7f4d2bce0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff92ace70 con 0x560ff91b76b0 2023-07-21T22:37:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.961+0000 7f4d1d0f6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff935b830 0x560ff935c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.961+0000 7f4d1d0f6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff935b830 0x560ff935c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51418/0 (socket says 172.21.15.107:51418) 2023-07-21T22:37:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.961+0000 7f4d1d0f6700 1 -- 172.21.15.107:0/4226301864 learned_addr learned my addr 172.21.15.107:0/4226301864 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.961+0000 7f4d1e0f8700 1 --2- 172.21.15.107:0/4226301864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935c580 0x560ff935c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.962+0000 7f4d1d8f7700 1 --2- 172.21.15.107:0/4226301864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff91b76b0 0x560ff935b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.962+0000 7f4d1d0f6700 1 -- 172.21.15.107:0/4226301864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff91b76b0 msgr2=0x560ff935b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.962+0000 7f4d1d0f6700 1 --2- 172.21.15.107:0/4226301864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff91b76b0 0x560ff935b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.962+0000 7f4d1d0f6700 1 -- 172.21.15.107:0/4226301864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935c580 msgr2=0x560ff935c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.962+0000 7f4d1d0f6700 1 --2- 172.21.15.107:0/4226301864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935c580 0x560ff935c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.962+0000 7f4d1d0f6700 1 -- 172.21.15.107:0/4226301864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ff90dad80 con 0x560ff935b830 2023-07-21T22:37:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.962+0000 7f4d1d0f6700 1 --2- 172.21.15.107:0/4226301864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff935b830 0x560ff935c040 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400cd70 tx=0x7f4d14018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dffdc7699fea59 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.962+0000 7f4d03fff700 1 -- 172.21.15.107:0/4226301864 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14024070 con 0x560ff935b830 2023-07-21T22:37:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.962+0000 7f4d03fff700 1 -- 172.21.15.107:0/4226301864 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d1400cda0 con 0x560ff935b830 2023-07-21T22:37:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.962+0000 7f4d03fff700 1 -- 172.21.15.107:0/4226301864 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14017040 con 0x560ff935b830 2023-07-21T22:37:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.963+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/4226301864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff935b830 msgr2=0x560ff935c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.963+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/4226301864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff935b830 0x560ff935c040 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400cd70 tx=0x7f4d14018000 comp rx=0 tx=0).stop 2023-07-21T22:37:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.963+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/4226301864 shutdown_connections 2023-07-21T22:37:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.963+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/4226301864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935c580 0x560ff935c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.963+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/4226301864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff91b76b0 0x560ff935b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.963+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/4226301864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff935b830 0x560ff935c040 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.963+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/4226301864 >> 172.21.15.107:0/4226301864 conn(0x560ff9350ae0 msgr2=0x560ff9355c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.963+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/4226301864 shutdown_connections 2023-07-21T22:37:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.963+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/4226301864 wait complete. 2023-07-21T22:37:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.963+0000 7f4d2bce0b80 1 -- start start 2023-07-21T22:37:17.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.963+0000 7f4d2bce0b80 1 Processor -- start 2023-07-21T22:37:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d2bce0b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9364760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d2bce0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 0x560ff9364ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d2bce0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 0x560ff936bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d2bce0b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff924ca60 con 0x560ff91b76b0 2023-07-21T22:37:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d2bce0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff924c760 con 0x560ff935b830 2023-07-21T22:37:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d2bce0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff91308c0 con 0x560ff935c580 2023-07-21T22:37:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d2bce0b80 10 client.0 _ll_register_callbacks cb 0x560ff924a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:17.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d1d8f7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9364760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d1d0f6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 0x560ff9364ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d1d8f7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9364760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51422/0 (socket says 172.21.15.107:51422) 2023-07-21T22:37:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d1d0f6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 0x560ff9364ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.107:42252/0 (socket says 172.21.15.107:42252) 2023-07-21T22:37:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/659512989 learned_addr learned my addr 172.21.15.107:0/659512989 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d1e0f8700 1 --2- 172.21.15.107:0/659512989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 0x560ff936bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/659512989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 msgr2=0x560ff936bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d1d8f7700 1 --2- 172.21.15.107:0/659512989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 0x560ff936bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/659512989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 msgr2=0x560ff9364ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d1d8f7700 1 --2- 172.21.15.107:0/659512989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 0x560ff9364ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/659512989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1400ca50 con 0x560ff91b76b0 2023-07-21T22:37:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d1d8f7700 1 --2- 172.21.15.107:0/659512989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9364760 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00f9e0 tx=0x7f4d0c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f28e8e751032137 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d017fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:37:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d017fa700 1 -- 172.21.15.107:0/659512989 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c09ee50 con 0x560ff91b76b0 2023-07-21T22:37:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.964+0000 7f4d017fa700 1 -- 172.21.15.107:0/659512989 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4d0c0a31e0 con 0x560ff91b76b0 2023-07-21T22:37:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.965+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/659512989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ff9206490 con 0x560ff91b76b0 2023-07-21T22:37:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.965+0000 7f4d017fa700 1 -- 172.21.15.107:0/659512989 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d0c09b080 con 0x560ff91b76b0 2023-07-21T22:37:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.965+0000 7f4d2bce0b80 10 client.16707 fetch_fsmap learned FSMap version 51 2023-07-21T22:37:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.965+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/659512989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x560ff90c3c00 con 0x560ff91b76b0 2023-07-21T22:37:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.965+0000 7f4d017fa700 1 -- 172.21.15.107:0/659512989 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560ff9206490 con 0x560ff91b76b0 2023-07-21T22:37:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.965+0000 7f4d017fa700 1 -- 172.21.15.107:0/659512989 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 51) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f4d0c0a2d50 con 0x560ff91b76b0 2023-07-21T22:37:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.965+0000 7f4d2bce0b80 10 client.16707 fetch_fsmap finished waiting for FSMap version 51 2023-07-21T22:37:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.965+0000 7f4d2bce0b80 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:37:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.965+0000 7f4d2bce0b80 20 client.16707 populate_metadata read hostname 'smithi107' 2023-07-21T22:37:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.965+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/659512989 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x560ff9370340 0x560ff9372750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.965+0000 7f4d2bce0b80 4 client.16707 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:37:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.965+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/659512989 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ff92a5e50 con 0x560ff9370340 2023-07-21T22:37:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.965+0000 7f4d1d0f6700 1 --2- 172.21.15.107:0/659512989 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x560ff9370340 0x560ff9372750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.966+0000 7f4d017fa700 10 client.16707 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.966+0000 7f4d1d0f6700 1 --2- 172.21.15.107:0/659512989 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x560ff9370340 0x560ff9372750 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f75e84d3df4eb128 server_cookie=d6e1744cfbb27229 in_seq=0 out_seq=0 2023-07-21T22:37:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d017fa700 1 -- 172.21.15.107:0/659512989 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560ff92a5e50 con 0x560ff9370340 2023-07-21T22:37:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d017fa700 10 client.16707 handle_command_reply: tid=0 2023-07-21T22:37:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d2bce0b80 1 client.16707 shutdown 2023-07-21T22:37:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/659512989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 msgr2=0x560ff9364760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/659512989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9364760 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00f9e0 tx=0x7f4d0c00ba10 comp rx=0 tx=0).stop 2023-07-21T22:37:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/659512989 shutdown_connections 2023-07-21T22:37:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/659512989 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x560ff9370340 0x560ff9372750 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/659512989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 0x560ff936bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/659512989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9364760 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/659512989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 0x560ff9364ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d017fa700 0 client.16707 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/659512989 >> 172.21.15.107:0/659512989 conn(0x560ff9350ae0 msgr2=0x560ff93553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/659512989 shutdown_connections 2023-07-21T22:37:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.967+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/659512989 wait complete. 2023-07-21T22:37:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.968+0000 7f4d2bce0b80 20 client.16707 trim_cache size 0 max 16384 2023-07-21T22:37:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.985+0000 7f4d2bce0b80 1 Processor -- start 2023-07-21T22:37:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.985+0000 7f4d2bce0b80 1 -- start start 2023-07-21T22:37:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.985+0000 7f4d2bce0b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9355be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.985+0000 7f4d2bce0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 0x560ff936bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.985+0000 7f4d2bce0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 0x560ff93770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.985+0000 7f4d2bce0b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff93632c0 con 0x560ff91b76b0 2023-07-21T22:37:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.985+0000 7f4d2bce0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff9363c40 con 0x560ff935b830 2023-07-21T22:37:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.985+0000 7f4d2bce0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff9357440 con 0x560ff935c580 2023-07-21T22:37:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.986+0000 7f4d1d8f7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9355be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.986+0000 7f4d1d8f7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9355be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51430/0 (socket says 172.21.15.107:51430) 2023-07-21T22:37:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.986+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/72893672 learned_addr learned my addr 172.21.15.107:0/72893672 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.986+0000 7f4d1e0f8700 1 --2- 172.21.15.107:0/72893672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 0x560ff93770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.986+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/72893672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 msgr2=0x560ff93770b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.986+0000 7f4d1d8f7700 1 --2- 172.21.15.107:0/72893672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 0x560ff93770b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.986+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/72893672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 msgr2=0x560ff936bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.986+0000 7f4d1d8f7700 1 --2- 172.21.15.107:0/72893672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 0x560ff936bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.986+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/72893672 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ff91011f0 con 0x560ff91b76b0 2023-07-21T22:37:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.986+0000 7f4d1d8f7700 1 --2- 172.21.15.107:0/72893672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9355be0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0cb4d0 tx=0x7f4d0c097290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a63ef078434addc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.986+0000 7f4d08dac700 1 -- 172.21.15.107:0/72893672 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c021a60 con 0x560ff91b76b0 2023-07-21T22:37:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.986+0000 7f4d08dac700 1 -- 172.21.15.107:0/72893672 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d0c0a31e0 con 0x560ff91b76b0 2023-07-21T22:37:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.986+0000 7f4d08dac700 1 -- 172.21.15.107:0/72893672 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c097530 con 0x560ff91b76b0 2023-07-21T22:37:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.987+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/72893672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 msgr2=0x560ff9355be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:17.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.987+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/72893672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9355be0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0cb4d0 tx=0x7f4d0c097290 comp rx=0 tx=0).stop 2023-07-21T22:37:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.987+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/72893672 shutdown_connections 2023-07-21T22:37:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.987+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/72893672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 0x560ff93770b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.987+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/72893672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9355be0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.987+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/72893672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 0x560ff936bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.987+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/72893672 >> 172.21.15.107:0/72893672 conn(0x560ff9350ae0 msgr2=0x560ff93634a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.987+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/72893672 shutdown_connections 2023-07-21T22:37:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.987+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/72893672 wait complete. 2023-07-21T22:37:17.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.987+0000 7f4d2bce0b80 1 -- start start 2023-07-21T22:37:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d2bce0b80 1 Processor -- start 2023-07-21T22:37:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d2bce0b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9367a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d2bce0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 0x560ff9367f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d2bce0b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 0x560ff936afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d2bce0b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff936b4e0 con 0x560ff91b76b0 2023-07-21T22:37:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d2bce0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff936b650 con 0x560ff935b830 2023-07-21T22:37:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d2bce0b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ff936b810 con 0x560ff935c580 2023-07-21T22:37:17.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d2bce0b80 10 client.0 _ll_register_callbacks cb 0x7f4d0c09aa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d1d8f7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9367a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d1d8f7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9367a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51436/0 (socket says 172.21.15.107:51436) 2023-07-21T22:37:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/2161073844 learned_addr learned my addr 172.21.15.107:0/2161073844 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d1d0f6700 1 --2- 172.21.15.107:0/2161073844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 0x560ff9367f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d1e0f8700 1 --2- 172.21.15.107:0/2161073844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 0x560ff936afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/2161073844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 msgr2=0x560ff936afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:17.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d1d8f7700 1 --2- 172.21.15.107:0/2161073844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 0x560ff936afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/2161073844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 msgr2=0x560ff9367f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d1d8f7700 1 --2- 172.21.15.107:0/2161073844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 0x560ff9367f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.988+0000 7f4d1d8f7700 1 -- 172.21.15.107:0/2161073844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ff90c3c00 con 0x560ff91b76b0 2023-07-21T22:37:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d1d8f7700 1 --2- 172.21.15.107:0/2161073844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9367a40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00f9e0 tx=0x7f4d0c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cc96def25909b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d017fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:37:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d017fa700 1 -- 172.21.15.107:0/2161073844 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c021a60 con 0x560ff91b76b0 2023-07-21T22:37:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d017fa700 1 -- 172.21.15.107:0/2161073844 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4d0c005690 con 0x560ff91b76b0 2023-07-21T22:37:17.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/2161073844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d1400cda0 con 0x560ff91b76b0 2023-07-21T22:37:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d017fa700 1 -- 172.21.15.107:0/2161073844 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d0c0cb2a0 con 0x560ff91b76b0 2023-07-21T22:37:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d2bce0b80 10 client.16713 fetch_fsmap learned FSMap version 51 2023-07-21T22:37:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/2161073844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x560ff90d9250 con 0x560ff91b76b0 2023-07-21T22:37:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d017fa700 1 -- 172.21.15.107:0/2161073844 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d1400cda0 con 0x560ff91b76b0 2023-07-21T22:37:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d017fa700 1 -- 172.21.15.107:0/2161073844 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 51) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f4d0c003b60 con 0x560ff91b76b0 2023-07-21T22:37:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d2bce0b80 10 client.16713 fetch_fsmap finished waiting for FSMap version 51 2023-07-21T22:37:17.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d2bce0b80 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:37:17.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d2bce0b80 20 client.16713 populate_metadata read hostname 'smithi107' 2023-07-21T22:37:17.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/2161073844 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x560ff93c4aa0 0x560ff93c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:17.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d2bce0b80 4 client.16713 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:37:17.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.989+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/2161073844 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560ff9267ff0 con 0x560ff93c4aa0 2023-07-21T22:37:17.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.990+0000 7f4d1d0f6700 1 --2- 172.21.15.107:0/2161073844 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x560ff93c4aa0 0x560ff93c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:17.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.990+0000 7f4d017fa700 10 client.16713 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:17.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:16.990+0000 7f4d1d0f6700 1 --2- 172.21.15.107:0/2161073844 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x560ff93c4aa0 0x560ff93c6e80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c3bed34d3b71c1f server_cookie=683e213cecf985ce in_seq=0 out_seq=0 2023-07-21T22:37:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.036+0000 7f4d017fa700 1 -- 172.21.15.107:0/2161073844 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560ff9267ff0 con 0x560ff93c4aa0 2023-07-21T22:37:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.036+0000 7f4d017fa700 10 client.16713 handle_command_reply: tid=0 2023-07-21T22:37:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.036+0000 7f4d2bce0b80 1 client.16713 shutdown 2023-07-21T22:37:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.036+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/2161073844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 msgr2=0x560ff9367a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:17.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.036+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/2161073844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9367a40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00f9e0 tx=0x7f4d0c009c00 comp rx=0 tx=0).stop 2023-07-21T22:37:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.036+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/2161073844 shutdown_connections 2023-07-21T22:37:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.036+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/2161073844 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x560ff93c4aa0 0x560ff93c6e80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.036+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/2161073844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ff935c580 0x560ff936afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.036+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/2161073844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ff91b76b0 0x560ff9367a40 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.036+0000 7f4d2bce0b80 1 --2- 172.21.15.107:0/2161073844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560ff935b830 0x560ff9367f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.036+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/2161073844 >> 172.21.15.107:0/2161073844 conn(0x560ff9350ae0 msgr2=0x560ff9355bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.036+0000 7f4d017fa700 0 client.16713 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.037+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/2161073844 shutdown_connections 2023-07-21T22:37:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.037+0000 7f4d2bce0b80 1 -- 172.21.15.107:0/2161073844 wait complete. 2023-07-21T22:37:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.037+0000 7f4d2bce0b80 20 client.16713 trim_cache size 0 max 16384 2023-07-21T22:37:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:37:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:37:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:37:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:37:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.038+0000 7f4d1fb5b700 1 -- 172.21.15.107:0/643847071 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4d040a1150 msgr2=0x7f4d040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.038+0000 7f4d1fb5b700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4d040a1150 0x7f4d040a3600 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4d14005fd0 tx=0x7f4d14014040 comp rx=0 tx=0).stop 2023-07-21T22:37:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.038+0000 7f4d1fb5b700 1 -- 172.21.15.107:0/643847071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d180fe180 msgr2=0x7f4d18195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.038+0000 7f4d1fb5b700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d180fe180 0x7f4d18195980 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c000c00 tx=0x7f4d0c00b730 comp rx=0 tx=0).stop 2023-07-21T22:37:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.038+0000 7f4d1fb5b700 1 -- 172.21.15.107:0/643847071 shutdown_connections 2023-07-21T22:37:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.038+0000 7f4d1fb5b700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d180ffd00 0x7f4d18196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.038+0000 7f4d1fb5b700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d180fe180 0x7f4d18195980 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.038+0000 7f4d1fb5b700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4d040a1150 0x7f4d040a3600 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.038+0000 7f4d1fb5b700 1 --2- 172.21.15.107:0/643847071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d180ff370 0x7f4d18195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.038+0000 7f4d1fb5b700 1 -- 172.21.15.107:0/643847071 >> 172.21.15.107:0/643847071 conn(0x7f4d180f9780 msgr2=0x7f4d18102890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.040+0000 7f4d1fb5b700 1 -- 172.21.15.107:0/643847071 shutdown_connections 2023-07-21T22:37:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:17.040+0000 7f4d1fb5b700 1 -- 172.21.15.107:0/643847071 wait complete. 2023-07-21T22:37:17.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:17 smithi190 ceph-mon[98334]: pgmap v540: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 423 op/s 2023-07-21T22:37:17.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:17 smithi132 ceph-mon[97863]: pgmap v540: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 423 op/s 2023-07-21T22:37:17.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:17 smithi107 ceph-mon[96543]: pgmap v540: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 423 op/s 2023-07-21T22:37:19.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:19 smithi190 ceph-mon[98334]: pgmap v541: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 495 op/s 2023-07-21T22:37:19.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:19 smithi132 ceph-mon[97863]: pgmap v541: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 495 op/s 2023-07-21T22:37:19.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:19 smithi107 ceph-mon[96543]: pgmap v541: 97 pgs: 97 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 495 op/s 2023-07-21T22:37:21.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:21 smithi190 ceph-mon[98334]: pgmap v542: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 388 op/s 2023-07-21T22:37:21.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:21 smithi132 ceph-mon[97863]: pgmap v542: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 388 op/s 2023-07-21T22:37:21.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:21 smithi107 ceph-mon[96543]: pgmap v542: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 388 op/s 2023-07-21T22:37:22.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:22 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:22.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:22 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:22.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:22 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:23.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:23 smithi190 ceph-mon[98334]: pgmap v543: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 475 op/s 2023-07-21T22:37:23.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:23 smithi132 ceph-mon[97863]: pgmap v543: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 475 op/s 2023-07-21T22:37:23.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:23 smithi107 ceph-mon[96543]: pgmap v543: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 475 op/s 2023-07-21T22:37:25.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:25 smithi190 ceph-mon[98334]: pgmap v544: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 453 op/s 2023-07-21T22:37:25.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:25 smithi132 ceph-mon[97863]: pgmap v544: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 453 op/s 2023-07-21T22:37:25.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:25 smithi107 ceph-mon[96543]: pgmap v544: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 453 op/s 2023-07-21T22:37:27.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:27 smithi190 ceph-mon[98334]: pgmap v545: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 432 op/s 2023-07-21T22:37:27.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:27 smithi132 ceph-mon[97863]: pgmap v545: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 432 op/s 2023-07-21T22:37:29.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:29 smithi190 ceph-mon[98334]: pgmap v546: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 509 op/s 2023-07-21T22:37:29.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:29 smithi132 ceph-mon[97863]: pgmap v546: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 509 op/s 2023-07-21T22:37:29.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:27 smithi107 ceph-mon[96543]: pgmap v545: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 432 op/s 2023-07-21T22:37:29.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:29 smithi107 ceph-mon[96543]: pgmap v546: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 509 op/s 2023-07-21T22:37:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.152+0000 7ff624a6d700 1 -- 172.21.15.107:0/3900090259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6200fea00 msgr2=0x7ff6200fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.152+0000 7ff624a6d700 1 --2- 172.21.15.107:0/3900090259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6200fea00 0x7ff6200fee10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff610009990 tx=0x7ff610009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.152+0000 7ff624a6d700 1 -- 172.21.15.107:0/3900090259 shutdown_connections 2023-07-21T22:37:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.152+0000 7ff624a6d700 1 --2- 172.21.15.107:0/3900090259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff620100640 0x7ff620106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.152+0000 7ff624a6d700 1 --2- 172.21.15.107:0/3900090259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6200fea00 0x7ff6200fee10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.152+0000 7ff624a6d700 1 --2- 172.21.15.107:0/3900090259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6200ffc90 0x7ff620100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.152+0000 7ff624a6d700 1 -- 172.21.15.107:0/3900090259 >> 172.21.15.107:0/3900090259 conn(0x7ff6200fa000 msgr2=0x7ff6200fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.152+0000 7ff624a6d700 1 -- 172.21.15.107:0/3900090259 shutdown_connections 2023-07-21T22:37:30.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.152+0000 7ff624a6d700 1 -- 172.21.15.107:0/3900090259 wait complete. 2023-07-21T22:37:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff624a6d700 1 Processor -- start 2023-07-21T22:37:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff624a6d700 1 -- start start 2023-07-21T22:37:30.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff624a6d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6200fea00 0x7ff620198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff624a6d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6200ffc90 0x7ff62019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff624a6d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff620100640 0x7ff62019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff624a6d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62019d5d0 con 0x7ff6200ffc90 2023-07-21T22:37:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff624a6d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62019d740 con 0x7ff6200fea00 2023-07-21T22:37:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff624a6d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62019d900 con 0x7ff620100640 2023-07-21T22:37:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff61dd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6200ffc90 0x7ff62019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff61dd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6200ffc90 0x7ff62019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51452/0 (socket says 172.21.15.107:51452) 2023-07-21T22:37:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff61dd9b700 1 -- 172.21.15.107:0/3118507749 learned_addr learned my addr 172.21.15.107:0/3118507749 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff61e59c700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6200fea00 0x7ff620198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff61ed9d700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff620100640 0x7ff62019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff61dd9b700 1 -- 172.21.15.107:0/3118507749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff620100640 msgr2=0x7ff62019cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff61dd9b700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff620100640 0x7ff62019cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff61dd9b700 1 -- 172.21.15.107:0/3118507749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6200fea00 msgr2=0x7ff620198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff61dd9b700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6200fea00 0x7ff620198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.153+0000 7ff61dd9b700 1 -- 172.21.15.107:0/3118507749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff618005e70 con 0x7ff6200ffc90 2023-07-21T22:37:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.154+0000 7ff61dd9b700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6200ffc90 0x7ff62019aa80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff61400e9a0 tx=0x7ff61400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.154+0000 7ff60f7fe700 1 -- 172.21.15.107:0/3118507749 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614009790 con 0x7ff6200ffc90 2023-07-21T22:37:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.154+0000 7ff60f7fe700 1 -- 172.21.15.107:0/3118507749 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff61400ce80 con 0x7ff6200ffc90 2023-07-21T22:37:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.154+0000 7ff624a6d700 1 -- 172.21.15.107:0/3118507749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6100095f0 con 0x7ff6200ffc90 2023-07-21T22:37:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.154+0000 7ff60f7fe700 1 -- 172.21.15.107:0/3118507749 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614018680 con 0x7ff6200ffc90 2023-07-21T22:37:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.154+0000 7ff624a6d700 1 -- 172.21.15.107:0/3118507749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62019df60 con 0x7ff6200ffc90 2023-07-21T22:37:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.156+0000 7ff60f7fe700 1 -- 172.21.15.107:0/3118507749 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff614010430 con 0x7ff6200ffc90 2023-07-21T22:37:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.156+0000 7ff60f7fe700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff6080a1150 0x7ff6080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.156+0000 7ff60f7fe700 1 -- 172.21.15.107:0/3118507749 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff614014070 con 0x7ff6200ffc90 2023-07-21T22:37:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.156+0000 7ff61e59c700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff6080a1150 0x7ff6080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.157+0000 7ff61e59c700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff6080a1150 0x7ff6080a3600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff61000b2e0 tx=0x7ff610005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.157+0000 7ff630bf2b80 1 Processor -- start 2023-07-21T22:37:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.157+0000 7ff630bf2b80 1 -- start start 2023-07-21T22:37:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff630bf2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6c4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff630bf2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6c5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff630bf2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff630bf2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e5c5c50 con 0x55c38e51c6b0 2023-07-21T22:37:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff630bf2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e6189e0 con 0x55c38e6c5a60 2023-07-21T22:37:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff630bf2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e611e70 con 0x55c38e6c4ba0 2023-07-21T22:37:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff61e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff61e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6c4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51458/0 (socket says 172.21.15.107:51458) 2023-07-21T22:37:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff61e59c700 1 -- 172.21.15.107:0/858336587 learned_addr learned my addr 172.21.15.107:0/858336587 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff61ed9d700 1 --2- 172.21.15.107:0/858336587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff61dd9b700 1 --2- 172.21.15.107:0/858336587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff61e59c700 1 -- 172.21.15.107:0/858336587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 msgr2=0x55c38e6c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff61e59c700 1 --2- 172.21.15.107:0/858336587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6c5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff61e59c700 1 -- 172.21.15.107:0/858336587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 msgr2=0x55c38e6c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff61e59c700 1 --2- 172.21.15.107:0/858336587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff61e59c700 1 -- 172.21.15.107:0/858336587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c38e43fd80 con 0x55c38e51c6b0 2023-07-21T22:37:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff61ed9d700 1 --2- 172.21.15.107:0/858336587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff61e59c700 1 --2- 172.21.15.107:0/858336587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6c4660 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff610009990 tx=0x7ff610020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994b829973aab5a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff60cff9700 1 -- 172.21.15.107:0/858336587 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61002b070 con 0x55c38e51c6b0 2023-07-21T22:37:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff60cff9700 1 -- 172.21.15.107:0/858336587 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff61001e040 con 0x55c38e51c6b0 2023-07-21T22:37:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.158+0000 7ff60cff9700 1 -- 172.21.15.107:0/858336587 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610024a00 con 0x55c38e51c6b0 2023-07-21T22:37:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.159+0000 7ff630bf2b80 1 -- 172.21.15.107:0/858336587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 msgr2=0x55c38e6c4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.159+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/858336587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6c4660 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff610009990 tx=0x7ff610020000 comp rx=0 tx=0).stop 2023-07-21T22:37:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.159+0000 7ff630bf2b80 1 -- 172.21.15.107:0/858336587 shutdown_connections 2023-07-21T22:37:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.159+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/858336587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.159+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/858336587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6c4660 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.159+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/858336587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.159+0000 7ff630bf2b80 1 -- 172.21.15.107:0/858336587 >> 172.21.15.107:0/858336587 conn(0x55c38e6b5ae0 msgr2=0x55c38e6bac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.159+0000 7ff630bf2b80 1 -- 172.21.15.107:0/858336587 shutdown_connections 2023-07-21T22:37:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.159+0000 7ff630bf2b80 1 -- 172.21.15.107:0/858336587 wait complete. 2023-07-21T22:37:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff630bf2b80 1 -- start start 2023-07-21T22:37:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff630bf2b80 1 Processor -- start 2023-07-21T22:37:30.215 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff630bf2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff630bf2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff630bf2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff630bf2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e5b1a60 con 0x55c38e51c6b0 2023-07-21T22:37:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff630bf2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e5b1760 con 0x55c38e6c5a60 2023-07-21T22:37:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff630bf2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e4958c0 con 0x55c38e6c4ba0 2023-07-21T22:37:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff630bf2b80 10 client.0 _ll_register_callbacks cb 0x55c38e5af700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff61e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff61e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6cdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51464/0 (socket says 172.21.15.107:51464) 2023-07-21T22:37:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff61dd9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff61e59c700 1 -- 172.21.15.107:0/3851395238 learned_addr learned my addr 172.21.15.107:0/3851395238 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:30.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.160+0000 7ff61ed9d700 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff61e59c700 1 -- 172.21.15.107:0/3851395238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 msgr2=0x55c38e6ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:30.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff61e59c700 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff61e59c700 1 -- 172.21.15.107:0/3851395238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 msgr2=0x55c38e6d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:30.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff61e59c700 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6d5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff61e59c700 1 -- 172.21.15.107:0/3851395238 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff618011230 con 0x55c38e51c6b0 2023-07-21T22:37:30.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff61dd9b700 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff61ed9d700 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff61e59c700 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6cdb10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff610031040 tx=0x7ff61000faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809d5e89b4dfffb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:30.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff605ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:37:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff605ffb700 1 -- 172.21.15.107:0/3851395238 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61002b040 con 0x55c38e51c6b0 2023-07-21T22:37:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff605ffb700 1 -- 172.21.15.107:0/3851395238 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff61002f610 con 0x55c38e51c6b0 2023-07-21T22:37:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3851395238 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c38e56b490 con 0x55c38e51c6b0 2023-07-21T22:37:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff605ffb700 1 -- 172.21.15.107:0/3851395238 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff61002f7d0 con 0x55c38e51c6b0 2023-07-21T22:37:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff630bf2b80 10 client.16725 fetch_fsmap learned FSMap version 52 2023-07-21T22:37:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3851395238 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7ff620107750 con 0x55c38e51c6b0 2023-07-21T22:37:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff605ffb700 1 -- 172.21.15.107:0/3851395238 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c38e56b490 con 0x55c38e51c6b0 2023-07-21T22:37:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff605ffb700 1 -- 172.21.15.107:0/3851395238 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 52) v1 ==== 7534+0+0 (secure 0 0 0) 0x7ff6100395c0 con 0x55c38e51c6b0 2023-07-21T22:37:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff630bf2b80 10 client.16725 fetch_fsmap finished waiting for FSMap version 52 2023-07-21T22:37:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff630bf2b80 10 client.16725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:37:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff630bf2b80 20 client.16725 populate_metadata read hostname 'smithi107' 2023-07-21T22:37:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c38e6d95d0 0x55c38e6db9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff630bf2b80 4 client.16725 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:37:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.161+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3851395238 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c38e60ae50 con 0x55c38e6d95d0 2023-07-21T22:37:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.162+0000 7ff61dd9b700 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c38e6d95d0 0x55c38e6db9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.162+0000 7ff605ffb700 10 client.16725 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.162+0000 7ff61dd9b700 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c38e6d95d0 0x55c38e6db9e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a86445937c99510e server_cookie=3e298759b844f114 in_seq=0 out_seq=0 2023-07-21T22:37:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff605ffb700 1 -- 172.21.15.107:0/3851395238 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c38e60ae50 con 0x55c38e6d95d0 2023-07-21T22:37:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff605ffb700 10 client.16725 handle_command_reply: tid=0 2023-07-21T22:37:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff630bf2b80 1 client.16725 shutdown 2023-07-21T22:37:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3851395238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 msgr2=0x55c38e6cdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6cdb10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff610031040 tx=0x7ff61000faf0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3851395238 shutdown_connections 2023-07-21T22:37:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c38e6d95d0 0x55c38e6db9e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff605ffb700 0 client.16725 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6cdb10 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/3851395238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3851395238 >> 172.21.15.107:0/3851395238 conn(0x55c38e6b5ae0 msgr2=0x55c38e6ba3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3851395238 shutdown_connections 2023-07-21T22:37:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3851395238 wait complete. 2023-07-21T22:37:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.163+0000 7ff630bf2b80 20 client.16725 trim_cache size 0 max 16384 2023-07-21T22:37:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.178+0000 7ff630bf2b80 1 Processor -- start 2023-07-21T22:37:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.178+0000 7ff630bf2b80 1 -- start start 2023-07-21T22:37:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff630bf2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff630bf2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6d5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff630bf2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6e0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff630bf2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e56b490 con 0x55c38e51c6b0 2023-07-21T22:37:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff630bf2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e6d9390 con 0x55c38e6c5a60 2023-07-21T22:37:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff630bf2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e6cc630 con 0x55c38e6c4ba0 2023-07-21T22:37:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff61e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff61e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51472/0 (socket says 172.21.15.107:51472) 2023-07-21T22:37:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff61e59c700 1 -- 172.21.15.107:0/2382194160 learned_addr learned my addr 172.21.15.107:0/2382194160 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff61ed9d700 1 --2- 172.21.15.107:0/2382194160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6e0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff61dd9b700 1 --2- 172.21.15.107:0/2382194160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6d5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff61e59c700 1 -- 172.21.15.107:0/2382194160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 msgr2=0x55c38e6d5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff61e59c700 1 --2- 172.21.15.107:0/2382194160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6d5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff61e59c700 1 -- 172.21.15.107:0/2382194160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 msgr2=0x55c38e6e0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff61e59c700 1 --2- 172.21.15.107:0/2382194160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6e0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff61e59c700 1 -- 172.21.15.107:0/2382194160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c38e428c00 con 0x55c38e51c6b0 2023-07-21T22:37:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff61e59c700 1 --2- 172.21.15.107:0/2382194160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6cd1c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff6100398f0 tx=0x7ff610009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bced4007045ae601 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff60d7fa700 1 -- 172.21.15.107:0/2382194160 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61002b070 con 0x55c38e51c6b0 2023-07-21T22:37:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff60d7fa700 1 -- 172.21.15.107:0/2382194160 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6100058e0 con 0x55c38e51c6b0 2023-07-21T22:37:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.179+0000 7ff60d7fa700 1 -- 172.21.15.107:0/2382194160 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61002f870 con 0x55c38e51c6b0 2023-07-21T22:37:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.180+0000 7ff630bf2b80 1 -- 172.21.15.107:0/2382194160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 msgr2=0x55c38e6cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.180+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/2382194160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6cd1c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff6100398f0 tx=0x7ff610009310 comp rx=0 tx=0).stop 2023-07-21T22:37:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.180+0000 7ff630bf2b80 1 -- 172.21.15.107:0/2382194160 shutdown_connections 2023-07-21T22:37:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.180+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/2382194160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e6c5a60 0x55c38e6e0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.180+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/2382194160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e51c6b0 0x55c38e6cd1c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.180+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/2382194160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6d5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.180+0000 7ff630bf2b80 1 -- 172.21.15.107:0/2382194160 >> 172.21.15.107:0/2382194160 conn(0x55c38e6b5ae0 msgr2=0x55c38e6babe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:30.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.180+0000 7ff630bf2b80 1 -- 172.21.15.107:0/2382194160 shutdown_connections 2023-07-21T22:37:30.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.180+0000 7ff630bf2b80 1 -- 172.21.15.107:0/2382194160 wait complete. 2023-07-21T22:37:30.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.180+0000 7ff630bf2b80 1 -- start start 2023-07-21T22:37:30.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.180+0000 7ff630bf2b80 1 Processor -- start 2023-07-21T22:37:30.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff630bf2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e51c6b0 0x55c38e6d0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff630bf2b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff630bf2b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e6c5a60 0x55c38e6d3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff630bf2b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e6d3fc0 con 0x55c38e6c5a60 2023-07-21T22:37:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff630bf2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e6d4130 con 0x55c38e51c6b0 2023-07-21T22:37:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff630bf2b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c38e6d42f0 con 0x55c38e6c4ba0 2023-07-21T22:37:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff630bf2b80 10 client.0 _ll_register_callbacks cb 0x7ff614096220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff61ed9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e6c5a60 0x55c38e6d3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff61e59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e51c6b0 0x55c38e6d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff61ed9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e6c5a60 0x55c38e6d3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51482/0 (socket says 172.21.15.107:51482) 2023-07-21T22:37:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff61ed9d700 1 -- 172.21.15.107:0/3143586860 learned_addr learned my addr 172.21.15.107:0/3143586860 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff61dd9b700 1 --2- 172.21.15.107:0/3143586860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff61ed9d700 1 -- 172.21.15.107:0/3143586860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 msgr2=0x55c38e6d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff61ed9d700 1 --2- 172.21.15.107:0/3143586860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff61ed9d700 1 -- 172.21.15.107:0/3143586860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e51c6b0 msgr2=0x55c38e6d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff61ed9d700 1 --2- 172.21.15.107:0/3143586860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e51c6b0 0x55c38e6d0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff61ed9d700 1 -- 172.21.15.107:0/3143586860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff620107750 con 0x55c38e6c5a60 2023-07-21T22:37:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff61ed9d700 1 --2- 172.21.15.107:0/3143586860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e6c5a60 0x55c38e6d3a80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff618004b40 tx=0x7ff6180077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45d3a95b7f3bca server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff605ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:37:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff605ffb700 1 -- 172.21.15.107:0/3143586860 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61801c070 con 0x55c38e6c5a60 2023-07-21T22:37:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff605ffb700 1 -- 172.21.15.107:0/3143586860 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff618021470 con 0x55c38e6c5a60 2023-07-21T22:37:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.181+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3143586860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff61001e040 con 0x55c38e6c5a60 2023-07-21T22:37:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff605ffb700 1 -- 172.21.15.107:0/3143586860 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff61800e040 con 0x55c38e6c5a60 2023-07-21T22:37:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff630bf2b80 10 client.16731 fetch_fsmap learned FSMap version 52 2023-07-21T22:37:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3143586860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55c38e4661f0 con 0x55c38e6c5a60 2023-07-21T22:37:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff605ffb700 1 -- 172.21.15.107:0/3143586860 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff61001e040 con 0x55c38e6c5a60 2023-07-21T22:37:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff605ffb700 1 -- 172.21.15.107:0/3143586860 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 52) v1 ==== 7534+0+0 (secure 0 0 0) 0x7ff61802a050 con 0x55c38e6c5a60 2023-07-21T22:37:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff630bf2b80 10 client.16731 fetch_fsmap finished waiting for FSMap version 52 2023-07-21T22:37:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff630bf2b80 10 client.16731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:37:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff630bf2b80 20 client.16731 populate_metadata read hostname 'smithi107' 2023-07-21T22:37:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/3143586860 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c38e72dd40 0x55c38e730150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff630bf2b80 4 client.16731 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:37:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3143586860 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c38e5ccff0 con 0x55c38e72dd40 2023-07-21T22:37:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff61e59c700 1 --2- 172.21.15.107:0/3143586860 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c38e72dd40 0x55c38e730150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff61e59c700 1 --2- 172.21.15.107:0/3143586860 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c38e72dd40 0x55c38e730150 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf836a9f1b8f423e server_cookie=292a38f7015b89ec in_seq=0 out_seq=0 2023-07-21T22:37:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.182+0000 7ff605ffb700 10 client.16731 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff605ffb700 1 -- 172.21.15.107:0/3143586860 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c38e5ccff0 con 0x55c38e72dd40 2023-07-21T22:37:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff605ffb700 10 client.16731 handle_command_reply: tid=0 2023-07-21T22:37:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff630bf2b80 1 client.16731 shutdown 2023-07-21T22:37:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3143586860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e6c5a60 msgr2=0x55c38e6d3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/3143586860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e6c5a60 0x55c38e6d3a80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff618004b40 tx=0x7ff6180077e0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.243 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3143586860 shutdown_connections 2023-07-21T22:37:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/3143586860 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55c38e72dd40 0x55c38e730150 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/3143586860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c38e6c5a60 0x55c38e6d3a80 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/3143586860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c38e51c6b0 0x55c38e6d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff630bf2b80 1 --2- 172.21.15.107:0/3143586860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c38e6c4ba0 0x55c38e6d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3143586860 >> 172.21.15.107:0/3143586860 conn(0x55c38e6b5ae0 msgr2=0x55c38e6cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff605ffb700 0 client.16731 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:30.244 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3143586860 shutdown_connections 2023-07-21T22:37:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff630bf2b80 1 -- 172.21.15.107:0/3143586860 wait complete. 2023-07-21T22:37:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff630bf2b80 20 client.16731 trim_cache size 0 max 16384 2023-07-21T22:37:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:37:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:37:30.245 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:37:30.246 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:37:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff624a6d700 1 -- 172.21.15.107:0/3118507749 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff6080a1150 msgr2=0x7ff6080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff624a6d700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff6080a1150 0x7ff6080a3600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff61000b2e0 tx=0x7ff610005fb0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff624a6d700 1 -- 172.21.15.107:0/3118507749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6200ffc90 msgr2=0x7ff62019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:30.247 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff624a6d700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6200ffc90 0x7ff62019aa80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff61400e9a0 tx=0x7ff61400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff624a6d700 1 -- 172.21.15.107:0/3118507749 shutdown_connections 2023-07-21T22:37:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff624a6d700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff6080a1150 0x7ff6080a3600 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.248 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff624a6d700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff620100640 0x7ff62019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff624a6d700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6200fea00 0x7ff620198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff624a6d700 1 --2- 172.21.15.107:0/3118507749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6200ffc90 0x7ff62019aa80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:30.249 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff624a6d700 1 -- 172.21.15.107:0/3118507749 >> 172.21.15.107:0/3118507749 conn(0x7ff6200fa000 msgr2=0x7ff6200fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:30.250 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff624a6d700 1 -- 172.21.15.107:0/3118507749 shutdown_connections 2023-07-21T22:37:30.250 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:30.198+0000 7ff624a6d700 1 -- 172.21.15.107:0/3118507749 wait complete. 2023-07-21T22:37:31.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:31 smithi190 ceph-mon[98334]: pgmap v547: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 404 op/s 2023-07-21T22:37:31.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:31 smithi132 ceph-mon[97863]: pgmap v547: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 404 op/s 2023-07-21T22:37:31.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:31 smithi107 ceph-mon[96543]: pgmap v547: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 404 op/s 2023-07-21T22:37:32.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:32 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:32.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:32 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:32.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:32 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:33.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:33 smithi190 ceph-mon[98334]: pgmap v548: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 474 op/s 2023-07-21T22:37:33.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:33 smithi132 ceph-mon[97863]: pgmap v548: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 474 op/s 2023-07-21T22:37:33.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:33 smithi107 ceph-mon[96543]: pgmap v548: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 474 op/s 2023-07-21T22:37:35.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:35 smithi190 ceph-mon[98334]: pgmap v549: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 432 op/s 2023-07-21T22:37:35.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:35 smithi132 ceph-mon[97863]: pgmap v549: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 432 op/s 2023-07-21T22:37:35.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:35 smithi107 ceph-mon[96543]: pgmap v549: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 432 op/s 2023-07-21T22:37:37.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:37 smithi190 ceph-mon[98334]: pgmap v550: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 405 op/s 2023-07-21T22:37:37.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:37 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:37.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:37 smithi132 ceph-mon[97863]: pgmap v550: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 405 op/s 2023-07-21T22:37:37.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:37 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:37.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:37 smithi107 ceph-mon[96543]: pgmap v550: 97 pgs: 97 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 405 op/s 2023-07-21T22:37:37.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:37 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:39.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:39 smithi190 ceph-mon[98334]: pgmap v551: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 473 op/s 2023-07-21T22:37:39.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:39 smithi132 ceph-mon[97863]: pgmap v551: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 473 op/s 2023-07-21T22:37:39.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:39 smithi107 ceph-mon[96543]: pgmap v551: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 473 op/s 2023-07-21T22:37:41.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:41 smithi190 ceph-mon[98334]: pgmap v552: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 364 op/s 2023-07-21T22:37:41.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:41 smithi132 ceph-mon[97863]: pgmap v552: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 364 op/s 2023-07-21T22:37:41.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:41 smithi107 ceph-mon[96543]: pgmap v552: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 364 op/s 2023-07-21T22:37:42.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:42 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:42.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:42 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:42.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:42 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.306+0000 7f9514206700 1 -- 172.21.15.107:0/1746353081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f950c0fea00 msgr2=0x7f950c0fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.306+0000 7f9514206700 1 --2- 172.21.15.107:0/1746353081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f950c0fea00 0x7f950c0fee10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9500009990 tx=0x7f9500009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.306+0000 7f9514206700 1 -- 172.21.15.107:0/1746353081 shutdown_connections 2023-07-21T22:37:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.306+0000 7f9514206700 1 --2- 172.21.15.107:0/1746353081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f950c100580 0x7f950c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.306+0000 7f9514206700 1 --2- 172.21.15.107:0/1746353081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f950c0fea00 0x7f950c0fee10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.306+0000 7f9514206700 1 --2- 172.21.15.107:0/1746353081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f950c0ffbf0 0x7f950c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.306+0000 7f9514206700 1 -- 172.21.15.107:0/1746353081 >> 172.21.15.107:0/1746353081 conn(0x7f950c0f9fe0 msgr2=0x7f950c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.306+0000 7f9514206700 1 -- 172.21.15.107:0/1746353081 shutdown_connections 2023-07-21T22:37:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.306+0000 7f9514206700 1 -- 172.21.15.107:0/1746353081 wait complete. 2023-07-21T22:37:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.306+0000 7f9514206700 1 Processor -- start 2023-07-21T22:37:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f9514206700 1 -- start start 2023-07-21T22:37:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f9514206700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f950c0fea00 0x7f950c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f9514206700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f950c0ffbf0 0x7f950c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f9514206700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f950c100580 0x7f950c19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f9514206700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c19d5f0 con 0x7f950c0ffbf0 2023-07-21T22:37:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f9514206700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c19d730 con 0x7f950c100580 2023-07-21T22:37:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f9514206700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c19d8f0 con 0x7f950c0fea00 2023-07-21T22:37:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f95117a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f950c0ffbf0 0x7f950c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f95117a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f950c0ffbf0 0x7f950c19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51494/0 (socket says 172.21.15.107:51494) 2023-07-21T22:37:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f9511fa2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f950c0fea00 0x7f950c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f95117a1700 1 -- 172.21.15.107:0/1984453812 learned_addr learned my addr 172.21.15.107:0/1984453812 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f95127a3700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f950c100580 0x7f950c19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f95117a1700 1 -- 172.21.15.107:0/1984453812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f950c0fea00 msgr2=0x7f950c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f95117a1700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f950c0fea00 0x7f950c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f95117a1700 1 -- 172.21.15.107:0/1984453812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f950c100580 msgr2=0x7f950c19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f95117a1700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f950c100580 0x7f950c19d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f95117a1700 1 -- 172.21.15.107:0/1984453812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95000095f0 con 0x7f950c0ffbf0 2023-07-21T22:37:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f95127a3700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f950c100580 0x7f950c19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:37:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f95117a1700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f950c0ffbf0 0x7f950c19aad0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f950800d950 tx=0x7f950800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f94feffd700 1 -- 172.21.15.107:0/1984453812 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508009940 con 0x7f950c0ffbf0 2023-07-21T22:37:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f94feffd700 1 -- 172.21.15.107:0/1984453812 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9508010460 con 0x7f950c0ffbf0 2023-07-21T22:37:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f94feffd700 1 -- 172.21.15.107:0/1984453812 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950800f620 con 0x7f950c0ffbf0 2023-07-21T22:37:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f9514206700 1 -- 172.21.15.107:0/1984453812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f950c19dbd0 con 0x7f950c0ffbf0 2023-07-21T22:37:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.307+0000 7f9514206700 1 -- 172.21.15.107:0/1984453812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950c19e060 con 0x7f950c0ffbf0 2023-07-21T22:37:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.309+0000 7f94feffd700 1 -- 172.21.15.107:0/1984453812 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f95080108d0 con 0x7f950c0ffbf0 2023-07-21T22:37:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.310+0000 7f94feffd700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f94f80a1150 0x7f94f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.310+0000 7f94feffd700 1 -- 172.21.15.107:0/1984453812 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f95080cc7b0 con 0x7f950c0ffbf0 2023-07-21T22:37:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.310+0000 7f9511fa2700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f94f80a1150 0x7f94f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.310+0000 7f9511fa2700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f94f80a1150 0x7f94f80a3600 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f950000b5c0 tx=0x7f9500005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.311+0000 7f952038bb80 1 Processor -- start 2023-07-21T22:37:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.311+0000 7f952038bb80 1 -- start start 2023-07-21T22:37:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.311+0000 7f952038bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddab8c6b0 0x562ddad302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.311+0000 7f952038bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad30830 0x562ddad31040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.311+0000 7f952038bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad31580 0x562ddad319f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.311+0000 7f952038bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddac35c50 con 0x562ddad31580 2023-07-21T22:37:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.311+0000 7f952038bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddac889e0 con 0x562ddad30830 2023-07-21T22:37:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.311+0000 7f952038bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddac81e70 con 0x562ddab8c6b0 2023-07-21T22:37:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.311+0000 7f95127a3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad31580 0x562ddad319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.311+0000 7f9511fa2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddab8c6b0 0x562ddad302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.311+0000 7f95127a3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad31580 0x562ddad319f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51504/0 (socket says 172.21.15.107:51504) 2023-07-21T22:37:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.311+0000 7f9511fa2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddab8c6b0 0x562ddad302f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:40058/0 (socket says 172.21.15.107:40058) 2023-07-21T22:37:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.312+0000 7f95127a3700 1 -- 172.21.15.107:0/507476117 learned_addr learned my addr 172.21.15.107:0/507476117 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.312+0000 7f95117a1700 1 --2- 172.21.15.107:0/507476117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad30830 0x562ddad31040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.312+0000 7f95127a3700 1 -- 172.21.15.107:0/507476117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddab8c6b0 msgr2=0x562ddad302f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.312+0000 7f95127a3700 1 --2- 172.21.15.107:0/507476117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddab8c6b0 0x562ddad302f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.312+0000 7f95127a3700 1 -- 172.21.15.107:0/507476117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad30830 msgr2=0x562ddad31040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.312+0000 7f95127a3700 1 --2- 172.21.15.107:0/507476117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad30830 0x562ddad31040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.312+0000 7f95127a3700 1 -- 172.21.15.107:0/507476117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ddaaafd80 con 0x562ddad31580 2023-07-21T22:37:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.312+0000 7f95127a3700 1 --2- 172.21.15.107:0/507476117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad31580 0x562ddad319f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9504009e60 tx=0x7f95040077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c95ae4a9b758420 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.312+0000 7f94f7fff700 1 -- 172.21.15.107:0/507476117 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504014610 con 0x562ddad31580 2023-07-21T22:37:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.312+0000 7f94f7fff700 1 -- 172.21.15.107:0/507476117 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f950400f050 con 0x562ddad31580 2023-07-21T22:37:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.312+0000 7f94f7fff700 1 -- 172.21.15.107:0/507476117 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504013970 con 0x562ddad31580 2023-07-21T22:37:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.313+0000 7f952038bb80 1 -- 172.21.15.107:0/507476117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad31580 msgr2=0x562ddad319f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.313+0000 7f952038bb80 1 --2- 172.21.15.107:0/507476117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad31580 0x562ddad319f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9504009e60 tx=0x7f95040077e0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.313+0000 7f952038bb80 1 -- 172.21.15.107:0/507476117 shutdown_connections 2023-07-21T22:37:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.313+0000 7f952038bb80 1 --2- 172.21.15.107:0/507476117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad31580 0x562ddad319f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.313+0000 7f952038bb80 1 --2- 172.21.15.107:0/507476117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddab8c6b0 0x562ddad302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.313+0000 7f952038bb80 1 --2- 172.21.15.107:0/507476117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad30830 0x562ddad31040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.313+0000 7f952038bb80 1 -- 172.21.15.107:0/507476117 >> 172.21.15.107:0/507476117 conn(0x562ddad25ae0 msgr2=0x562ddad2ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.313+0000 7f952038bb80 1 -- 172.21.15.107:0/507476117 shutdown_connections 2023-07-21T22:37:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.313+0000 7f952038bb80 1 -- 172.21.15.107:0/507476117 wait complete. 2023-07-21T22:37:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.313+0000 7f952038bb80 1 -- start start 2023-07-21T22:37:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.313+0000 7f952038bb80 1 Processor -- start 2023-07-21T22:37:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f952038bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 0x562ddad39760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f952038bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddad30830 0x562ddad39ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f952038bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad31580 0x562ddad40ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f952038bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddac21a60 con 0x562ddab8c6b0 2023-07-21T22:37:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f952038bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddac21760 con 0x562ddad31580 2023-07-21T22:37:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f952038bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddab058c0 con 0x562ddad30830 2023-07-21T22:37:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f952038bb80 10 client.0 _ll_register_callbacks cb 0x562ddac1f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f9511fa2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 0x562ddad39760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f9511fa2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 0x562ddad39760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51506/0 (socket says 172.21.15.107:51506) 2023-07-21T22:37:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f9511fa2700 1 -- 172.21.15.107:0/769738559 learned_addr learned my addr 172.21.15.107:0/769738559 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f95117a1700 1 --2- 172.21.15.107:0/769738559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddad30830 0x562ddad39ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f95127a3700 1 --2- 172.21.15.107:0/769738559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad31580 0x562ddad40ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f9511fa2700 1 -- 172.21.15.107:0/769738559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddad30830 msgr2=0x562ddad39ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f9511fa2700 1 --2- 172.21.15.107:0/769738559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddad30830 0x562ddad39ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f9511fa2700 1 -- 172.21.15.107:0/769738559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad31580 msgr2=0x562ddad40ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f9511fa2700 1 --2- 172.21.15.107:0/769738559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad31580 0x562ddad40ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f9511fa2700 1 -- 172.21.15.107:0/769738559 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9504009a90 con 0x562ddab8c6b0 2023-07-21T22:37:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f9511fa2700 1 --2- 172.21.15.107:0/769738559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 0x562ddad39760 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9500005f50 tx=0x7f950001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5867176e5245cd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f94f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:37:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.314+0000 7f94f57fa700 1 -- 172.21.15.107:0/769738559 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950002b050 con 0x562ddab8c6b0 2023-07-21T22:37:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f94f57fa700 1 -- 172.21.15.107:0/769738559 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f950000be60 con 0x562ddab8c6b0 2023-07-21T22:37:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f952038bb80 1 -- 172.21.15.107:0/769738559 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ddabdb490 con 0x562ddab8c6b0 2023-07-21T22:37:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f94f57fa700 1 -- 172.21.15.107:0/769738559 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f950000f870 con 0x562ddab8c6b0 2023-07-21T22:37:43.369 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f952038bb80 10 client.16743 fetch_fsmap learned FSMap version 55 2023-07-21T22:37:43.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f952038bb80 1 -- 172.21.15.107:0/769738559 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x562ddaa98c00 con 0x562ddab8c6b0 2023-07-21T22:37:43.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f94f57fa700 1 -- 172.21.15.107:0/769738559 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f950003f020 con 0x562ddab8c6b0 2023-07-21T22:37:43.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f94f57fa700 1 -- 172.21.15.107:0/769738559 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 55) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f95000249a0 con 0x562ddab8c6b0 2023-07-21T22:37:43.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f952038bb80 10 client.16743 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:37:43.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f952038bb80 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:37:43.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f952038bb80 20 client.16743 populate_metadata read hostname 'smithi107' 2023-07-21T22:37:43.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f952038bb80 1 --2- 172.21.15.107:0/769738559 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562ddad45690 0x562ddad47aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f952038bb80 4 client.16743 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:37:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f952038bb80 1 -- 172.21.15.107:0/769738559 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ddac7ae50 con 0x562ddad45690 2023-07-21T22:37:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.315+0000 7f95117a1700 1 --2- 172.21.15.107:0/769738559 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562ddad45690 0x562ddad47aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.316+0000 7f94f57fa700 10 client.16743 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.316+0000 7f95117a1700 1 --2- 172.21.15.107:0/769738559 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562ddad45690 0x562ddad47aa0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2931b80b598d81bb server_cookie=3390a1606a63bf96 in_seq=0 out_seq=0 2023-07-21T22:37:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f94f57fa700 1 -- 172.21.15.107:0/769738559 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562ddac7ae50 con 0x562ddad45690 2023-07-21T22:37:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f94f57fa700 10 client.16743 handle_command_reply: tid=0 2023-07-21T22:37:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f952038bb80 1 client.16743 shutdown 2023-07-21T22:37:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f952038bb80 1 -- 172.21.15.107:0/769738559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 msgr2=0x562ddad39760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f952038bb80 1 --2- 172.21.15.107:0/769738559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 0x562ddad39760 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9500005f50 tx=0x7f950001a050 comp rx=0 tx=0).stop 2023-07-21T22:37:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f952038bb80 1 -- 172.21.15.107:0/769738559 shutdown_connections 2023-07-21T22:37:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f952038bb80 1 --2- 172.21.15.107:0/769738559 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562ddad45690 0x562ddad47aa0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f952038bb80 1 --2- 172.21.15.107:0/769738559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad31580 0x562ddad40ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f94f57fa700 0 client.16743 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f952038bb80 1 --2- 172.21.15.107:0/769738559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 0x562ddad39760 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f952038bb80 1 --2- 172.21.15.107:0/769738559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddad30830 0x562ddad39ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f952038bb80 1 -- 172.21.15.107:0/769738559 >> 172.21.15.107:0/769738559 conn(0x562ddad25ae0 msgr2=0x562ddad2a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f952038bb80 1 -- 172.21.15.107:0/769738559 shutdown_connections 2023-07-21T22:37:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f952038bb80 1 -- 172.21.15.107:0/769738559 wait complete. 2023-07-21T22:37:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.317+0000 7f952038bb80 20 client.16743 trim_cache size 0 max 16384 2023-07-21T22:37:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.331+0000 7f952038bb80 1 Processor -- start 2023-07-21T22:37:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.331+0000 7f952038bb80 1 -- start start 2023-07-21T22:37:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.331+0000 7f952038bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 0x562ddad31950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.331+0000 7f952038bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad30830 0x562ddad34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.331+0000 7f952038bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddad31580 0x562ddad34890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.331+0000 7f952038bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddad30c00 con 0x562ddab8c6b0 2023-07-21T22:37:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.331+0000 7f952038bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddad34ec0 con 0x562ddad30830 2023-07-21T22:37:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.331+0000 7f952038bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddad41090 con 0x562ddad31580 2023-07-21T22:37:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.332+0000 7f9511fa2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 0x562ddad31950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.332+0000 7f9511fa2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 0x562ddad31950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51514/0 (socket says 172.21.15.107:51514) 2023-07-21T22:37:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.332+0000 7f9511fa2700 1 -- 172.21.15.107:0/3031067090 learned_addr learned my addr 172.21.15.107:0/3031067090 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.332+0000 7f95127a3700 1 --2- 172.21.15.107:0/3031067090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddad31580 0x562ddad34890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.332+0000 7f9511fa2700 1 -- 172.21.15.107:0/3031067090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddad31580 msgr2=0x562ddad34890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.332+0000 7f9511fa2700 1 --2- 172.21.15.107:0/3031067090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddad31580 0x562ddad34890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.332+0000 7f9511fa2700 1 -- 172.21.15.107:0/3031067090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad30830 msgr2=0x562ddad34350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.332+0000 7f9511fa2700 1 --2- 172.21.15.107:0/3031067090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad30830 0x562ddad34350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.332+0000 7f9511fa2700 1 -- 172.21.15.107:0/3031067090 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ddaad61f0 con 0x562ddab8c6b0 2023-07-21T22:37:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.332+0000 7f9511fa2700 1 --2- 172.21.15.107:0/3031067090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 0x562ddad31950 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9500000c00 tx=0x7f950000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c8f69142463028 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.332+0000 7f94fcff9700 1 -- 172.21.15.107:0/3031067090 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950002b070 con 0x562ddab8c6b0 2023-07-21T22:37:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.332+0000 7f94fcff9700 1 -- 172.21.15.107:0/3031067090 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f950003f070 con 0x562ddab8c6b0 2023-07-21T22:37:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.332+0000 7f94fcff9700 1 -- 172.21.15.107:0/3031067090 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950000fd30 con 0x562ddab8c6b0 2023-07-21T22:37:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 -- 172.21.15.107:0/3031067090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 msgr2=0x562ddad31950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 --2- 172.21.15.107:0/3031067090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 0x562ddad31950 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9500000c00 tx=0x7f950000b830 comp rx=0 tx=0).stop 2023-07-21T22:37:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 -- 172.21.15.107:0/3031067090 shutdown_connections 2023-07-21T22:37:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 --2- 172.21.15.107:0/3031067090 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddad31580 0x562ddad34890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 --2- 172.21.15.107:0/3031067090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddab8c6b0 0x562ddad31950 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 --2- 172.21.15.107:0/3031067090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad30830 0x562ddad34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 -- 172.21.15.107:0/3031067090 >> 172.21.15.107:0/3031067090 conn(0x562ddad25ae0 msgr2=0x562ddad4c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 -- 172.21.15.107:0/3031067090 shutdown_connections 2023-07-21T22:37:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 -- 172.21.15.107:0/3031067090 wait complete. 2023-07-21T22:37:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 -- start start 2023-07-21T22:37:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 Processor -- start 2023-07-21T22:37:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddab8c6b0 0x562ddad95790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad30830 0x562ddad95cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad31580 0x562ddad9c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddad9cbf0 con 0x562ddad30830 2023-07-21T22:37:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddad9cd60 con 0x562ddad31580 2023-07-21T22:37:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddad9cf20 con 0x562ddab8c6b0 2023-07-21T22:37:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.333+0000 7f952038bb80 10 client.0 _ll_register_callbacks cb 0x562ddac1f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f95117a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad30830 0x562ddad95cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f95117a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad30830 0x562ddad95cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51522/0 (socket says 172.21.15.107:51522) 2023-07-21T22:37:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f95117a1700 1 -- 172.21.15.107:0/3962052620 learned_addr learned my addr 172.21.15.107:0/3962052620 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f9511fa2700 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddab8c6b0 0x562ddad95790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f95127a3700 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad31580 0x562ddad9c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f95117a1700 1 -- 172.21.15.107:0/3962052620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddab8c6b0 msgr2=0x562ddad95790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f95117a1700 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddab8c6b0 0x562ddad95790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f95117a1700 1 -- 172.21.15.107:0/3962052620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad31580 msgr2=0x562ddad9c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f95117a1700 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad31580 0x562ddad9c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f95117a1700 1 -- 172.21.15.107:0/3962052620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ddaa98c00 con 0x562ddad30830 2023-07-21T22:37:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f9511fa2700 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddab8c6b0 0x562ddad95790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:37:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f95117a1700 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad30830 0x562ddad95cd0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f95080ca290 tx=0x7f95080a20a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d058dcd1b2b3344 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f94f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:37:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f94f57fa700 1 -- 172.21.15.107:0/3962052620 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508099e00 con 0x562ddad30830 2023-07-21T22:37:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f94f57fa700 1 -- 172.21.15.107:0/3962052620 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9508099f60 con 0x562ddad30830 2023-07-21T22:37:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f952038bb80 1 -- 172.21.15.107:0/3962052620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f950400f050 con 0x562ddad30830 2023-07-21T22:37:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f94f57fa700 1 -- 172.21.15.107:0/3962052620 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9508099090 con 0x562ddad30830 2023-07-21T22:37:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f952038bb80 10 client.16749 fetch_fsmap learned FSMap version 55 2023-07-21T22:37:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f952038bb80 1 -- 172.21.15.107:0/3962052620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) v3 -- 0x562ddaaae250 con 0x562ddad30830 2023-07-21T22:37:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f94f57fa700 1 -- 172.21.15.107:0/3962052620 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f950400f050 con 0x562ddad30830 2023-07-21T22:37:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.334+0000 7f94f57fa700 1 -- 172.21.15.107:0/3962052620 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 55) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f95080b20d0 con 0x562ddad30830 2023-07-21T22:37:43.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.335+0000 7f952038bb80 10 client.16749 fetch_fsmap finished waiting for FSMap version 55 2023-07-21T22:37:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.335+0000 7f952038bb80 10 client.16749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:37:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.335+0000 7f952038bb80 20 client.16749 populate_metadata read hostname 'smithi107' 2023-07-21T22:37:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.335+0000 7f952038bb80 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562ddad9e8c0 0x562ddada0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.335+0000 7f952038bb80 4 client.16749 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:37:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.335+0000 7f952038bb80 1 -- 172.21.15.107:0/3962052620 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562ddac3cff0 con 0x562ddad9e8c0 2023-07-21T22:37:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.335+0000 7f9511fa2700 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562ddad9e8c0 0x562ddada0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.335+0000 7f94f57fa700 10 client.16749 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:43.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.335+0000 7f9511fa2700 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562ddad9e8c0 0x562ddada0cd0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeccbe67cca1028e server_cookie=75af4513b504f4be in_seq=0 out_seq=0 2023-07-21T22:37:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.345+0000 7f94f57fa700 1 -- 172.21.15.107:0/3962052620 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562ddac3cff0 con 0x562ddad9e8c0 2023-07-21T22:37:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.345+0000 7f94f57fa700 10 client.16749 handle_command_reply: tid=0 2023-07-21T22:37:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.345+0000 7f952038bb80 1 client.16749 shutdown 2023-07-21T22:37:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.345+0000 7f952038bb80 1 -- 172.21.15.107:0/3962052620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad30830 msgr2=0x562ddad95cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.345+0000 7f952038bb80 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad30830 0x562ddad95cd0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f95080ca290 tx=0x7f95080a20a0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f952038bb80 1 -- 172.21.15.107:0/3962052620 shutdown_connections 2023-07-21T22:37:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f952038bb80 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x562ddad9e8c0 0x562ddada0cd0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f952038bb80 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562ddad31580 0x562ddad9c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f952038bb80 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ddab8c6b0 0x562ddad95790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f94f57fa700 0 client.16749 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f952038bb80 1 --2- 172.21.15.107:0/3962052620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ddad30830 0x562ddad95cd0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f952038bb80 1 -- 172.21.15.107:0/3962052620 >> 172.21.15.107:0/3962052620 conn(0x562ddad25ae0 msgr2=0x562ddad34350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f952038bb80 1 -- 172.21.15.107:0/3962052620 shutdown_connections 2023-07-21T22:37:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f952038bb80 1 -- 172.21.15.107:0/3962052620 wait complete. 2023-07-21T22:37:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f952038bb80 20 client.16749 trim_cache size 0 max 16384 2023-07-21T22:37:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:37:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:37:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:37:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:37:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f9514206700 1 -- 172.21.15.107:0/1984453812 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f94f80a1150 msgr2=0x7f94f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f9514206700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f94f80a1150 0x7f94f80a3600 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f950000b5c0 tx=0x7f9500005960 comp rx=0 tx=0).stop 2023-07-21T22:37:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f9514206700 1 -- 172.21.15.107:0/1984453812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f950c0ffbf0 msgr2=0x7f950c19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f9514206700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f950c0ffbf0 0x7f950c19aad0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f950800d950 tx=0x7f950800dc60 comp rx=0 tx=0).stop 2023-07-21T22:37:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f9514206700 1 -- 172.21.15.107:0/1984453812 shutdown_connections 2023-07-21T22:37:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f9514206700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f950c100580 0x7f950c19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f9514206700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f94f80a1150 0x7f94f80a3600 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f9514206700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f950c0fea00 0x7f950c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f9514206700 1 --2- 172.21.15.107:0/1984453812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f950c0ffbf0 0x7f950c19aad0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f9514206700 1 -- 172.21.15.107:0/1984453812 >> 172.21.15.107:0/1984453812 conn(0x7f950c0f9fe0 msgr2=0x7f950c1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f9514206700 1 -- 172.21.15.107:0/1984453812 shutdown_connections 2023-07-21T22:37:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:43.346+0000 7f9514206700 1 -- 172.21.15.107:0/1984453812 wait complete. 2023-07-21T22:37:43.888 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:43 smithi190 ceph-mon[98334]: pgmap v553: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 444 op/s 2023-07-21T22:37:43.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:43 smithi132 ceph-mon[97863]: pgmap v553: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 444 op/s 2023-07-21T22:37:43.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:43 smithi107 ceph-mon[96543]: pgmap v553: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 444 op/s 2023-07-21T22:37:45.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:45 smithi190 ceph-mon[98334]: pgmap v554: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 422 op/s 2023-07-21T22:37:45.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:45 smithi132 ceph-mon[97863]: pgmap v554: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 422 op/s 2023-07-21T22:37:45.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:45 smithi107 ceph-mon[96543]: pgmap v554: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 422 op/s 2023-07-21T22:37:47.888 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:47 smithi190 ceph-mon[98334]: pgmap v555: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 403 op/s 2023-07-21T22:37:47.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:47 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:47.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:47 smithi132 ceph-mon[97863]: pgmap v555: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 403 op/s 2023-07-21T22:37:47.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:47 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:47.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:47 smithi107 ceph-mon[96543]: pgmap v555: 97 pgs: 97 active+clean; 1.0 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 403 op/s 2023-07-21T22:37:47.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:47 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:37:49.888 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:49 smithi190 ceph-mon[98334]: pgmap v556: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 460 op/s 2023-07-21T22:37:49.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:49 smithi132 ceph-mon[97863]: pgmap v556: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 460 op/s 2023-07-21T22:37:49.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:49 smithi107 ceph-mon[96543]: pgmap v556: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 460 op/s 2023-07-21T22:37:51.889 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:51 smithi190 ceph-mon[98334]: pgmap v557: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 351 op/s 2023-07-21T22:37:51.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:51 smithi132 ceph-mon[97863]: pgmap v557: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 351 op/s 2023-07-21T22:37:51.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:51 smithi107 ceph-mon[96543]: pgmap v557: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 351 op/s 2023-07-21T22:37:53.888 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:53 smithi190 ceph-mon[98334]: pgmap v558: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 424 op/s 2023-07-21T22:37:53.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:53 smithi132 ceph-mon[97863]: pgmap v558: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 424 op/s 2023-07-21T22:37:53.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:53 smithi107 ceph-mon[96543]: pgmap v558: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 424 op/s 2023-07-21T22:37:55.888 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:55 smithi190 ceph-mon[98334]: pgmap v559: 97 pgs: 97 active+clean; 1021 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 393 op/s 2023-07-21T22:37:55.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:55 smithi132 ceph-mon[97863]: pgmap v559: 97 pgs: 97 active+clean; 1021 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 393 op/s 2023-07-21T22:37:55.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:55 smithi107 ceph-mon[96543]: pgmap v559: 97 pgs: 97 active+clean; 1021 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 393 op/s 2023-07-21T22:37:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.455+0000 7f9e38d78700 1 -- 172.21.15.107:0/4186838750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e34102f20 msgr2=0x7f9e34105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.455+0000 7f9e38d78700 1 --2- 172.21.15.107:0/4186838750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e34102f20 0x7f9e34105300 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009990 tx=0x7f9e24009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.455+0000 7f9e38d78700 1 -- 172.21.15.107:0/4186838750 shutdown_connections 2023-07-21T22:37:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.455+0000 7f9e38d78700 1 --2- 172.21.15.107:0/4186838750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e34105840 0x7f9e34107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.455+0000 7f9e38d78700 1 --2- 172.21.15.107:0/4186838750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e34100600 0x7f9e341029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.455+0000 7f9e38d78700 1 --2- 172.21.15.107:0/4186838750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e34102f20 0x7f9e34105300 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.455+0000 7f9e38d78700 1 -- 172.21.15.107:0/4186838750 >> 172.21.15.107:0/4186838750 conn(0x7f9e340fa020 msgr2=0x7f9e340fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.455+0000 7f9e38d78700 1 -- 172.21.15.107:0/4186838750 shutdown_connections 2023-07-21T22:37:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.456+0000 7f9e38d78700 1 -- 172.21.15.107:0/4186838750 wait complete. 2023-07-21T22:37:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.456+0000 7f9e38d78700 1 Processor -- start 2023-07-21T22:37:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.456+0000 7f9e38d78700 1 -- start start 2023-07-21T22:37:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.456+0000 7f9e38d78700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e34100600 0x7f9e340fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.456+0000 7f9e38d78700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e34102f20 0x7f9e340feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.456+0000 7f9e38d78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e34105840 0x7f9e340ff030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.456+0000 7f9e38d78700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3419f8d0 con 0x7f9e34100600 2023-07-21T22:37:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.456+0000 7f9e38d78700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3419fa10 con 0x7f9e34105840 2023-07-21T22:37:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.456+0000 7f9e38d78700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3419fb50 con 0x7f9e34102f20 2023-07-21T22:37:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e3259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e34100600 0x7f9e340fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e3259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e34100600 0x7f9e340fe5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51532/0 (socket says 172.21.15.107:51532) 2023-07-21T22:37:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e3259c700 1 -- 172.21.15.107:0/774686755 learned_addr learned my addr 172.21.15.107:0/774686755 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e31d9b700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e34102f20 0x7f9e340feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e32d9d700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e34105840 0x7f9e340ff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e3259c700 1 -- 172.21.15.107:0/774686755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e34102f20 msgr2=0x7f9e340feaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e3259c700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e34102f20 0x7f9e340feaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e3259c700 1 -- 172.21.15.107:0/774686755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e34105840 msgr2=0x7f9e340ff030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e3259c700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e34105840 0x7f9e340ff030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e3259c700 1 -- 172.21.15.107:0/774686755 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e240095f0 con 0x7f9e34100600 2023-07-21T22:37:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e3259c700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e34100600 0x7f9e340fe5b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00c920 tx=0x7f9e1c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e2b7fe700 1 -- 172.21.15.107:0/774686755 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c004a30 con 0x7f9e34100600 2023-07-21T22:37:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e2b7fe700 1 -- 172.21.15.107:0/774686755 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e1c0054e0 con 0x7f9e34100600 2023-07-21T22:37:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e38d78700 1 -- 172.21.15.107:0/774686755 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3419fde0 con 0x7f9e34100600 2023-07-21T22:37:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e2b7fe700 1 -- 172.21.15.107:0/774686755 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c007bd0 con 0x7f9e34100600 2023-07-21T22:37:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.457+0000 7f9e38d78700 1 -- 172.21.15.107:0/774686755 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e341a0200 con 0x7f9e34100600 2023-07-21T22:37:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.459+0000 7f9e2b7fe700 1 -- 172.21.15.107:0/774686755 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9e1c005000 con 0x7f9e34100600 2023-07-21T22:37:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.460+0000 7f9e2b7fe700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9e200a1150 0x7f9e200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.460+0000 7f9e2b7fe700 1 -- 172.21.15.107:0/774686755 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9e1c0cc440 con 0x7f9e34100600 2023-07-21T22:37:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.460+0000 7f9e31d9b700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9e200a1150 0x7f9e200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.460+0000 7f9e31d9b700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9e200a1150 0x7f9e200a3600 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400b2e0 tx=0x7f9e24005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e44efdb80 1 Processor -- start 2023-07-21T22:37:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e44efdb80 1 -- start start 2023-07-21T22:37:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e44efdb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e4932f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e44efdb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e493830 0x56163e494040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e44efdb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e494580 0x56163e4949f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e44efdb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163e398c50 con 0x56163e2ef6b0 2023-07-21T22:37:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e44efdb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163e3eb9e0 con 0x56163e493830 2023-07-21T22:37:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e44efdb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163e3e4e70 con 0x56163e494580 2023-07-21T22:37:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e3259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e4932f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e3259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e4932f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51540/0 (socket says 172.21.15.107:51540) 2023-07-21T22:37:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e3259c700 1 -- 172.21.15.107:0/2084128393 learned_addr learned my addr 172.21.15.107:0/2084128393 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e32d9d700 1 --2- 172.21.15.107:0/2084128393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e494580 0x56163e4949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e31d9b700 1 --2- 172.21.15.107:0/2084128393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e493830 0x56163e494040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e3259c700 1 -- 172.21.15.107:0/2084128393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e494580 msgr2=0x56163e4949f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e3259c700 1 --2- 172.21.15.107:0/2084128393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e494580 0x56163e4949f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e3259c700 1 -- 172.21.15.107:0/2084128393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e493830 msgr2=0x56163e494040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e3259c700 1 --2- 172.21.15.107:0/2084128393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e493830 0x56163e494040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.461+0000 7f9e3259c700 1 -- 172.21.15.107:0/2084128393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56163e212d80 con 0x56163e2ef6b0 2023-07-21T22:37:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.462+0000 7f9e3259c700 1 --2- 172.21.15.107:0/2084128393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e4932f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0997c0 tx=0x7f9e1c0993c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1ef14aa26ce631 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.462+0000 7f9e28ff9700 1 -- 172.21.15.107:0/2084128393 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c099830 con 0x56163e2ef6b0 2023-07-21T22:37:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.462+0000 7f9e28ff9700 1 -- 172.21.15.107:0/2084128393 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e1c09d860 con 0x56163e2ef6b0 2023-07-21T22:37:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.462+0000 7f9e28ff9700 1 -- 172.21.15.107:0/2084128393 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c007d50 con 0x56163e2ef6b0 2023-07-21T22:37:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.462+0000 7f9e44efdb80 1 -- 172.21.15.107:0/2084128393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 msgr2=0x56163e4932f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.462+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/2084128393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e4932f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0997c0 tx=0x7f9e1c0993c0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.462+0000 7f9e44efdb80 1 -- 172.21.15.107:0/2084128393 shutdown_connections 2023-07-21T22:37:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.462+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/2084128393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e494580 0x56163e4949f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.462+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/2084128393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e4932f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.462+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/2084128393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e493830 0x56163e494040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.462+0000 7f9e44efdb80 1 -- 172.21.15.107:0/2084128393 >> 172.21.15.107:0/2084128393 conn(0x56163e488ae0 msgr2=0x56163e48dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e44efdb80 1 -- 172.21.15.107:0/2084128393 shutdown_connections 2023-07-21T22:37:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e44efdb80 1 -- 172.21.15.107:0/2084128393 wait complete. 2023-07-21T22:37:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e44efdb80 1 -- start start 2023-07-21T22:37:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e44efdb80 1 Processor -- start 2023-07-21T22:37:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e44efdb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e44efdb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e493830 0x56163e49cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e44efdb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e494580 0x56163e4a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e44efdb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163e384a60 con 0x56163e2ef6b0 2023-07-21T22:37:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e44efdb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163e384760 con 0x56163e494580 2023-07-21T22:37:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e44efdb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163e2688c0 con 0x56163e493830 2023-07-21T22:37:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e44efdb80 10 client.0 _ll_register_callbacks cb 0x56163e382700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e3259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e3259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51546/0 (socket says 172.21.15.107:51546) 2023-07-21T22:37:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e3259c700 1 -- 172.21.15.107:0/3763619726 learned_addr learned my addr 172.21.15.107:0/3763619726 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e31d9b700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e493830 0x56163e49cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.463+0000 7f9e32d9d700 1 --2- 172.21.15.107:0/3763619726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e494580 0x56163e4a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e3259c700 1 -- 172.21.15.107:0/3763619726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e493830 msgr2=0x56163e49cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e3259c700 1 --2- 172.21.15.107:0/3763619726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e493830 0x56163e49cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e3259c700 1 -- 172.21.15.107:0/3763619726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e494580 msgr2=0x56163e4a3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e3259c700 1 --2- 172.21.15.107:0/3763619726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e494580 0x56163e4a3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e3259c700 1 -- 172.21.15.107:0/3763619726 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e341a0200 con 0x56163e2ef6b0 2023-07-21T22:37:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e32d9d700 1 --2- 172.21.15.107:0/3763619726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e494580 0x56163e4a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:37:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e3259c700 1 --2- 172.21.15.107:0/3763619726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49c710 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0c9f70 tx=0x7f9e1c004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe36d27c09926463 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:37:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e19ffb700 1 -- 172.21.15.107:0/3763619726 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c097970 con 0x56163e2ef6b0 2023-07-21T22:37:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e19ffb700 1 -- 172.21.15.107:0/3763619726 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9e1c099830 con 0x56163e2ef6b0 2023-07-21T22:37:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e44efdb80 1 -- 172.21.15.107:0/3763619726 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56163e33e490 con 0x56163e2ef6b0 2023-07-21T22:37:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e19ffb700 1 -- 172.21.15.107:0/3763619726 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e1c098d20 con 0x56163e2ef6b0 2023-07-21T22:37:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e44efdb80 10 client.16761 fetch_fsmap learned FSMap version 56 2023-07-21T22:37:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e44efdb80 1 -- 172.21.15.107:0/3763619726 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x56163e1fbc00 con 0x56163e2ef6b0 2023-07-21T22:37:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e19ffb700 1 -- 172.21.15.107:0/3763619726 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e1c065020 con 0x56163e2ef6b0 2023-07-21T22:37:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e19ffb700 1 -- 172.21.15.107:0/3763619726 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f9e1c0b4070 con 0x56163e2ef6b0 2023-07-21T22:37:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e44efdb80 10 client.16761 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T22:37:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e44efdb80 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:37:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e44efdb80 20 client.16761 populate_metadata read hostname 'smithi107' 2023-07-21T22:37:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/3763619726 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56163e4a81e0 0x56163e4aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e44efdb80 4 client.16761 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:37:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.464+0000 7f9e44efdb80 1 -- 172.21.15.107:0/3763619726 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56163e3dde50 con 0x56163e4a81e0 2023-07-21T22:37:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.465+0000 7f9e31d9b700 1 --2- 172.21.15.107:0/3763619726 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56163e4a81e0 0x56163e4aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.465+0000 7f9e19ffb700 10 client.16761 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.465+0000 7f9e31d9b700 1 --2- 172.21.15.107:0/3763619726 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56163e4a81e0 0x56163e4aa5f0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d99d4274609f71f server_cookie=ff657d6a42879865 in_seq=0 out_seq=0 2023-07-21T22:37:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e19ffb700 1 -- 172.21.15.107:0/3763619726 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56163e3dde50 con 0x56163e4a81e0 2023-07-21T22:37:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e19ffb700 10 client.16761 handle_command_reply: tid=0 2023-07-21T22:37:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e44efdb80 1 client.16761 shutdown 2023-07-21T22:37:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e44efdb80 1 -- 172.21.15.107:0/3763619726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 msgr2=0x56163e49c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/3763619726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49c710 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0c9f70 tx=0x7f9e1c004dc0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e44efdb80 1 -- 172.21.15.107:0/3763619726 shutdown_connections 2023-07-21T22:37:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/3763619726 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56163e4a81e0 0x56163e4aa5f0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/3763619726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e494580 0x56163e4a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/3763619726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49c710 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/3763619726 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e493830 0x56163e49cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e19ffb700 0 client.16761 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e44efdb80 1 -- 172.21.15.107:0/3763619726 >> 172.21.15.107:0/3763619726 conn(0x56163e488ae0 msgr2=0x56163e48d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e44efdb80 1 -- 172.21.15.107:0/3763619726 shutdown_connections 2023-07-21T22:37:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e44efdb80 1 -- 172.21.15.107:0/3763619726 wait complete. 2023-07-21T22:37:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.466+0000 7f9e44efdb80 20 client.16761 trim_cache size 0 max 16384 2023-07-21T22:37:56.525 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.480+0000 7f9e44efdb80 1 Processor -- start 2023-07-21T22:37:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.480+0000 7f9e44efdb80 1 -- start start 2023-07-21T22:37:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.480+0000 7f9e44efdb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.480+0000 7f9e44efdb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e493830 0x56163e4a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.480+0000 7f9e44efdb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e494580 0x56163e49ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.480+0000 7f9e44efdb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163e492f70 con 0x56163e2ef6b0 2023-07-21T22:37:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.480+0000 7f9e44efdb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163e493c30 con 0x56163e494580 2023-07-21T22:37:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.480+0000 7f9e44efdb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163e48f740 con 0x56163e493830 2023-07-21T22:37:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e3259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e3259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51554/0 (socket says 172.21.15.107:51554) 2023-07-21T22:37:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e3259c700 1 -- 172.21.15.107:0/847344263 learned_addr learned my addr 172.21.15.107:0/847344263 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e31d9b700 1 --2- 172.21.15.107:0/847344263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e493830 0x56163e4a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e3259c700 1 -- 172.21.15.107:0/847344263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e493830 msgr2=0x56163e4a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e3259c700 1 --2- 172.21.15.107:0/847344263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e493830 0x56163e4a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e3259c700 1 -- 172.21.15.107:0/847344263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e494580 msgr2=0x56163e49ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e32d9d700 1 --2- 172.21.15.107:0/847344263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e494580 0x56163e49ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e3259c700 1 --2- 172.21.15.107:0/847344263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e494580 0x56163e49ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e3259c700 1 -- 172.21.15.107:0/847344263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56163e2391f0 con 0x56163e2ef6b0 2023-07-21T22:37:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e3259c700 1 --2- 172.21.15.107:0/847344263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49c4c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0a1e40 tx=0x7f9e1c0c5f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea92654cb84f173 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e297fa700 1 -- 172.21.15.107:0/847344263 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0b91a0 con 0x56163e2ef6b0 2023-07-21T22:37:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e297fa700 1 -- 172.21.15.107:0/847344263 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e1c065040 con 0x56163e2ef6b0 2023-07-21T22:37:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.481+0000 7f9e297fa700 1 -- 172.21.15.107:0/847344263 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0a17f0 con 0x56163e2ef6b0 2023-07-21T22:37:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 -- 172.21.15.107:0/847344263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 msgr2=0x56163e49c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/847344263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49c4c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0a1e40 tx=0x7f9e1c0c5f30 comp rx=0 tx=0).stop 2023-07-21T22:37:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 -- 172.21.15.107:0/847344263 shutdown_connections 2023-07-21T22:37:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/847344263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e494580 0x56163e49ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/847344263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49c4c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/847344263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e493830 0x56163e4a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 -- 172.21.15.107:0/847344263 >> 172.21.15.107:0/847344263 conn(0x56163e488ae0 msgr2=0x56163e497490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 -- 172.21.15.107:0/847344263 shutdown_connections 2023-07-21T22:37:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 -- 172.21.15.107:0/847344263 wait complete. 2023-07-21T22:37:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 -- start start 2023-07-21T22:37:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 Processor -- start 2023-07-21T22:37:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e493830 0x56163e4a0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e494580 0x56163e4a3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163e4a3770 con 0x56163e2ef6b0 2023-07-21T22:37:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163e4a38e0 con 0x56163e493830 2023-07-21T22:37:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163e4a3aa0 con 0x56163e494580 2023-07-21T22:37:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e44efdb80 10 client.0 _ll_register_callbacks cb 0x7f9e2c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:37:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e3259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e3259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51560/0 (socket says 172.21.15.107:51560) 2023-07-21T22:37:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e3259c700 1 -- 172.21.15.107:0/2835958409 learned_addr learned my addr 172.21.15.107:0/2835958409 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:37:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.482+0000 7f9e32d9d700 1 --2- 172.21.15.107:0/2835958409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e494580 0x56163e4a3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e31d9b700 1 --2- 172.21.15.107:0/2835958409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e493830 0x56163e4a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e3259c700 1 -- 172.21.15.107:0/2835958409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e494580 msgr2=0x56163e4a3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e3259c700 1 --2- 172.21.15.107:0/2835958409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e494580 0x56163e4a3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e3259c700 1 -- 172.21.15.107:0/2835958409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e493830 msgr2=0x56163e4a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:37:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e3259c700 1 --2- 172.21.15.107:0/2835958409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e493830 0x56163e4a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e3259c700 1 -- 172.21.15.107:0/2835958409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56163e1fbc00 con 0x56163e2ef6b0 2023-07-21T22:37:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e3259c700 1 --2- 172.21.15.107:0/2835958409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49fcd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c095d30 tx=0x7f9e1c097970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321cf64ace684962 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:37:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e19ffb700 1 -- 172.21.15.107:0/2835958409 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0b91a0 con 0x56163e2ef6b0 2023-07-21T22:37:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e19ffb700 1 -- 172.21.15.107:0/2835958409 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9e1c099830 con 0x56163e2ef6b0 2023-07-21T22:37:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e44efdb80 1 -- 172.21.15.107:0/2835958409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e1c09d860 con 0x56163e2ef6b0 2023-07-21T22:37:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e19ffb700 1 -- 172.21.15.107:0/2835958409 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e1c095b70 con 0x56163e2ef6b0 2023-07-21T22:37:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e44efdb80 10 client.16767 fetch_fsmap learned FSMap version 56 2023-07-21T22:37:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e44efdb80 1 -- 172.21.15.107:0/2835958409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x56163e211250 con 0x56163e2ef6b0 2023-07-21T22:37:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e19ffb700 1 -- 172.21.15.107:0/2835958409 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e1c09d860 con 0x56163e2ef6b0 2023-07-21T22:37:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e19ffb700 1 -- 172.21.15.107:0/2835958409 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f9e1c0b4070 con 0x56163e2ef6b0 2023-07-21T22:37:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e44efdb80 10 client.16767 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T22:37:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e44efdb80 10 client.16767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:37:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.483+0000 7f9e44efdb80 20 client.16767 populate_metadata read hostname 'smithi107' 2023-07-21T22:37:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.484+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/2835958409 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56163e4fce10 0x56163e4ff220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.484+0000 7f9e44efdb80 4 client.16767 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:37:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.484+0000 7f9e44efdb80 1 -- 172.21.15.107:0/2835958409 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56163e39fff0 con 0x56163e4fce10 2023-07-21T22:37:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.484+0000 7f9e31d9b700 1 --2- 172.21.15.107:0/2835958409 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56163e4fce10 0x56163e4ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.484+0000 7f9e31d9b700 1 --2- 172.21.15.107:0/2835958409 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56163e4fce10 0x56163e4ff220 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae5e7692bdfdfde5 server_cookie=d9592df97651660e in_seq=0 out_seq=0 2023-07-21T22:37:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.484+0000 7f9e19ffb700 10 client.16767 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:56.538 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e19ffb700 1 -- 172.21.15.107:0/2835958409 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56163e39fff0 con 0x56163e4fce10 2023-07-21T22:37:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e19ffb700 10 client.16767 handle_command_reply: tid=0 2023-07-21T22:37:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e44efdb80 1 client.16767 shutdown 2023-07-21T22:37:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e44efdb80 1 -- 172.21.15.107:0/2835958409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 msgr2=0x56163e49fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/2835958409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49fcd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c095d30 tx=0x7f9e1c097970 comp rx=0 tx=0).stop 2023-07-21T22:37:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e44efdb80 1 -- 172.21.15.107:0/2835958409 shutdown_connections 2023-07-21T22:37:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/2835958409 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x56163e4fce10 0x56163e4ff220 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/2835958409 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56163e494580 0x56163e4a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e19ffb700 0 client.16767 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:37:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/2835958409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56163e2ef6b0 0x56163e49fcd0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e44efdb80 1 --2- 172.21.15.107:0/2835958409 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56163e493830 0x56163e4a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e44efdb80 1 -- 172.21.15.107:0/2835958409 >> 172.21.15.107:0/2835958409 conn(0x56163e488ae0 msgr2=0x56163e49c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e44efdb80 1 -- 172.21.15.107:0/2835958409 shutdown_connections 2023-07-21T22:37:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e44efdb80 1 -- 172.21.15.107:0/2835958409 wait complete. 2023-07-21T22:37:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.492+0000 7f9e44efdb80 20 client.16767 trim_cache size 0 max 16384 2023-07-21T22:37:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:37:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:37:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:37:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:37:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.493+0000 7f9e38d78700 1 -- 172.21.15.107:0/774686755 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9e200a1150 msgr2=0x7f9e200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.493+0000 7f9e38d78700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9e200a1150 0x7f9e200a3600 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400b2e0 tx=0x7f9e24005fb0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.493+0000 7f9e38d78700 1 -- 172.21.15.107:0/774686755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e34100600 msgr2=0x7f9e340fe5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.493+0000 7f9e38d78700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e34100600 0x7f9e340fe5b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00c920 tx=0x7f9e1c00cc30 comp rx=0 tx=0).stop 2023-07-21T22:37:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.493+0000 7f9e38d78700 1 -- 172.21.15.107:0/774686755 shutdown_connections 2023-07-21T22:37:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.493+0000 7f9e38d78700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e34105840 0x7f9e340ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.493+0000 7f9e38d78700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e34100600 0x7f9e340fe5b0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.493+0000 7f9e38d78700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9e200a1150 0x7f9e200a3600 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.493+0000 7f9e38d78700 1 --2- 172.21.15.107:0/774686755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e34102f20 0x7f9e340feaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.493+0000 7f9e38d78700 1 -- 172.21.15.107:0/774686755 >> 172.21.15.107:0/774686755 conn(0x7f9e340fa020 msgr2=0x7f9e34101260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.493+0000 7f9e38d78700 1 -- 172.21.15.107:0/774686755 shutdown_connections 2023-07-21T22:37:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:37:56.493+0000 7f9e38d78700 1 -- 172.21.15.107:0/774686755 wait complete. 2023-07-21T22:37:57.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:57 smithi132 ceph-mon[97863]: pgmap v560: 97 pgs: 97 active+clean; 1008 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 380 op/s 2023-07-21T22:37:57.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:57 smithi107 ceph-mon[96543]: pgmap v560: 97 pgs: 97 active+clean; 1008 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 380 op/s 2023-07-21T22:37:58.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:57 smithi190 ceph-mon[98334]: pgmap v560: 97 pgs: 97 active+clean; 1008 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 380 op/s 2023-07-21T22:37:59.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:37:59 smithi132 ceph-mon[97863]: pgmap v561: 97 pgs: 97 active+clean; 991 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 453 op/s 2023-07-21T22:37:59.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:37:59 smithi107 ceph-mon[96543]: pgmap v561: 97 pgs: 97 active+clean; 991 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 453 op/s 2023-07-21T22:38:00.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:37:59 smithi190 ceph-mon[98334]: pgmap v561: 97 pgs: 97 active+clean; 991 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 453 op/s 2023-07-21T22:38:01.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:01 smithi132 ceph-mon[97863]: pgmap v562: 97 pgs: 97 active+clean; 991 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 365 op/s 2023-07-21T22:38:01.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:01 smithi107 ceph-mon[96543]: pgmap v562: 97 pgs: 97 active+clean; 991 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 365 op/s 2023-07-21T22:38:02.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:01 smithi190 ceph-mon[98334]: pgmap v562: 97 pgs: 97 active+clean; 991 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 365 op/s 2023-07-21T22:38:03.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:03 smithi132 ceph-mon[97863]: pgmap v563: 97 pgs: 97 active+clean; 979 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 441 op/s 2023-07-21T22:38:03.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:03 smithi107 ceph-mon[96543]: pgmap v563: 97 pgs: 97 active+clean; 979 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 441 op/s 2023-07-21T22:38:04.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:03 smithi190 ceph-mon[98334]: pgmap v563: 97 pgs: 97 active+clean; 979 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 441 op/s 2023-07-21T22:38:05.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:05 smithi132 ceph-mon[97863]: pgmap v564: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 414 op/s 2023-07-21T22:38:05.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:05 smithi107 ceph-mon[96543]: pgmap v564: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 414 op/s 2023-07-21T22:38:06.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:05 smithi190 ceph-mon[98334]: pgmap v564: 97 pgs: 97 active+clean; 963 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 414 op/s 2023-07-21T22:38:07.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:07 smithi132 ceph-mon[97863]: pgmap v565: 97 pgs: 97 active+clean; 952 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 391 op/s 2023-07-21T22:38:07.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:07 smithi107 ceph-mon[96543]: pgmap v565: 97 pgs: 97 active+clean; 952 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 391 op/s 2023-07-21T22:38:08.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:07 smithi190 ceph-mon[98334]: pgmap v565: 97 pgs: 97 active+clean; 952 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 391 op/s 2023-07-21T22:38:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.601+0000 7f9bf54ab700 1 -- 172.21.15.107:0/621042717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf00fe180 msgr2=0x7f9bf00fe590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.601+0000 7f9bf54ab700 1 --2- 172.21.15.107:0/621042717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf00fe180 0x7f9bf00fe590 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009990 tx=0x7f9be0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.601+0000 7f9bf54ab700 1 -- 172.21.15.107:0/621042717 shutdown_connections 2023-07-21T22:38:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.601+0000 7f9bf54ab700 1 --2- 172.21.15.107:0/621042717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf00ffd00 0x7f9bf01065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.601+0000 7f9bf54ab700 1 --2- 172.21.15.107:0/621042717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf00fe180 0x7f9bf00fe590 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.601+0000 7f9bf54ab700 1 --2- 172.21.15.107:0/621042717 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf00ff370 0x7f9bf00ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.601+0000 7f9bf54ab700 1 -- 172.21.15.107:0/621042717 >> 172.21.15.107:0/621042717 conn(0x7f9bf00f9780 msgr2=0x7f9bf00fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.601+0000 7f9bf54ab700 1 -- 172.21.15.107:0/621042717 shutdown_connections 2023-07-21T22:38:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.601+0000 7f9bf54ab700 1 -- 172.21.15.107:0/621042717 wait complete. 2023-07-21T22:38:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.602+0000 7f9bf54ab700 1 Processor -- start 2023-07-21T22:38:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.602+0000 7f9bf54ab700 1 -- start start 2023-07-21T22:38:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.602+0000 7f9bf54ab700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf00fe180 0x7f9bf0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.602+0000 7f9bf54ab700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf00ff370 0x7f9bf0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.602+0000 7f9bf54ab700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf00ffd00 0x7f9bf0198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.602+0000 7f9bf54ab700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0199530 con 0x7f9bf00ff370 2023-07-21T22:38:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.602+0000 7f9bf54ab700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf01996a0 con 0x7f9bf00ffd00 2023-07-21T22:38:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.602+0000 7f9bf54ab700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0199860 con 0x7f9bf00fe180 2023-07-21T22:38:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.602+0000 7f9beeffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf00fe180 0x7f9bf0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.602+0000 7f9bee7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf00ff370 0x7f9bf0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.602+0000 7f9beeffd700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf00fe180 0x7f9bf0197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:40132/0 (socket says 172.21.15.107:40132) 2023-07-21T22:38:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.602+0000 7f9beeffd700 1 -- 172.21.15.107:0/3216151471 learned_addr learned my addr 172.21.15.107:0/3216151471 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.602+0000 7f9bef7fe700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf00ffd00 0x7f9bf0198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.603+0000 7f9beeffd700 1 -- 172.21.15.107:0/3216151471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf00ffd00 msgr2=0x7f9bf0198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.603+0000 7f9beeffd700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf00ffd00 0x7f9bf0198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.603+0000 7f9beeffd700 1 -- 172.21.15.107:0/3216151471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf00ff370 msgr2=0x7f9bf0198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.603+0000 7f9beeffd700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf00ff370 0x7f9bf0198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.603+0000 7f9beeffd700 1 -- 172.21.15.107:0/3216151471 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be00095f0 con 0x7f9bf00fe180 2023-07-21T22:38:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.603+0000 7f9bee7fc700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf00ff370 0x7f9bf0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.603+0000 7f9beeffd700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf00fe180 0x7f9bf0197b10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9be0000c00 tx=0x7f9be00047a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.603+0000 7f9bd7fff700 1 -- 172.21.15.107:0/3216151471 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be001d070 con 0x7f9bf00fe180 2023-07-21T22:38:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.603+0000 7f9bf54ab700 1 -- 172.21.15.107:0/3216151471 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf0199ae0 con 0x7f9bf00fe180 2023-07-21T22:38:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.603+0000 7f9bd7fff700 1 -- 172.21.15.107:0/3216151471 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9be0022470 con 0x7f9bf00fe180 2023-07-21T22:38:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.603+0000 7f9bd7fff700 1 -- 172.21.15.107:0/3216151471 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0017650 con 0x7f9bf00fe180 2023-07-21T22:38:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.603+0000 7f9bf54ab700 1 -- 172.21.15.107:0/3216151471 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf01a0300 con 0x7f9bf00fe180 2023-07-21T22:38:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.605+0000 7f9bd7fff700 1 -- 172.21.15.107:0/3216151471 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9be00225e0 con 0x7f9bf00fe180 2023-07-21T22:38:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.606+0000 7f9bd7fff700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9bd80a11d0 0x7f9bd80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.606+0000 7f9bee7fc700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9bd80a11d0 0x7f9bd80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.606+0000 7f9bd7fff700 1 -- 172.21.15.107:0/3216151471 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9be00cdf40 con 0x7f9bf00fe180 2023-07-21T22:38:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.606+0000 7f9bee7fc700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9bd80a11d0 0x7f9bd80a3680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9be400e510 tx=0x7f9be4010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9c01630b80 1 Processor -- start 2023-07-21T22:38:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9c01630b80 1 -- start start 2023-07-21T22:38:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9c01630b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a70c6b0 0x55e48a8b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9c01630b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 0x55e48a8b1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9c01630b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 0x55e48a8b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9c01630b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48a7b5c50 con 0x55e48a8b0830 2023-07-21T22:38:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9c01630b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48a8089e0 con 0x55e48a8b1580 2023-07-21T22:38:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9c01630b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48a801e70 con 0x55e48a70c6b0 2023-07-21T22:38:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9bee7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 0x55e48a8b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9bee7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 0x55e48a8b1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51584/0 (socket says 172.21.15.107:51584) 2023-07-21T22:38:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9bee7fc700 1 -- 172.21.15.107:0/2503626554 learned_addr learned my addr 172.21.15.107:0/2503626554 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9beeffd700 1 --2- 172.21.15.107:0/2503626554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a70c6b0 0x55e48a8b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9bef7fe700 1 --2- 172.21.15.107:0/2503626554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 0x55e48a8b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9bee7fc700 1 -- 172.21.15.107:0/2503626554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a70c6b0 msgr2=0x55e48a8b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9bee7fc700 1 --2- 172.21.15.107:0/2503626554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a70c6b0 0x55e48a8b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9bee7fc700 1 -- 172.21.15.107:0/2503626554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 msgr2=0x55e48a8b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9bee7fc700 1 --2- 172.21.15.107:0/2503626554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 0x55e48a8b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.607+0000 7f9bee7fc700 1 -- 172.21.15.107:0/2503626554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e48a62fd80 con 0x55e48a8b0830 2023-07-21T22:38:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.608+0000 7f9bee7fc700 1 --2- 172.21.15.107:0/2503626554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 0x55e48a8b1040 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9be400aca0 tx=0x7f9be4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ae443f9b6d97 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.610+0000 7f9bd57fa700 1 -- 172.21.15.107:0/2503626554 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be4013040 con 0x55e48a8b0830 2023-07-21T22:38:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.610+0000 7f9bd57fa700 1 -- 172.21.15.107:0/2503626554 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9be4003df0 con 0x55e48a8b0830 2023-07-21T22:38:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.610+0000 7f9bd57fa700 1 -- 172.21.15.107:0/2503626554 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be401c660 con 0x55e48a8b0830 2023-07-21T22:38:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 -- 172.21.15.107:0/2503626554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 msgr2=0x55e48a8b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 --2- 172.21.15.107:0/2503626554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 0x55e48a8b1040 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9be400aca0 tx=0x7f9be4014000 comp rx=0 tx=0).stop 2023-07-21T22:38:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 -- 172.21.15.107:0/2503626554 shutdown_connections 2023-07-21T22:38:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 --2- 172.21.15.107:0/2503626554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 0x55e48a8b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 --2- 172.21.15.107:0/2503626554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a70c6b0 0x55e48a8b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 --2- 172.21.15.107:0/2503626554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 0x55e48a8b1040 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 -- 172.21.15.107:0/2503626554 >> 172.21.15.107:0/2503626554 conn(0x55e48a8a5ae0 msgr2=0x55e48a8aac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 -- 172.21.15.107:0/2503626554 shutdown_connections 2023-07-21T22:38:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 -- 172.21.15.107:0/2503626554 wait complete. 2023-07-21T22:38:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 -- start start 2023-07-21T22:38:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 Processor -- start 2023-07-21T22:38:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 0x55e48a8b9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b0830 0x55e48a8b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a8b1580 0x55e48a8c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48a7a1a60 con 0x55e48a70c6b0 2023-07-21T22:38:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48a7a1760 con 0x55e48a8b0830 2023-07-21T22:38:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48a6858c0 con 0x55e48a8b1580 2023-07-21T22:38:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9c01630b80 10 client.0 _ll_register_callbacks cb 0x55e48a79f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9beeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 0x55e48a8b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9beeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 0x55e48a8b9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51588/0 (socket says 172.21.15.107:51588) 2023-07-21T22:38:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.611+0000 7f9beeffd700 1 -- 172.21.15.107:0/3178276037 learned_addr learned my addr 172.21.15.107:0/3178276037 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9bef7fe700 1 --2- 172.21.15.107:0/3178276037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a8b1580 0x55e48a8c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9bee7fc700 1 --2- 172.21.15.107:0/3178276037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b0830 0x55e48a8b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9beeffd700 1 -- 172.21.15.107:0/3178276037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a8b1580 msgr2=0x55e48a8c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9beeffd700 1 --2- 172.21.15.107:0/3178276037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a8b1580 0x55e48a8c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9beeffd700 1 -- 172.21.15.107:0/3178276037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b0830 msgr2=0x55e48a8b9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9beeffd700 1 --2- 172.21.15.107:0/3178276037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b0830 0x55e48a8b9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9beeffd700 1 -- 172.21.15.107:0/3178276037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be4004130 con 0x55e48a70c6b0 2023-07-21T22:38:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9beeffd700 1 --2- 172.21.15.107:0/3178276037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 0x55e48a8b9850 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9be0097dd0 tx=0x7f9be0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482614f5d7bd0011 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9bd2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:38:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9bd2ff5700 1 -- 172.21.15.107:0/3178276037 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be00a3720 con 0x55e48a70c6b0 2023-07-21T22:38:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9bd2ff5700 1 -- 172.21.15.107:0/3178276037 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9be000fa90 con 0x55e48a70c6b0 2023-07-21T22:38:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9c01630b80 1 -- 172.21.15.107:0/3178276037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e48a75b490 con 0x55e48a70c6b0 2023-07-21T22:38:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9bd2ff5700 1 -- 172.21.15.107:0/3178276037 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9be000fc50 con 0x55e48a70c6b0 2023-07-21T22:38:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9c01630b80 10 client.16779 fetch_fsmap learned FSMap version 56 2023-07-21T22:38:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9c01630b80 1 -- 172.21.15.107:0/3178276037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55e48a618c00 con 0x55e48a70c6b0 2023-07-21T22:38:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.612+0000 7f9bd2ff5700 1 -- 172.21.15.107:0/3178276037 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9be00c9080 con 0x55e48a70c6b0 2023-07-21T22:38:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.613+0000 7f9bd2ff5700 1 -- 172.21.15.107:0/3178276037 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f9be009a8e0 con 0x55e48a70c6b0 2023-07-21T22:38:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.613+0000 7f9c01630b80 10 client.16779 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T22:38:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.613+0000 7f9c01630b80 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:38:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.613+0000 7f9c01630b80 20 client.16779 populate_metadata read hostname 'smithi107' 2023-07-21T22:38:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.613+0000 7f9c01630b80 1 --2- 172.21.15.107:0/3178276037 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e48a8c5340 0x55e48a8c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.613+0000 7f9c01630b80 4 client.16779 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:38:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.613+0000 7f9c01630b80 1 -- 172.21.15.107:0/3178276037 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e48a7fae50 con 0x55e48a8c5340 2023-07-21T22:38:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.613+0000 7f9bee7fc700 1 --2- 172.21.15.107:0/3178276037 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e48a8c5340 0x55e48a8c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.613+0000 7f9bee7fc700 1 --2- 172.21.15.107:0/3178276037 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e48a8c5340 0x55e48a8c7750 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b99e4e17ab6f99 server_cookie=1ca74039de369dcf in_seq=0 out_seq=0 2023-07-21T22:38:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.613+0000 7f9bd2ff5700 10 client.16779 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:38:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9bd2ff5700 1 -- 172.21.15.107:0/3178276037 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e48a7fae50 con 0x55e48a8c5340 2023-07-21T22:38:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9bd2ff5700 10 client.16779 handle_command_reply: tid=0 2023-07-21T22:38:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9c01630b80 1 client.16779 shutdown 2023-07-21T22:38:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9c01630b80 1 -- 172.21.15.107:0/3178276037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 msgr2=0x55e48a8b9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9c01630b80 1 --2- 172.21.15.107:0/3178276037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 0x55e48a8b9850 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9be0097dd0 tx=0x7f9be0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9c01630b80 1 -- 172.21.15.107:0/3178276037 shutdown_connections 2023-07-21T22:38:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9c01630b80 1 --2- 172.21.15.107:0/3178276037 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e48a8c5340 0x55e48a8c7750 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9c01630b80 1 --2- 172.21.15.107:0/3178276037 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a8b1580 0x55e48a8c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9bd2ff5700 0 client.16779 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:38:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9c01630b80 1 --2- 172.21.15.107:0/3178276037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 0x55e48a8b9850 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9c01630b80 1 --2- 172.21.15.107:0/3178276037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b0830 0x55e48a8b9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9c01630b80 1 -- 172.21.15.107:0/3178276037 >> 172.21.15.107:0/3178276037 conn(0x55e48a8a5ae0 msgr2=0x55e48a8aa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9c01630b80 1 -- 172.21.15.107:0/3178276037 shutdown_connections 2023-07-21T22:38:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9c01630b80 1 -- 172.21.15.107:0/3178276037 wait complete. 2023-07-21T22:38:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.614+0000 7f9c01630b80 20 client.16779 trim_cache size 0 max 16384 2023-07-21T22:38:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.628+0000 7f9c01630b80 1 Processor -- start 2023-07-21T22:38:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9c01630b80 1 -- start start 2023-07-21T22:38:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9c01630b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 0x55e48a8aab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9c01630b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a8b0830 0x55e48a8c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9c01630b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 0x55e48a8b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9c01630b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48a8b9420 con 0x55e48a70c6b0 2023-07-21T22:38:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9c01630b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48a8aff40 con 0x55e48a8b1580 2023-07-21T22:38:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9c01630b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48a8a7c00 con 0x55e48a8b0830 2023-07-21T22:38:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9beeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 0x55e48a8aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9beeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 0x55e48a8aab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51596/0 (socket says 172.21.15.107:51596) 2023-07-21T22:38:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9beeffd700 1 -- 172.21.15.107:0/1493193453 learned_addr learned my addr 172.21.15.107:0/1493193453 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9bee7fc700 1 --2- 172.21.15.107:0/1493193453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a8b0830 0x55e48a8c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9bef7fe700 1 --2- 172.21.15.107:0/1493193453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 0x55e48a8b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9beeffd700 1 -- 172.21.15.107:0/1493193453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a8b0830 msgr2=0x55e48a8c0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9beeffd700 1 --2- 172.21.15.107:0/1493193453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a8b0830 0x55e48a8c0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9beeffd700 1 -- 172.21.15.107:0/1493193453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 msgr2=0x55e48a8b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9beeffd700 1 --2- 172.21.15.107:0/1493193453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 0x55e48a8b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9beeffd700 1 -- 172.21.15.107:0/1493193453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e48a6561f0 con 0x55e48a70c6b0 2023-07-21T22:38:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9beeffd700 1 --2- 172.21.15.107:0/1493193453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 0x55e48a8aab10 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9be00996b0 tx=0x7f9be00977d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b34560bc5297170 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9bd5ffb700 1 -- 172.21.15.107:0/1493193453 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be00cb900 con 0x55e48a70c6b0 2023-07-21T22:38:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9bd5ffb700 1 -- 172.21.15.107:0/1493193453 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e48a75b490 con 0x55e48a70c6b0 2023-07-21T22:38:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.629+0000 7f9bd5ffb700 1 -- 172.21.15.107:0/1493193453 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be009b820 con 0x55e48a70c6b0 2023-07-21T22:38:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 -- 172.21.15.107:0/1493193453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 msgr2=0x55e48a8aab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 --2- 172.21.15.107:0/1493193453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 0x55e48a8aab10 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9be00996b0 tx=0x7f9be00977d0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 -- 172.21.15.107:0/1493193453 shutdown_connections 2023-07-21T22:38:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 --2- 172.21.15.107:0/1493193453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 0x55e48a8b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 --2- 172.21.15.107:0/1493193453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a70c6b0 0x55e48a8aab10 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 --2- 172.21.15.107:0/1493193453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a8b0830 0x55e48a8c0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 -- 172.21.15.107:0/1493193453 >> 172.21.15.107:0/1493193453 conn(0x55e48a8a5ae0 msgr2=0x55e48a8cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 -- 172.21.15.107:0/1493193453 shutdown_connections 2023-07-21T22:38:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 -- 172.21.15.107:0/1493193453 wait complete. 2023-07-21T22:38:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 -- start start 2023-07-21T22:38:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 Processor -- start 2023-07-21T22:38:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a70c6b0 0x55e48a8bc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 0x55e48a8bcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 0x55e48a8bfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48a8c02c0 con 0x55e48a8b0830 2023-07-21T22:38:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48a8c0430 con 0x55e48a8b1580 2023-07-21T22:38:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.630+0000 7f9c01630b80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48a8c05f0 con 0x55e48a70c6b0 2023-07-21T22:38:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9c01630b80 10 client.0 _ll_register_callbacks cb 0x7f9be009af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bee7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 0x55e48a8bcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bee7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 0x55e48a8bcd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51604/0 (socket says 172.21.15.107:51604) 2023-07-21T22:38:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bee7fc700 1 -- 172.21.15.107:0/710194352 learned_addr learned my addr 172.21.15.107:0/710194352 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9beeffd700 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a70c6b0 0x55e48a8bc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bef7fe700 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 0x55e48a8bfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bee7fc700 1 -- 172.21.15.107:0/710194352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a70c6b0 msgr2=0x55e48a8bc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bee7fc700 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a70c6b0 0x55e48a8bc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bee7fc700 1 -- 172.21.15.107:0/710194352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 msgr2=0x55e48a8bfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bee7fc700 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 0x55e48a8bfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bee7fc700 1 -- 172.21.15.107:0/710194352 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e48a618c00 con 0x55e48a8b0830 2023-07-21T22:38:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9beeffd700 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a70c6b0 0x55e48a8bc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:38:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bef7fe700 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 0x55e48a8bfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:38:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bee7fc700 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 0x55e48a8bcd60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9be4003fd0 tx=0x7f9be400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62e3872f042861 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bd2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:38:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bd2ff5700 1 -- 172.21.15.107:0/710194352 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be4013040 con 0x55e48a8b0830 2023-07-21T22:38:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bd2ff5700 1 -- 172.21.15.107:0/710194352 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9be40323a0 con 0x55e48a8b0830 2023-07-21T22:38:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9c01630b80 1 -- 172.21.15.107:0/710194352 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9be4003df0 con 0x55e48a8b0830 2023-07-21T22:38:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9bd2ff5700 1 -- 172.21.15.107:0/710194352 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9be40054b0 con 0x55e48a8b0830 2023-07-21T22:38:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9c01630b80 10 client.16785 fetch_fsmap learned FSMap version 56 2023-07-21T22:38:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.631+0000 7f9c01630b80 1 -- 172.21.15.107:0/710194352 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55e48a62e250 con 0x55e48a8b0830 2023-07-21T22:38:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.632+0000 7f9bd2ff5700 1 -- 172.21.15.107:0/710194352 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9be4040020 con 0x55e48a8b0830 2023-07-21T22:38:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.632+0000 7f9bd2ff5700 1 -- 172.21.15.107:0/710194352 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f9be403a020 con 0x55e48a8b0830 2023-07-21T22:38:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.632+0000 7f9c01630b80 10 client.16785 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T22:38:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.632+0000 7f9c01630b80 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:38:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.632+0000 7f9c01630b80 20 client.16785 populate_metadata read hostname 'smithi107' 2023-07-21T22:38:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.632+0000 7f9c01630b80 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e48a919f40 0x55e48a91c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.632+0000 7f9c01630b80 4 client.16785 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:38:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.632+0000 7f9c01630b80 1 -- 172.21.15.107:0/710194352 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e48a7bcff0 con 0x55e48a919f40 2023-07-21T22:38:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.632+0000 7f9beeffd700 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e48a919f40 0x55e48a91c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.632+0000 7f9bd2ff5700 10 client.16785 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:38:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.632+0000 7f9beeffd700 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e48a919f40 0x55e48a91c350 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc2164ba183b51f server_cookie=4eea6ef8c69168 in_seq=0 out_seq=0 2023-07-21T22:38:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9bd2ff5700 1 -- 172.21.15.107:0/710194352 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e48a7bcff0 con 0x55e48a919f40 2023-07-21T22:38:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9bd2ff5700 10 client.16785 handle_command_reply: tid=0 2023-07-21T22:38:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9c01630b80 1 client.16785 shutdown 2023-07-21T22:38:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9c01630b80 1 -- 172.21.15.107:0/710194352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 msgr2=0x55e48a8bcd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9c01630b80 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 0x55e48a8bcd60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9be4003fd0 tx=0x7f9be400eda0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9c01630b80 1 -- 172.21.15.107:0/710194352 shutdown_connections 2023-07-21T22:38:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9c01630b80 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x55e48a919f40 0x55e48a91c350 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9c01630b80 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e48a8b1580 0x55e48a8bfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9bd2ff5700 0 client.16785 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:38:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9c01630b80 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e48a70c6b0 0x55e48a8bc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9c01630b80 1 --2- 172.21.15.107:0/710194352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e48a8b0830 0x55e48a8bcd60 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9c01630b80 1 -- 172.21.15.107:0/710194352 >> 172.21.15.107:0/710194352 conn(0x55e48a8a5ae0 msgr2=0x55e48a8b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9c01630b80 1 -- 172.21.15.107:0/710194352 shutdown_connections 2023-07-21T22:38:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.641+0000 7f9c01630b80 1 -- 172.21.15.107:0/710194352 wait complete. 2023-07-21T22:38:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.642+0000 7f9c01630b80 20 client.16785 trim_cache size 0 max 16384 2023-07-21T22:38:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:38:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:38:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:38:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:38:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.642+0000 7f9bf54ab700 1 -- 172.21.15.107:0/3216151471 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9bd80a11d0 msgr2=0x7f9bd80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.642+0000 7f9bf54ab700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9bd80a11d0 0x7f9bd80a3680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9be400e510 tx=0x7f9be4010040 comp rx=0 tx=0).stop 2023-07-21T22:38:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.642+0000 7f9bf54ab700 1 -- 172.21.15.107:0/3216151471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf00fe180 msgr2=0x7f9bf0197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.642+0000 7f9bf54ab700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf00fe180 0x7f9bf0197b10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9be0000c00 tx=0x7f9be00047a0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.642+0000 7f9bf54ab700 1 -- 172.21.15.107:0/3216151471 shutdown_connections 2023-07-21T22:38:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.642+0000 7f9bf54ab700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf00ffd00 0x7f9bf0198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.642+0000 7f9bf54ab700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f9bd80a11d0 0x7f9bd80a3680 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.642+0000 7f9bf54ab700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bf00fe180 0x7f9bf0197b10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.642+0000 7f9bf54ab700 1 --2- 172.21.15.107:0/3216151471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bf00ff370 0x7f9bf0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.642+0000 7f9bf54ab700 1 -- 172.21.15.107:0/3216151471 >> 172.21.15.107:0/3216151471 conn(0x7f9bf00f9780 msgr2=0x7f9bf00fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.642+0000 7f9bf54ab700 1 -- 172.21.15.107:0/3216151471 shutdown_connections 2023-07-21T22:38:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:09.642+0000 7f9bf54ab700 1 -- 172.21.15.107:0/3216151471 wait complete. 2023-07-21T22:38:09.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:09 smithi132 ceph-mon[97863]: pgmap v566: 97 pgs: 97 active+clean; 936 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 459 op/s 2023-07-21T22:38:09.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:09 smithi107 ceph-mon[96543]: pgmap v566: 97 pgs: 97 active+clean; 936 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 459 op/s 2023-07-21T22:38:10.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:09 smithi190 ceph-mon[98334]: pgmap v566: 97 pgs: 97 active+clean; 936 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 459 op/s 2023-07-21T22:38:11.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:11 smithi132 ceph-mon[97863]: pgmap v567: 97 pgs: 97 active+clean; 935 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 357 op/s 2023-07-21T22:38:11.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:11 smithi107 ceph-mon[96543]: pgmap v567: 97 pgs: 97 active+clean; 935 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 357 op/s 2023-07-21T22:38:12.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:11 smithi190 ceph-mon[98334]: pgmap v567: 97 pgs: 97 active+clean; 935 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 357 op/s 2023-07-21T22:38:13.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:13 smithi132 ceph-mon[97863]: pgmap v568: 97 pgs: 97 active+clean; 924 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 406 op/s 2023-07-21T22:38:13.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:13 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:38:13.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:13 smithi107 ceph-mon[96543]: pgmap v568: 97 pgs: 97 active+clean; 924 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 406 op/s 2023-07-21T22:38:13.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:13 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:38:14.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:13 smithi190 ceph-mon[98334]: pgmap v568: 97 pgs: 97 active+clean; 924 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 406 op/s 2023-07-21T22:38:14.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:13 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:38:14.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:38:14.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:38:14.930 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:38:14.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:38:14.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:38:14.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:38:15.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:38:15.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:38:15.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:38:15.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:15 smithi132 ceph-mon[97863]: pgmap v569: 97 pgs: 97 active+clean; 920 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 966 KiB/s wr, 340 op/s 2023-07-21T22:38:15.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:15 smithi107 ceph-mon[96543]: pgmap v569: 97 pgs: 97 active+clean; 920 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 966 KiB/s wr, 340 op/s 2023-07-21T22:38:16.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:15 smithi190 ceph-mon[98334]: pgmap v569: 97 pgs: 97 active+clean; 920 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 965 KiB/s rd, 966 KiB/s wr, 340 op/s 2023-07-21T22:38:17.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:17 smithi132 ceph-mon[97863]: pgmap v570: 97 pgs: 97 active+clean; 918 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 292 op/s 2023-07-21T22:38:17.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:17 smithi107 ceph-mon[96543]: pgmap v570: 97 pgs: 97 active+clean; 918 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 292 op/s 2023-07-21T22:38:18.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:17 smithi190 ceph-mon[98334]: pgmap v570: 97 pgs: 97 active+clean; 918 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 292 op/s 2023-07-21T22:38:19.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:19 smithi132 ceph-mon[97863]: pgmap v571: 97 pgs: 97 active+clean; 905 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 330 op/s 2023-07-21T22:38:19.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:19 smithi107 ceph-mon[96543]: pgmap v571: 97 pgs: 97 active+clean; 905 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 330 op/s 2023-07-21T22:38:20.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:19 smithi190 ceph-mon[98334]: pgmap v571: 97 pgs: 97 active+clean; 905 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 330 op/s 2023-07-21T22:38:21.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:21 smithi132 ceph-mon[97863]: pgmap v572: 97 pgs: 97 active+clean; 902 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 754 KiB/s wr, 225 op/s 2023-07-21T22:38:21.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:21 smithi107 ceph-mon[96543]: pgmap v572: 97 pgs: 97 active+clean; 902 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 754 KiB/s wr, 225 op/s 2023-07-21T22:38:22.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:21 smithi190 ceph-mon[98334]: pgmap v572: 97 pgs: 97 active+clean; 902 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 754 KiB/s wr, 225 op/s 2023-07-21T22:38:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.754+0000 7f7752336700 1 -- 172.21.15.107:0/3867211722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f774c0ffc90 msgr2=0x7f774c100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.754+0000 7f7752336700 1 --2- 172.21.15.107:0/3867211722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f774c0ffc90 0x7f774c100100 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f773c009990 tx=0x7f773c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.754+0000 7f7752336700 1 -- 172.21.15.107:0/3867211722 shutdown_connections 2023-07-21T22:38:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.754+0000 7f7752336700 1 --2- 172.21.15.107:0/3867211722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f774c100640 0x7f774c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.754+0000 7f7752336700 1 --2- 172.21.15.107:0/3867211722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f774c0fea00 0x7f774c0fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.754+0000 7f7752336700 1 --2- 172.21.15.107:0/3867211722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f774c0ffc90 0x7f774c100100 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.754+0000 7f7752336700 1 -- 172.21.15.107:0/3867211722 >> 172.21.15.107:0/3867211722 conn(0x7f774c0fa000 msgr2=0x7f774c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.754+0000 7f7752336700 1 -- 172.21.15.107:0/3867211722 shutdown_connections 2023-07-21T22:38:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.754+0000 7f7752336700 1 -- 172.21.15.107:0/3867211722 wait complete. 2023-07-21T22:38:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.754+0000 7f7752336700 1 Processor -- start 2023-07-21T22:38:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.754+0000 7f7752336700 1 -- start start 2023-07-21T22:38:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f7752336700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f774c0fea00 0x7f774c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f7752336700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f774c0ffc90 0x7f774c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f7752336700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f774c100640 0x7f774c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f7752336700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c068850 con 0x7f774c0ffc90 2023-07-21T22:38:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f7752336700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c19f960 con 0x7f774c0fea00 2023-07-21T22:38:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f7752336700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c19fad0 con 0x7f774c100640 2023-07-21T22:38:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f774b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f774c0ffc90 0x7f774c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f774b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f774c0ffc90 0x7f774c19ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51618/0 (socket says 172.21.15.107:51618) 2023-07-21T22:38:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f774b7fe700 1 -- 172.21.15.107:0/1855725312 learned_addr learned my addr 172.21.15.107:0/1855725312 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f77508d3700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f774c100640 0x7f774c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f774bfff700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f774c0fea00 0x7f774c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f774b7fe700 1 -- 172.21.15.107:0/1855725312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f774c100640 msgr2=0x7f774c19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f774b7fe700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f774c100640 0x7f774c19f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f774b7fe700 1 -- 172.21.15.107:0/1855725312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f774c0fea00 msgr2=0x7f774c19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f774b7fe700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f774c0fea00 0x7f774c19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f774b7fe700 1 -- 172.21.15.107:0/1855725312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c0095f0 con 0x7f774c0ffc90 2023-07-21T22:38:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f77508d3700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f774c100640 0x7f774c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f774b7fe700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f774c0ffc90 0x7f774c19ced0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f773c0059d0 tx=0x7f773c0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.755+0000 7f77497fa700 1 -- 172.21.15.107:0/1855725312 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c01d070 con 0x7f774c0ffc90 2023-07-21T22:38:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.756+0000 7f77497fa700 1 -- 172.21.15.107:0/1855725312 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f773c00fb90 con 0x7f774c0ffc90 2023-07-21T22:38:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.756+0000 7f7752336700 1 -- 172.21.15.107:0/1855725312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774c19fd50 con 0x7f774c0ffc90 2023-07-21T22:38:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.756+0000 7f77497fa700 1 -- 172.21.15.107:0/1855725312 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c017650 con 0x7f774c0ffc90 2023-07-21T22:38:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.756+0000 7f7752336700 1 -- 172.21.15.107:0/1855725312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f774c1a01d0 con 0x7f774c0ffc90 2023-07-21T22:38:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.758+0000 7f77497fa700 1 -- 172.21.15.107:0/1855725312 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f773c00fd00 con 0x7f774c0ffc90 2023-07-21T22:38:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.758+0000 7f77497fa700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f77340a11d0 0x7f77340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.758+0000 7f77497fa700 1 -- 172.21.15.107:0/1855725312 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f773c0cdf90 con 0x7f774c0ffc90 2023-07-21T22:38:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.758+0000 7f774bfff700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f77340a11d0 0x7f77340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.759+0000 7f774bfff700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f77340a11d0 0x7f77340a3680 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7740005950 tx=0x7f774000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.759+0000 7f775e4bbb80 1 Processor -- start 2023-07-21T22:38:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.759+0000 7f775e4bbb80 1 -- start start 2023-07-21T22:38:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.759+0000 7f775e4bbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608a686b0 0x564608c0c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.759+0000 7f775e4bbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608c0c830 0x564608c0d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.759+0000 7f775e4bbb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c0d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.759+0000 7f775e4bbb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564608b11c50 con 0x564608c0d580 2023-07-21T22:38:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.759+0000 7f775e4bbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564608b649e0 con 0x564608a686b0 2023-07-21T22:38:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.759+0000 7f775e4bbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564608b5de70 con 0x564608c0c830 2023-07-21T22:38:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f77508d3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c0d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f774b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608c0c830 0x564608c0d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f774b7fe700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608c0c830 0x564608c0d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:40184/0 (socket says 172.21.15.107:40184) 2023-07-21T22:38:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f774b7fe700 1 -- 172.21.15.107:0/2999763983 learned_addr learned my addr 172.21.15.107:0/2999763983 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f774bfff700 1 --2- 172.21.15.107:0/2999763983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608a686b0 0x564608c0c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f774b7fe700 1 -- 172.21.15.107:0/2999763983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608a686b0 msgr2=0x564608c0c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f774b7fe700 1 --2- 172.21.15.107:0/2999763983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608a686b0 0x564608c0c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f774b7fe700 1 -- 172.21.15.107:0/2999763983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 msgr2=0x564608c0d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f774b7fe700 1 --2- 172.21.15.107:0/2999763983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c0d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f774b7fe700 1 -- 172.21.15.107:0/2999763983 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56460898bd80 con 0x564608c0c830 2023-07-21T22:38:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f77508d3700 1 --2- 172.21.15.107:0/2999763983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c0d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f774b7fe700 1 --2- 172.21.15.107:0/2999763983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608c0c830 0x564608c0d040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f773c097910 tx=0x7f773c09ad70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fd7ef2e73d0b3ba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f77327fc700 1 -- 172.21.15.107:0/2999763983 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c09f440 con 0x564608c0c830 2023-07-21T22:38:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f77327fc700 1 -- 172.21.15.107:0/2999763983 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f773c0a37d0 con 0x564608c0c830 2023-07-21T22:38:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.760+0000 7f77327fc700 1 -- 172.21.15.107:0/2999763983 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c0a12b0 con 0x564608c0c830 2023-07-21T22:38:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.761+0000 7f775e4bbb80 1 -- 172.21.15.107:0/2999763983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608c0c830 msgr2=0x564608c0d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.761+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/2999763983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608c0c830 0x564608c0d040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f773c097910 tx=0x7f773c09ad70 comp rx=0 tx=0).stop 2023-07-21T22:38:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.761+0000 7f775e4bbb80 1 -- 172.21.15.107:0/2999763983 shutdown_connections 2023-07-21T22:38:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.761+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/2999763983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c0d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.761+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/2999763983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608a686b0 0x564608c0c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.761+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/2999763983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608c0c830 0x564608c0d040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.761+0000 7f775e4bbb80 1 -- 172.21.15.107:0/2999763983 >> 172.21.15.107:0/2999763983 conn(0x564608c01ae0 msgr2=0x564608c06c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.761+0000 7f775e4bbb80 1 -- 172.21.15.107:0/2999763983 shutdown_connections 2023-07-21T22:38:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.761+0000 7f775e4bbb80 1 -- 172.21.15.107:0/2999763983 wait complete. 2023-07-21T22:38:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.761+0000 7f775e4bbb80 1 -- start start 2023-07-21T22:38:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f775e4bbb80 1 Processor -- start 2023-07-21T22:38:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f775e4bbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c15450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f775e4bbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0c830 0x564608c15990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f775e4bbb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c1c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f775e4bbb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564608afda60 con 0x564608c0d580 2023-07-21T22:38:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f775e4bbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564608afd760 con 0x564608c0c830 2023-07-21T22:38:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f775e4bbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646089e18c0 con 0x564608a686b0 2023-07-21T22:38:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f775e4bbb80 10 client.0 _ll_register_callbacks cb 0x564608afb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f77508d3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c1c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f774bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c15450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f77508d3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c1c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51634/0 (socket says 172.21.15.107:51634) 2023-07-21T22:38:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f774bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c15450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:40188/0 (socket says 172.21.15.107:40188) 2023-07-21T22:38:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f77508d3700 1 -- 172.21.15.107:0/2899401949 learned_addr learned my addr 172.21.15.107:0/2899401949 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f77508d3700 1 -- 172.21.15.107:0/2899401949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 msgr2=0x564608c15450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f77508d3700 1 --2- 172.21.15.107:0/2899401949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c15450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f774b7fe700 1 --2- 172.21.15.107:0/2899401949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0c830 0x564608c15990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f77508d3700 1 -- 172.21.15.107:0/2899401949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0c830 msgr2=0x564608c15990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f77508d3700 1 --2- 172.21.15.107:0/2899401949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0c830 0x564608c15990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f77508d3700 1 -- 172.21.15.107:0/2899401949 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7744005e70 con 0x564608c0d580 2023-07-21T22:38:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f774b7fe700 1 --2- 172.21.15.107:0/2899401949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0c830 0x564608c15990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:38:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f77508d3700 1 --2- 172.21.15.107:0/2899401949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c1c9d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f77440042c0 tx=0x7f77440045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b09a3717fa1638 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.762+0000 7f772b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:38:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f772b7fe700 1 -- 172.21.15.107:0/2899401949 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744007500 con 0x564608c0d580 2023-07-21T22:38:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f772b7fe700 1 -- 172.21.15.107:0/2899401949 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7744007660 con 0x564608c0d580 2023-07-21T22:38:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f775e4bbb80 1 -- 172.21.15.107:0/2899401949 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564608ab7490 con 0x564608c0d580 2023-07-21T22:38:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f772b7fe700 1 -- 172.21.15.107:0/2899401949 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7744007820 con 0x564608c0d580 2023-07-21T22:38:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f775e4bbb80 10 client.16797 fetch_fsmap learned FSMap version 56 2023-07-21T22:38:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f775e4bbb80 1 -- 172.21.15.107:0/2899401949 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x7f774c1a01d0 con 0x564608c0d580 2023-07-21T22:38:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f772b7fe700 1 -- 172.21.15.107:0/2899401949 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564608ab7490 con 0x564608c0d580 2023-07-21T22:38:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f772b7fe700 1 -- 172.21.15.107:0/2899401949 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f774401c020 con 0x564608c0d580 2023-07-21T22:38:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f775e4bbb80 10 client.16797 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T22:38:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f775e4bbb80 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:38:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f775e4bbb80 20 client.16797 populate_metadata read hostname 'smithi107' 2023-07-21T22:38:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/2899401949 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x564608c213b0 0x564608c237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f775e4bbb80 4 client.16797 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:38:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f775e4bbb80 1 -- 172.21.15.107:0/2899401949 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564608b56e50 con 0x564608c213b0 2023-07-21T22:38:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.763+0000 7f774bfff700 1 --2- 172.21.15.107:0/2899401949 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x564608c213b0 0x564608c237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.764+0000 7f772b7fe700 10 client.16797 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:38:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.764+0000 7f774bfff700 1 --2- 172.21.15.107:0/2899401949 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x564608c213b0 0x564608c237c0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e685416052fcc457 server_cookie=7dfbed24552e0386 in_seq=0 out_seq=0 2023-07-21T22:38:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.764+0000 7f772b7fe700 1 -- 172.21.15.107:0/2899401949 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564608b56e50 con 0x564608c213b0 2023-07-21T22:38:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.764+0000 7f772b7fe700 10 client.16797 handle_command_reply: tid=0 2023-07-21T22:38:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.764+0000 7f775e4bbb80 1 client.16797 shutdown 2023-07-21T22:38:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.764+0000 7f775e4bbb80 1 -- 172.21.15.107:0/2899401949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 msgr2=0x564608c1c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.764+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/2899401949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c1c9d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f77440042c0 tx=0x7f77440045d0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.765+0000 7f775e4bbb80 1 -- 172.21.15.107:0/2899401949 shutdown_connections 2023-07-21T22:38:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.765+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/2899401949 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x564608c213b0 0x564608c237c0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.765+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/2899401949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c1c9d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.765+0000 7f772b7fe700 0 client.16797 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:38:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.765+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/2899401949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c15450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.765+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/2899401949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0c830 0x564608c15990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.765+0000 7f775e4bbb80 1 -- 172.21.15.107:0/2899401949 >> 172.21.15.107:0/2899401949 conn(0x564608c01ae0 msgr2=0x564608c063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.765+0000 7f775e4bbb80 1 -- 172.21.15.107:0/2899401949 shutdown_connections 2023-07-21T22:38:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.765+0000 7f775e4bbb80 1 -- 172.21.15.107:0/2899401949 wait complete. 2023-07-21T22:38:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.765+0000 7f775e4bbb80 20 client.16797 trim_cache size 0 max 16384 2023-07-21T22:38:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f775e4bbb80 1 Processor -- start 2023-07-21T22:38:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f775e4bbb80 1 -- start start 2023-07-21T22:38:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f775e4bbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c0b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f775e4bbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0c830 0x564608c1c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f775e4bbb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c05b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f775e4bbb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564608c14900 con 0x564608c0d580 2023-07-21T22:38:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f775e4bbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564608c14fc0 con 0x564608c0c830 2023-07-21T22:38:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f775e4bbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564608c07700 con 0x564608a686b0 2023-07-21T22:38:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f77508d3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c05b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f774b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0c830 0x564608c1c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f77508d3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c05b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51642/0 (socket says 172.21.15.107:51642) 2023-07-21T22:38:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f77508d3700 1 -- 172.21.15.107:0/3393531692 learned_addr learned my addr 172.21.15.107:0/3393531692 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f774bfff700 1 --2- 172.21.15.107:0/3393531692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c0b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f77508d3700 1 -- 172.21.15.107:0/3393531692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 msgr2=0x564608c0b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f77508d3700 1 --2- 172.21.15.107:0/3393531692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c0b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f77508d3700 1 -- 172.21.15.107:0/3393531692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0c830 msgr2=0x564608c1c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f77508d3700 1 --2- 172.21.15.107:0/3393531692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0c830 0x564608c1c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f77508d3700 1 -- 172.21.15.107:0/3393531692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564608974c00 con 0x564608c0d580 2023-07-21T22:38:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f77508d3700 1 --2- 172.21.15.107:0/3393531692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c05b50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f774401ffd0 tx=0x7f77440049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e70ea33d47d1261 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f7732ffd700 1 -- 172.21.15.107:0/3393531692 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744006000 con 0x564608c0d580 2023-07-21T22:38:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f7732ffd700 1 -- 172.21.15.107:0/3393531692 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f774400f040 con 0x564608c0d580 2023-07-21T22:38:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.780+0000 7f7732ffd700 1 -- 172.21.15.107:0/3393531692 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744007e60 con 0x564608c0d580 2023-07-21T22:38:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.781+0000 7f775e4bbb80 1 -- 172.21.15.107:0/3393531692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 msgr2=0x564608c05b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.781+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/3393531692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c05b50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f774401ffd0 tx=0x7f77440049d0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.781+0000 7f775e4bbb80 1 -- 172.21.15.107:0/3393531692 shutdown_connections 2023-07-21T22:38:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.781+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/3393531692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0d580 0x564608c05b50 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.781+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/3393531692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c0b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.781+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/3393531692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0c830 0x564608c1c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.781+0000 7f775e4bbb80 1 -- 172.21.15.107:0/3393531692 >> 172.21.15.107:0/3393531692 conn(0x564608c01ae0 msgr2=0x564608c0af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.781+0000 7f775e4bbb80 1 -- 172.21.15.107:0/3393531692 shutdown_connections 2023-07-21T22:38:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.781+0000 7f775e4bbb80 1 -- 172.21.15.107:0/3393531692 wait complete. 2023-07-21T22:38:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.781+0000 7f775e4bbb80 1 -- start start 2023-07-21T22:38:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f775e4bbb80 1 Processor -- start 2023-07-21T22:38:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f775e4bbb80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c18b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f775e4bbb80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0c830 0x564608c19060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f775e4bbb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0d580 0x564608c1ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f775e4bbb80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564608c1bfd0 con 0x564608c0c830 2023-07-21T22:38:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f775e4bbb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564608c1c140 con 0x564608c0d580 2023-07-21T22:38:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f775e4bbb80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564608c1c300 con 0x564608a686b0 2023-07-21T22:38:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f775e4bbb80 10 client.0 _ll_register_callbacks cb 0x7f774400eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f774b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0c830 0x564608c19060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f774bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c18b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f77508d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0d580 0x564608c1ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f774b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0c830 0x564608c19060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51646/0 (socket says 172.21.15.107:51646) 2023-07-21T22:38:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f774b7fe700 1 -- 172.21.15.107:0/340897991 learned_addr learned my addr 172.21.15.107:0/340897991 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f774bfff700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c18b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.107:40202/0 (socket says 172.21.15.107:40202) 2023-07-21T22:38:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f774b7fe700 1 -- 172.21.15.107:0/340897991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 msgr2=0x564608c18b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f774b7fe700 1 --2- 172.21.15.107:0/340897991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c18b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f774b7fe700 1 -- 172.21.15.107:0/340897991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0d580 msgr2=0x564608c1ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f774b7fe700 1 --2- 172.21.15.107:0/340897991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0d580 0x564608c1ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f774b7fe700 1 -- 172.21.15.107:0/340897991 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f774c1a01d0 con 0x564608c0c830 2023-07-21T22:38:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f774bfff700 1 --2- 172.21.15.107:0/340897991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c18b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f77508d3700 1 --2- 172.21.15.107:0/340897991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0d580 0x564608c1ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f774b7fe700 1 --2- 172.21.15.107:0/340897991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0c830 0x564608c19060 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f773c09ba70 tx=0x7f773c0a1420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4b26f8a2ad1ad7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f772b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:38:22.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.782+0000 7f772b7fe700 1 -- 172.21.15.107:0/340897991 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c0b9070 con 0x564608c0c830 2023-07-21T22:38:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f775e4bbb80 1 -- 172.21.15.107:0/340897991 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f773c0a37d0 con 0x564608c0c830 2023-07-21T22:38:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f772b7fe700 1 -- 172.21.15.107:0/340897991 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f773c09f120 con 0x564608c0c830 2023-07-21T22:38:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f772b7fe700 1 -- 172.21.15.107:0/340897991 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f773c0b3050 con 0x564608c0c830 2023-07-21T22:38:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f775e4bbb80 10 client.16803 fetch_fsmap learned FSMap version 56 2023-07-21T22:38:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f775e4bbb80 1 -- 172.21.15.107:0/340897991 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5646089b21f0 con 0x564608c0c830 2023-07-21T22:38:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f772b7fe700 1 -- 172.21.15.107:0/340897991 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f773c0a37d0 con 0x564608c0c830 2023-07-21T22:38:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f772b7fe700 1 -- 172.21.15.107:0/340897991 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f773c09a370 con 0x564608c0c830 2023-07-21T22:38:22.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f775e4bbb80 10 client.16803 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T22:38:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f775e4bbb80 10 client.16803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:38:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f775e4bbb80 20 client.16803 populate_metadata read hostname 'smithi107' 2023-07-21T22:38:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/340897991 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x564608c75cc0 0x564608c780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f775e4bbb80 4 client.16803 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:38:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f775e4bbb80 1 -- 172.21.15.107:0/340897991 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564608b18ff0 con 0x564608c75cc0 2023-07-21T22:38:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.783+0000 7f774bfff700 1 --2- 172.21.15.107:0/340897991 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x564608c75cc0 0x564608c780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.784+0000 7f772b7fe700 10 client.16803 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:38:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.784+0000 7f774bfff700 1 --2- 172.21.15.107:0/340897991 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x564608c75cc0 0x564608c780a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d1be14d8e81544 server_cookie=d3f4f4339dc85b4a in_seq=0 out_seq=0 2023-07-21T22:38:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f772b7fe700 1 -- 172.21.15.107:0/340897991 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564608b18ff0 con 0x564608c75cc0 2023-07-21T22:38:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f772b7fe700 10 client.16803 handle_command_reply: tid=0 2023-07-21T22:38:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f775e4bbb80 1 client.16803 shutdown 2023-07-21T22:38:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f775e4bbb80 1 -- 172.21.15.107:0/340897991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0c830 msgr2=0x564608c19060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/340897991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0c830 0x564608c19060 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f773c09ba70 tx=0x7f773c0a1420 comp rx=0 tx=0).stop 2023-07-21T22:38:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f775e4bbb80 1 -- 172.21.15.107:0/340897991 shutdown_connections 2023-07-21T22:38:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/340897991 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x564608c75cc0 0x564608c780a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/340897991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564608c0d580 0x564608c1ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/340897991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564608a686b0 0x564608c18b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f775e4bbb80 1 --2- 172.21.15.107:0/340897991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564608c0c830 0x564608c19060 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f772b7fe700 0 client.16803 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:38:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f775e4bbb80 1 -- 172.21.15.107:0/340897991 >> 172.21.15.107:0/340897991 conn(0x564608c01ae0 msgr2=0x564608c15900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f775e4bbb80 1 -- 172.21.15.107:0/340897991 shutdown_connections 2023-07-21T22:38:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f775e4bbb80 1 -- 172.21.15.107:0/340897991 wait complete. 2023-07-21T22:38:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.792+0000 7f775e4bbb80 20 client.16803 trim_cache size 0 max 16384 2023-07-21T22:38:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:38:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:38:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:38:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:38:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.793+0000 7f7752336700 1 -- 172.21.15.107:0/1855725312 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f77340a11d0 msgr2=0x7f77340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.793+0000 7f7752336700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f77340a11d0 0x7f77340a3680 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7740005950 tx=0x7f774000a300 comp rx=0 tx=0).stop 2023-07-21T22:38:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.793+0000 7f7752336700 1 -- 172.21.15.107:0/1855725312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f774c0ffc90 msgr2=0x7f774c19ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.793+0000 7f7752336700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f774c0ffc90 0x7f774c19ced0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f773c0059d0 tx=0x7f773c0047a0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.793+0000 7f7752336700 1 -- 172.21.15.107:0/1855725312 shutdown_connections 2023-07-21T22:38:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.793+0000 7f7752336700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f77340a11d0 0x7f77340a3680 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.793+0000 7f7752336700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f774c100640 0x7f774c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.793+0000 7f7752336700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f774c0fea00 0x7f774c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.793+0000 7f7752336700 1 --2- 172.21.15.107:0/1855725312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f774c0ffc90 0x7f774c19ced0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.793+0000 7f7752336700 1 -- 172.21.15.107:0/1855725312 >> 172.21.15.107:0/1855725312 conn(0x7f774c0fa000 msgr2=0x7f774c0fc210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.793+0000 7f7752336700 1 -- 172.21.15.107:0/1855725312 shutdown_connections 2023-07-21T22:38:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:22.793+0000 7f7752336700 1 -- 172.21.15.107:0/1855725312 wait complete. 2023-07-21T22:38:23.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:23 smithi132 ceph-mon[97863]: pgmap v573: 97 pgs: 97 active+clean; 892 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 302 op/s 2023-07-21T22:38:23.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:23 smithi107 ceph-mon[96543]: pgmap v573: 97 pgs: 97 active+clean; 892 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 302 op/s 2023-07-21T22:38:24.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:23 smithi190 ceph-mon[98334]: pgmap v573: 97 pgs: 97 active+clean; 892 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 302 op/s 2023-07-21T22:38:25.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:25 smithi132 ceph-mon[97863]: pgmap v574: 97 pgs: 97 active+clean; 881 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 296 op/s 2023-07-21T22:38:25.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:25 smithi107 ceph-mon[96543]: pgmap v574: 97 pgs: 97 active+clean; 881 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 296 op/s 2023-07-21T22:38:26.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:25 smithi190 ceph-mon[98334]: pgmap v574: 97 pgs: 97 active+clean; 881 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 296 op/s 2023-07-21T22:38:27.929 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:27 smithi132 ceph-mon[97863]: pgmap v575: 97 pgs: 97 active+clean; 869 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2023-07-21T22:38:27.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:27 smithi107 ceph-mon[96543]: pgmap v575: 97 pgs: 97 active+clean; 869 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2023-07-21T22:38:28.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:27 smithi190 ceph-mon[98334]: pgmap v575: 97 pgs: 97 active+clean; 869 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2023-07-21T22:38:29.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:29 smithi107 ceph-mon[96543]: pgmap v576: 97 pgs: 97 active+clean; 850 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 417 op/s 2023-07-21T22:38:30.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:29 smithi190 ceph-mon[98334]: pgmap v576: 97 pgs: 97 active+clean; 850 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 417 op/s 2023-07-21T22:38:30.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:29 smithi132 ceph-mon[97863]: pgmap v576: 97 pgs: 97 active+clean; 850 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 417 op/s 2023-07-21T22:38:31.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:31 smithi107 ceph-mon[96543]: pgmap v577: 97 pgs: 97 active+clean; 851 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 346 op/s 2023-07-21T22:38:32.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:31 smithi190 ceph-mon[98334]: pgmap v577: 97 pgs: 97 active+clean; 851 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 346 op/s 2023-07-21T22:38:32.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:31 smithi132 ceph-mon[97863]: pgmap v577: 97 pgs: 97 active+clean; 851 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 346 op/s 2023-07-21T22:38:33.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:33 smithi107 ceph-mon[96543]: pgmap v578: 97 pgs: 97 active+clean; 842 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 380 op/s 2023-07-21T22:38:34.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:33 smithi190 ceph-mon[98334]: pgmap v578: 97 pgs: 97 active+clean; 842 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 380 op/s 2023-07-21T22:38:34.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:33 smithi132 ceph-mon[97863]: pgmap v578: 97 pgs: 97 active+clean; 842 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 380 op/s 2023-07-21T22:38:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.903+0000 7fb227f55700 1 -- 172.21.15.107:0/2467786183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2200fec80 msgr2=0x7fb2200ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.903+0000 7fb227f55700 1 --2- 172.21.15.107:0/2467786183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2200fec80 0x7fb2200ff0d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb210009990 tx=0x7fb210009ca0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.903+0000 7fb227f55700 1 -- 172.21.15.107:0/2467786183 shutdown_connections 2023-07-21T22:38:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.903+0000 7fb227f55700 1 --2- 172.21.15.107:0/2467786183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2200ff610 0x7fb220106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.903+0000 7fb227f55700 1 --2- 172.21.15.107:0/2467786183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2200fffe0 0x7fb2201003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.903+0000 7fb227f55700 1 --2- 172.21.15.107:0/2467786183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2200fec80 0x7fb2200ff0d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.903+0000 7fb227f55700 1 -- 172.21.15.107:0/2467786183 >> 172.21.15.107:0/2467786183 conn(0x7fb2200f9780 msgr2=0x7fb2200fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.903+0000 7fb227f55700 1 -- 172.21.15.107:0/2467786183 shutdown_connections 2023-07-21T22:38:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.903+0000 7fb227f55700 1 -- 172.21.15.107:0/2467786183 wait complete. 2023-07-21T22:38:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb227f55700 1 Processor -- start 2023-07-21T22:38:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb227f55700 1 -- start start 2023-07-21T22:38:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb227f55700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2200fec80 0x7fb220194820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb227f55700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2200ff610 0x7fb220194d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb227f55700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2200fffe0 0x7fb220195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb227f55700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb220196240 con 0x7fb2200fffe0 2023-07-21T22:38:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb227f55700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2201963b0 con 0x7fb2200fec80 2023-07-21T22:38:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb227f55700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb220196570 con 0x7fb2200ff610 2023-07-21T22:38:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb2264f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2200fffe0 0x7fb220195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb2264f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2200fffe0 0x7fb220195be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51662/0 (socket says 172.21.15.107:51662) 2023-07-21T22:38:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb2264f2700 1 -- 172.21.15.107:0/817191183 learned_addr learned my addr 172.21.15.107:0/817191183 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb2254f0700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2200ff610 0x7fb220194d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb225cf1700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2200fec80 0x7fb220194820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb2264f2700 1 -- 172.21.15.107:0/817191183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2200ff610 msgr2=0x7fb220194d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb2264f2700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2200ff610 0x7fb220194d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb2264f2700 1 -- 172.21.15.107:0/817191183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2200fec80 msgr2=0x7fb220194820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb2264f2700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2200fec80 0x7fb220194820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb2264f2700 1 -- 172.21.15.107:0/817191183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2100095f0 con 0x7fb2200fffe0 2023-07-21T22:38:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.904+0000 7fb2264f2700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2200fffe0 0x7fb220195be0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00ef30 tx=0x7fb21c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.905+0000 7fb216ffd700 1 -- 172.21.15.107:0/817191183 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c018070 con 0x7fb2200fffe0 2023-07-21T22:38:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.905+0000 7fb216ffd700 1 -- 172.21.15.107:0/817191183 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb21c00f040 con 0x7fb2200fffe0 2023-07-21T22:38:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.905+0000 7fb227f55700 1 -- 172.21.15.107:0/817191183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2201967f0 con 0x7fb2200fffe0 2023-07-21T22:38:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.905+0000 7fb216ffd700 1 -- 172.21.15.107:0/817191183 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00cd00 con 0x7fb2200fffe0 2023-07-21T22:38:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.905+0000 7fb227f55700 1 -- 172.21.15.107:0/817191183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22019d010 con 0x7fb2200fffe0 2023-07-21T22:38:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.907+0000 7fb216ffd700 1 -- 172.21.15.107:0/817191183 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb21c003a00 con 0x7fb2200fffe0 2023-07-21T22:38:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.908+0000 7fb216ffd700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb20c0a1150 0x7fb20c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.908+0000 7fb225cf1700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb20c0a1150 0x7fb20c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.908+0000 7fb216ffd700 1 -- 172.21.15.107:0/817191183 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb21c0cc480 con 0x7fb2200fffe0 2023-07-21T22:38:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.908+0000 7fb225cf1700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb20c0a1150 0x7fb20c0a3600 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb2200676c0 tx=0x7fb218009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.908+0000 7fb2340dab80 1 Processor -- start 2023-07-21T22:38:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2340dab80 1 -- start start 2023-07-21T22:38:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2340dab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9bf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2340dab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609af9c0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2340dab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9c0580 0x5609af9c09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2340dab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609af8c4c50 con 0x5609af9bf830 2023-07-21T22:38:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2340dab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609af9179e0 con 0x5609af81b6b0 2023-07-21T22:38:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2340dab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609af910e70 con 0x5609af9c0580 2023-07-21T22:38:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2254f0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609af9c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2254f0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609af9c0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51668/0 (socket says 172.21.15.107:51668) 2023-07-21T22:38:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2254f0700 1 -- 172.21.15.107:0/2862816785 learned_addr learned my addr 172.21.15.107:0/2862816785 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2264f2700 1 --2- 172.21.15.107:0/2862816785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9c0580 0x5609af9c09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb225cf1700 1 --2- 172.21.15.107:0/2862816785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9bf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2254f0700 1 -- 172.21.15.107:0/2862816785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9c0580 msgr2=0x5609af9c09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2254f0700 1 --2- 172.21.15.107:0/2862816785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9c0580 0x5609af9c09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2254f0700 1 -- 172.21.15.107:0/2862816785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 msgr2=0x5609af9bf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2254f0700 1 --2- 172.21.15.107:0/2862816785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9bf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2254f0700 1 -- 172.21.15.107:0/2862816785 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609af73ed80 con 0x5609af9bf830 2023-07-21T22:38:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb225cf1700 1 --2- 172.21.15.107:0/2862816785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9bf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:38:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb2254f0700 1 --2- 172.21.15.107:0/2862816785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609af9c0040 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb210015000 tx=0x7fb210005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddda1ea2bec3dfc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.909+0000 7fb20bfff700 1 -- 172.21.15.107:0/2862816785 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21001d070 con 0x5609af9bf830 2023-07-21T22:38:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.910+0000 7fb20bfff700 1 -- 172.21.15.107:0/2862816785 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb21001e040 con 0x5609af9bf830 2023-07-21T22:38:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.910+0000 7fb20bfff700 1 -- 172.21.15.107:0/2862816785 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210004950 con 0x5609af9bf830 2023-07-21T22:38:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.910+0000 7fb2340dab80 1 -- 172.21.15.107:0/2862816785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 msgr2=0x5609af9c0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.910+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2862816785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609af9c0040 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb210015000 tx=0x7fb210005ea0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.910+0000 7fb2340dab80 1 -- 172.21.15.107:0/2862816785 shutdown_connections 2023-07-21T22:38:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.910+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2862816785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9c0580 0x5609af9c09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.910+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2862816785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9bf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.910+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2862816785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609af9c0040 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.910+0000 7fb2340dab80 1 -- 172.21.15.107:0/2862816785 >> 172.21.15.107:0/2862816785 conn(0x5609af9b4ae0 msgr2=0x5609af9b9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.910+0000 7fb2340dab80 1 -- 172.21.15.107:0/2862816785 shutdown_connections 2023-07-21T22:38:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.910+0000 7fb2340dab80 1 -- 172.21.15.107:0/2862816785 wait complete. 2023-07-21T22:38:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb2340dab80 1 -- start start 2023-07-21T22:38:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb2340dab80 1 Processor -- start 2023-07-21T22:38:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb2340dab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9c8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb2340dab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609af9c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb2340dab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9c0580 0x5609af9d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb2340dab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609af8b0a60 con 0x5609af9bf830 2023-07-21T22:38:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb2340dab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609af8b0760 con 0x5609af81b6b0 2023-07-21T22:38:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb2340dab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609af7948c0 con 0x5609af9c0580 2023-07-21T22:38:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb2340dab80 10 client.0 _ll_register_callbacks cb 0x5609af8ae700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb2254f0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609af9c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb2254f0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609af9c8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51674/0 (socket says 172.21.15.107:51674) 2023-07-21T22:38:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb2264f2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9c0580 0x5609af9d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb2254f0700 1 -- 172.21.15.107:0/1213402538 learned_addr learned my addr 172.21.15.107:0/1213402538 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.911+0000 7fb225cf1700 1 --2- 172.21.15.107:0/1213402538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2254f0700 1 -- 172.21.15.107:0/1213402538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9c0580 msgr2=0x5609af9d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2254f0700 1 --2- 172.21.15.107:0/1213402538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9c0580 0x5609af9d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2254f0700 1 -- 172.21.15.107:0/1213402538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 msgr2=0x5609af9c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2254f0700 1 --2- 172.21.15.107:0/1213402538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2254f0700 1 -- 172.21.15.107:0/1213402538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb220107750 con 0x5609af9bf830 2023-07-21T22:38:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2254f0700 1 --2- 172.21.15.107:0/1213402538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609af9c8c90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb210009350 tx=0x7fb210022bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830a2d741636133 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2097fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:38:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2097fa700 1 -- 172.21.15.107:0/1213402538 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21001d030 con 0x5609af9bf830 2023-07-21T22:38:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2097fa700 1 -- 172.21.15.107:0/1213402538 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb21000f460 con 0x5609af9bf830 2023-07-21T22:38:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2340dab80 1 -- 172.21.15.107:0/1213402538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609af86a490 con 0x5609af9bf830 2023-07-21T22:38:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2097fa700 1 -- 172.21.15.107:0/1213402538 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb210005bc0 con 0x5609af9bf830 2023-07-21T22:38:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2340dab80 10 client.16815 fetch_fsmap learned FSMap version 56 2023-07-21T22:38:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2340dab80 1 -- 172.21.15.107:0/1213402538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5609af727c00 con 0x5609af9bf830 2023-07-21T22:38:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2097fa700 1 -- 172.21.15.107:0/1213402538 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb21003f080 con 0x5609af9bf830 2023-07-21T22:38:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2097fa700 1 -- 172.21.15.107:0/1213402538 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) v1 ==== 7542+0+0 (secure 0 0 0) 0x7fb210039030 con 0x5609af9bf830 2023-07-21T22:38:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2340dab80 10 client.16815 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T22:38:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2340dab80 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:38:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2340dab80 20 client.16815 populate_metadata read hostname 'smithi107' 2023-07-21T22:38:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2340dab80 1 --2- 172.21.15.107:0/1213402538 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5609af9d63f0 0x5609af9d8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2340dab80 4 client.16815 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:38:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.912+0000 7fb2340dab80 1 -- 172.21.15.107:0/1213402538 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609af909e50 con 0x5609af9d63f0 2023-07-21T22:38:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.913+0000 7fb225cf1700 1 --2- 172.21.15.107:0/1213402538 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5609af9d63f0 0x5609af9d8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.913+0000 7fb2097fa700 10 client.16815 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:38:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.913+0000 7fb225cf1700 1 --2- 172.21.15.107:0/1213402538 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5609af9d63f0 0x5609af9d8800 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a43515e9219145fa server_cookie=6344577aaa86a6fa in_seq=0 out_seq=0 2023-07-21T22:38:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2097fa700 1 -- 172.21.15.107:0/1213402538 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5609af909e50 con 0x5609af9d63f0 2023-07-21T22:38:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2097fa700 10 client.16815 handle_command_reply: tid=0 2023-07-21T22:38:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2340dab80 1 client.16815 shutdown 2023-07-21T22:38:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2340dab80 1 -- 172.21.15.107:0/1213402538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 msgr2=0x5609af9c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2340dab80 1 --2- 172.21.15.107:0/1213402538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609af9c8c90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb210009350 tx=0x7fb210022bd0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2340dab80 1 -- 172.21.15.107:0/1213402538 shutdown_connections 2023-07-21T22:38:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2340dab80 1 --2- 172.21.15.107:0/1213402538 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5609af9d63f0 0x5609af9d8800 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2340dab80 1 --2- 172.21.15.107:0/1213402538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9c0580 0x5609af9d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2340dab80 1 --2- 172.21.15.107:0/1213402538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2340dab80 1 --2- 172.21.15.107:0/1213402538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609af9c8c90 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2097fa700 0 client.16815 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:38:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2340dab80 1 -- 172.21.15.107:0/1213402538 >> 172.21.15.107:0/1213402538 conn(0x5609af9b4ae0 msgr2=0x5609af9b93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2340dab80 1 -- 172.21.15.107:0/1213402538 shutdown_connections 2023-07-21T22:38:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2340dab80 1 -- 172.21.15.107:0/1213402538 wait complete. 2023-07-21T22:38:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.914+0000 7fb2340dab80 20 client.16815 trim_cache size 0 max 16384 2023-07-21T22:38:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2340dab80 1 Processor -- start 2023-07-21T22:38:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2340dab80 1 -- start start 2023-07-21T22:38:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2340dab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2340dab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9bf830 0x5609af9b9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2340dab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9c0580 0x5609af9dd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2340dab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609af9c7290 con 0x5609af9c0580 2023-07-21T22:38:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2340dab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609af9c93d0 con 0x5609af81b6b0 2023-07-21T22:38:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2340dab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609af9ba120 con 0x5609af9bf830 2023-07-21T22:38:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2264f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9c0580 0x5609af9dd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2254f0700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9bf830 0x5609af9b9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2264f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9c0580 0x5609af9dd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51684/0 (socket says 172.21.15.107:51684) 2023-07-21T22:38:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2264f2700 1 -- 172.21.15.107:0/2408849229 learned_addr learned my addr 172.21.15.107:0/2408849229 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb225cf1700 1 --2- 172.21.15.107:0/2408849229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2264f2700 1 -- 172.21.15.107:0/2408849229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9bf830 msgr2=0x5609af9b9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2264f2700 1 --2- 172.21.15.107:0/2408849229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9bf830 0x5609af9b9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2264f2700 1 -- 172.21.15.107:0/2408849229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 msgr2=0x5609af9d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2264f2700 1 --2- 172.21.15.107:0/2408849229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2264f2700 1 -- 172.21.15.107:0/2408849229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609af7651f0 con 0x5609af9c0580 2023-07-21T22:38:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb225cf1700 1 --2- 172.21.15.107:0/2408849229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:38:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb2264f2700 1 --2- 172.21.15.107:0/2408849229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9c0580 0x5609af9dd120 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0c9f60 tx=0x7fb21c09f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9d56133489c9a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb214ff9700 1 -- 172.21.15.107:0/2408849229 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0b7ea0 con 0x5609af9c0580 2023-07-21T22:38:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb214ff9700 1 -- 172.21.15.107:0/2408849229 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb21c099900 con 0x5609af9c0580 2023-07-21T22:38:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.930+0000 7fb214ff9700 1 -- 172.21.15.107:0/2408849229 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0cc310 con 0x5609af9c0580 2023-07-21T22:38:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.931+0000 7fb2340dab80 1 -- 172.21.15.107:0/2408849229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9c0580 msgr2=0x5609af9dd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.931+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2408849229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9c0580 0x5609af9dd120 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0c9f60 tx=0x7fb21c09f040 comp rx=0 tx=0).stop 2023-07-21T22:38:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.931+0000 7fb2340dab80 1 -- 172.21.15.107:0/2408849229 shutdown_connections 2023-07-21T22:38:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.931+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2408849229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9c0580 0x5609af9dd120 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.931+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2408849229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af81b6b0 0x5609af9d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.931+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2408849229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af9bf830 0x5609af9b9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.931+0000 7fb2340dab80 1 -- 172.21.15.107:0/2408849229 >> 172.21.15.107:0/2408849229 conn(0x5609af9b4ae0 msgr2=0x5609af9c7500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.931+0000 7fb2340dab80 1 -- 172.21.15.107:0/2408849229 shutdown_connections 2023-07-21T22:38:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.931+0000 7fb2340dab80 1 -- 172.21.15.107:0/2408849229 wait complete. 2023-07-21T22:38:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.931+0000 7fb2340dab80 1 -- start start 2023-07-21T22:38:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.931+0000 7fb2340dab80 1 Processor -- start 2023-07-21T22:38:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2340dab80 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af81b6b0 0x5609af9e6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2340dab80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609afa269d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2340dab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af9c0580 0x5609afa28f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2340dab80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609afa29630 con 0x5609af9bf830 2023-07-21T22:38:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2340dab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609afa297a0 con 0x5609af9c0580 2023-07-21T22:38:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2340dab80 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609afa29960 con 0x5609af81b6b0 2023-07-21T22:38:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2340dab80 10 client.0 _ll_register_callbacks cb 0x7fb21c009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:38:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2254f0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609afa269d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2254f0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609afa269d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51688/0 (socket says 172.21.15.107:51688) 2023-07-21T22:38:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2254f0700 1 -- 172.21.15.107:0/2618486664 learned_addr learned my addr 172.21.15.107:0/2618486664 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb225cf1700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af81b6b0 0x5609af9e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2264f2700 1 --2- 172.21.15.107:0/2618486664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af9c0580 0x5609afa28f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2254f0700 1 -- 172.21.15.107:0/2618486664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af81b6b0 msgr2=0x5609af9e6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2254f0700 1 --2- 172.21.15.107:0/2618486664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af81b6b0 0x5609af9e6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2254f0700 1 -- 172.21.15.107:0/2618486664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af9c0580 msgr2=0x5609afa28f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:38:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2254f0700 1 --2- 172.21.15.107:0/2618486664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af9c0580 0x5609afa28f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2254f0700 1 -- 172.21.15.107:0/2618486664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22019d010 con 0x5609af9bf830 2023-07-21T22:38:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb225cf1700 1 --2- 172.21.15.107:0/2618486664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af81b6b0 0x5609af9e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2254f0700 1 --2- 172.21.15.107:0/2618486664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609afa269d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb21003b550 tx=0x7fb21002c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbeeba43fc86efd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2097fa700 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2023-07-21T22:38:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2097fa700 1 -- 172.21.15.107:0/2618486664 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21001d040 con 0x5609af9bf830 2023-07-21T22:38:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2097fa700 1 -- 172.21.15.107:0/2618486664 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb210000e40 con 0x5609af9bf830 2023-07-21T22:38:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2340dab80 1 -- 172.21.15.107:0/2618486664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb21001e040 con 0x5609af9bf830 2023-07-21T22:38:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2097fa700 1 -- 172.21.15.107:0/2618486664 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb210005bc0 con 0x5609af9bf830 2023-07-21T22:38:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2340dab80 10 client.16821 fetch_fsmap learned FSMap version 56 2023-07-21T22:38:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.932+0000 7fb2340dab80 1 -- 172.21.15.107:0/2618486664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5609af73d250 con 0x5609af9bf830 2023-07-21T22:38:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.933+0000 7fb2097fa700 1 -- 172.21.15.107:0/2618486664 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb21001e040 con 0x5609af9bf830 2023-07-21T22:38:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.933+0000 7fb2097fa700 1 -- 172.21.15.107:0/2618486664 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 56) v1 ==== 7542+0+0 (secure 0 0 0) 0x7fb21000b850 con 0x5609af9bf830 2023-07-21T22:38:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.933+0000 7fb2340dab80 10 client.16821 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T22:38:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.933+0000 7fb2340dab80 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T22:38:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.933+0000 7fb2340dab80 20 client.16821 populate_metadata read hostname 'smithi107' 2023-07-21T22:38:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.933+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2618486664 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5609af9cc0f0 0x5609af9cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.933+0000 7fb2340dab80 4 client.16821 mds_command: new command op to 24533 tid=0[{"prefix": "flush journal"}] 2023-07-21T22:38:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.933+0000 7fb2340dab80 1 -- 172.21.15.107:0/2618486664 --> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5609af8cbff0 con 0x5609af9cc0f0 2023-07-21T22:38:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.933+0000 7fb225cf1700 1 --2- 172.21.15.107:0/2618486664 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5609af9cc0f0 0x5609af9cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.933+0000 7fb2097fa700 10 client.16821 ms_handle_connect on v2:172.21.15.132:6838/523672176 2023-07-21T22:38:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.933+0000 7fb225cf1700 1 --2- 172.21.15.107:0/2618486664 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5609af9cc0f0 0x5609af9cc4f0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51a75b193dffbc3a server_cookie=c512d84e1c25cb2f in_seq=0 out_seq=0 2023-07-21T22:38:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2097fa700 1 -- 172.21.15.107:0/2618486664 <== mds.0 v2:172.21.15.132:6838/523672176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5609af8cbff0 con 0x5609af9cc0f0 2023-07-21T22:38:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2097fa700 10 client.16821 handle_command_reply: tid=0 2023-07-21T22:38:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2340dab80 1 client.16821 shutdown 2023-07-21T22:38:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2340dab80 1 -- 172.21.15.107:0/2618486664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 msgr2=0x5609afa269d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2618486664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609afa269d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb21003b550 tx=0x7fb21002c910 comp rx=0 tx=0).stop 2023-07-21T22:38:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2340dab80 1 -- 172.21.15.107:0/2618486664 shutdown_connections 2023-07-21T22:38:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2618486664 >> [v2:172.21.15.132:6838/523672176,v1:172.21.15.132:6839/523672176] conn(0x5609af9cc0f0 0x5609af9cc4f0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2618486664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5609af9c0580 0x5609afa28f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2618486664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5609af81b6b0 0x5609af9e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2340dab80 1 --2- 172.21.15.107:0/2618486664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5609af9bf830 0x5609afa269d0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2097fa700 0 client.16821 ms_handle_reset on v2:172.21.15.132:6838/523672176 2023-07-21T22:38:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2340dab80 1 -- 172.21.15.107:0/2618486664 >> 172.21.15.107:0/2618486664 conn(0x5609af9b4ae0 msgr2=0x5609af9dd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2340dab80 1 -- 172.21.15.107:0/2618486664 shutdown_connections 2023-07-21T22:38:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2340dab80 1 -- 172.21.15.107:0/2618486664 wait complete. 2023-07-21T22:38:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.942+0000 7fb2340dab80 20 client.16821 trim_cache size 0 max 16384 2023-07-21T22:38:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2023-07-21T22:38:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2023-07-21T22:38:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2023-07-21T22:38:35.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2023-07-21T22:38:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.943+0000 7fb227f55700 1 -- 172.21.15.107:0/817191183 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb20c0a1150 msgr2=0x7fb20c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.943+0000 7fb227f55700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb20c0a1150 0x7fb20c0a3600 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb2200676c0 tx=0x7fb218009260 comp rx=0 tx=0).stop 2023-07-21T22:38:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.943+0000 7fb227f55700 1 -- 172.21.15.107:0/817191183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2200fffe0 msgr2=0x7fb220195be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.943+0000 7fb227f55700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2200fffe0 0x7fb220195be0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb21c00ef30 tx=0x7fb21c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.943+0000 7fb227f55700 1 -- 172.21.15.107:0/817191183 shutdown_connections 2023-07-21T22:38:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.943+0000 7fb227f55700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2200fffe0 0x7fb220195be0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.943+0000 7fb227f55700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2200fec80 0x7fb220194820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.943+0000 7fb227f55700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fb20c0a1150 0x7fb20c0a3600 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.943+0000 7fb227f55700 1 --2- 172.21.15.107:0/817191183 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2200ff610 0x7fb220194d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.943+0000 7fb227f55700 1 -- 172.21.15.107:0/817191183 >> 172.21.15.107:0/817191183 conn(0x7fb2200f9780 msgr2=0x7fb2201031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.943+0000 7fb227f55700 1 -- 172.21.15.107:0/817191183 shutdown_connections 2023-07-21T22:38:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2023-07-21T22:38:35.943+0000 7fb227f55700 1 -- 172.21.15.107:0/817191183 wait complete. 2023-07-21T22:38:36.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:35 smithi190 ceph-mon[98334]: pgmap v579: 97 pgs: 97 active+clean; 840 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 932 KiB/s wr, 299 op/s 2023-07-21T22:38:36.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:35 smithi132 ceph-mon[97863]: pgmap v579: 97 pgs: 97 active+clean; 840 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 932 KiB/s wr, 299 op/s 2023-07-21T22:38:36.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:35 smithi107 ceph-mon[96543]: pgmap v579: 97 pgs: 97 active+clean; 840 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 932 KiB/s wr, 299 op/s 2023-07-21T22:38:38.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:37 smithi190 ceph-mon[98334]: pgmap v580: 97 pgs: 97 active+clean; 836 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 837 KiB/s wr, 263 op/s 2023-07-21T22:38:38.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:37 smithi132 ceph-mon[97863]: pgmap v580: 97 pgs: 97 active+clean; 836 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 837 KiB/s wr, 263 op/s 2023-07-21T22:38:38.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:37 smithi107 ceph-mon[96543]: pgmap v580: 97 pgs: 97 active+clean; 836 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 837 KiB/s wr, 263 op/s 2023-07-21T22:38:40.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:39 smithi190 ceph-mon[98334]: pgmap v581: 97 pgs: 97 active+clean; 823 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 313 op/s 2023-07-21T22:38:40.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:39 smithi132 ceph-mon[97863]: pgmap v581: 97 pgs: 97 active+clean; 823 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 313 op/s 2023-07-21T22:38:40.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:39 smithi107 ceph-mon[96543]: pgmap v581: 97 pgs: 97 active+clean; 823 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 313 op/s 2023-07-21T22:38:42.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:41 smithi190 ceph-mon[98334]: pgmap v582: 97 pgs: 97 active+clean; 820 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1001 KiB/s wr, 214 op/s 2023-07-21T22:38:42.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:41 smithi132 ceph-mon[97863]: pgmap v582: 97 pgs: 97 active+clean; 820 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1001 KiB/s wr, 214 op/s 2023-07-21T22:38:42.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:41 smithi107 ceph-mon[96543]: pgmap v582: 97 pgs: 97 active+clean; 820 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1001 KiB/s wr, 214 op/s 2023-07-21T22:38:44.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:43 smithi190 ceph-mon[98334]: pgmap v583: 97 pgs: 97 active+clean; 801 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 807 KiB/s wr, 290 op/s 2023-07-21T22:38:44.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:43 smithi132 ceph-mon[97863]: pgmap v583: 97 pgs: 97 active+clean; 801 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 807 KiB/s wr, 290 op/s 2023-07-21T22:38:44.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:43 smithi107 ceph-mon[96543]: pgmap v583: 97 pgs: 97 active+clean; 801 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 807 KiB/s wr, 290 op/s 2023-07-21T22:38:44.628 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2023-07-21T22:38:44.629 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-07-21T22:38:44.919 DEBUG:teuthology.parallel:result is None 2023-07-21T22:38:44.920 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-07-21T22:38:44.959 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-07-21T22:38:44.959 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-07-21T22:38:44.970 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi107.front.sepia.ceph.com 2023-07-21T22:38:45.007 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-07-21T22:38:45.019 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-07-21T22:38:45.019 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-07-21T22:38:45.020 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-07-21T22:38:45.034 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:38:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.127+0000 7f4f054e8700 1 -- 172.21.15.107:0/4150804636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f000ff410 msgr2=0x7f4f000ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.127+0000 7f4f054e8700 1 --2- 172.21.15.107:0/4150804636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f000ff410 0x7f4f000ff860 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400aaa0 tx=0x7f4ef400adb0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.170 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.127+0000 7f4f054e8700 1 -- 172.21.15.107:0/4150804636 shutdown_connections 2023-07-21T22:38:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.127+0000 7f4f054e8700 1 --2- 172.21.15.107:0/4150804636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f000ffda0 0x7f4f00106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.127+0000 7f4f054e8700 1 --2- 172.21.15.107:0/4150804636 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f000fe220 0x7f4f000fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.127+0000 7f4f054e8700 1 --2- 172.21.15.107:0/4150804636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f000ff410 0x7f4f000ff860 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.127+0000 7f4f054e8700 1 -- 172.21.15.107:0/4150804636 >> 172.21.15.107:0/4150804636 conn(0x7f4f000f9820 msgr2=0x7f4f000fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.127+0000 7f4f054e8700 1 -- 172.21.15.107:0/4150804636 shutdown_connections 2023-07-21T22:38:45.171 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.127+0000 7f4f054e8700 1 -- 172.21.15.107:0/4150804636 wait complete. 2023-07-21T22:38:45.173 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.128+0000 7f4f054e8700 1 Processor -- start 2023-07-21T22:38:45.173 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.128+0000 7f4f054e8700 1 -- start start 2023-07-21T22:38:45.173 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.128+0000 7f4f054e8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f000fe220 0x7f4f00195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:45.173 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.128+0000 7f4f054e8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f000ff410 0x7f4f00195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.128+0000 7f4f054e8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f000ffda0 0x7f4f0019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.128+0000 7f4f054e8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0019d300 con 0x7f4f000ffda0 2023-07-21T22:38:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.128+0000 7f4f054e8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0019d440 con 0x7f4f000fe220 2023-07-21T22:38:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.128+0000 7f4f054e8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0019d5d0 con 0x7f4f000ff410 2023-07-21T22:38:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.128+0000 7f4eff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f000ffda0 0x7f4f0019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.128+0000 7f4efeffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f000fe220 0x7f4f00195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.128+0000 7f4eff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f000ffda0 0x7f4f0019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51704/0 (socket says 172.21.15.107:51704) 2023-07-21T22:38:45.174 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.128+0000 7f4eff7fe700 1 -- 172.21.15.107:0/2333459282 learned_addr learned my addr 172.21.15.107:0/2333459282 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.128+0000 7f4efe7fc700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f000ff410 0x7f4f00195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.129+0000 7f4efeffd700 1 -- 172.21.15.107:0/2333459282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f000ff410 msgr2=0x7f4f00195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.129+0000 7f4efeffd700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f000ff410 0x7f4f00195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.129+0000 7f4efeffd700 1 -- 172.21.15.107:0/2333459282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f000ffda0 msgr2=0x7f4f0019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.129+0000 7f4efeffd700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f000ffda0 0x7f4f0019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.129+0000 7f4efeffd700 1 -- 172.21.15.107:0/2333459282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef400a700 con 0x7f4f000fe220 2023-07-21T22:38:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.129+0000 7f4eff7fe700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f000ffda0 0x7f4f0019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:45.175 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.129+0000 7f4efeffd700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f000fe220 0x7f4f00195780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000e9c0 tx=0x7f4ef000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:45.176 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.129+0000 7f4ee7fff700 1 -- 172.21.15.107:0/2333459282 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0004220 con 0x7f4f000fe220 2023-07-21T22:38:45.176 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.129+0000 7f4ee7fff700 1 -- 172.21.15.107:0/2333459282 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ef0004380 con 0x7f4f000fe220 2023-07-21T22:38:45.176 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.129+0000 7f4f054e8700 1 -- 172.21.15.107:0/2333459282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f0019d8b0 con 0x7f4f000fe220 2023-07-21T22:38:45.176 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.129+0000 7f4ee7fff700 1 -- 172.21.15.107:0/2333459282 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0003be0 con 0x7f4f000fe220 2023-07-21T22:38:45.176 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.129+0000 7f4f054e8700 1 -- 172.21.15.107:0/2333459282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0019ddc0 con 0x7f4f000fe220 2023-07-21T22:38:45.176 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.130+0000 7f4f054e8700 1 -- 172.21.15.107:0/2333459282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f001091c0 con 0x7f4f000fe220 2023-07-21T22:38:45.177 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.131+0000 7f4ee7fff700 1 -- 172.21.15.107:0/2333459282 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4ef0003d40 con 0x7f4f000fe220 2023-07-21T22:38:45.177 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.132+0000 7f4ee7fff700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4ee80a11a0 0x7f4ee80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:45.177 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.132+0000 7f4efe7fc700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4ee80a11a0 0x7f4ee80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:45.177 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.132+0000 7f4ee7fff700 1 -- 172.21.15.107:0/2333459282 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4ef0014070 con 0x7f4f000fe220 2023-07-21T22:38:45.178 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.132+0000 7f4efe7fc700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4ee80a11a0 0x7f4ee80a3650 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400c6d0 tx=0x7f4ef4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:45.178 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.135+0000 7f4ee7fff700 1 -- 172.21.15.107:0/2333459282 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4ef00968d0 con 0x7f4f000fe220 2023-07-21T22:38:45.420 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.378+0000 7f4f054e8700 1 -- 172.21.15.107:0/2333459282 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f00105080 con 0x7f4f000fe220 2023-07-21T22:38:45.422 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.379+0000 7f4ee7fff700 1 -- 172.21.15.107:0/2333459282 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22727 (secure 0 0 0) 0x7f4ef001f340 con 0x7f4f000fe220 2023-07-21T22:38:45.423 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:38:45.423 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":85,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:31:05.959573+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:38:45.427 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.384+0000 7f4f054e8700 1 -- 172.21.15.107:0/2333459282 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4ee80a11a0 msgr2=0x7f4ee80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:45.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.384+0000 7f4f054e8700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4ee80a11a0 0x7f4ee80a3650 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400c6d0 tx=0x7f4ef4005960 comp rx=0 tx=0).stop 2023-07-21T22:38:45.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.384+0000 7f4f054e8700 1 -- 172.21.15.107:0/2333459282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f000fe220 msgr2=0x7f4f00195780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:45.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.384+0000 7f4f054e8700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f000fe220 0x7f4f00195780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000e9c0 tx=0x7f4ef000ecd0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.384+0000 7f4f054e8700 1 -- 172.21.15.107:0/2333459282 shutdown_connections 2023-07-21T22:38:45.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.384+0000 7f4f054e8700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f000ffda0 0x7f4f0019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.428 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.384+0000 7f4f054e8700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f000fe220 0x7f4f00195780 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.384+0000 7f4f054e8700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4ee80a11a0 0x7f4ee80a3650 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.384+0000 7f4f054e8700 1 --2- 172.21.15.107:0/2333459282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f000ff410 0x7f4f00195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.384+0000 7f4f054e8700 1 -- 172.21.15.107:0/2333459282 >> 172.21.15.107:0/2333459282 conn(0x7f4f000f9820 msgr2=0x7f4f000fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:45.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.385+0000 7f4f054e8700 1 -- 172.21.15.107:0/2333459282 shutdown_connections 2023-07-21T22:38:45.429 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.385+0000 7f4f054e8700 1 -- 172.21.15.107:0/2333459282 wait complete. 2023-07-21T22:38:45.444 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:23:52.031819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:27:39.494209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:27:40.672303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:38:45.445 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:38:45.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.526+0000 7f7a403d3700 1 -- 172.21.15.107:0/1722701652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a380fea00 msgr2=0x7f7a380fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:45.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.526+0000 7f7a403d3700 1 --2- 172.21.15.107:0/1722701652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a380fea00 0x7f7a380fee10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009990 tx=0x7f7a34009ca0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.526+0000 7f7a403d3700 1 -- 172.21.15.107:0/1722701652 shutdown_connections 2023-07-21T22:38:45.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.526+0000 7f7a403d3700 1 --2- 172.21.15.107:0/1722701652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a38100640 0x7f7a38106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.526+0000 7f7a403d3700 1 --2- 172.21.15.107:0/1722701652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a380fea00 0x7f7a380fee10 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.526+0000 7f7a403d3700 1 --2- 172.21.15.107:0/1722701652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a380ffc90 0x7f7a38100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.526+0000 7f7a403d3700 1 -- 172.21.15.107:0/1722701652 >> 172.21.15.107:0/1722701652 conn(0x7f7a380fa020 msgr2=0x7f7a380fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:45.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.526+0000 7f7a403d3700 1 -- 172.21.15.107:0/1722701652 shutdown_connections 2023-07-21T22:38:45.570 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a403d3700 1 -- 172.21.15.107:0/1722701652 wait complete. 2023-07-21T22:38:45.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a403d3700 1 Processor -- start 2023-07-21T22:38:45.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a403d3700 1 -- start start 2023-07-21T22:38:45.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a403d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a380fea00 0x7f7a38069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:45.572 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a403d3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a380ffc90 0x7f7a3819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:45.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a403d3700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a38100640 0x7f7a3819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:45.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a403d3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3819d450 con 0x7f7a380ffc90 2023-07-21T22:38:45.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a403d3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3819d5c0 con 0x7f7a380fea00 2023-07-21T22:38:45.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a403d3700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3819d780 con 0x7f7a38100640 2023-07-21T22:38:45.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a3d96e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a380ffc90 0x7f7a3819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:45.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a3e16f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a380fea00 0x7f7a38069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:45.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a3d96e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a380ffc90 0x7f7a3819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51716/0 (socket says 172.21.15.107:51716) 2023-07-21T22:38:45.573 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a3d96e700 1 -- 172.21.15.107:0/1156548886 learned_addr learned my addr 172.21.15.107:0/1156548886 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.527+0000 7f7a3e970700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a38100640 0x7f7a3819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.528+0000 7f7a3d96e700 1 -- 172.21.15.107:0/1156548886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a38100640 msgr2=0x7f7a3819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.528+0000 7f7a3d96e700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a38100640 0x7f7a3819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.528+0000 7f7a3d96e700 1 -- 172.21.15.107:0/1156548886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a380fea00 msgr2=0x7f7a38069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.528+0000 7f7a3d96e700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a380fea00 0x7f7a38069390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.528+0000 7f7a3d96e700 1 -- 172.21.15.107:0/1156548886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a340095f0 con 0x7f7a380ffc90 2023-07-21T22:38:45.574 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.528+0000 7f7a3e970700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a38100640 0x7f7a3819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:38:45.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.528+0000 7f7a3e16f700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a380fea00 0x7f7a38069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:45.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.528+0000 7f7a3d96e700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a380ffc90 0x7f7a3819a670 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00e9a0 tx=0x7f7a2c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:45.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.528+0000 7f7a2b7fe700 1 -- 172.21.15.107:0/1156548886 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c009790 con 0x7f7a380ffc90 2023-07-21T22:38:45.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.528+0000 7f7a2b7fe700 1 -- 172.21.15.107:0/1156548886 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a2c00cbd0 con 0x7f7a380ffc90 2023-07-21T22:38:45.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.528+0000 7f7a403d3700 1 -- 172.21.15.107:0/1156548886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a381944a0 con 0x7f7a380ffc90 2023-07-21T22:38:45.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.528+0000 7f7a2b7fe700 1 -- 172.21.15.107:0/1156548886 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c018680 con 0x7f7a380ffc90 2023-07-21T22:38:45.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.528+0000 7f7a403d3700 1 -- 172.21.15.107:0/1156548886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a381949b0 con 0x7f7a380ffc90 2023-07-21T22:38:45.575 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.529+0000 7f7a403d3700 1 -- 172.21.15.107:0/1156548886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a381060e0 con 0x7f7a380ffc90 2023-07-21T22:38:45.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.531+0000 7f7a2b7fe700 1 -- 172.21.15.107:0/1156548886 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7a2c00cd40 con 0x7f7a380ffc90 2023-07-21T22:38:45.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.531+0000 7f7a2b7fe700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7a240a1150 0x7f7a240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:45.576 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.531+0000 7f7a3e16f700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7a240a1150 0x7f7a240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:45.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.531+0000 7f7a2b7fe700 1 -- 172.21.15.107:0/1156548886 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7a2c014070 con 0x7f7a380ffc90 2023-07-21T22:38:45.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.532+0000 7f7a3e16f700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7a240a1150 0x7f7a240a3600 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400b2e0 tx=0x7f7a3401e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:45.577 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.535+0000 7f7a2b7fe700 1 -- 172.21.15.107:0/1156548886 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7a2c096590 con 0x7f7a380ffc90 2023-07-21T22:38:45.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.725+0000 7f7a403d3700 1 -- 172.21.15.107:0/1156548886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a3802cc50 con 0x7f7a380ffc90 2023-07-21T22:38:45.768 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.726+0000 7f7a2b7fe700 1 -- 172.21.15.107:0/1156548886 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a2c094e60 con 0x7f7a380ffc90 2023-07-21T22:38:45.769 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2023-07-21T22:38:45.770 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.728+0000 7f7a403d3700 1 -- 172.21.15.107:0/1156548886 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7a240a1150 msgr2=0x7f7a240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:45.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.728+0000 7f7a403d3700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7a240a1150 0x7f7a240a3600 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400b2e0 tx=0x7f7a3401e040 comp rx=0 tx=0).stop 2023-07-21T22:38:45.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.728+0000 7f7a403d3700 1 -- 172.21.15.107:0/1156548886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a380ffc90 msgr2=0x7f7a3819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:45.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.728+0000 7f7a403d3700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a380ffc90 0x7f7a3819a670 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00e9a0 tx=0x7f7a2c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.728+0000 7f7a403d3700 1 -- 172.21.15.107:0/1156548886 shutdown_connections 2023-07-21T22:38:45.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.728+0000 7f7a403d3700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f7a240a1150 0x7f7a240a3600 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.771 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.728+0000 7f7a403d3700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a38100640 0x7f7a3819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.728+0000 7f7a403d3700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a380fea00 0x7f7a38069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.728+0000 7f7a403d3700 1 --2- 172.21.15.107:0/1156548886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a380ffc90 0x7f7a3819a670 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.728+0000 7f7a403d3700 1 -- 172.21.15.107:0/1156548886 >> 172.21.15.107:0/1156548886 conn(0x7f7a380fa020 msgr2=0x7f7a38103220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:45.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.728+0000 7f7a403d3700 1 -- 172.21.15.107:0/1156548886 shutdown_connections 2023-07-21T22:38:45.772 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.728+0000 7f7a403d3700 1 -- 172.21.15.107:0/1156548886 wait complete. 2023-07-21T22:38:45.783 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:38:45.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.866+0000 7efca0ed2700 1 -- 172.21.15.107:0/933744371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9c0ffbd0 msgr2=0x7efc9c100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:45.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.866+0000 7efca0ed2700 1 --2- 172.21.15.107:0/933744371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9c0ffbd0 0x7efc9c100020 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009990 tx=0x7efc8c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.866+0000 7efca0ed2700 1 -- 172.21.15.107:0/933744371 shutdown_connections 2023-07-21T22:38:45.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.866+0000 7efca0ed2700 1 --2- 172.21.15.107:0/933744371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc9c100560 0x7efc9c106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.909 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.866+0000 7efca0ed2700 1 --2- 172.21.15.107:0/933744371 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc9c0fe9e0 0x7efc9c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.866+0000 7efca0ed2700 1 --2- 172.21.15.107:0/933744371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9c0ffbd0 0x7efc9c100020 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.866+0000 7efca0ed2700 1 -- 172.21.15.107:0/933744371 >> 172.21.15.107:0/933744371 conn(0x7efc9c0fa000 msgr2=0x7efc9c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:45.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.866+0000 7efca0ed2700 1 -- 172.21.15.107:0/933744371 shutdown_connections 2023-07-21T22:38:45.910 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.866+0000 7efca0ed2700 1 -- 172.21.15.107:0/933744371 wait complete. 2023-07-21T22:38:45.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efca0ed2700 1 Processor -- start 2023-07-21T22:38:45.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efca0ed2700 1 -- start start 2023-07-21T22:38:45.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efca0ed2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc9c0fe9e0 0x7efc9c1957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:45.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efca0ed2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9c0ffbd0 0x7efc9c195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:45.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efca0ed2700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc9c100560 0x7efc9c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:45.912 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efca0ed2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c19d360 con 0x7efc9c0ffbd0 2023-07-21T22:38:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efca0ed2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c19d4a0 con 0x7efc9c0fe9e0 2023-07-21T22:38:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efca0ed2700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c19d630 con 0x7efc9c100560 2023-07-21T22:38:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efc99d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9c0ffbd0 0x7efc9c195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efc99d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9c0ffbd0 0x7efc9c195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51730/0 (socket says 172.21.15.107:51730) 2023-07-21T22:38:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efc99d9b700 1 -- 172.21.15.107:0/4284927935 learned_addr learned my addr 172.21.15.107:0/4284927935 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efc9a59c700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc9c0fe9e0 0x7efc9c1957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efc9ad9d700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc9c100560 0x7efc9c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:45.913 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efc99d9b700 1 -- 172.21.15.107:0/4284927935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc9c100560 msgr2=0x7efc9c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efc99d9b700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc9c100560 0x7efc9c19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efc99d9b700 1 -- 172.21.15.107:0/4284927935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc9c0fe9e0 msgr2=0x7efc9c1957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efc99d9b700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc9c0fe9e0 0x7efc9c1957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.867+0000 7efc99d9b700 1 -- 172.21.15.107:0/4284927935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c0095f0 con 0x7efc9c0ffbd0 2023-07-21T22:38:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.868+0000 7efc99d9b700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9c0ffbd0 0x7efc9c195d20 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00b620 tx=0x7efc8c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.868+0000 7efc937fe700 1 -- 172.21.15.107:0/4284927935 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c00fb00 con 0x7efc9c0ffbd0 2023-07-21T22:38:45.914 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.868+0000 7efc937fe700 1 -- 172.21.15.107:0/4284927935 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc8c023070 con 0x7efc9c0ffbd0 2023-07-21T22:38:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.868+0000 7efca0ed2700 1 -- 172.21.15.107:0/4284927935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc9c100df0 con 0x7efc9c0ffbd0 2023-07-21T22:38:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.868+0000 7efc937fe700 1 -- 172.21.15.107:0/4284927935 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c01fea0 con 0x7efc9c0ffbd0 2023-07-21T22:38:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.868+0000 7efca0ed2700 1 -- 172.21.15.107:0/4284927935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9c1012a0 con 0x7efc9c0ffbd0 2023-07-21T22:38:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.869+0000 7efca0ed2700 1 -- 172.21.15.107:0/4284927935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9c10a200 con 0x7efc9c0ffbd0 2023-07-21T22:38:45.915 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.870+0000 7efc937fe700 1 -- 172.21.15.107:0/4284927935 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7efc8c025030 con 0x7efc9c0ffbd0 2023-07-21T22:38:45.916 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.870+0000 7efc937fe700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efc880a1150 0x7efc880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:45.916 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.871+0000 7efc937fe700 1 -- 172.21.15.107:0/4284927935 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efc8c0d2040 con 0x7efc9c0ffbd0 2023-07-21T22:38:45.916 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.871+0000 7efc9a59c700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efc880a1150 0x7efc880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:45.916 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.871+0000 7efc9a59c700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efc880a1150 0x7efc880a3600 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0ffa30 tx=0x7efc8400a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:45.917 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:45.874+0000 7efc937fe700 1 -- 172.21.15.107:0/4284927935 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7efc8c09b800 con 0x7efc9c0ffbd0 2023-07-21T22:38:46.108 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.066+0000 7efca0ed2700 1 -- 172.21.15.107:0/4284927935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efc9c101c90 con 0x7efc9c0ffbd0 2023-07-21T22:38:46.109 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.066+0000 7efc937fe700 1 -- 172.21.15.107:0/4284927935 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7efc8c09ee20 con 0x7efc9c0ffbd0 2023-07-21T22:38:46.109 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2023-07-21T22:38:46.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.068+0000 7efca0ed2700 1 -- 172.21.15.107:0/4284927935 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efc880a1150 msgr2=0x7efc880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:46.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.068+0000 7efca0ed2700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efc880a1150 0x7efc880a3600 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0ffa30 tx=0x7efc8400a3b0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.068+0000 7efca0ed2700 1 -- 172.21.15.107:0/4284927935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9c0ffbd0 msgr2=0x7efc9c195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:46.111 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.068+0000 7efca0ed2700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9c0ffbd0 0x7efc9c195d20 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00b620 tx=0x7efc8c00bea0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.068+0000 7efca0ed2700 1 -- 172.21.15.107:0/4284927935 shutdown_connections 2023-07-21T22:38:46.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.068+0000 7efca0ed2700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7efc880a1150 0x7efc880a3600 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.068+0000 7efca0ed2700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc9c100560 0x7efc9c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.068+0000 7efca0ed2700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc9c0fe9e0 0x7efc9c1957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.068+0000 7efca0ed2700 1 --2- 172.21.15.107:0/4284927935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc9c0ffbd0 0x7efc9c195d20 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.068+0000 7efca0ed2700 1 -- 172.21.15.107:0/4284927935 >> 172.21.15.107:0/4284927935 conn(0x7efc9c0fa000 msgr2=0x7efc9c067690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:46.112 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.068+0000 7efca0ed2700 1 -- 172.21.15.107:0/4284927935 shutdown_connections 2023-07-21T22:38:46.113 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.068+0000 7efca0ed2700 1 -- 172.21.15.107:0/4284927935 wait complete. 2023-07-21T22:38:46.122 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:45 smithi107 ceph-mon[96543]: pgmap v584: 97 pgs: 97 active+clean; 793 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 311 op/s 2023-07-21T22:38:46.122 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:45 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2333459282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:38:46.123 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:38:46.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:45 smithi190 ceph-mon[98334]: pgmap v584: 97 pgs: 97 active+clean; 793 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 311 op/s 2023-07-21T22:38:46.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:45 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2333459282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:38:46.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:45 smithi132 ceph-mon[97863]: pgmap v584: 97 pgs: 97 active+clean; 793 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 311 op/s 2023-07-21T22:38:46.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:45 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2333459282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:38:46.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.206+0000 7f6b2beb8700 1 -- 172.21.15.107:0/2438142196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b24100590 msgr2=0x7f6b24106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:46.249 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.206+0000 7f6b2beb8700 1 --2- 172.21.15.107:0/2438142196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b24100590 0x7f6b24106e50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00b0a0 tx=0x7f6b1c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 -- 172.21.15.107:0/2438142196 shutdown_connections 2023-07-21T22:38:46.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 --2- 172.21.15.107:0/2438142196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b24100590 0x7f6b24106e50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 --2- 172.21.15.107:0/2438142196 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b240fea10 0x7f6b240fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 --2- 172.21.15.107:0/2438142196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b240ffc00 0x7f6b24100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 -- 172.21.15.107:0/2438142196 >> 172.21.15.107:0/2438142196 conn(0x7f6b240fa030 msgr2=0x7f6b240fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:46.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 -- 172.21.15.107:0/2438142196 shutdown_connections 2023-07-21T22:38:46.250 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 -- 172.21.15.107:0/2438142196 wait complete. 2023-07-21T22:38:46.252 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 Processor -- start 2023-07-21T22:38:46.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 -- start start 2023-07-21T22:38:46.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b240fea10 0x7f6b241029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:46.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b240ffc00 0x7f6b24100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:46.253 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b24100590 0x7f6b24101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:46.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b24101cc0 con 0x7f6b240fea10 2023-07-21T22:38:46.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b24101e30 con 0x7f6b240ffc00 2023-07-21T22:38:46.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b2beb8700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2419f910 con 0x7f6b24100590 2023-07-21T22:38:46.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b29c54700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b240fea10 0x7f6b241029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:46.254 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b29c54700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b240fea10 0x7f6b241029a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51742/0 (socket says 172.21.15.107:51742) 2023-07-21T22:38:46.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.207+0000 7f6b29c54700 1 -- 172.21.15.107:0/4267933365 learned_addr learned my addr 172.21.15.107:0/4267933365 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:46.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.208+0000 7f6b29453700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b240ffc00 0x7f6b24100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:46.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.208+0000 7f6b29c54700 1 -- 172.21.15.107:0/4267933365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b24100590 msgr2=0x7f6b24101520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:46.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.208+0000 7f6b29c54700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b24100590 0x7f6b24101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.208+0000 7f6b29c54700 1 -- 172.21.15.107:0/4267933365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b240ffc00 msgr2=0x7f6b24100fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:46.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.208+0000 7f6b29c54700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b240ffc00 0x7f6b24100fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.208+0000 7f6b29c54700 1 -- 172.21.15.107:0/4267933365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c009d00 con 0x7f6b240fea10 2023-07-21T22:38:46.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.208+0000 7f6b29453700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b240ffc00 0x7f6b24100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:38:46.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.208+0000 7f6b29c54700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b240fea10 0x7f6b241029a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000d980 tx=0x7f6b2000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:46.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.208+0000 7f6b1affd700 1 -- 172.21.15.107:0/4267933365 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000de60 con 0x7f6b240fea10 2023-07-21T22:38:46.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.208+0000 7f6b1affd700 1 -- 172.21.15.107:0/4267933365 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b200044e0 con 0x7f6b240fea10 2023-07-21T22:38:46.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.208+0000 7f6b2beb8700 1 -- 172.21.15.107:0/4267933365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2419fbf0 con 0x7f6b240fea10 2023-07-21T22:38:46.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.208+0000 7f6b1affd700 1 -- 172.21.15.107:0/4267933365 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20003be0 con 0x7f6b240fea10 2023-07-21T22:38:46.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.208+0000 7f6b2beb8700 1 -- 172.21.15.107:0/4267933365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b241a0100 con 0x7f6b240fea10 2023-07-21T22:38:46.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.209+0000 7f6b2beb8700 1 -- 172.21.15.107:0/4267933365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2410a1f0 con 0x7f6b240fea10 2023-07-21T22:38:46.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.210+0000 7f6b1affd700 1 -- 172.21.15.107:0/4267933365 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6b20009bb0 con 0x7f6b240fea10 2023-07-21T22:38:46.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.210+0000 7f6b1affd700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6b100a1150 0x7f6b100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:46.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.211+0000 7f6b1affd700 1 -- 172.21.15.107:0/4267933365 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6b200cc6d0 con 0x7f6b240fea10 2023-07-21T22:38:46.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.211+0000 7f6b29453700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6b100a1150 0x7f6b100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:46.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.211+0000 7f6b29453700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6b100a1150 0x7f6b100a3600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b14005fd0 tx=0x7f6b14005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:46.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.215+0000 7f6b1affd700 1 -- 172.21.15.107:0/4267933365 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6b20095e00 con 0x7f6b240fea10 2023-07-21T22:38:46.450 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.408+0000 7f6b2beb8700 1 -- 172.21.15.107:0/4267933365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b241a0710 con 0x7f6b240fea10 2023-07-21T22:38:46.451 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.408+0000 7f6b1affd700 1 -- 172.21.15.107:0/4267933365 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b20009940 con 0x7f6b240fea10 2023-07-21T22:38:46.451 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2023-07-21T22:38:46.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.410+0000 7f6b2beb8700 1 -- 172.21.15.107:0/4267933365 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6b100a1150 msgr2=0x7f6b100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:46.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.410+0000 7f6b2beb8700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6b100a1150 0x7f6b100a3600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6b14005fd0 tx=0x7f6b14005e30 comp rx=0 tx=0).stop 2023-07-21T22:38:46.453 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.410+0000 7f6b2beb8700 1 -- 172.21.15.107:0/4267933365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b240fea10 msgr2=0x7f6b241029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:46.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.410+0000 7f6b2beb8700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b240fea10 0x7f6b241029a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000d980 tx=0x7f6b2000dd40 comp rx=0 tx=0).stop 2023-07-21T22:38:46.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.411+0000 7f6b2beb8700 1 -- 172.21.15.107:0/4267933365 shutdown_connections 2023-07-21T22:38:46.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.411+0000 7f6b2beb8700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f6b100a1150 0x7f6b100a3600 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.411+0000 7f6b2beb8700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b24100590 0x7f6b24101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.411+0000 7f6b2beb8700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b240fea10 0x7f6b241029a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.411+0000 7f6b2beb8700 1 --2- 172.21.15.107:0/4267933365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b240ffc00 0x7f6b24100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.454 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.411+0000 7f6b2beb8700 1 -- 172.21.15.107:0/4267933365 >> 172.21.15.107:0/4267933365 conn(0x7f6b240fa030 msgr2=0x7f6b240686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:46.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.411+0000 7f6b2beb8700 1 -- 172.21.15.107:0/4267933365 shutdown_connections 2023-07-21T22:38:46.455 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.411+0000 7f6b2beb8700 1 -- 172.21.15.107:0/4267933365 wait complete. 2023-07-21T22:38:46.465 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-07-21T22:38:46.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.545+0000 7ff1331e5700 1 -- 172.21.15.107:0/1848764437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff12c0fea00 msgr2=0x7ff12c0fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:46.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.545+0000 7ff1331e5700 1 --2- 172.21.15.107:0/1848764437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff12c0fea00 0x7ff12c0fee10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff120009990 tx=0x7ff120009ca0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.545+0000 7ff1331e5700 1 -- 172.21.15.107:0/1848764437 shutdown_connections 2023-07-21T22:38:46.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.545+0000 7ff1331e5700 1 --2- 172.21.15.107:0/1848764437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff12c100580 0x7ff12c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.545+0000 7ff1331e5700 1 --2- 172.21.15.107:0/1848764437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff12c0fea00 0x7ff12c0fee10 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.545+0000 7ff1331e5700 1 --2- 172.21.15.107:0/1848764437 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12c0ffbf0 0x7ff12c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.545+0000 7ff1331e5700 1 -- 172.21.15.107:0/1848764437 >> 172.21.15.107:0/1848764437 conn(0x7ff12c0fa020 msgr2=0x7ff12c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:46.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.545+0000 7ff1331e5700 1 -- 172.21.15.107:0/1848764437 shutdown_connections 2023-07-21T22:38:46.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.545+0000 7ff1331e5700 1 -- 172.21.15.107:0/1848764437 wait complete. 2023-07-21T22:38:46.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff1331e5700 1 Processor -- start 2023-07-21T22:38:46.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff1331e5700 1 -- start start 2023-07-21T22:38:46.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff1331e5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff12c0fea00 0x7ff12c194630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:46.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff1331e5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff12c0ffbf0 0x7ff12c194b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:46.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff1331e5700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12c100580 0x7ff12c19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:46.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff1331e5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c19c1b0 con 0x7ff12c0fea00 2023-07-21T22:38:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff1331e5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c19c2f0 con 0x7ff12c0ffbf0 2023-07-21T22:38:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff1331e5700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c19c480 con 0x7ff12c100580 2023-07-21T22:38:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff130f81700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff12c0fea00 0x7ff12c194630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff130f81700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff12c0fea00 0x7ff12c194630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51754/0 (socket says 172.21.15.107:51754) 2023-07-21T22:38:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff130f81700 1 -- 172.21.15.107:0/3325318892 learned_addr learned my addr 172.21.15.107:0/3325318892 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:38:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff131782700 1 --2- 172.21.15.107:0/3325318892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12c100580 0x7ff12c19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff130f81700 1 -- 172.21.15.107:0/3325318892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12c100580 msgr2=0x7ff12c19bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff130f81700 1 --2- 172.21.15.107:0/3325318892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12c100580 0x7ff12c19bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff130f81700 1 -- 172.21.15.107:0/3325318892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff12c0ffbf0 msgr2=0x7ff12c194b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff130f81700 1 --2- 172.21.15.107:0/3325318892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff12c0ffbf0 0x7ff12c194b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.546+0000 7ff130f81700 1 -- 172.21.15.107:0/3325318892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1200095f0 con 0x7ff12c0fea00 2023-07-21T22:38:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.547+0000 7ff130f81700 1 --2- 172.21.15.107:0/3325318892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff12c0fea00 0x7ff12c194630 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff1200059d0 tx=0x7ff120005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.547+0000 7ff129ffb700 1 -- 172.21.15.107:0/3325318892 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12001d070 con 0x7ff12c0fea00 2023-07-21T22:38:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.547+0000 7ff129ffb700 1 -- 172.21.15.107:0/3325318892 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff120004120 con 0x7ff12c0fea00 2023-07-21T22:38:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.547+0000 7ff1331e5700 1 -- 172.21.15.107:0/3325318892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12c19c700 con 0x7ff12c0fea00 2023-07-21T22:38:46.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.547+0000 7ff129ffb700 1 -- 172.21.15.107:0/3325318892 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120022de0 con 0x7ff12c0fea00 2023-07-21T22:38:46.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.547+0000 7ff1331e5700 1 -- 172.21.15.107:0/3325318892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c19cb20 con 0x7ff12c0fea00 2023-07-21T22:38:46.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.548+0000 7ff1331e5700 1 -- 172.21.15.107:0/3325318892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12c105840 con 0x7ff12c0fea00 2023-07-21T22:38:46.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.549+0000 7ff129ffb700 1 -- 172.21.15.107:0/3325318892 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff120022470 con 0x7ff12c0fea00 2023-07-21T22:38:46.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.550+0000 7ff129ffb700 1 --2- 172.21.15.107:0/3325318892 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff11c0a11a0 0x7ff11c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:46.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.550+0000 7ff12bfff700 1 --2- 172.21.15.107:0/3325318892 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff11c0a11a0 0x7ff11c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:46.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.550+0000 7ff129ffb700 1 -- 172.21.15.107:0/3325318892 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff1200ce4c0 con 0x7ff12c0fea00 2023-07-21T22:38:46.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.550+0000 7ff12bfff700 1 --2- 172.21.15.107:0/3325318892 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff11c0a11a0 0x7ff11c0a3650 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff118005fd0 tx=0x7ff118005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:46.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.553+0000 7ff129ffb700 1 -- 172.21.15.107:0/3325318892 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff120097c20 con 0x7ff12c0fea00 2023-07-21T22:38:46.841 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.798+0000 7ff1331e5700 1 -- 172.21.15.107:0/3325318892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7ff12c02c850 con 0x7ff12c0fea00 2023-07-21T22:38:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.806+0000 7ff129ffb700 1 -- 172.21.15.107:0/3325318892 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733250 (secure 0 0 0) 0x7ff12009b240 con 0x7ff12c0fea00 2023-07-21T22:38:46.856 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:38:46.856 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:38:46.856 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 18, 2023-07-21T22:38:46.856 INFO:teuthology.orchestra.run.smithi107.stdout: "active_gid": 14150, 2023-07-21T22:38:46.857 INFO:teuthology.orchestra.run.smithi107.stdout: "active_name": "x", 2023-07-21T22:38:46.857 INFO:teuthology.orchestra.run.smithi107.stdout: "active_addrs": { 2023-07-21T22:38:46.857 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2023-07-21T22:38:46.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:46.857 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2023-07-21T22:38:46.857 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2023-07-21T22:38:46.858 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1657248266 2023-07-21T22:38:46.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.858 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:46.858 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2023-07-21T22:38:46.858 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2023-07-21T22:38:46.858 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1657248266 2023-07-21T22:38:46.858 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:46.858 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2023-07-21T22:38:46.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.859 INFO:teuthology.orchestra.run.smithi107.stdout: "active_addr": "172.21.15.107:6801/1657248266", 2023-07-21T22:38:46.859 INFO:teuthology.orchestra.run.smithi107.stdout: "active_change": "2023-07-21T22:21:45.714467+0000", 2023-07-21T22:38:46.859 INFO:teuthology.orchestra.run.smithi107.stdout: "active_mgr_features": 4540138322906710015, 2023-07-21T22:38:46.859 INFO:teuthology.orchestra.run.smithi107.stdout: "available": true, 2023-07-21T22:38:46.859 INFO:teuthology.orchestra.run.smithi107.stdout: "standbys": [ 2023-07-21T22:38:46.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:46.860 INFO:teuthology.orchestra.run.smithi107.stdout: "gid": 24109, 2023-07-21T22:38:46.860 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "z", 2023-07-21T22:38:46.860 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_features": 4540138322906710015, 2023-07-21T22:38:46.860 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2023-07-21T22:38:46.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:46.860 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2023-07-21T22:38:46.861 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:46.861 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:46.861 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:46.861 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:46.861 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:46.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:46.861 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.861 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.862 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:46.862 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.862 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.862 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.862 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T22:38:46.862 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.862 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.863 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.863 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:46.863 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:46.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.863 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.863 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.864 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:46.864 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.864 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.864 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:46.864 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:46.864 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:46.864 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:46.865 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:46.865 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:46.865 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:46.865 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:46.865 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:46.865 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.865 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.865 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.866 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:46.866 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:46.866 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.866 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.866 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.866 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:46.866 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.867 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.867 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.867 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:46.867 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.867 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.867 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.869 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.869 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:46.869 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:46.869 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:46.870 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:46.870 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:46.870 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:46.870 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:46.870 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:46.870 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:46.870 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.871 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.871 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.871 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:46.871 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:46.871 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.871 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.871 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.872 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:46.872 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.872 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.872 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.872 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:46.872 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.872 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.873 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.873 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2023-07-21T22:38:46.873 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2023-07-21T22:38:46.873 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.873 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.873 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.874 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:46.874 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.874 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.874 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.874 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2023-07-21T22:38:46.874 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.875 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.875 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.875 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2023-07-21T22:38:46.875 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2023-07-21T22:38:46.875 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.875 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.876 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.876 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2023-07-21T22:38:46.876 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.876 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.876 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.876 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2023-07-21T22:38:46.876 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.876 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.877 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.877 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2023-07-21T22:38:46.877 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2023-07-21T22:38:46.877 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.877 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.877 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.878 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:46.878 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.878 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.878 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.878 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2023-07-21T22:38:46.878 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.878 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.879 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.879 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2023-07-21T22:38:46.879 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2023-07-21T22:38:46.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.879 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.879 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.880 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:46.880 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.880 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.880 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.880 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2023-07-21T22:38:46.880 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.881 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.881 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.881 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2023-07-21T22:38:46.881 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2023-07-21T22:38:46.881 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:46.881 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.881 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.882 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2023-07-21T22:38:46.882 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.882 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.882 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.882 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2023-07-21T22:38:46.882 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.882 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.882 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.883 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2023-07-21T22:38:46.883 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2023-07-21T22:38:46.883 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.883 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.883 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.883 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:46.883 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.884 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.884 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.884 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2023-07-21T22:38:46.884 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.884 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.885 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2023-07-21T22:38:46.885 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2023-07-21T22:38:46.885 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.885 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.885 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.885 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:46.885 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.886 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.886 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.886 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T22:38:46.886 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.886 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.886 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.887 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2023-07-21T22:38:46.887 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2023-07-21T22:38:46.887 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.887 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.887 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.887 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:46.887 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.888 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.888 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.888 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2023-07-21T22:38:46.888 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.888 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.888 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.888 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:46.889 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:46.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.889 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:46.889 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2023-07-21T22:38:46.889 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:46.889 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:46.889 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:46.889 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2023-07-21T22:38:46.890 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2023-07-21T22:38:46.890 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.890 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.890 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.890 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:46.890 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.891 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.891 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.891 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T22:38:46.891 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.891 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.891 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.891 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2023-07-21T22:38:46.892 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2023-07-21T22:38:46.892 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.892 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.892 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.892 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2023-07-21T22:38:46.892 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.892 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.893 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.893 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T22:38:46.893 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:38:46.893 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.893 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.893 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2023-07-21T22:38:46.893 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2023-07-21T22:38:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2023-07-21T22:38:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2023-07-21T22:38:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T22:38:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:38:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2023-07-21T22:38:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2023-07-21T22:38:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:46.896 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.896 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.896 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2023-07-21T22:38:46.896 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2023-07-21T22:38:46.896 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2023-07-21T22:38:46.896 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.896 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T22:38:46.897 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.897 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.897 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.897 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2023-07-21T22:38:46.897 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2023-07-21T22:38:46.897 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.897 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.898 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.898 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T22:38:46.898 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.898 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.898 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.898 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T22:38:46.898 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T22:38:46.898 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.899 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.899 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2023-07-21T22:38:46.899 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2023-07-21T22:38:46.899 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:46.899 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.899 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.900 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2023-07-21T22:38:46.900 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2023-07-21T22:38:46.900 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2023-07-21T22:38:46.900 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.900 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2023-07-21T22:38:46.900 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T22:38:46.900 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.901 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.901 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2023-07-21T22:38:46.901 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2023-07-21T22:38:46.901 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.901 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.901 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.902 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2023-07-21T22:38:46.902 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.902 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.902 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.902 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T22:38:46.902 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:38:46.902 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.902 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.903 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2023-07-21T22:38:46.903 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2023-07-21T22:38:46.903 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:46.903 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.903 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.903 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:46.904 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2023-07-21T22:38:46.904 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2023-07-21T22:38:46.904 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.904 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T22:38:46.904 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:38:46.904 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.904 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.905 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:46.905 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:46.905 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.905 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.905 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.905 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:46.905 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.906 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.906 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:46.906 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:46.906 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:46.906 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:46.906 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:46.906 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:46.907 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:46.907 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:46.907 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:46.907 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.907 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.907 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.907 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:46.908 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:46.908 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.908 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.908 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.908 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:46.908 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.909 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.909 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.909 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:46.909 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.909 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.909 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.909 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:46.910 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:46.910 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.910 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.910 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.910 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:46.910 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.910 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.910 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:46.911 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:46.911 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:46.911 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:46.911 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:46.912 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:46.912 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:46.912 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:46.912 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:46.912 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.912 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.912 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.913 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:46.913 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:46.913 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.913 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.913 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.913 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:46.913 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.914 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.914 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.914 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:46.914 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.914 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.914 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.914 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2023-07-21T22:38:46.915 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2023-07-21T22:38:46.915 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:46.915 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.915 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.915 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:46.915 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.915 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.916 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.916 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T22:38:46.916 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.916 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.916 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.916 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2023-07-21T22:38:46.916 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2023-07-21T22:38:46.917 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.917 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.917 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.917 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2023-07-21T22:38:46.917 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.917 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.917 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:46.918 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2023-07-21T22:38:46.918 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2023-07-21T22:38:46.918 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap" 2023-07-21T22:38:46.918 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:46.918 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2023-07-21T22:38:46.918 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.918 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.918 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.919 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2023-07-21T22:38:46.919 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2023-07-21T22:38:46.919 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.919 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.919 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.919 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:46.919 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.920 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.920 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.920 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T22:38:46.920 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.920 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.920 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.920 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:46.921 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:46.921 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:46.921 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.921 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.921 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:46.921 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.921 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.922 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.922 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T22:38:46.922 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.922 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.922 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.922 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2023-07-21T22:38:46.923 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2023-07-21T22:38:46.923 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:46.923 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.923 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.923 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2023-07-21T22:38:46.923 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2023-07-21T22:38:46.923 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.923 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.924 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T22:38:46.924 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T22:38:46.924 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.924 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.924 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2023-07-21T22:38:46.925 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2023-07-21T22:38:46.925 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:46.925 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.925 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.925 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:46.925 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.925 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.925 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.926 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T22:38:46.926 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.926 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.926 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.926 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:46.926 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:46.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:46.927 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2023-07-21T22:38:46.927 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:46.927 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:46.927 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:46.927 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2023-07-21T22:38:46.927 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2023-07-21T22:38:46.927 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:46.927 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.928 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.928 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2023-07-21T22:38:46.928 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.928 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.928 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.928 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T22:38:46.928 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.929 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.929 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.929 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2023-07-21T22:38:46.929 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2023-07-21T22:38:46.929 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:46.929 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.929 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.930 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2023-07-21T22:38:46.930 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.930 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.930 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.930 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T22:38:46.930 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.930 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.930 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.931 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2023-07-21T22:38:46.931 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2023-07-21T22:38:46.931 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:46.931 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.931 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.931 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2023-07-21T22:38:46.931 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.932 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.932 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.932 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T22:38:46.932 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.932 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.933 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.933 INFO:teuthology.orchestra.run.smithi107.stdout: "alertmanager_web_password": { 2023-07-21T22:38:46.933 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alertmanager_web_password", 2023-07-21T22:38:46.933 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.933 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.933 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.934 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:46.934 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.934 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.934 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.934 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Alertmanager web password", 2023-07-21T22:38:46.934 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.934 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.935 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.935 INFO:teuthology.orchestra.run.smithi107.stdout: "alertmanager_web_user": { 2023-07-21T22:38:46.935 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alertmanager_web_user", 2023-07-21T22:38:46.935 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.935 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.935 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.935 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:46.936 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.936 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.936 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.936 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Alertmanager web user", 2023-07-21T22:38:46.936 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.936 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.936 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.937 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2023-07-21T22:38:46.937 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2023-07-21T22:38:46.937 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.937 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.937 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.937 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:46.937 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.938 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.938 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.938 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T22:38:46.938 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T22:38:46.938 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.938 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.939 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2023-07-21T22:38:46.939 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2023-07-21T22:38:46.939 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:46.939 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.939 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.939 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:46.939 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.939 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.940 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.940 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T22:38:46.940 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.940 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.940 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.940 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2023-07-21T22:38:46.940 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T22:38:46.941 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:46.941 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.941 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.941 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2023-07-21T22:38:46.941 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.941 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.941 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.941 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T22:38:46.942 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.942 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.942 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.942 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2023-07-21T22:38:46.942 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2023-07-21T22:38:46.943 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.943 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.943 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.943 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:46.943 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.943 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.943 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.943 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T22:38:46.944 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.944 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.944 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.944 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2023-07-21T22:38:46.944 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2023-07-21T22:38:46.944 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.944 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.945 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.945 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:46.945 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.945 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.945 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.945 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T22:38:46.945 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.946 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.946 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.946 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2023-07-21T22:38:46.946 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2023-07-21T22:38:46.946 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.946 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.946 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.947 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:46.947 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.947 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.947 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.947 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T22:38:46.947 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.947 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.947 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.948 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2023-07-21T22:38:46.948 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2023-07-21T22:38:46.948 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.948 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.948 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.948 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T22:38:46.948 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.949 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.949 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.949 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2023-07-21T22:38:46.949 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.949 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.949 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.949 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2023-07-21T22:38:46.950 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2023-07-21T22:38:46.950 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.950 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.950 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.950 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T22:38:46.950 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.950 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.950 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.951 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2023-07-21T22:38:46.951 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.951 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.951 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.951 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2023-07-21T22:38:46.951 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2023-07-21T22:38:46.951 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.952 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.952 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.952 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T22:38:46.952 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.952 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.952 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.952 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2023-07-21T22:38:46.953 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.953 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.953 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.953 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2023-07-21T22:38:46.953 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2023-07-21T22:38:46.953 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.954 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.954 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.954 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T22:38:46.954 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.954 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.954 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.954 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2023-07-21T22:38:46.954 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.955 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.955 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.955 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2023-07-21T22:38:46.955 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2023-07-21T22:38:46.955 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.955 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.956 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.956 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T22:38:46.956 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.956 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.956 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.956 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2023-07-21T22:38:46.956 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.957 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.957 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.957 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2023-07-21T22:38:46.957 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2023-07-21T22:38:46.957 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.957 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.958 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.958 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T22:38:46.958 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.958 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.958 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.958 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2023-07-21T22:38:46.958 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.959 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.959 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.959 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2023-07-21T22:38:46.959 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2023-07-21T22:38:46.959 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.959 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.959 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.960 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T22:38:46.960 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.960 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.960 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.960 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2023-07-21T22:38:46.960 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.960 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.961 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.961 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2023-07-21T22:38:46.961 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2023-07-21T22:38:46.961 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.961 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.961 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.961 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T22:38:46.962 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.962 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.962 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.962 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2023-07-21T22:38:46.962 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.962 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.962 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.963 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2023-07-21T22:38:46.963 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2023-07-21T22:38:46.963 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.963 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.963 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.963 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T22:38:46.963 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.963 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.964 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.964 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2023-07-21T22:38:46.964 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.964 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.964 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.964 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2023-07-21T22:38:46.964 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2023-07-21T22:38:46.965 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.965 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.965 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.965 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T22:38:46.965 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.965 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.965 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.965 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2023-07-21T22:38:46.966 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.966 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.966 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.966 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2023-07-21T22:38:46.966 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2023-07-21T22:38:46.966 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.967 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.967 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.967 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T22:38:46.967 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.967 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.967 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.967 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2023-07-21T22:38:46.967 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.968 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.968 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2023-07-21T22:38:46.968 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2023-07-21T22:38:46.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.968 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.969 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.969 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T22:38:46.969 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.969 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.969 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.969 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2023-07-21T22:38:46.969 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.969 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.970 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.970 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2023-07-21T22:38:46.970 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2023-07-21T22:38:46.970 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.970 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.970 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.971 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T22:38:46.971 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.971 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.971 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.971 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2023-07-21T22:38:46.971 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.971 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.971 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.972 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2023-07-21T22:38:46.972 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2023-07-21T22:38:46.972 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.972 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.972 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.972 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T22:38:46.972 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.973 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.973 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.973 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2023-07-21T22:38:46.973 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.973 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.973 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.973 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2023-07-21T22:38:46.974 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2023-07-21T22:38:46.974 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.974 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.974 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.974 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:46.974 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.974 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.974 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.975 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T22:38:46.975 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.975 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.975 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.975 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2023-07-21T22:38:46.975 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2023-07-21T22:38:46.975 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:46.976 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.976 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.976 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:46.976 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.976 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.976 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.976 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T22:38:46.976 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.977 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.977 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.977 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2023-07-21T22:38:46.977 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T22:38:46.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:46.977 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.978 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.978 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2023-07-21T22:38:46.978 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.978 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.978 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.978 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T22:38:46.978 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.979 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.979 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.979 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2023-07-21T22:38:46.979 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2023-07-21T22:38:46.979 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.979 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.980 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.980 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2023-07-21T22:38:46.980 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.980 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.980 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.980 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T22:38:46.980 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.980 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.981 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.981 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2023-07-21T22:38:46.981 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2023-07-21T22:38:46.981 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:46.981 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.981 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.981 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2023-07-21T22:38:46.982 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.982 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.982 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.982 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2023-07-21T22:38:46.982 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.982 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.982 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.983 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2023-07-21T22:38:46.983 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2023-07-21T22:38:46.983 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.983 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.983 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.984 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:46.984 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.984 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.984 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.984 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T22:38:46.984 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.985 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.985 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.985 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2023-07-21T22:38:46.985 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2023-07-21T22:38:46.985 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:46.985 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.985 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.986 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:46.986 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.986 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.986 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.986 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2023-07-21T22:38:46.986 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.986 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.986 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.987 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2023-07-21T22:38:46.987 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2023-07-21T22:38:46.987 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:46.987 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.987 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.987 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:46.988 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.988 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.988 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.988 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2023-07-21T22:38:46.988 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.988 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.988 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.989 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:46.989 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:46.989 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.989 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.989 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.989 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:46.989 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.989 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.990 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:46.990 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:46.990 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:46.990 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:46.990 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:46.990 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:46.990 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:46.990 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:46.991 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:46.991 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.991 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.991 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.991 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2023-07-21T22:38:46.991 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2023-07-21T22:38:46.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.992 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.992 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.992 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:46.992 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.992 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.992 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.992 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T22:38:46.993 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.993 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.993 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.993 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:46.993 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:46.993 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.993 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.994 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:46.994 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:46.994 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.994 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.994 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.994 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T22:38:46.994 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.994 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.995 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.995 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:46.995 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:46.995 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:46.995 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.995 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.996 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:46.996 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.996 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.996 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:46.996 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:46.996 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:46.996 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:46.996 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:46.997 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:46.997 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:46.997 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:46.997 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:46.997 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.997 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.997 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.998 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:46.998 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:46.998 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:46.998 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:46.998 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:46.998 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:46.998 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:46.999 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:46.999 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:46.999 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:46.999 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:46.999 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:46.999 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:46.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:46.999 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T22:38:47.000 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T22:38:47.000 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.000 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.000 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.000 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.000 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.000 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.001 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.001 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T22:38:47.001 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.001 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.001 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.001 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T22:38:47.001 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T22:38:47.002 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.002 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.002 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.002 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2023-07-21T22:38:47.002 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.002 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.002 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.002 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T22:38:47.003 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.003 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.003 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.003 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2023-07-21T22:38:47.003 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2023-07-21T22:38:47.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.003 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.004 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:47.004 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.004 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.004 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.004 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2023-07-21T22:38:47.004 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.005 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.005 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.005 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2023-07-21T22:38:47.005 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2023-07-21T22:38:47.005 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.005 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.005 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.006 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:47.006 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.006 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.006 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.006 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T22:38:47.006 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.006 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.006 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.007 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2023-07-21T22:38:47.007 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2023-07-21T22:38:47.007 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.007 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.007 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.007 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.008 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.008 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.008 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.008 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2023-07-21T22:38:47.008 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.008 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.008 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.009 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2023-07-21T22:38:47.009 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2023-07-21T22:38:47.009 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.009 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.009 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.009 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2023-07-21T22:38:47.009 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.010 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.010 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.010 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2023-07-21T22:38:47.010 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2023-07-21T22:38:47.010 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.010 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T22:38:47.010 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.010 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2023-07-21T22:38:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2023-07-21T22:38:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.011 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T22:38:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T22:38:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_web_password": { 2023-07-21T22:38:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_web_password", 2023-07-21T22:38:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.013 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.014 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.014 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus web password", 2023-07-21T22:38:47.014 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.014 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.014 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.014 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_web_user": { 2023-07-21T22:38:47.015 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_web_user", 2023-07-21T22:38:47.015 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.015 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.015 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.015 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:47.015 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.015 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.015 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.016 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus web user", 2023-07-21T22:38:47.016 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.016 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.016 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.016 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2023-07-21T22:38:47.016 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2023-07-21T22:38:47.016 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.017 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.017 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.017 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.017 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.017 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.017 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.017 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T22:38:47.018 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.018 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.018 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.018 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2023-07-21T22:38:47.018 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2023-07-21T22:38:47.018 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.018 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.019 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.019 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.019 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.019 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.019 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.019 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T22:38:47.019 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.020 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.020 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.020 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2023-07-21T22:38:47.020 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2023-07-21T22:38:47.020 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.020 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.020 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.021 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.021 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.021 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.021 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.021 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T22:38:47.021 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.021 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.021 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.022 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2023-07-21T22:38:47.022 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2023-07-21T22:38:47.022 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.022 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.022 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.022 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.022 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.023 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.023 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.023 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T22:38:47.023 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.023 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.023 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.024 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2023-07-21T22:38:47.024 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2023-07-21T22:38:47.024 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.024 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.024 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.024 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.024 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.024 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.025 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.025 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T22:38:47.025 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.025 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.025 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.026 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2023-07-21T22:38:47.026 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2023-07-21T22:38:47.026 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.026 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.026 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.026 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2023-07-21T22:38:47.026 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.026 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.027 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.027 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2023-07-21T22:38:47.027 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.027 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.027 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.027 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2023-07-21T22:38:47.027 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2023-07-21T22:38:47.028 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.028 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.028 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.028 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.028 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.028 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.028 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.029 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T22:38:47.029 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.029 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.029 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.029 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2023-07-21T22:38:47.029 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2023-07-21T22:38:47.029 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.030 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.030 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.030 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.030 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.030 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.030 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.030 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T22:38:47.030 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.031 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.031 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.031 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2023-07-21T22:38:47.031 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2023-07-21T22:38:47.031 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.031 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.031 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.032 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.032 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.032 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.032 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.032 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T22:38:47.032 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.032 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.032 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.033 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2023-07-21T22:38:47.033 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2023-07-21T22:38:47.033 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.033 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.033 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.033 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.034 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.034 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.034 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.034 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T22:38:47.034 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.034 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.034 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.035 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2023-07-21T22:38:47.035 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2023-07-21T22:38:47.035 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.035 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.035 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.035 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.035 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.036 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.036 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.036 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T22:38:47.036 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.036 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.036 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.037 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2023-07-21T22:38:47.037 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2023-07-21T22:38:47.037 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.037 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.037 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.037 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.037 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.037 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.038 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.038 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T22:38:47.038 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.038 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.038 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.038 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.038 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.039 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.039 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2023-07-21T22:38:47.039 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.039 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.039 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.039 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.039 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.040 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.040 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.040 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.040 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.040 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.040 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.041 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.041 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.041 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.041 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.041 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.041 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.041 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.041 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.042 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.042 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.042 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.042 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.042 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.042 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.042 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.043 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.043 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.043 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.043 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.043 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.043 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.043 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.044 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.044 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.044 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.044 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.044 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.044 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.045 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.045 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.045 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.045 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.045 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.045 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.045 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.046 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.046 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.046 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.046 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.046 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.046 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.046 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.046 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.047 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.047 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.047 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.047 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.047 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.047 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.048 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.048 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.048 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.048 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.048 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.048 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.048 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.049 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2023-07-21T22:38:47.049 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2023-07-21T22:38:47.049 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.049 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.049 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.049 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2023-07-21T22:38:47.049 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.049 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.050 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.050 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T22:38:47.050 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.050 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.050 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.050 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2023-07-21T22:38:47.051 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2023-07-21T22:38:47.051 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.051 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.051 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.051 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2023-07-21T22:38:47.051 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.051 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.051 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.052 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T22:38:47.052 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.052 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.052 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.052 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.052 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.053 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2023-07-21T22:38:47.053 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.053 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.053 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.053 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T22:38:47.053 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T22:38:47.053 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.053 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.054 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.054 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:47.054 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.054 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.054 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.054 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.055 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.055 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.055 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.055 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T22:38:47.055 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T22:38:47.055 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.055 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.056 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.056 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.056 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.056 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.056 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.056 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.057 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.057 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.057 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T22:38:47.057 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T22:38:47.057 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.057 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.057 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.058 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.058 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.058 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.058 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.058 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.058 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.058 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.059 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.059 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2023-07-21T22:38:47.059 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T22:38:47.059 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.059 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.059 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.059 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.060 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.060 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.060 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.060 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.060 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.060 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.061 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T22:38:47.061 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T22:38:47.061 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.061 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.061 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.061 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.061 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.062 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.062 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.062 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.062 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.062 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.062 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.062 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T22:38:47.063 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T22:38:47.063 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.063 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.063 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.063 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.063 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.063 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.063 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.064 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.064 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.064 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.064 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T22:38:47.064 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T22:38:47.064 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.065 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.065 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.065 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.065 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.065 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.065 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.065 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.065 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.066 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.066 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.066 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T22:38:47.066 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T22:38:47.066 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.066 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.067 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.067 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.067 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.067 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.067 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.067 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.067 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.067 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.068 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.068 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T22:38:47.068 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T22:38:47.068 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.068 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.068 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.069 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.069 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.069 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.069 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.069 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.069 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.069 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.070 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.070 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T22:38:47.070 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T22:38:47.070 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.070 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.070 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.070 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.071 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.071 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.071 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.071 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.071 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.071 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.071 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.072 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T22:38:47.072 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T22:38:47.072 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.072 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.072 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.072 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.072 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.073 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.073 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.073 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.073 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.073 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.073 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.073 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T22:38:47.074 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T22:38:47.074 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.074 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.074 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.074 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.074 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.074 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.075 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.075 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.075 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.075 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.075 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.075 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T22:38:47.075 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T22:38:47.076 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.076 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.076 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.076 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.076 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.076 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.076 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.076 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.077 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.077 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.077 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.077 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T22:38:47.077 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T22:38:47.077 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.078 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.078 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.078 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.078 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.078 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.078 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.078 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.078 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.079 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.079 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.079 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T22:38:47.079 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T22:38:47.079 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.079 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.079 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.080 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:47.080 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.080 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.080 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.080 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.080 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.080 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.081 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.081 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T22:38:47.081 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T22:38:47.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.081 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.081 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.082 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.082 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.082 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.082 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.082 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.082 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.082 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.083 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2023-07-21T22:38:47.083 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2023-07-21T22:38:47.083 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.083 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.083 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.083 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.083 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.083 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.084 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.084 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.084 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.084 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.084 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.084 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T22:38:47.085 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T22:38:47.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.085 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.085 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.085 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:47.085 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.085 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.086 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.086 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.086 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.086 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.086 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T22:38:47.086 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T22:38:47.087 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.087 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.087 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.087 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.087 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.087 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.087 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.087 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.088 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.088 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.088 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.088 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T22:38:47.088 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T22:38:47.088 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.089 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.089 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.089 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.089 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.089 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.089 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.090 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.090 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.090 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.090 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.090 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T22:38:47.091 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T22:38:47.091 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.091 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.091 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.091 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.092 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.092 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.092 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.092 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.092 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.092 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.093 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.093 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T22:38:47.093 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T22:38:47.093 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.093 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.093 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.093 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.094 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.094 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.094 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.094 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.094 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.094 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.094 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.095 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T22:38:47.095 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T22:38:47.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.095 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.095 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.095 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.095 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.096 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.096 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.096 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.096 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.096 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.096 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.097 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T22:38:47.097 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T22:38:47.097 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.097 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.097 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.097 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.097 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.098 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.098 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.098 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.098 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.098 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.098 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.098 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T22:38:47.099 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T22:38:47.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.099 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.099 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.099 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.099 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.099 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.099 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.100 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.100 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.100 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.100 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.100 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T22:38:47.100 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T22:38:47.101 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.101 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.101 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.101 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.101 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.101 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.101 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.101 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.102 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.102 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.102 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.102 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T22:38:47.102 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T22:38:47.102 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.102 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.103 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.103 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.103 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.103 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.103 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.104 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.104 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.104 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.104 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.105 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T22:38:47.105 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T22:38:47.106 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.106 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.106 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.106 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.107 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.107 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.107 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.108 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.108 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.108 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.109 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.109 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T22:38:47.109 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T22:38:47.110 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.110 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.110 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.110 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.111 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.111 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.111 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.111 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.112 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.112 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.112 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.112 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T22:38:47.113 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T22:38:47.113 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.113 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.114 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.114 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.114 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.114 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.114 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.114 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.114 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.115 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.115 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.115 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T22:38:47.115 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T22:38:47.115 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.115 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.116 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.116 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.116 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.116 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.116 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.116 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.116 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.116 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.117 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.117 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T22:38:47.117 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T22:38:47.117 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.117 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.117 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.118 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.118 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.118 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.118 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.118 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.118 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.118 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.118 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.119 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T22:38:47.119 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T22:38:47.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.119 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.119 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.119 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T22:38:47.119 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.120 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.120 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.120 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.120 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.120 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.120 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.121 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T22:38:47.121 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T22:38:47.121 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.121 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.121 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.121 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:47.121 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.121 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.122 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.122 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.122 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.122 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.122 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.122 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T22:38:47.122 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T22:38:47.123 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.123 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.123 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.123 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2023-07-21T22:38:47.123 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.123 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.123 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.123 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.124 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.124 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.124 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.124 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T22:38:47.124 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T22:38:47.124 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.125 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.125 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.125 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2023-07-21T22:38:47.125 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.125 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.125 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.125 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.126 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.126 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.126 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.126 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T22:38:47.126 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T22:38:47.126 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.127 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.127 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.127 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.127 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.127 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.127 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.127 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.127 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.128 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.128 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.128 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T22:38:47.128 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T22:38:47.128 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.128 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.129 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.129 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:47.129 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.129 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.129 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.129 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.129 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.129 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.130 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.130 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T22:38:47.130 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T22:38:47.130 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.130 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.130 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.130 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.131 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.131 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.131 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.131 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.131 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.131 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.131 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.132 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T22:38:47.132 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T22:38:47.132 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.132 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.132 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.132 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.132 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.133 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.133 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.133 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.133 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.133 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.133 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.134 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T22:38:47.134 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T22:38:47.134 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.134 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.135 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.135 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:47.135 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.136 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.136 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.136 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.136 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.137 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.137 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.137 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T22:38:47.138 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T22:38:47.138 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.138 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.138 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.139 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:47.139 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.139 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.139 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.140 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.140 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.140 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.141 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.141 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T22:38:47.141 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T22:38:47.142 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.142 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.142 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.142 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2023-07-21T22:38:47.143 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.143 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.143 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.143 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.144 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.144 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.144 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.145 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2023-07-21T22:38:47.145 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2023-07-21T22:38:47.145 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.145 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.146 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.146 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.146 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.146 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.147 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.147 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.147 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.147 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.148 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.148 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2023-07-21T22:38:47.148 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2023-07-21T22:38:47.149 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.149 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.149 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.149 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.150 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.150 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.150 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.151 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.151 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.151 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.151 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.152 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2023-07-21T22:38:47.152 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2023-07-21T22:38:47.152 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.153 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.153 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.153 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.153 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.154 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.154 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.154 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2023-07-21T22:38:47.154 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.155 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.155 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.155 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2023-07-21T22:38:47.156 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2023-07-21T22:38:47.156 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.156 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.157 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.157 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2023-07-21T22:38:47.157 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.157 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.158 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.158 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.158 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.158 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.159 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.159 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2023-07-21T22:38:47.159 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2023-07-21T22:38:47.160 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.160 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.160 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.160 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.161 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.161 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.161 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.162 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.162 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.162 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.162 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.163 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.163 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.163 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.164 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.164 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.164 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.165 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.165 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.165 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.165 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.165 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.165 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.165 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.166 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.166 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.166 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.166 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.166 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.166 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.166 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.167 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.167 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.167 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.167 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.167 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.167 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.167 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.168 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.168 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.168 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.168 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.168 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.168 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.169 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.169 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.169 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.169 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.169 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.169 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.170 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.170 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.170 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.170 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.170 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.170 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.170 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.170 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.171 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.171 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.171 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.171 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.171 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.171 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.171 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.172 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.172 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.172 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.172 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.172 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.172 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.172 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.173 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.173 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.173 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.173 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.173 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.173 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2023-07-21T22:38:47.173 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2023-07-21T22:38:47.174 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.174 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.174 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.174 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.174 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.174 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.174 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.174 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2023-07-21T22:38:47.175 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.175 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.175 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.175 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2023-07-21T22:38:47.175 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T22:38:47.175 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.176 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.176 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.176 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.176 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.176 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.176 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.176 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.176 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.177 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.177 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.177 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2023-07-21T22:38:47.177 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2023-07-21T22:38:47.177 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.177 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.178 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.178 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2023-07-21T22:38:47.178 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.178 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.178 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.178 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.178 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.178 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.179 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.179 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2023-07-21T22:38:47.179 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2023-07-21T22:38:47.179 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.180 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.180 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.180 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2023-07-21T22:38:47.181 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.181 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.181 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.181 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.182 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.182 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.182 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.183 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2023-07-21T22:38:47.183 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2023-07-21T22:38:47.183 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.184 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.184 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.184 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.185 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.185 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.185 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.186 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.186 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.186 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.186 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.187 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2023-07-21T22:38:47.187 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2023-07-21T22:38:47.187 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.187 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.188 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.188 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2023-07-21T22:38:47.188 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.188 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.189 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.189 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.189 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.189 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.190 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.190 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2023-07-21T22:38:47.190 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2023-07-21T22:38:47.190 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.190 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.190 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.191 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2023-07-21T22:38:47.191 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.191 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.191 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.191 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.191 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2023-07-21T22:38:47.191 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.191 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.192 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.192 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.192 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.192 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2023-07-21T22:38:47.192 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2023-07-21T22:38:47.192 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.193 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.193 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.193 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2023-07-21T22:38:47.193 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2023-07-21T22:38:47.193 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2023-07-21T22:38:47.193 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.193 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.193 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.194 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.194 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.194 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2023-07-21T22:38:47.194 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2023-07-21T22:38:47.194 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.194 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.194 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.195 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2023-07-21T22:38:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.196 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2023-07-21T22:38:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2023-07-21T22:38:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.197 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2023-07-21T22:38:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.199 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.200 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.201 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.202 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.203 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.204 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.205 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.206 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.207 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.208 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.209 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.209 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.209 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.209 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.209 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.210 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.210 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.210 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.210 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.211 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.211 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.211 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.211 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.211 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.211 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.212 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2023-07-21T22:38:47.212 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2023-07-21T22:38:47.212 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.212 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.212 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.212 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2023-07-21T22:38:47.212 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.213 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.213 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.213 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T22:38:47.213 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.213 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.213 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.213 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2023-07-21T22:38:47.214 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2023-07-21T22:38:47.214 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.214 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.214 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.214 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2023-07-21T22:38:47.214 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.214 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.214 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.215 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T22:38:47.215 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.215 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.215 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.215 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2023-07-21T22:38:47.215 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2023-07-21T22:38:47.215 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.216 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.216 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.216 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2023-07-21T22:38:47.216 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.216 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.216 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.216 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2023-07-21T22:38:47.217 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.217 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.217 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.217 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2023-07-21T22:38:47.217 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2023-07-21T22:38:47.217 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.217 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.218 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.218 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2023-07-21T22:38:47.218 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.218 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.218 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.218 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T22:38:47.218 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.218 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.219 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.219 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2023-07-21T22:38:47.219 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2023-07-21T22:38:47.219 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.219 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.219 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.219 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T22:38:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.221 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:47.221 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:47.221 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.221 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.221 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.221 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:47.221 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.222 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.222 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.222 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T22:38:47.222 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.222 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.222 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.222 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2023-07-21T22:38:47.223 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2023-07-21T22:38:47.223 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.223 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.223 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.223 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2023-07-21T22:38:47.223 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.223 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.223 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.224 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T22:38:47.224 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.224 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.224 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.224 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.224 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.225 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.225 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2023-07-21T22:38:47.225 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.225 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.225 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.225 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.225 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.226 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.226 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.226 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.226 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.226 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.226 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.226 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.226 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.227 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.227 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.227 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.227 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.227 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.227 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.227 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.227 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.228 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.228 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.228 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.228 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.228 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.228 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.229 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.229 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.229 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.229 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.229 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.229 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.229 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.229 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.230 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.230 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.230 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.230 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.230 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.230 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.230 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.231 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.231 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.231 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.231 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.231 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.231 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.231 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.231 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.232 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.232 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.232 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.232 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.232 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.232 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.233 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.233 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.233 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.233 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.233 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.233 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.233 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.233 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.234 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.234 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.234 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.234 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.234 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.234 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2023-07-21T22:38:47.235 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2023-07-21T22:38:47.235 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.235 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.235 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.235 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2023-07-21T22:38:47.235 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.235 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.235 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.236 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.236 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.236 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.236 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.236 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2023-07-21T22:38:47.236 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2023-07-21T22:38:47.236 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.237 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.237 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.237 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2023-07-21T22:38:47.237 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.237 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.237 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.237 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.237 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.238 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.238 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.238 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:47.238 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:47.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.238 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.239 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.239 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:47.239 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.239 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.239 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.239 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.239 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.239 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.240 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.240 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.240 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.240 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2023-07-21T22:38:47.240 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2023-07-21T22:38:47.240 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2023-07-21T22:38:47.241 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.241 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2023-07-21T22:38:47.241 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2023-07-21T22:38:47.241 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.241 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.241 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.241 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2023-07-21T22:38:47.241 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.242 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.242 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.242 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T22:38:47.242 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.242 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.242 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.243 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2023-07-21T22:38:47.243 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2023-07-21T22:38:47.243 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.243 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.243 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.243 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2023-07-21T22:38:47.243 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.243 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.244 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.244 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T22:38:47.244 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.244 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.244 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.244 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2023-07-21T22:38:47.244 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2023-07-21T22:38:47.245 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.245 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.245 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.245 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.245 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.245 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.245 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.246 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2023-07-21T22:38:47.246 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.246 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.246 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.246 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:47.246 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:47.246 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.247 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.247 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.247 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2023-07-21T22:38:47.247 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2023-07-21T22:38:47.247 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.247 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.247 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T22:38:47.247 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.248 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.248 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.248 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.248 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.248 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.248 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.248 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.249 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.249 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.249 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.250 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.250 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.250 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.251 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.251 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.251 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.251 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.252 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.252 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.252 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.253 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.253 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.253 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.253 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.253 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.253 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.254 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.254 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.254 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.254 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.254 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.254 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.254 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.255 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.255 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.255 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.255 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.255 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.255 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.256 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.256 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.256 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.256 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.256 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.256 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.256 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.257 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.257 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.257 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.257 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.257 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.257 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.257 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.257 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.258 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.258 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.258 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.258 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.258 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.258 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.258 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.259 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.259 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.259 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.259 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.259 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.259 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.259 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2023-07-21T22:38:47.260 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2023-07-21T22:38:47.260 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.260 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.260 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.260 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.260 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.260 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.261 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.261 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2023-07-21T22:38:47.261 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.261 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.261 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.261 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2023-07-21T22:38:47.261 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2023-07-21T22:38:47.262 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.262 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.262 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.262 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2023-07-21T22:38:47.262 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.262 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.262 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.262 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2023-07-21T22:38:47.263 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.263 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.263 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.263 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2023-07-21T22:38:47.263 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2023-07-21T22:38:47.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.263 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.264 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.264 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2023-07-21T22:38:47.264 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.264 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.264 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.264 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:38:47.264 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.265 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.265 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.265 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2023-07-21T22:38:47.265 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2023-07-21T22:38:47.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.265 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.265 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.266 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2023-07-21T22:38:47.266 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2023-07-21T22:38:47.266 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2023-07-21T22:38:47.266 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.266 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T22:38:47.266 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.266 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.266 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.267 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2023-07-21T22:38:47.267 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2023-07-21T22:38:47.267 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.267 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.267 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.267 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.268 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.268 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.268 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.268 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2023-07-21T22:38:47.268 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.268 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.268 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.269 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2023-07-21T22:38:47.269 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2023-07-21T22:38:47.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.269 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.269 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.269 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2023-07-21T22:38:47.269 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.270 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.270 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.270 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:38:47.270 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.270 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.270 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.270 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.270 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.271 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.271 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2023-07-21T22:38:47.271 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.271 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.271 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.271 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.271 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.272 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.272 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.272 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.272 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.272 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.272 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.272 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.273 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.273 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.273 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.273 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.273 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.273 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.273 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.273 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.274 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.274 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.274 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.274 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.274 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.275 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.275 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.275 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.275 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.275 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.275 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.275 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.276 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.276 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.276 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.276 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.276 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.276 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.277 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.277 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.277 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.277 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.277 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.277 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.277 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.278 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.278 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.278 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.278 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.278 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.278 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.278 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.278 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.279 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.279 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.279 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.279 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.279 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.279 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.280 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.280 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.280 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.280 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.281 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.281 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.281 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.281 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.281 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.281 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.282 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.282 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2023-07-21T22:38:47.282 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.282 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.282 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.282 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.282 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.283 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.283 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.283 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.283 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.283 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.283 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.283 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.283 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.284 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.284 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.284 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.284 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.284 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.284 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.284 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.285 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.285 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.285 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.285 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.285 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.285 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.286 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.286 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.286 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.286 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.286 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.286 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.286 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.286 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.287 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.287 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.287 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.287 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.287 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.287 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.288 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.288 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.288 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.288 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.288 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.288 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.288 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.289 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.289 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.289 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.289 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.289 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.289 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.289 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.290 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.290 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.290 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.290 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.290 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.290 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.290 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.290 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.291 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.291 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.291 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.291 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.291 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.292 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.292 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2023-07-21T22:38:47.292 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.292 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.292 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.292 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2023-07-21T22:38:47.292 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2023-07-21T22:38:47.293 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.293 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.293 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.293 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2023-07-21T22:38:47.293 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.293 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.293 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.293 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T22:38:47.294 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.294 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.294 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.294 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2023-07-21T22:38:47.294 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2023-07-21T22:38:47.294 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.294 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.295 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:47.295 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2023-07-21T22:38:47.295 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.295 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.295 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T22:38:47.295 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.295 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.296 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.296 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.296 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.296 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.296 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.296 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.296 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.297 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.297 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.297 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.297 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.297 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.297 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.297 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.298 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.298 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.298 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.298 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.298 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.298 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.298 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.299 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.299 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.299 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.299 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.299 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.299 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.299 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.300 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.300 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.300 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.300 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.300 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.300 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.300 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.301 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.301 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.301 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.301 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.301 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.301 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.301 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.301 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.302 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.302 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.302 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.302 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.302 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.302 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.303 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.303 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.303 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.303 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.303 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.303 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.303 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.304 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.304 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.304 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.304 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.304 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.304 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.304 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.305 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.305 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.305 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.305 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.305 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.305 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2023-07-21T22:38:47.306 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.306 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.306 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.306 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2023-07-21T22:38:47.306 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2023-07-21T22:38:47.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.306 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.307 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2023-07-21T22:38:47.307 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.307 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.307 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.307 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2023-07-21T22:38:47.307 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T22:38:47.307 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.307 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.308 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.308 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.308 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.308 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.308 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.308 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.308 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.309 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.309 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.309 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.309 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.309 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.309 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.309 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.310 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.310 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.310 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.310 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.310 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.310 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.310 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.311 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.311 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.311 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.311 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.311 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.311 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.311 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.312 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.312 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.312 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.312 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.312 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.312 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.312 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.313 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.313 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.313 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.313 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.313 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.313 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.313 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.314 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.314 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.314 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.314 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.314 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.314 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.314 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.314 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.315 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.315 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.315 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.315 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.315 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.316 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.316 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.316 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.316 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.316 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.316 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.316 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.316 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.317 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2023-07-21T22:38:47.317 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2023-07-21T22:38:47.317 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.317 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.317 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.317 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.317 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.318 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.318 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.318 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2023-07-21T22:38:47.318 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T22:38:47.318 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.318 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.318 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2023-07-21T22:38:47.319 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2023-07-21T22:38:47.319 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.319 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.319 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.319 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2023-07-21T22:38:47.319 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.319 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.319 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.320 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2023-07-21T22:38:47.320 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.320 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.320 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.320 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2023-07-21T22:38:47.320 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2023-07-21T22:38:47.321 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.321 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.321 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.321 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2023-07-21T22:38:47.321 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.321 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.321 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.321 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T22:38:47.322 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.322 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.322 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.322 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2023-07-21T22:38:47.322 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2023-07-21T22:38:47.322 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.322 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.323 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.323 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.323 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.323 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.323 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.323 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2023-07-21T22:38:47.323 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.323 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.324 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.324 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2023-07-21T22:38:47.324 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2023-07-21T22:38:47.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.324 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.324 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.325 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2023-07-21T22:38:47.325 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.325 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.325 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.325 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T22:38:47.325 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T22:38:47.325 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.325 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.326 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.326 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.326 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.326 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2023-07-21T22:38:47.326 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.326 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.327 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.327 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.327 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.327 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.327 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.327 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.327 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.328 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.328 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.328 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.328 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.328 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.328 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.328 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.328 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.329 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.329 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.329 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.329 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.329 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.329 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.330 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.330 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.330 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.330 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.330 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.330 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.330 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.331 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.331 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.331 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.331 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.331 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.331 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.331 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.332 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.332 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.332 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.332 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.332 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.332 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.332 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.332 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.333 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.333 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.333 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.333 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.333 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.333 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.333 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.334 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.334 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.334 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.334 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.334 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.334 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.334 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.335 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.335 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.335 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.335 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.335 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.335 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.335 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.335 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.336 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.336 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.336 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.336 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2023-07-21T22:38:47.336 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.336 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.337 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.337 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.337 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.337 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.337 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.337 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.337 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.337 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.338 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.338 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.338 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.338 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.338 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.338 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.338 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.338 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.339 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.339 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.339 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.339 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.339 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.339 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.340 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.340 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.340 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.340 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.340 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.340 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.340 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.340 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.341 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.341 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.341 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.341 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.341 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.341 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.341 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.342 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.342 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.342 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.342 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.342 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.342 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.342 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.342 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.343 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.343 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.343 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.343 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.343 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.343 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.343 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.343 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.344 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.344 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.344 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.344 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.344 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.344 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.345 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.345 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.345 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.345 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.345 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.345 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.345 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.345 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.346 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.346 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.346 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2023-07-21T22:38:47.346 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.346 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.346 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.347 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.347 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.347 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.347 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.347 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.347 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.347 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.347 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.348 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.348 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.348 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.348 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.348 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.348 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.348 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.349 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.349 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.349 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.349 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.349 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.349 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.349 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.350 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.350 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.350 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.350 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.350 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.350 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.350 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.350 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.351 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.351 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.351 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.351 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.351 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.351 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.351 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.352 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.352 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.352 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.352 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.352 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.352 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.352 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.353 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.353 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.353 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.353 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.353 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.353 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.353 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.353 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.354 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.354 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.354 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.354 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.354 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.354 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.354 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.355 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.355 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.355 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.355 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.355 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.355 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.355 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.356 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.356 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.356 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2023-07-21T22:38:47.356 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.356 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.356 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.357 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2023-07-21T22:38:47.357 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2023-07-21T22:38:47.357 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.357 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.357 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.357 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.357 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.357 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.358 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.358 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T22:38:47.358 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.358 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.358 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.358 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.358 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.359 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.359 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.359 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.359 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.359 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.359 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.359 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.359 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.360 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.360 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.360 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.360 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.360 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.360 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.360 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.361 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.361 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.361 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.361 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.361 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.361 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.361 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.362 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.362 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.362 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.362 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.362 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.362 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.362 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.362 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.363 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.363 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.363 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.363 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.363 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.363 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.363 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.364 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.364 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.364 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.364 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.364 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.364 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.364 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.364 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.365 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.365 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.365 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.365 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.365 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.365 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.365 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.366 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.366 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.366 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.366 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.366 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.366 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.366 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.366 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.367 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.367 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.367 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.367 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.367 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2023-07-21T22:38:47.367 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2023-07-21T22:38:47.368 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.368 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.368 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.368 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.368 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.368 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.368 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.368 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2023-07-21T22:38:47.369 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2023-07-21T22:38:47.369 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2023-07-21T22:38:47.369 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.369 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2023-07-21T22:38:47.369 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.369 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.369 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.369 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.370 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.370 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.370 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2023-07-21T22:38:47.370 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.370 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.370 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.370 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.371 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.371 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.371 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.371 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.371 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.371 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.371 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.372 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.372 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.372 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.372 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.372 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.372 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.372 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.372 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.373 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.373 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.373 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.373 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.373 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.373 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.373 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.374 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.374 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.374 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.374 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.374 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.374 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.374 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.374 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.375 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.375 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.375 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.375 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.375 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.375 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.376 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.376 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.376 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.376 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.376 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.376 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.376 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.376 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.377 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.377 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.377 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.377 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.377 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.377 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.377 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.378 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.378 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.378 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.378 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.378 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.378 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.379 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.379 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.379 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.379 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.379 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.379 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.379 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.379 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.380 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.380 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2023-07-21T22:38:47.380 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.380 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.380 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.380 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.380 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.381 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.381 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.381 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.381 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.381 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.381 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.381 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.382 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.382 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.382 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.382 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.382 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.382 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.382 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.382 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.383 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.383 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.383 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.383 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.383 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.383 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.383 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.384 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.384 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.384 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.384 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.384 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.384 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.384 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.384 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.385 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.385 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.385 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.385 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.385 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.385 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.385 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.386 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.386 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.386 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.386 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.386 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.386 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.386 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.386 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.387 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.387 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.387 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.387 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.387 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.387 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.388 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.388 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.388 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.388 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.388 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.388 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.388 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.388 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.389 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.389 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.389 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.389 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.389 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.389 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.389 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.390 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.390 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2023-07-21T22:38:47.390 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.390 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.390 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.390 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.390 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.390 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.391 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.391 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.391 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.391 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.391 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.391 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.391 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.391 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.392 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.392 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.392 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.392 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.392 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.392 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.392 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.393 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.393 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.393 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.393 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.393 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.393 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.393 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.394 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.394 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.394 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.394 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.394 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.394 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.394 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.394 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.395 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.395 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.395 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.395 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.395 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.395 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.395 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.396 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.396 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.396 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.396 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.396 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.396 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.396 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.396 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.397 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.397 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.397 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.397 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.397 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.397 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.397 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.398 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.398 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.398 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.398 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.398 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.398 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.398 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.399 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.399 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.399 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.399 INFO:teuthology.orchestra.run.smithi107.stdout: "noautoscale": { 2023-07-21T22:38:47.399 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "noautoscale", 2023-07-21T22:38:47.399 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.399 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.400 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.400 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.400 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.400 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.400 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.400 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "global autoscale flag", 2023-07-21T22:38:47.400 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T22:38:47.401 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.401 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.401 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:47.401 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:47.401 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.401 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.401 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.402 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:47.402 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.402 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.402 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.402 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.402 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.402 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.403 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.403 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2023-07-21T22:38:47.403 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2023-07-21T22:38:47.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:47.403 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.403 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.403 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2023-07-21T22:38:47.404 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2023-07-21T22:38:47.404 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.404 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.404 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2023-07-21T22:38:47.404 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T22:38:47.404 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.404 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.404 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.405 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.405 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.405 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2023-07-21T22:38:47.405 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.405 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.405 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.405 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2023-07-21T22:38:47.406 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2023-07-21T22:38:47.406 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.406 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.406 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.406 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.406 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.406 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.406 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.407 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T22:38:47.407 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.407 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.407 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.407 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2023-07-21T22:38:47.407 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2023-07-21T22:38:47.408 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.408 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.408 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.408 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.408 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.408 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.408 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.408 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.409 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.409 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.409 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.409 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.409 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.409 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.409 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.410 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.410 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.410 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.410 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.410 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.410 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.410 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.410 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.411 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.411 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.411 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.411 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.411 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.411 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.411 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.412 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.412 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.412 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.412 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.412 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.412 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.412 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.413 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.413 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.413 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.413 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.413 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.413 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.413 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.414 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.414 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.414 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.414 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.414 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.414 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.414 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.414 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.415 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.415 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.415 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.415 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.415 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.415 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.415 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.415 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.416 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.416 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.416 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.416 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.416 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.416 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.416 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.417 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.417 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.417 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.417 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.417 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.417 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.417 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.417 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.418 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.418 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.418 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2023-07-21T22:38:47.418 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2023-07-21T22:38:47.418 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.418 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.418 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.419 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2023-07-21T22:38:47.419 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.419 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.419 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.419 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2023-07-21T22:38:47.419 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.419 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.420 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.420 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:47.420 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:47.420 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.420 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.420 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.420 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2023-07-21T22:38:47.421 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.421 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.421 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.421 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2023-07-21T22:38:47.421 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.421 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.421 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.421 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.422 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.422 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.422 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2023-07-21T22:38:47.422 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.422 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.422 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.422 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2023-07-21T22:38:47.423 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2023-07-21T22:38:47.423 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.423 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.423 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.423 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.423 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.423 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.423 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.424 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.424 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.424 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.424 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.424 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2023-07-21T22:38:47.424 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2023-07-21T22:38:47.424 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.425 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.425 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.425 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.425 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.425 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.425 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.425 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T22:38:47.426 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T22:38:47.426 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.426 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.426 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.426 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.426 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.427 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.427 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.427 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.427 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.427 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.427 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.427 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.427 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.428 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.428 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.428 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.428 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.428 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.428 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.428 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.428 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.429 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.429 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.429 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.429 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.429 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.429 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.430 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.430 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.430 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.430 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.430 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.430 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.430 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.430 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.431 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.431 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.431 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.431 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.431 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.431 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.431 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.432 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.432 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.432 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.432 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.432 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.432 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.432 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.432 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.433 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.434 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2023-07-21T22:38:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2023-07-21T22:38:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.435 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.436 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T22:38:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T22:38:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.437 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2023-07-21T22:38:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.438 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2023-07-21T22:38:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2023-07-21T22:38:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2023-07-21T22:38:47.439 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.440 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2023-07-21T22:38:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2023-07-21T22:38:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2023-07-21T22:38:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.441 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T22:38:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2023-07-21T22:38:47.442 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2023-07-21T22:38:47.443 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.443 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.443 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.443 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2023-07-21T22:38:47.443 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.443 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.443 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.443 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T22:38:47.444 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.444 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.444 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.444 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2023-07-21T22:38:47.444 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2023-07-21T22:38:47.444 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.444 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.445 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.445 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2023-07-21T22:38:47.445 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.445 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.445 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.445 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.445 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.445 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.446 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.446 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2023-07-21T22:38:47.446 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2023-07-21T22:38:47.446 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.446 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.446 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.447 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2023-07-21T22:38:47.447 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.447 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.447 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.447 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2023-07-21T22:38:47.447 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2023-07-21T22:38:47.447 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.447 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.448 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.448 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.448 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.448 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2023-07-21T22:38:47.448 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2023-07-21T22:38:47.448 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.448 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.449 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.449 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2023-07-21T22:38:47.449 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2023-07-21T22:38:47.449 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2023-07-21T22:38:47.449 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.449 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.449 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.449 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.450 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.450 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.450 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.450 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.450 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2023-07-21T22:38:47.451 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.451 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.451 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.451 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.451 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.451 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.451 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.451 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.452 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.452 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.452 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.452 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.452 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.452 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.452 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.452 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.453 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.453 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.453 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.453 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.453 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.453 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.453 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.454 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.454 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.454 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.454 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.454 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.454 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.454 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.455 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.455 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.455 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.455 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.455 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.455 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.455 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.456 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.456 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.456 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.456 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.456 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.456 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.456 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.457 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.457 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.457 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.457 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.457 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.457 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.457 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.457 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.458 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.458 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.458 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.458 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.458 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.458 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.459 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.459 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.459 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.459 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.459 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.459 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.459 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.459 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.460 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.460 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.460 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.460 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2023-07-21T22:38:47.460 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2023-07-21T22:38:47.460 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.461 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.461 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.461 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:47.461 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.461 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.461 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.461 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.461 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.462 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.462 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.462 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2023-07-21T22:38:47.462 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T22:38:47.462 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.462 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.463 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.463 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.463 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.463 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.463 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.463 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.463 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.463 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.464 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.464 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2023-07-21T22:38:47.464 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2023-07-21T22:38:47.464 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.464 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.464 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.465 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.465 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.465 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.465 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.465 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.465 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.465 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.465 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.466 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.466 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.466 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.466 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2023-07-21T22:38:47.466 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.466 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.467 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.467 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2023-07-21T22:38:47.467 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2023-07-21T22:38:47.467 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.467 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.467 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.467 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.468 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.468 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.468 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.468 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.468 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.468 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.468 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.469 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2023-07-21T22:38:47.469 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2023-07-21T22:38:47.469 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.469 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.469 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.469 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.470 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.470 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.470 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.470 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.470 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.470 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.470 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.471 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.471 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.471 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.471 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.471 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.471 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.471 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.472 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.472 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.472 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.472 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.472 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.472 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.472 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.473 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.473 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.473 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.473 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.473 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.473 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.474 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.474 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.474 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.474 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.474 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.474 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.474 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.475 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.475 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.475 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.475 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.475 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.475 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.475 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.476 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.476 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.476 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.476 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.476 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.477 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.477 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.477 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.477 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.477 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.477 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.477 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.478 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.478 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.478 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.478 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.478 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.478 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.478 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.479 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.479 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.479 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.479 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.479 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.479 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.479 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.479 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.480 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.480 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.480 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.480 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.480 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.480 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2023-07-21T22:38:47.481 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2023-07-21T22:38:47.481 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.481 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.481 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.481 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.481 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.481 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.481 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.482 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.482 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.482 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.482 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.482 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2023-07-21T22:38:47.482 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2023-07-21T22:38:47.482 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.483 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.483 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.483 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.483 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.483 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.483 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.483 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.483 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.484 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.484 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.484 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.484 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.484 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.484 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2023-07-21T22:38:47.485 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.485 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.485 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.485 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.485 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.485 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.485 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.485 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.486 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.486 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.486 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.486 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.486 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.486 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.486 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.486 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.487 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.487 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.487 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.487 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.487 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.487 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.487 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.488 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.488 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.488 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.488 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.488 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.488 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.488 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.489 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.489 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.489 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.489 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.489 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.489 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.489 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.490 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.490 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.490 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.490 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.490 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.490 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.490 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.490 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.491 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.491 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.491 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.491 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.491 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.491 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.491 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.491 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.492 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.492 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.492 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.492 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.492 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.492 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.493 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.493 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.493 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.493 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.493 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.493 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.493 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.493 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.494 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.494 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.494 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.494 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.494 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.494 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2023-07-21T22:38:47.494 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.495 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.495 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.495 INFO:teuthology.orchestra.run.smithi107.stdout: "drive_group_interval": { 2023-07-21T22:38:47.495 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "drive_group_interval", 2023-07-21T22:38:47.495 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:47.495 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.495 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.495 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300.0", 2023-07-21T22:38:47.496 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.496 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.496 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.496 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T22:38:47.496 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.496 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.496 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.497 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.497 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.497 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.497 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.497 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.497 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.497 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.498 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.498 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.498 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.498 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.498 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.498 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.498 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.498 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.499 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.499 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.499 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.499 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.499 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.499 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.500 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.500 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.500 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.500 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.500 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.500 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.500 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.501 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.501 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.501 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.501 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.501 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.501 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.501 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.502 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.502 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.502 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.502 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.502 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.502 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.503 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.503 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.503 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.503 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.503 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.503 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.503 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.504 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.504 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.504 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.504 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.504 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.504 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.505 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.505 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.505 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.505 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.505 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.505 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.505 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.506 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.506 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.506 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.506 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.506 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.506 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2023-07-21T22:38:47.507 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2023-07-21T22:38:47.507 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.507 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.507 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.507 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2023-07-21T22:38:47.507 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.507 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.508 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.508 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T22:38:47.508 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.508 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.508 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.508 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.508 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.509 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2023-07-21T22:38:47.509 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.509 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.509 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.509 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.509 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.510 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.510 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.510 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.510 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.510 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.510 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.510 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.511 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.511 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.511 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.511 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.511 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.511 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.511 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.512 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.512 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.512 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.512 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.512 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.512 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.513 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.513 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.513 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.513 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.513 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.513 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.513 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.514 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.514 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.514 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.514 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.514 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.514 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.515 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.515 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.515 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.515 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.515 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.515 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.515 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.516 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.516 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.516 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.516 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.516 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.516 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.516 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.517 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.517 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.517 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.517 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.517 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.517 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.518 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.518 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.518 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.518 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.518 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.518 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.518 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.518 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.519 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.519 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.519 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.519 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2023-07-21T22:38:47.519 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2023-07-21T22:38:47.519 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.520 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.520 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.520 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.520 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.520 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.520 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.520 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.521 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.521 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.521 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.521 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2023-07-21T22:38:47.521 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2023-07-21T22:38:47.521 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.522 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.522 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.522 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2023-07-21T22:38:47.522 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.522 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.522 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.522 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.523 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.523 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.523 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.523 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2023-07-21T22:38:47.523 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2023-07-21T22:38:47.523 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.524 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.524 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.524 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.524 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.524 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.524 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.524 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.525 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.525 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.525 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.525 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2023-07-21T22:38:47.525 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2023-07-21T22:38:47.525 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.526 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.526 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.526 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.526 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.526 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.526 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.526 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.526 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.527 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.527 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.527 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2023-07-21T22:38:47.527 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2023-07-21T22:38:47.527 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:47.527 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.528 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.528 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.528 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.528 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.528 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.528 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.528 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.529 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.529 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.529 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2023-07-21T22:38:47.529 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2023-07-21T22:38:47.529 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.529 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.530 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.530 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.530 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.530 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.530 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.530 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.530 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.530 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.531 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.531 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2023-07-21T22:38:47.531 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2023-07-21T22:38:47.531 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.531 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.531 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.531 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.532 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.532 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.532 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.532 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.532 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.532 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.532 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.533 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2023-07-21T22:38:47.533 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2023-07-21T22:38:47.533 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.533 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.533 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.533 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.533 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.534 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.534 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.534 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.534 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.534 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.534 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.534 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2023-07-21T22:38:47.535 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2023-07-21T22:38:47.535 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.535 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.535 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.535 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.535 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2023-07-21T22:38:47.535 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2023-07-21T22:38:47.535 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.536 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.536 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.536 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.536 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.536 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2023-07-21T22:38:47.536 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2023-07-21T22:38:47.536 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.537 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.537 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.537 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.537 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.537 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.537 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.537 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.537 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.538 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.538 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.538 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2023-07-21T22:38:47.538 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2023-07-21T22:38:47.538 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.538 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.538 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.539 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.539 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.539 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.539 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.539 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.539 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.539 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.540 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.540 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2023-07-21T22:38:47.540 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2023-07-21T22:38:47.540 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.540 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.540 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.540 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.541 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.541 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.541 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.541 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.541 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.541 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.541 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.541 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.542 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.542 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.542 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2023-07-21T22:38:47.542 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.542 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.542 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.542 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2023-07-21T22:38:47.543 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2023-07-21T22:38:47.543 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.543 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.543 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.543 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.543 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.543 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.543 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.544 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T22:38:47.544 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.544 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.544 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.544 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2023-07-21T22:38:47.544 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2023-07-21T22:38:47.545 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.545 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.545 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.545 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.545 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.545 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.545 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.545 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2023-07-21T22:38:47.546 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.546 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.546 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.546 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.546 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.546 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.547 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.547 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.547 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.547 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.547 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.547 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.547 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.547 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.548 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.548 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.548 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.548 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.548 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.548 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.548 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.549 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.549 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.549 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.549 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.549 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.549 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.549 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.550 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.550 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.550 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.550 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.550 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.550 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.550 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.551 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.551 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.551 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.551 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.551 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.551 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.551 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.551 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.552 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.552 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.552 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.552 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.552 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.552 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.552 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.553 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.553 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.553 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.553 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.553 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.553 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.553 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.554 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.554 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.554 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.554 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.554 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.554 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.555 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.555 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.555 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.555 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.555 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.555 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.555 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.556 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.556 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2023-07-21T22:38:47.556 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.556 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.556 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.556 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.557 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.557 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.557 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.557 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.557 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.557 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.557 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.558 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.558 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.558 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.558 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.558 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.558 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.558 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.558 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.559 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.559 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.559 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.559 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.559 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.559 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.560 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.560 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.560 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.560 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.560 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.560 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.560 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.561 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.561 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.561 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.561 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.561 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.561 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.562 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.562 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.562 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.562 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.562 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.562 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.562 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.562 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.563 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.563 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.563 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.563 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.563 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.563 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.563 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.564 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.564 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.564 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.564 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.564 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.564 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.565 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.565 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.565 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.565 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.565 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.565 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.565 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.566 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.566 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.566 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.566 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.566 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.566 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2023-07-21T22:38:47.567 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.567 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.567 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.567 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.567 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.567 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.567 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.568 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.568 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.568 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.568 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.568 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.568 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.568 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.568 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.569 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.569 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.569 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.569 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.569 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.569 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.569 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.569 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.570 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.570 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.570 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.570 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.570 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.570 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.570 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.571 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.571 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.571 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.571 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.571 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.571 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.571 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.572 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.572 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.572 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.572 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.572 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.572 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.572 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.573 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.573 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.573 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.573 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.573 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.573 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.573 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.573 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.574 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.574 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.574 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.574 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.574 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.574 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.574 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.575 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.575 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.575 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.575 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.575 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.575 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.575 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.575 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.576 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.576 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.576 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.576 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.576 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.576 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2023-07-21T22:38:47.576 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.577 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.577 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.577 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2023-07-21T22:38:47.577 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2023-07-21T22:38:47.577 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.577 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.577 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.578 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T22:38:47.578 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.578 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.578 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.578 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.578 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.578 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.578 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.579 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:47.579 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:47.579 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.579 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.579 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.579 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2023-07-21T22:38:47.579 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.580 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.580 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.580 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.580 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.580 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.580 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.581 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.581 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.581 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.581 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.581 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.581 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.581 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.582 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.582 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.582 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.582 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.582 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.582 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.582 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.582 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.583 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.583 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.583 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.583 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.583 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.583 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.583 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.584 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.584 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.584 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.584 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.584 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.584 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.584 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.585 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.585 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.585 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.585 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.585 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.585 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.585 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.586 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.586 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.586 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.586 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.586 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.586 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.586 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.586 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.587 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.587 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.587 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.587 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.587 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.587 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.587 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.587 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.588 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.588 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.588 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.588 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.588 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.588 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.589 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.589 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.589 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.589 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.589 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.589 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.589 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.589 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.590 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.590 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2023-07-21T22:38:47.590 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.590 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.590 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.590 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2023-07-21T22:38:47.591 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2023-07-21T22:38:47.591 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.591 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.591 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.591 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.591 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.591 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.591 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.592 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T22:38:47.592 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.592 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.592 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.592 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2023-07-21T22:38:47.592 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2023-07-21T22:38:47.593 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.593 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.593 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.593 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.593 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.593 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.593 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.593 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T22:38:47.594 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.594 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.594 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.594 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2023-07-21T22:38:47.594 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2023-07-21T22:38:47.594 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.594 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.595 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.595 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.595 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.595 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.595 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.595 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T22:38:47.595 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.595 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.596 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.596 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2023-07-21T22:38:47.596 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2023-07-21T22:38:47.596 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.596 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.596 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.596 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.597 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.597 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.597 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.597 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T22:38:47.597 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.597 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.597 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.598 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2023-07-21T22:38:47.598 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2023-07-21T22:38:47.598 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.598 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.598 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.598 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.598 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.599 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.599 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.599 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T22:38:47.599 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.599 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.599 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.599 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2023-07-21T22:38:47.600 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2023-07-21T22:38:47.600 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.600 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.600 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.600 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.600 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.600 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.601 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.601 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.601 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.601 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.601 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.601 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2023-07-21T22:38:47.601 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2023-07-21T22:38:47.602 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.602 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.602 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.602 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.602 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.602 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.602 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.602 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.603 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.603 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.603 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.603 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2023-07-21T22:38:47.603 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2023-07-21T22:38:47.603 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.603 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.604 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.604 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T22:38:47.604 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.604 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.604 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.604 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.604 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.604 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.605 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.605 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2023-07-21T22:38:47.605 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2023-07-21T22:38:47.605 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.605 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.605 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.606 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.606 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.606 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.606 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.606 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.606 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.606 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.606 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.607 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:47.607 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:47.607 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.607 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.607 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.607 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2023-07-21T22:38:47.607 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2023-07-21T22:38:47.608 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.608 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.608 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.608 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.608 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.608 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.608 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2023-07-21T22:38:47.609 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2023-07-21T22:38:47.609 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.609 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.609 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.609 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2023-07-21T22:38:47.609 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.609 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.610 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.610 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.610 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.610 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.610 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.610 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2023-07-21T22:38:47.610 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2023-07-21T22:38:47.611 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.611 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.611 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.611 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.611 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.611 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.611 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.611 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.612 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.612 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.612 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.612 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2023-07-21T22:38:47.612 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2023-07-21T22:38:47.612 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.612 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.613 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.613 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.613 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.613 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.613 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.613 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.613 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.613 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.614 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.614 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.614 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.614 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.614 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.614 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.614 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.615 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.615 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.615 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.615 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.615 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.615 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.615 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.616 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.616 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.616 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.616 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.616 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.616 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.616 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.617 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.617 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.617 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.617 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.617 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.617 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.617 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.617 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.618 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.618 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.618 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.618 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.618 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.618 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.618 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.619 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.619 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.619 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.619 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.619 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.619 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.619 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.619 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.620 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.620 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.620 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.620 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.620 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.620 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.620 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.620 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.621 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.621 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.621 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.621 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.621 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.621 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.621 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.622 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.622 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.622 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.622 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.622 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.622 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.622 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.622 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.623 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2023-07-21T22:38:47.623 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2023-07-21T22:38:47.623 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.623 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.623 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.623 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.623 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.624 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.624 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.624 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.624 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.624 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.624 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.625 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2023-07-21T22:38:47.625 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2023-07-21T22:38:47.625 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.625 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.625 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.625 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.625 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.625 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.626 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.626 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.626 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.626 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.626 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.626 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2023-07-21T22:38:47.626 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2023-07-21T22:38:47.627 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.627 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.627 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.627 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T22:38:47.627 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.627 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.627 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.627 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.628 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.628 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.628 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.628 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.628 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.628 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.628 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2023-07-21T22:38:47.629 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.629 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.629 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.629 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.629 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.629 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.629 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.629 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.630 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.630 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.630 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.630 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.630 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.630 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.630 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.631 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.631 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.631 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.631 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.631 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.631 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.631 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.631 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.632 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.632 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.632 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.632 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.632 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.632 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.632 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.633 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.633 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.633 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.633 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.633 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.633 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.634 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.634 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.634 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.634 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.634 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.634 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.634 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.634 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.635 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.635 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.635 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.635 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.635 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.635 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.635 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.635 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.636 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.636 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.636 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.636 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.636 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.636 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.636 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.637 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.637 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.637 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.637 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.637 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.637 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.637 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.637 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.638 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.638 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.638 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.638 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.638 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.638 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2023-07-21T22:38:47.638 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.639 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.639 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.639 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.639 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.639 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.639 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.639 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.639 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.640 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.640 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.640 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.640 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.640 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.640 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.640 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.641 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.641 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.641 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.641 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.641 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.641 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.641 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.642 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.642 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.642 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.642 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.642 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.642 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.642 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.642 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.643 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.643 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.643 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.643 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.643 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.643 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.643 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.644 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.644 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.644 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.644 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.644 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.644 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.644 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.644 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.645 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.645 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.645 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.645 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.645 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.645 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.645 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.645 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.646 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.646 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.646 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.646 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.646 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.646 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.646 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.647 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.647 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.647 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.647 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.647 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.647 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.647 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.647 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.648 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2023-07-21T22:38:47.648 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2023-07-21T22:38:47.648 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.648 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.648 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.648 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2023-07-21T22:38:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T22:38:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2023-07-21T22:38:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2023-07-21T22:38:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.651 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.651 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T22:38:47.651 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.651 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.651 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.651 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.651 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.652 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.652 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2023-07-21T22:38:47.652 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.652 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.652 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.652 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2023-07-21T22:38:47.652 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2023-07-21T22:38:47.653 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:47.653 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.653 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.653 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2023-07-21T22:38:47.653 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.653 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.653 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.653 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.654 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.654 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.654 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.654 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2023-07-21T22:38:47.654 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2023-07-21T22:38:47.654 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.654 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.655 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.655 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.655 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.655 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.655 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.655 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.655 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.656 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.656 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.656 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:47.656 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:47.656 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.656 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.656 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.657 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:47.657 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.657 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.657 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.657 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.657 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.657 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.657 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.658 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.658 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.658 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.658 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.658 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.658 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.659 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.659 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.659 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.659 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.659 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.659 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.659 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.659 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.660 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.660 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.660 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.660 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.660 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.660 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.660 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.661 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.661 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.661 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.661 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.661 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.661 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.661 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.661 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.662 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.662 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.662 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.662 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.662 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.662 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.662 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.663 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.663 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.663 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.663 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.663 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.663 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.663 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.663 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.664 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.664 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.664 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.664 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.664 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.664 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.664 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.665 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.665 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.665 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.665 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.665 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.665 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.665 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.666 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.666 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.666 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.666 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.666 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.666 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.666 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.666 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.667 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2023-07-21T22:38:47.667 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2023-07-21T22:38:47.667 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.667 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.667 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.667 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.667 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.668 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.668 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.668 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.668 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.668 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.668 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.668 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2023-07-21T22:38:47.669 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2023-07-21T22:38:47.669 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.669 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.669 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.669 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2023-07-21T22:38:47.669 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.669 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.670 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.670 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.670 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.670 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.670 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.670 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2023-07-21T22:38:47.670 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2023-07-21T22:38:47.671 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.671 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.671 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.671 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T22:38:47.671 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.671 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.671 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.671 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.672 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.672 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.672 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.672 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.672 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.672 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.672 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2023-07-21T22:38:47.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.673 INFO:teuthology.orchestra.run.smithi107.stdout: "gid": 24116, 2023-07-21T22:38:47.673 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "y", 2023-07-21T22:38:47.673 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_features": 4540138322906710015, 2023-07-21T22:38:47.673 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2023-07-21T22:38:47.673 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.673 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2023-07-21T22:38:47.674 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.674 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.674 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.674 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:47.674 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:47.674 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.674 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.674 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.675 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:47.675 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.675 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.675 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.675 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T22:38:47.675 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.675 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.675 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.676 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.676 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.676 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.676 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.676 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.676 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.676 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.677 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.677 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.677 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.677 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.677 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.677 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.677 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.678 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.678 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.678 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.678 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.678 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.678 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.678 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.679 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.679 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.679 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.679 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.679 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.679 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.679 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.679 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.680 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.680 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.680 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.680 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.680 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.680 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.681 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.681 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.681 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.681 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.681 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.681 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.681 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.681 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.682 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.682 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.682 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.682 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.682 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.682 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.682 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.682 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.683 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.683 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.683 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.683 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.683 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.683 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.684 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.684 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.684 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.684 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.684 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.684 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.684 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.684 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.685 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.685 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2023-07-21T22:38:47.685 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2023-07-21T22:38:47.685 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.685 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.685 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.685 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.686 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.686 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.686 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.686 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2023-07-21T22:38:47.686 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.686 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.686 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.687 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2023-07-21T22:38:47.687 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2023-07-21T22:38:47.687 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.687 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.687 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.687 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2023-07-21T22:38:47.687 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.687 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.688 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.688 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2023-07-21T22:38:47.688 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.688 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.688 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.688 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2023-07-21T22:38:47.689 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2023-07-21T22:38:47.689 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.689 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.689 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.689 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.689 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.689 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.689 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.690 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2023-07-21T22:38:47.690 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.690 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.690 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.690 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2023-07-21T22:38:47.690 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2023-07-21T22:38:47.690 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2023-07-21T22:38:47.691 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.692 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.692 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.692 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2023-07-21T22:38:47.692 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2023-07-21T22:38:47.692 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.692 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.693 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.693 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2023-07-21T22:38:47.693 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.693 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.693 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.693 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2023-07-21T22:38:47.693 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.693 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.694 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.694 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2023-07-21T22:38:47.694 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2023-07-21T22:38:47.694 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.694 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.694 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.694 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.695 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.695 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.695 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.695 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2023-07-21T22:38:47.695 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.695 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.695 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.696 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2023-07-21T22:38:47.696 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2023-07-21T22:38:47.696 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.696 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.696 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.696 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.696 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.696 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.697 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.697 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T22:38:47.697 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.697 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.697 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.697 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2023-07-21T22:38:47.698 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2023-07-21T22:38:47.698 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.698 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.698 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.698 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.698 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.698 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.698 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.699 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2023-07-21T22:38:47.699 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.699 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.699 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.699 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.699 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.700 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2023-07-21T22:38:47.700 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.700 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.700 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.700 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2023-07-21T22:38:47.700 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2023-07-21T22:38:47.700 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.700 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.701 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.701 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.701 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.701 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.701 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.701 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T22:38:47.701 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.701 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.702 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.702 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2023-07-21T22:38:47.702 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2023-07-21T22:38:47.702 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.702 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.702 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.703 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2023-07-21T22:38:47.703 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.703 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.703 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.703 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T22:38:47.703 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:38:47.703 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.703 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.704 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2023-07-21T22:38:47.704 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2023-07-21T22:38:47.704 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:47.704 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.704 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.704 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:47.705 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2023-07-21T22:38:47.705 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2023-07-21T22:38:47.705 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.705 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T22:38:47.705 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:38:47.705 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.705 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.706 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2023-07-21T22:38:47.706 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2023-07-21T22:38:47.706 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:47.706 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.706 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.706 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2023-07-21T22:38:47.706 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2023-07-21T22:38:47.706 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2023-07-21T22:38:47.707 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.707 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T22:38:47.707 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.707 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.707 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.707 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2023-07-21T22:38:47.707 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2023-07-21T22:38:47.708 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.708 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.708 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.708 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T22:38:47.708 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.708 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.708 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.709 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T22:38:47.709 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T22:38:47.709 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.709 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.709 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2023-07-21T22:38:47.709 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2023-07-21T22:38:47.709 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:47.710 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.710 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.710 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2023-07-21T22:38:47.710 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2023-07-21T22:38:47.710 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2023-07-21T22:38:47.710 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.710 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2023-07-21T22:38:47.710 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T22:38:47.711 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.711 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.711 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2023-07-21T22:38:47.711 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2023-07-21T22:38:47.711 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.711 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.712 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.712 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2023-07-21T22:38:47.712 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.712 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.712 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.712 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T22:38:47.712 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:38:47.712 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.713 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.713 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2023-07-21T22:38:47.713 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2023-07-21T22:38:47.713 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:47.713 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.713 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.713 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:47.714 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2023-07-21T22:38:47.714 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2023-07-21T22:38:47.714 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.714 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T22:38:47.714 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:38:47.714 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.714 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.715 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.715 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.715 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.715 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.715 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.715 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.715 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.715 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.716 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.716 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.716 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.716 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.716 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.716 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.716 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.716 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.717 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.717 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.717 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.717 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.717 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.717 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.718 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.718 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.718 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.718 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.718 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.718 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.718 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.718 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.719 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.719 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.719 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.719 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.719 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.719 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.719 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.720 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.720 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.720 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.720 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.720 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.720 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.720 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.720 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.721 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.721 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.721 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.721 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.721 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.721 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.721 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.722 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.722 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.722 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.722 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.722 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.722 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.722 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.723 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.723 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.723 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.723 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.723 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.723 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.723 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.724 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2023-07-21T22:38:47.724 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2023-07-21T22:38:47.724 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:47.724 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.724 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.724 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:47.724 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.725 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.725 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.725 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T22:38:47.725 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.725 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.725 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.725 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2023-07-21T22:38:47.726 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2023-07-21T22:38:47.726 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.726 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.726 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.726 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2023-07-21T22:38:47.726 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.726 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.726 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.727 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2023-07-21T22:38:47.727 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2023-07-21T22:38:47.727 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap" 2023-07-21T22:38:47.727 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.727 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2023-07-21T22:38:47.727 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.727 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.728 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.728 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2023-07-21T22:38:47.728 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2023-07-21T22:38:47.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.728 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.728 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.728 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.729 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.729 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.729 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.729 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T22:38:47.729 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.729 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.729 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.730 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:47.730 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:47.730 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.730 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.730 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.730 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:47.730 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.731 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.731 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.731 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T22:38:47.731 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.731 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.731 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.731 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2023-07-21T22:38:47.732 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2023-07-21T22:38:47.732 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.732 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.732 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.732 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2023-07-21T22:38:47.732 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2023-07-21T22:38:47.732 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.733 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.733 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T22:38:47.733 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T22:38:47.733 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.733 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.733 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2023-07-21T22:38:47.734 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2023-07-21T22:38:47.734 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:47.734 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.734 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.734 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:47.734 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.734 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.734 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.735 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T22:38:47.735 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.735 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.735 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.735 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.735 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.735 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.736 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2023-07-21T22:38:47.736 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.736 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.736 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.736 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2023-07-21T22:38:47.736 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2023-07-21T22:38:47.736 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:47.736 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.737 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.737 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2023-07-21T22:38:47.737 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.737 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.737 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.737 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T22:38:47.737 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.737 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.738 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.738 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2023-07-21T22:38:47.738 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2023-07-21T22:38:47.738 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.738 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.738 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.738 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2023-07-21T22:38:47.739 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.739 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.739 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.739 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T22:38:47.739 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.739 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.739 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.740 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2023-07-21T22:38:47.740 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2023-07-21T22:38:47.740 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.740 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.740 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.740 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2023-07-21T22:38:47.740 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.741 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.741 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.741 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T22:38:47.741 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.741 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.741 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.741 INFO:teuthology.orchestra.run.smithi107.stdout: "alertmanager_web_password": { 2023-07-21T22:38:47.742 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alertmanager_web_password", 2023-07-21T22:38:47.742 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.742 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.742 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.742 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:47.742 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.742 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.742 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.743 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Alertmanager web password", 2023-07-21T22:38:47.743 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.743 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.743 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.743 INFO:teuthology.orchestra.run.smithi107.stdout: "alertmanager_web_user": { 2023-07-21T22:38:47.743 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alertmanager_web_user", 2023-07-21T22:38:47.743 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.744 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.744 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.744 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:47.744 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.744 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.744 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.744 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Alertmanager web user", 2023-07-21T22:38:47.744 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.745 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.745 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.745 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2023-07-21T22:38:47.745 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2023-07-21T22:38:47.745 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.745 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.745 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.746 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.746 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.746 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.746 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.746 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T22:38:47.746 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T22:38:47.746 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.746 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.747 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2023-07-21T22:38:47.747 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2023-07-21T22:38:47.747 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.747 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.747 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:47.748 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.748 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.748 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.748 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T22:38:47.748 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.748 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.748 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.749 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2023-07-21T22:38:47.749 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T22:38:47.749 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:47.749 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.749 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.749 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2023-07-21T22:38:47.749 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.750 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.750 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.750 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T22:38:47.750 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.750 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.750 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.751 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2023-07-21T22:38:47.751 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2023-07-21T22:38:47.751 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.751 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.751 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.751 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.752 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.752 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.752 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.752 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T22:38:47.752 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.752 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.752 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.753 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2023-07-21T22:38:47.753 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2023-07-21T22:38:47.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.753 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.753 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.753 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.754 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.754 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.754 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T22:38:47.754 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.754 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.754 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.754 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2023-07-21T22:38:47.755 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2023-07-21T22:38:47.755 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.755 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.755 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.755 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.755 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.755 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.755 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.756 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T22:38:47.756 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.756 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.756 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.756 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2023-07-21T22:38:47.756 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2023-07-21T22:38:47.756 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.757 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.757 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.757 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T22:38:47.757 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.757 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.757 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.757 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2023-07-21T22:38:47.758 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.758 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.758 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.758 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2023-07-21T22:38:47.758 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2023-07-21T22:38:47.758 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.758 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.759 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.759 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T22:38:47.759 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.759 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.759 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.759 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2023-07-21T22:38:47.760 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.760 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.760 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.760 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2023-07-21T22:38:47.760 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2023-07-21T22:38:47.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.760 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.761 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.761 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T22:38:47.761 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.761 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.761 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.761 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2023-07-21T22:38:47.761 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.761 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.762 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.762 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2023-07-21T22:38:47.762 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2023-07-21T22:38:47.762 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.762 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.762 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.762 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T22:38:47.763 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.763 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.763 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.763 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2023-07-21T22:38:47.763 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.763 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.763 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.764 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2023-07-21T22:38:47.764 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2023-07-21T22:38:47.764 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.764 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.764 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.764 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T22:38:47.765 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.765 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.765 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.765 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2023-07-21T22:38:47.765 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.765 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.765 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.766 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2023-07-21T22:38:47.766 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2023-07-21T22:38:47.766 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.766 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.766 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.766 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T22:38:47.766 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.767 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.767 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.767 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2023-07-21T22:38:47.767 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.767 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.767 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.767 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2023-07-21T22:38:47.768 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2023-07-21T22:38:47.768 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.768 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.768 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.768 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T22:38:47.768 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.768 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.769 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.769 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2023-07-21T22:38:47.769 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.769 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.769 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2023-07-21T22:38:47.769 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2023-07-21T22:38:47.770 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.770 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.770 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.770 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T22:38:47.770 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.770 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.770 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.770 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2023-07-21T22:38:47.771 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.771 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.771 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.771 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2023-07-21T22:38:47.771 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2023-07-21T22:38:47.771 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.772 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.772 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.772 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T22:38:47.772 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.772 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.772 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.772 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2023-07-21T22:38:47.773 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.773 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.773 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.773 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2023-07-21T22:38:47.773 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2023-07-21T22:38:47.773 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.773 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.774 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.774 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T22:38:47.774 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.774 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.774 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.774 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2023-07-21T22:38:47.774 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.774 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.775 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.775 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2023-07-21T22:38:47.775 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2023-07-21T22:38:47.775 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.775 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.775 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.775 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T22:38:47.776 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.776 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.776 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.776 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2023-07-21T22:38:47.776 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.776 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.776 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.777 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2023-07-21T22:38:47.777 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2023-07-21T22:38:47.777 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.777 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.777 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.777 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T22:38:47.777 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.778 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.778 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.778 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2023-07-21T22:38:47.778 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.778 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.778 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.778 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2023-07-21T22:38:47.779 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2023-07-21T22:38:47.779 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.779 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.779 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.779 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T22:38:47.779 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.779 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.779 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.780 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2023-07-21T22:38:47.780 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.780 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.780 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.780 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2023-07-21T22:38:47.780 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2023-07-21T22:38:47.781 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.781 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.781 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.781 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T22:38:47.781 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.781 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.781 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.781 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2023-07-21T22:38:47.782 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.782 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.782 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.782 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2023-07-21T22:38:47.782 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2023-07-21T22:38:47.782 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.782 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.783 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.783 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.783 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.783 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.783 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.783 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T22:38:47.783 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.783 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.784 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.784 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2023-07-21T22:38:47.784 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2023-07-21T22:38:47.784 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.784 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.784 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T22:38:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2023-07-21T22:38:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T22:38:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2023-07-21T22:38:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T22:38:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2023-07-21T22:38:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2023-07-21T22:38:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2023-07-21T22:38:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T22:38:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2023-07-21T22:38:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2023-07-21T22:38:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2023-07-21T22:38:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2023-07-21T22:38:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2023-07-21T22:38:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2023-07-21T22:38:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T22:38:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2023-07-21T22:38:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2023-07-21T22:38:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2023-07-21T22:38:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2023-07-21T22:38:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2023-07-21T22:38:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2023-07-21T22:38:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.797 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.797 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.797 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.797 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.797 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.797 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.797 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.798 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.798 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.798 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.798 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.798 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.798 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.798 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.798 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.799 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.799 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.799 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.799 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.799 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.799 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2023-07-21T22:38:47.800 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2023-07-21T22:38:47.800 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.800 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.800 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.800 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.800 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.800 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.800 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.801 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T22:38:47.801 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.801 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.801 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.801 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.801 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.802 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.802 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.802 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.802 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.802 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.802 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.802 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.802 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T22:38:47.803 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.803 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.803 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.803 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.803 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.803 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.803 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.804 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.804 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.804 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.804 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.804 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.804 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.804 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.805 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.805 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.805 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.805 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.805 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.805 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.805 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.805 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.806 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.806 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.806 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.806 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.806 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.806 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.806 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.807 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.807 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.807 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.807 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.807 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.807 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.807 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.808 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T22:38:47.808 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T22:38:47.808 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.808 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.808 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.808 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.808 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.809 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.809 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.809 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T22:38:47.809 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.809 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.809 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.809 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T22:38:47.810 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T22:38:47.810 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.810 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.810 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.810 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2023-07-21T22:38:47.810 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.810 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.811 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.811 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T22:38:47.811 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.811 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.811 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.811 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2023-07-21T22:38:47.811 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2023-07-21T22:38:47.812 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.812 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.812 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.812 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:47.812 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.812 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.812 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.813 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2023-07-21T22:38:47.813 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.813 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.813 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.813 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2023-07-21T22:38:47.813 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2023-07-21T22:38:47.813 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.814 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.814 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.814 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:47.814 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.814 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.814 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.814 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T22:38:47.814 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.815 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.815 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.815 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2023-07-21T22:38:47.815 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2023-07-21T22:38:47.815 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.815 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.815 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.816 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.816 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.816 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.816 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.816 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2023-07-21T22:38:47.816 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.816 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.817 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.817 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2023-07-21T22:38:47.817 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2023-07-21T22:38:47.817 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.817 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.817 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.817 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2023-07-21T22:38:47.818 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.818 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.818 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.818 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2023-07-21T22:38:47.818 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2023-07-21T22:38:47.818 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.818 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T22:38:47.819 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.819 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.819 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.819 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2023-07-21T22:38:47.819 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2023-07-21T22:38:47.819 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.819 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.820 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.820 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T22:38:47.820 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.820 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.820 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.820 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T22:38:47.820 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.821 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.821 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_web_password": { 2023-07-21T22:38:47.821 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_web_password", 2023-07-21T22:38:47.821 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.821 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.821 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.822 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:47.822 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.822 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.822 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.822 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus web password", 2023-07-21T22:38:47.822 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.822 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.822 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.823 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_web_user": { 2023-07-21T22:38:47.823 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_web_user", 2023-07-21T22:38:47.823 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.823 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.823 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.823 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:47.823 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.824 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.824 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.824 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus web user", 2023-07-21T22:38:47.824 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.824 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.824 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.824 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2023-07-21T22:38:47.825 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2023-07-21T22:38:47.825 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.825 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.825 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.825 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.825 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.825 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.826 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T22:38:47.826 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.826 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.826 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.826 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2023-07-21T22:38:47.826 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2023-07-21T22:38:47.826 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.827 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.827 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.827 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.827 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.827 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.827 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.827 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T22:38:47.827 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.828 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.828 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.828 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2023-07-21T22:38:47.828 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2023-07-21T22:38:47.828 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.828 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.829 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.829 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.829 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.829 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.829 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.829 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T22:38:47.829 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.829 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.830 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.830 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2023-07-21T22:38:47.830 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2023-07-21T22:38:47.830 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.830 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.830 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.830 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.831 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.831 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.831 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.831 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T22:38:47.831 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.831 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.831 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.832 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2023-07-21T22:38:47.832 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2023-07-21T22:38:47.832 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.832 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.832 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.832 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.832 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.833 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.833 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.833 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T22:38:47.833 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.833 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.833 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.833 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2023-07-21T22:38:47.834 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2023-07-21T22:38:47.834 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.834 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.834 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.834 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2023-07-21T22:38:47.834 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.834 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.835 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.835 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2023-07-21T22:38:47.835 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.835 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.835 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.835 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2023-07-21T22:38:47.835 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2023-07-21T22:38:47.836 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.836 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.836 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.836 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.836 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.836 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.836 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.836 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T22:38:47.837 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.837 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.837 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.837 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2023-07-21T22:38:47.837 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2023-07-21T22:38:47.837 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.837 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.838 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.838 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.838 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.838 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.838 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.838 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T22:38:47.838 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.839 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.839 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.839 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2023-07-21T22:38:47.839 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2023-07-21T22:38:47.839 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.839 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.839 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.840 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.840 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.840 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.840 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.840 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T22:38:47.840 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.840 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.841 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.841 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2023-07-21T22:38:47.841 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2023-07-21T22:38:47.841 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.841 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.841 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.841 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.842 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.842 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.842 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.842 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T22:38:47.842 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.842 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.842 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.843 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2023-07-21T22:38:47.843 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2023-07-21T22:38:47.843 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.843 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.843 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.843 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.843 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.844 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.844 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.844 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T22:38:47.844 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.844 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.844 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.844 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2023-07-21T22:38:47.845 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2023-07-21T22:38:47.845 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.845 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.845 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.845 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.845 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.845 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.845 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.846 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T22:38:47.846 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.846 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.846 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.846 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.846 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.847 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2023-07-21T22:38:47.847 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.847 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.847 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.847 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.847 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.847 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.848 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.848 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.848 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.848 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.848 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.848 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.848 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.848 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.849 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.849 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.849 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.849 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.849 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.849 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.849 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.849 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.850 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.850 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.850 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.850 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.850 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.851 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.851 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.851 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.851 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.851 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.851 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.851 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.851 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.852 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.852 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.852 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.852 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.852 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.852 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.852 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.853 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.853 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.853 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.853 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.853 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.853 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.853 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.854 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.854 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.854 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.854 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.854 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.854 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.854 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.855 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.855 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.855 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.855 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.855 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.855 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.855 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.855 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.856 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.856 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.856 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.856 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.856 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2023-07-21T22:38:47.856 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2023-07-21T22:38:47.856 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.857 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.857 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.857 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2023-07-21T22:38:47.857 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.857 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.857 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.857 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T22:38:47.858 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.858 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.858 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.858 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2023-07-21T22:38:47.858 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2023-07-21T22:38:47.858 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.858 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.859 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.859 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2023-07-21T22:38:47.859 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.859 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.859 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.859 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T22:38:47.859 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.859 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.860 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.860 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.860 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.860 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.860 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2023-07-21T22:38:47.860 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.860 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.861 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.861 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T22:38:47.861 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T22:38:47.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.861 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.861 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.861 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:47.862 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.862 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.862 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.862 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.862 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.862 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.862 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.863 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T22:38:47.863 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T22:38:47.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.863 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.863 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.863 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.863 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.864 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.864 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.864 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.864 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.864 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.864 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.864 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T22:38:47.865 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T22:38:47.865 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.865 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.865 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.865 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.865 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.865 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.865 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.866 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.866 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.866 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.866 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.866 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2023-07-21T22:38:47.866 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T22:38:47.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.867 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.867 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.867 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.867 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.867 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.867 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.868 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.868 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.868 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.868 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T22:38:47.868 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T22:38:47.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.868 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.869 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.869 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.869 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.869 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.869 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.869 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.869 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.869 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.870 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.870 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T22:38:47.870 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T22:38:47.870 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.870 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.870 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.871 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.871 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.871 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.871 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.871 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.871 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.871 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.871 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.872 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T22:38:47.872 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T22:38:47.872 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.872 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.872 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.872 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.873 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.873 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.873 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.873 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.873 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.873 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.873 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.874 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T22:38:47.874 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T22:38:47.874 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.874 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.874 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.874 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.874 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.874 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.875 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.875 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.875 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.875 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.875 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.875 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T22:38:47.876 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T22:38:47.876 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.876 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.876 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.876 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.876 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.876 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.876 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.877 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.877 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.877 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.877 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.877 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T22:38:47.877 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T22:38:47.877 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.878 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.878 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.878 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.878 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.878 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.878 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.878 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.879 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.879 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.879 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.879 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T22:38:47.879 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T22:38:47.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.879 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.880 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.880 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.880 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.880 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.880 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.880 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.880 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.881 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.881 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.881 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T22:38:47.881 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T22:38:47.881 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.881 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.881 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.882 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.882 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.882 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.882 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.882 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.882 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.882 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.882 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.883 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T22:38:47.883 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T22:38:47.883 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.883 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.883 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.883 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.884 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.884 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.884 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.884 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.884 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.884 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.885 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T22:38:47.885 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T22:38:47.885 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.885 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.885 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.885 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.885 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.885 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.886 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.886 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.886 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.886 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.886 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.886 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T22:38:47.887 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T22:38:47.887 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.887 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.887 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.887 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:47.887 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.887 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.888 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.888 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.888 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.888 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.888 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.888 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T22:38:47.888 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T22:38:47.889 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.889 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.889 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.889 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.889 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.889 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.889 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.890 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.890 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.890 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.890 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.890 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2023-07-21T22:38:47.890 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2023-07-21T22:38:47.890 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.891 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.891 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.891 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.891 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.891 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.891 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.891 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.891 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.892 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.892 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.892 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T22:38:47.892 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T22:38:47.892 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.892 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.893 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.893 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:47.893 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.893 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.893 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.893 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.893 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.893 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.894 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.894 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T22:38:47.894 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T22:38:47.894 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.894 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.894 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.894 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.895 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.895 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.895 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.895 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.895 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.895 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.895 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.896 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T22:38:47.896 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T22:38:47.896 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.896 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.896 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.896 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.896 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.897 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.897 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.897 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.897 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.897 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.897 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.898 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T22:38:47.898 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T22:38:47.898 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.898 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.898 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.898 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.898 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.898 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.899 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.899 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.899 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.899 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.899 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.899 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T22:38:47.900 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T22:38:47.900 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.900 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.900 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.900 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.900 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.900 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.901 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.901 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.901 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.901 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.901 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.901 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T22:38:47.901 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T22:38:47.902 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.902 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.902 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.902 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.902 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.902 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.902 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.903 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.903 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.903 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.903 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.903 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T22:38:47.903 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T22:38:47.904 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.904 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.904 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.904 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.904 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.904 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.904 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.904 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.905 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.905 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.905 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.905 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T22:38:47.905 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T22:38:47.905 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.905 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.906 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.906 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.906 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.906 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.906 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.906 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.906 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.906 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.907 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.907 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T22:38:47.907 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T22:38:47.907 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.907 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.907 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.908 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.908 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.908 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.908 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.908 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.908 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.908 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.908 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.909 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T22:38:47.909 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T22:38:47.909 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.909 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.909 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.909 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.909 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.910 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.910 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.910 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.910 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.910 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.910 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.911 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T22:38:47.911 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T22:38:47.911 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.911 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.911 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.911 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.911 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.911 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.912 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.912 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.912 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.912 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.912 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.912 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T22:38:47.913 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T22:38:47.913 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.913 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.913 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.913 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.913 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.913 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.913 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.914 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.914 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.914 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.914 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.914 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T22:38:47.914 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T22:38:47.914 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.915 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.915 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.915 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.915 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.915 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.915 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.915 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.916 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.916 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.916 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.916 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T22:38:47.916 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T22:38:47.916 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.916 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.917 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.917 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.917 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.917 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.917 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.917 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.917 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.917 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.918 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.918 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T22:38:47.918 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T22:38:47.918 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.918 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.918 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.918 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.919 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.919 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.919 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.919 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.919 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.919 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.919 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.920 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T22:38:47.920 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T22:38:47.920 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.920 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.920 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.920 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T22:38:47.921 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.921 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.921 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.921 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.921 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.921 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.921 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.922 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T22:38:47.922 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T22:38:47.922 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.922 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.922 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.922 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:47.922 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.922 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.923 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.923 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.923 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.923 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.923 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.923 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T22:38:47.924 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T22:38:47.924 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.924 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.924 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.924 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2023-07-21T22:38:47.924 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.924 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.924 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.925 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.925 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.925 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.925 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.925 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T22:38:47.925 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T22:38:47.926 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.926 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.926 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.926 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2023-07-21T22:38:47.926 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.926 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.926 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.926 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.927 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.927 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.927 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.927 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T22:38:47.927 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T22:38:47.927 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.927 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.928 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.928 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.928 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.928 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.928 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.928 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.928 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.928 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.929 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.929 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T22:38:47.929 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T22:38:47.929 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.929 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.929 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.930 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:47.930 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.930 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.930 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.930 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.930 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.930 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.930 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.931 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T22:38:47.931 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T22:38:47.931 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.931 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.931 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.931 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.931 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.932 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.932 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.932 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.932 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.932 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.932 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.932 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T22:38:47.933 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T22:38:47.933 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.933 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.933 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.933 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.933 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.933 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.934 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.934 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.934 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.934 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.934 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.934 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T22:38:47.934 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T22:38:47.935 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.935 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.935 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.935 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:47.935 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.935 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.935 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.936 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.936 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.936 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.936 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.936 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T22:38:47.936 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T22:38:47.936 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.937 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.937 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.937 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:47.937 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.937 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.937 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.937 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.938 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.938 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.938 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.938 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T22:38:47.938 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T22:38:47.938 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.938 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.939 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.939 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2023-07-21T22:38:47.939 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.939 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.939 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.939 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.939 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.939 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.940 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.940 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2023-07-21T22:38:47.940 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2023-07-21T22:38:47.940 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.940 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.940 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.941 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.941 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.941 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.941 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.941 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.941 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.941 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.941 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.942 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2023-07-21T22:38:47.942 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2023-07-21T22:38:47.942 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.942 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.942 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.942 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.942 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.943 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.943 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.943 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.943 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.943 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.943 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.944 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2023-07-21T22:38:47.944 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2023-07-21T22:38:47.944 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.944 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.944 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.944 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.944 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.944 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.945 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.945 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2023-07-21T22:38:47.945 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.945 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.945 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.945 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2023-07-21T22:38:47.945 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2023-07-21T22:38:47.946 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.946 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.946 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.946 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2023-07-21T22:38:47.946 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.946 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.946 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.946 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.947 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.947 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.947 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.947 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2023-07-21T22:38:47.947 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2023-07-21T22:38:47.947 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.948 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.948 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.948 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.948 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.948 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.948 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.948 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.948 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.949 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.949 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.949 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.949 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.949 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.949 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.949 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.950 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.950 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.950 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.950 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.950 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.950 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.950 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.950 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.951 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.951 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.951 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.951 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.951 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.951 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.951 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.952 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.952 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.952 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.952 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.952 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.952 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.952 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.953 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.953 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.953 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.953 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.953 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.953 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.953 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.954 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.954 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.954 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.954 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.954 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.954 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.954 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.955 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.955 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.955 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.955 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.955 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.955 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.955 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.955 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.956 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.956 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.956 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.956 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.956 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.956 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.956 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.957 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.957 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.957 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.957 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.957 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.957 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.957 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.958 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.958 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.958 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.958 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2023-07-21T22:38:47.958 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2023-07-21T22:38:47.958 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.958 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.959 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.959 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.959 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.959 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.959 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.959 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2023-07-21T22:38:47.959 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.959 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.960 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.960 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2023-07-21T22:38:47.960 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T22:38:47.960 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.960 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.960 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.961 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.961 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.961 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.961 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.961 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.961 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.961 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.962 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.962 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2023-07-21T22:38:47.962 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2023-07-21T22:38:47.962 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.962 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.962 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.962 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2023-07-21T22:38:47.963 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.963 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.963 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.963 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.963 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.963 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.963 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.964 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2023-07-21T22:38:47.964 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2023-07-21T22:38:47.964 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.964 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.964 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.964 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2023-07-21T22:38:47.964 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.965 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.965 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.965 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.965 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.965 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.965 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.965 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2023-07-21T22:38:47.966 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2023-07-21T22:38:47.966 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.966 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.966 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.966 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.966 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.966 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.966 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.967 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.967 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.967 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.967 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.967 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2023-07-21T22:38:47.967 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2023-07-21T22:38:47.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.968 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.968 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.968 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2023-07-21T22:38:47.968 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.968 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.968 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.968 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.969 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.969 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.969 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.969 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2023-07-21T22:38:47.969 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2023-07-21T22:38:47.969 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.969 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.970 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.970 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2023-07-21T22:38:47.970 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.970 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.970 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.970 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.970 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2023-07-21T22:38:47.970 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.971 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.971 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.971 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.971 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.971 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2023-07-21T22:38:47.971 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2023-07-21T22:38:47.972 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:47.972 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.972 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.972 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2023-07-21T22:38:47.972 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2023-07-21T22:38:47.972 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2023-07-21T22:38:47.972 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.972 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.973 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.973 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.973 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.973 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2023-07-21T22:38:47.973 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2023-07-21T22:38:47.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.973 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.974 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:47.974 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.974 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.974 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.974 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.974 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.974 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.975 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.975 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.975 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.975 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.975 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:47.975 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2023-07-21T22:38:47.975 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:47.976 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:47.976 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:47.976 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2023-07-21T22:38:47.976 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2023-07-21T22:38:47.976 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.976 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.976 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.976 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.977 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.977 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.977 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.977 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2023-07-21T22:38:47.977 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.977 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.977 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.978 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:47.978 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:47.978 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.978 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.978 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.978 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:47.978 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.979 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.979 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.979 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.979 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.979 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.979 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.979 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.979 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.980 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.980 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.980 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.980 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.980 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.980 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:47.980 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:47.981 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.981 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.981 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.981 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.981 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.981 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.981 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.981 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.982 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.982 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.982 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.982 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:47.982 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:47.982 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.983 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.983 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.983 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:47.983 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.983 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.983 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:47.983 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:47.983 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:47.984 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:47.984 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:47.984 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:47.984 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:47.984 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:47.984 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.984 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.984 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.985 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.985 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:47.985 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:47.985 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.985 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.985 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.986 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:47.986 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.986 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.986 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.986 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:47.986 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.986 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.986 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.987 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2023-07-21T22:38:47.987 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2023-07-21T22:38:47.987 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.987 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.987 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.987 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2023-07-21T22:38:47.987 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.988 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.988 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.988 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T22:38:47.988 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.988 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.988 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.989 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2023-07-21T22:38:47.989 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2023-07-21T22:38:47.989 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:47.989 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.989 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.989 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2023-07-21T22:38:47.989 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.989 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.990 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.990 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T22:38:47.990 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.990 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.990 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.990 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2023-07-21T22:38:47.990 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2023-07-21T22:38:47.991 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.991 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.991 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.991 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2023-07-21T22:38:47.991 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.991 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.991 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.992 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2023-07-21T22:38:47.992 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.992 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.992 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.992 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2023-07-21T22:38:47.992 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2023-07-21T22:38:47.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.993 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.993 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.993 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2023-07-21T22:38:47.993 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.993 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.993 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.993 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T22:38:47.994 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.994 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.994 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.994 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2023-07-21T22:38:47.994 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2023-07-21T22:38:47.994 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:47.994 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.995 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.995 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:47.995 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.995 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.995 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.995 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T22:38:47.995 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.995 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.996 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.996 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:47.996 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:47.996 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.996 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.996 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.997 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:47.997 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.997 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.997 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.997 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T22:38:47.997 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.997 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.997 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:47.998 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2023-07-21T22:38:47.998 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2023-07-21T22:38:47.998 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:47.998 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:47.998 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:47.998 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2023-07-21T22:38:47.998 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:47.999 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:47.999 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:47.999 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T22:38:47.999 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:47.999 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:47.999 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:47.999 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:47.999 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.000 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2023-07-21T22:38:48.000 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.000 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.000 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.000 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.001 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.001 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.001 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.001 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.001 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.001 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.001 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.001 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.002 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.002 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.002 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.002 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.002 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.002 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.002 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.002 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.003 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.003 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.003 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.003 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.003 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.003 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.004 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.004 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.004 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.004 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.004 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.004 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.005 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.005 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.005 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.005 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.005 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.005 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.005 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.006 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.006 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.006 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.006 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.006 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.006 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.006 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.006 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.007 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.007 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.007 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.007 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.007 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.007 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.007 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.008 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.008 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.008 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.008 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.008 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.008 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.008 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.009 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.009 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.009 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.009 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.009 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.009 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.010 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2023-07-21T22:38:48.010 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2023-07-21T22:38:48.010 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.010 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.010 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.010 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2023-07-21T22:38:48.010 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.010 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.011 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.011 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.011 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.011 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.011 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.011 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2023-07-21T22:38:48.011 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2023-07-21T22:38:48.012 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.012 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.012 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.012 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2023-07-21T22:38:48.012 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.012 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.012 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.013 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.013 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.013 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.013 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.013 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:48.013 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:48.013 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.014 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.014 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.014 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:48.014 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.014 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.014 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.014 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.014 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.015 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.015 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.015 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.015 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.015 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.015 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2023-07-21T22:38:48.016 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2023-07-21T22:38:48.016 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2023-07-21T22:38:48.016 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.016 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2023-07-21T22:38:48.016 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2023-07-21T22:38:48.016 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.016 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.016 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.017 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2023-07-21T22:38:48.017 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.017 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.017 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.017 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T22:38:48.017 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.017 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.017 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.018 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2023-07-21T22:38:48.018 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2023-07-21T22:38:48.018 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.018 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.018 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.018 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2023-07-21T22:38:48.018 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.019 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.019 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.019 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T22:38:48.019 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.019 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.019 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.020 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2023-07-21T22:38:48.020 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2023-07-21T22:38:48.020 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.020 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.020 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.020 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.020 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.021 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.021 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.021 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2023-07-21T22:38:48.021 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.021 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.021 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.021 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:48.022 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:48.022 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.022 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.022 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.022 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2023-07-21T22:38:48.022 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2023-07-21T22:38:48.022 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.022 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.023 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T22:38:48.023 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.023 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.023 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.023 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.023 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.023 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.024 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.024 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.024 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.024 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.024 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.024 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.024 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.025 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.025 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.025 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.025 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.025 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.025 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.025 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.025 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.026 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.026 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.026 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.026 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.026 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.026 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.027 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.027 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.027 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.027 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.027 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.027 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.027 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.027 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.028 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.028 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.028 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.028 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.028 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.028 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.029 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.029 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.029 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.029 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.029 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.029 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.029 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.029 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.030 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.030 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.030 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.030 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.030 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.030 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.030 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.031 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.031 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.031 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.031 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.031 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.031 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.031 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.031 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.032 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.032 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.032 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.032 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.032 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.032 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2023-07-21T22:38:48.033 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2023-07-21T22:38:48.033 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.033 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.033 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.033 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.033 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.033 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.033 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.034 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2023-07-21T22:38:48.034 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.034 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.034 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.034 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2023-07-21T22:38:48.034 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2023-07-21T22:38:48.035 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.035 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.035 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.035 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2023-07-21T22:38:48.035 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.035 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.035 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.035 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2023-07-21T22:38:48.036 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.036 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.036 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.036 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2023-07-21T22:38:48.036 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2023-07-21T22:38:48.036 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.037 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.037 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.037 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2023-07-21T22:38:48.037 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.037 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.037 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.037 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:38:48.037 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.038 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.038 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.038 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2023-07-21T22:38:48.038 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2023-07-21T22:38:48.038 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.038 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.039 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.039 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2023-07-21T22:38:48.039 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2023-07-21T22:38:48.039 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2023-07-21T22:38:48.039 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.039 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T22:38:48.039 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.039 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.040 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.040 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2023-07-21T22:38:48.040 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2023-07-21T22:38:48.040 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.040 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.040 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.041 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.041 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.041 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.041 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2023-07-21T22:38:48.041 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.041 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.041 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.042 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2023-07-21T22:38:48.042 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2023-07-21T22:38:48.042 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.042 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.042 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.042 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2023-07-21T22:38:48.042 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.043 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.043 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.043 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:38:48.043 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.043 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.043 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.043 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.043 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.044 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2023-07-21T22:38:48.044 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.044 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.044 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.044 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.045 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.045 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.045 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.045 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.045 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.045 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.046 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.046 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.046 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.046 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.046 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.046 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.046 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.046 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.047 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.047 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.047 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.047 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.047 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.047 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.048 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.048 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.048 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.048 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.048 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.048 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.048 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.048 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.049 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.049 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.049 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.049 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.049 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.049 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.049 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.050 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.050 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.050 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.050 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.050 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.050 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.050 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.051 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.051 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.051 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.051 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.051 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.051 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.051 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.051 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.052 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.052 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.052 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.052 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.052 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.052 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.052 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.053 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.053 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.053 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.053 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.053 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.053 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.053 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.054 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.054 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.054 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2023-07-21T22:38:48.054 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.054 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.054 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.054 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.055 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.055 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.055 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.055 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.055 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.055 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.055 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.055 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.056 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.056 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.056 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.056 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.056 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.056 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.056 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.056 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.057 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.057 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.057 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.057 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.057 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.057 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.058 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.058 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.058 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.058 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.058 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.058 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.058 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.058 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.059 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.060 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.061 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.061 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.061 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.061 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.061 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.062 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.062 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.062 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.062 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.062 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.062 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.063 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.063 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.063 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.063 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.063 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.063 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.063 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.064 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.064 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.064 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.064 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.064 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.065 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2023-07-21T22:38:48.065 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.065 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.065 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.065 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2023-07-21T22:38:48.065 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2023-07-21T22:38:48.065 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.065 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.066 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.066 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2023-07-21T22:38:48.066 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.066 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.066 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.066 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T22:38:48.066 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.066 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.067 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.067 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2023-07-21T22:38:48.067 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2023-07-21T22:38:48.067 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.067 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.068 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.068 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:48.068 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2023-07-21T22:38:48.068 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.069 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.069 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T22:38:48.069 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.069 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.070 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.070 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.070 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.071 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.071 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.071 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.072 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.072 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.072 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.072 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.073 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.073 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.073 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.074 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.074 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.074 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.074 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.075 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.075 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.075 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.075 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.076 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.076 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.077 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.077 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.077 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.077 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.077 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.077 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.077 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.078 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.078 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.078 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.078 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.078 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.078 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.079 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.079 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.079 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.079 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.079 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.079 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.079 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.080 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.080 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.080 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.080 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.080 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.081 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.081 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.081 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.081 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.081 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.081 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.082 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.082 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.082 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.082 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.082 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.082 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.082 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.082 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.083 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.083 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.083 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.083 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.083 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.083 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.083 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.084 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.084 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2023-07-21T22:38:48.084 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.084 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.084 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.084 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2023-07-21T22:38:48.085 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2023-07-21T22:38:48.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.085 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.085 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.085 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2023-07-21T22:38:48.086 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.086 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.086 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.086 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2023-07-21T22:38:48.086 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T22:38:48.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.086 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.087 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.087 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.087 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.087 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.087 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.087 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.087 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.087 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.088 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.088 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.088 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.088 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.088 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.088 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.088 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.089 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.089 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.089 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.089 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.089 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.089 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.089 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.090 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.090 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.090 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.090 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.090 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.091 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.091 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.091 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.091 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.091 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.092 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.092 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.092 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.092 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.092 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.092 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.093 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.093 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.093 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.093 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.093 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.093 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.093 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.093 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.094 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.094 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.094 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.094 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.094 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.094 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.094 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.095 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.095 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.095 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.095 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.095 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.095 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.095 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.096 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.096 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.096 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.096 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.096 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.096 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2023-07-21T22:38:48.097 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2023-07-21T22:38:48.097 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.097 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.097 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.097 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.097 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.097 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.097 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.098 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2023-07-21T22:38:48.098 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T22:38:48.098 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.098 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.098 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2023-07-21T22:38:48.098 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2023-07-21T22:38:48.098 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.099 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.099 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.099 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2023-07-21T22:38:48.099 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.099 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.099 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.099 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2023-07-21T22:38:48.099 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.100 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.100 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.100 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2023-07-21T22:38:48.100 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2023-07-21T22:38:48.100 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.100 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.101 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.101 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2023-07-21T22:38:48.101 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.101 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.101 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.101 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T22:38:48.102 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.102 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.102 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.102 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2023-07-21T22:38:48.102 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2023-07-21T22:38:48.102 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.103 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.103 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.103 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.103 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.103 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.103 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.104 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2023-07-21T22:38:48.104 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.104 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.104 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.105 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2023-07-21T22:38:48.105 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2023-07-21T22:38:48.105 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.105 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.105 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.106 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2023-07-21T22:38:48.106 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.106 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.106 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.106 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T22:38:48.107 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T22:38:48.107 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.107 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.108 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.108 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.108 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2023-07-21T22:38:48.108 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.109 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.109 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.109 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.109 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.109 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.110 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.110 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.110 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.110 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.111 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.111 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.111 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.111 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.111 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.112 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.112 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.112 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.112 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.112 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.113 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.113 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.113 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.114 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.114 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.114 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.114 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.114 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.115 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.115 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.115 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.115 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.116 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.116 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.116 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.116 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.117 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.117 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.117 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.117 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.118 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.118 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.118 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.118 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.118 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.118 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.119 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.119 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.119 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.119 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.119 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.119 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.120 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.120 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.120 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.120 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.120 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.120 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.121 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.121 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.121 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.121 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.121 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.121 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.121 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.122 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.122 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.122 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.122 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.122 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.122 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.123 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.123 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2023-07-21T22:38:48.123 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.123 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.123 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.123 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.123 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.124 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.124 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.124 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.124 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.124 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.124 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.124 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.125 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.126 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.126 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.126 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.126 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.126 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.126 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.127 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.127 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.127 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.127 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.127 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.127 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.127 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.127 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.128 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.128 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.128 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.128 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.128 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.128 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.128 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.129 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.129 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.129 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.129 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.129 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.129 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.129 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.130 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.130 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.130 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.130 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.130 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.131 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.131 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.131 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.131 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.131 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.131 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.132 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.132 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.132 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.132 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.132 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.132 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.132 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.132 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.133 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.133 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.133 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.133 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.133 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2023-07-21T22:38:48.134 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.134 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.134 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.134 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.134 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.134 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.134 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.134 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.135 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.135 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.135 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.135 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.135 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.135 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.135 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.135 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.136 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.136 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.136 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.136 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.136 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.136 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.136 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.137 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.137 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.137 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.137 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.137 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.137 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.137 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.137 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.138 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.138 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.138 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.138 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.138 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.138 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.139 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.139 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.139 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.139 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.139 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.139 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.139 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.139 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.140 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.140 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.140 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.140 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.140 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.140 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.140 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.141 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.141 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.141 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.141 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.141 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.141 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.142 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.142 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.142 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.142 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.142 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.142 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.142 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.142 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.143 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.143 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.143 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.143 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.143 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.143 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.143 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2023-07-21T22:38:48.144 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.144 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.144 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.144 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2023-07-21T22:38:48.144 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2023-07-21T22:38:48.144 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.144 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.144 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.145 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.145 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.145 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.145 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.145 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T22:38:48.145 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.145 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.145 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.146 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.146 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.146 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.146 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.146 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.146 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.147 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.147 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.147 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.147 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.147 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.147 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.147 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.147 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.148 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.148 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.148 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.148 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.148 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.148 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.148 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.149 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.149 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.149 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.149 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.149 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.149 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.150 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.150 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.150 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.150 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.150 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.150 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.151 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.151 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.151 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.151 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.151 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.151 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.151 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.151 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.152 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.152 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.152 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.152 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.152 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.152 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.152 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.153 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.153 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.153 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.153 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.153 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.153 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.153 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.154 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.154 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.154 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.154 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.154 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.154 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.154 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.154 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.155 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.155 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.155 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.155 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2023-07-21T22:38:48.155 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2023-07-21T22:38:48.155 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.155 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.156 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.156 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.156 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.156 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.156 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.156 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2023-07-21T22:38:48.156 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2023-07-21T22:38:48.157 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2023-07-21T22:38:48.157 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.157 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2023-07-21T22:38:48.157 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.157 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.157 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.157 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.157 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.158 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.158 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2023-07-21T22:38:48.158 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.158 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.158 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.158 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.158 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.159 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.159 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.159 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.159 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.159 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.159 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.159 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.159 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.160 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.160 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.160 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.160 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.160 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.160 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.160 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.161 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.161 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.161 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.161 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.162 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.162 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.162 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.162 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.162 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.162 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.162 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.162 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.163 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.163 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.163 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.163 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.163 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.163 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.164 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.164 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.164 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.164 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.164 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.164 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.164 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.165 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.165 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.165 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.165 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.165 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.165 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.165 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.165 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.166 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.166 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.166 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.166 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.166 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.166 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.166 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.167 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.167 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.167 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.167 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.167 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.167 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.167 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.167 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.168 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.168 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.168 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.168 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.168 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2023-07-21T22:38:48.168 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.169 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.169 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.169 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.169 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.169 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.169 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.169 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.169 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.170 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.170 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.170 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.170 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.170 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.170 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.170 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.171 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.171 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.171 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.171 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.171 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.171 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.171 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.172 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.172 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.172 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.172 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.172 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.172 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.172 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.173 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.173 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.173 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.173 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.173 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.173 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.173 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.174 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.174 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.174 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.174 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.174 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.174 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.174 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.174 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.175 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.175 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.175 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.175 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.175 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.175 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.175 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.176 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.176 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.176 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.176 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.176 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.176 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.176 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.177 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.177 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.177 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.177 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.177 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.177 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.177 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.177 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.178 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.178 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.178 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.178 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.178 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.178 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2023-07-21T22:38:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.181 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.181 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.181 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.181 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.181 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.181 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.181 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.181 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.182 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.182 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.182 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.182 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.182 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.182 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.183 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.183 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.183 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.183 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.183 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.183 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.183 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.184 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.184 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.184 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.184 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.184 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.184 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.184 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.185 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.185 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.185 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.185 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.185 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.185 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.185 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.186 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.186 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.186 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.186 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.186 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.186 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.186 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.186 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.187 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.187 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.187 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.187 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.187 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.187 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.187 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.187 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.188 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.188 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.188 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.188 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.188 INFO:teuthology.orchestra.run.smithi107.stdout: "noautoscale": { 2023-07-21T22:38:48.188 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "noautoscale", 2023-07-21T22:38:48.189 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.189 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.189 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.189 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.189 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.189 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.189 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.190 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "global autoscale flag", 2023-07-21T22:38:48.190 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T22:38:48.190 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.190 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.190 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:48.190 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:48.190 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.191 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.191 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.191 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:48.191 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.191 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.191 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.191 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.192 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.192 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.192 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.192 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2023-07-21T22:38:48.192 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2023-07-21T22:38:48.192 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:48.192 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.193 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.193 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2023-07-21T22:38:48.193 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2023-07-21T22:38:48.193 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.193 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.193 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2023-07-21T22:38:48.193 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T22:38:48.194 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.194 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.194 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.194 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.194 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.194 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2023-07-21T22:38:48.195 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.195 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.195 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.195 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2023-07-21T22:38:48.195 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2023-07-21T22:38:48.195 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.195 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.195 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.196 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.196 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.196 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.196 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.196 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T22:38:48.196 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.196 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.196 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.197 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2023-07-21T22:38:48.197 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2023-07-21T22:38:48.197 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.197 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.197 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.197 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.197 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.198 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.198 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.198 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.198 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.198 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.198 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.198 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.199 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.199 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.199 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.199 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.199 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.199 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.199 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.199 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.200 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.200 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.200 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.200 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.200 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.200 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.201 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.201 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.201 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.201 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.201 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.201 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.201 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.202 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.202 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.202 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.202 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.202 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.202 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.202 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.202 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.203 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.203 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.203 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.203 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.203 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.203 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.203 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.204 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.204 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.204 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.204 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.204 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.204 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.204 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.204 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.205 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.205 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.205 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.205 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.205 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.205 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.205 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.205 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.206 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.206 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.206 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.206 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.206 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.206 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.207 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.207 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.207 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.207 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.207 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.207 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.207 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.208 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2023-07-21T22:38:48.208 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2023-07-21T22:38:48.208 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.208 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.208 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.208 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2023-07-21T22:38:48.208 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.208 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.209 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.209 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2023-07-21T22:38:48.209 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.209 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.209 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.209 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:48.209 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:48.210 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.210 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.210 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.210 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2023-07-21T22:38:48.210 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.210 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.210 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.211 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2023-07-21T22:38:48.211 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.211 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.211 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.211 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.211 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.212 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2023-07-21T22:38:48.212 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.212 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.212 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.212 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2023-07-21T22:38:48.212 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2023-07-21T22:38:48.212 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.212 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.213 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.213 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.213 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.213 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.213 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.213 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.213 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.213 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.214 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.214 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2023-07-21T22:38:48.214 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2023-07-21T22:38:48.214 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.214 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.214 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.214 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.215 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.215 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.215 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.215 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T22:38:48.215 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T22:38:48.215 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.215 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.216 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.216 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.216 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.216 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.216 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.216 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.216 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.217 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.217 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.217 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.217 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.217 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.217 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.217 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.217 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.218 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.218 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.218 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.218 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.218 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.218 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.218 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.219 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.219 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.219 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.219 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.219 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.219 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.219 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.219 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.220 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.220 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.220 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.220 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.220 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.220 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.221 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.221 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.221 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.221 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.221 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.221 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.221 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.221 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.222 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.222 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.222 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.222 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.222 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.222 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.222 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.222 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.223 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.223 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.223 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.223 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.223 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.223 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.223 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.224 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.224 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.224 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.224 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.224 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.224 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.224 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.225 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2023-07-21T22:38:48.225 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2023-07-21T22:38:48.225 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.225 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.225 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.225 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.225 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.226 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.226 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.226 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.226 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.226 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.226 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.226 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T22:38:48.227 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T22:38:48.227 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.227 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.227 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.227 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2023-07-21T22:38:48.227 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.227 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.227 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.228 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.228 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.228 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.228 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.228 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2023-07-21T22:38:48.228 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2023-07-21T22:38:48.229 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:48.229 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.229 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.229 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2023-07-21T22:38:48.229 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.229 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.229 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.229 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.230 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.230 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.230 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.230 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2023-07-21T22:38:48.230 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2023-07-21T22:38:48.230 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.230 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.231 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.231 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2023-07-21T22:38:48.231 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.231 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.231 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.231 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T22:38:48.231 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.231 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.232 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.232 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2023-07-21T22:38:48.232 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2023-07-21T22:38:48.232 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.232 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.232 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.233 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2023-07-21T22:38:48.233 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.233 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.233 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.233 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T22:38:48.233 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.233 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.233 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.234 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2023-07-21T22:38:48.234 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2023-07-21T22:38:48.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.234 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.234 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2023-07-21T22:38:48.234 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.235 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.235 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.235 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.235 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.235 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.235 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.235 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2023-07-21T22:38:48.236 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2023-07-21T22:38:48.236 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.236 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.236 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.236 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2023-07-21T22:38:48.236 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.236 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.236 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.237 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2023-07-21T22:38:48.237 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2023-07-21T22:38:48.237 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.237 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.237 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.237 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.237 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.238 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2023-07-21T22:38:48.238 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2023-07-21T22:38:48.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.238 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.238 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.238 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2023-07-21T22:38:48.238 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2023-07-21T22:38:48.238 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2023-07-21T22:38:48.239 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.239 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.239 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.239 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.239 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.239 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.239 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.240 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.240 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2023-07-21T22:38:48.240 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.240 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.240 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.240 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.240 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.240 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.241 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.241 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.241 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.241 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.241 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.241 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.241 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.242 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.242 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.242 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.242 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.242 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.242 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.242 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.242 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.243 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.243 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.243 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.243 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.243 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.243 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.244 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.244 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.244 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.244 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.244 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.244 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.245 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.245 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.245 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.245 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.245 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.245 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.246 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.246 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.246 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.246 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.246 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.246 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.246 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.247 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.247 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.247 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.247 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.247 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.247 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.247 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.248 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.248 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.248 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.248 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.248 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.249 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.249 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.249 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.249 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.249 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.249 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.250 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.250 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.250 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.250 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.250 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.251 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2023-07-21T22:38:48.251 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2023-07-21T22:38:48.251 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.251 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.252 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.252 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:48.252 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.253 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.253 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.253 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.254 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.254 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.254 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.255 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2023-07-21T22:38:48.255 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T22:38:48.256 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.256 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.256 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.257 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.257 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.257 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.257 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.258 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.258 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.258 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.258 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.259 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2023-07-21T22:38:48.259 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2023-07-21T22:38:48.259 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.260 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.260 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.260 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.261 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.261 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.261 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.261 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.262 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.262 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.262 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.263 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.263 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.263 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.264 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2023-07-21T22:38:48.264 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.264 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.265 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.265 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2023-07-21T22:38:48.265 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2023-07-21T22:38:48.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.266 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.266 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.266 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.266 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.267 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.267 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.267 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.267 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.268 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.268 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.269 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2023-07-21T22:38:48.269 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2023-07-21T22:38:48.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.269 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.270 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.270 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.270 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.270 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.271 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.271 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.271 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.272 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.272 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.272 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.273 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.273 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.273 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.273 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.274 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.274 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.274 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.275 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.275 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.275 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.275 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.276 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.276 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.276 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.277 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.277 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.277 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.277 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.278 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.278 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.278 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.279 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.279 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.279 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.279 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.279 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.280 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.280 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.280 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.280 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.280 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.280 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.281 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.281 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.281 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.281 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.281 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.281 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.281 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.282 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.282 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.282 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.282 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.282 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.282 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.282 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.283 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.283 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.283 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.283 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.283 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.283 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.283 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.284 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.284 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.284 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.284 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.284 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.284 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.284 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.284 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.285 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.285 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.285 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2023-07-21T22:38:48.285 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2023-07-21T22:38:48.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.285 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.286 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.286 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.286 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.286 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.286 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.286 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.286 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.287 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.287 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.287 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2023-07-21T22:38:48.287 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2023-07-21T22:38:48.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.287 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.287 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.288 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.288 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.288 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.288 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.288 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.288 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.288 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.288 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.289 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.289 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.289 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.289 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2023-07-21T22:38:48.289 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.290 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.290 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.290 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.290 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.290 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.290 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.290 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.290 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.291 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.291 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.291 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.291 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.291 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.291 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.291 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.292 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.292 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.292 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.292 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.292 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.292 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.292 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.293 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.293 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.293 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.293 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.293 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.293 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.293 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.294 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.294 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.294 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.294 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.294 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.294 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.294 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.295 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.295 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.295 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.295 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.296 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.296 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.296 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.296 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.296 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.296 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.296 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.297 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.297 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.297 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.297 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.298 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.298 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.298 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.299 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.299 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.300 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.300 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.300 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.300 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.301 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.301 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.301 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.302 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.302 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.302 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.302 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.303 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.303 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2023-07-21T22:38:48.303 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.304 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.304 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.304 INFO:teuthology.orchestra.run.smithi107.stdout: "drive_group_interval": { 2023-07-21T22:38:48.305 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "drive_group_interval", 2023-07-21T22:38:48.305 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:48.305 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.305 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.306 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300.0", 2023-07-21T22:38:48.306 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.306 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.306 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.307 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T22:38:48.307 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.307 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.308 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.308 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.308 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.309 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.309 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.309 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.310 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.310 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.310 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.310 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.311 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.311 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.311 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.311 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.312 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.312 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.312 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.313 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.313 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.313 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.313 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.314 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.314 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.315 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.315 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.315 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.315 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.316 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.316 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.316 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.316 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.316 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.316 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.316 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.317 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.317 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.317 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.317 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.317 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.317 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.317 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.317 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.318 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.318 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.318 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.318 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.318 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.318 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.318 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.318 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.319 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.319 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.319 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.319 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.319 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.319 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.319 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.320 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.320 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.320 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.320 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.320 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.320 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.320 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.321 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.321 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.321 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2023-07-21T22:38:48.321 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2023-07-21T22:38:48.321 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.321 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.321 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.322 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2023-07-21T22:38:48.322 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.322 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.322 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.322 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T22:38:48.322 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.322 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.322 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.323 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.323 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.323 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.323 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2023-07-21T22:38:48.323 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.323 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.323 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.324 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.324 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.324 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.324 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.324 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.324 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.324 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.325 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.325 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.325 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.325 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.325 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.325 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.325 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.325 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.326 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.326 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.326 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.326 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.326 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.326 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.327 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.327 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.327 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.327 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.327 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.327 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.327 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.327 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.328 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.328 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.328 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.328 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.328 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.328 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.328 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.329 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.329 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.329 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.329 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.329 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.329 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.329 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.330 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.330 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.330 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.330 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.330 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.330 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.330 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.330 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.331 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.331 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.331 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.331 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.331 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.331 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.331 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.332 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.332 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.332 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.332 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.332 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.332 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.332 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.333 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2023-07-21T22:38:48.333 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2023-07-21T22:38:48.333 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.333 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.333 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.333 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.333 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.333 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.334 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.334 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.334 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.334 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.334 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.334 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2023-07-21T22:38:48.334 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2023-07-21T22:38:48.335 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.335 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.335 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.335 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2023-07-21T22:38:48.335 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.335 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.335 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.335 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.336 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.336 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.336 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.336 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2023-07-21T22:38:48.336 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2023-07-21T22:38:48.336 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.337 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.337 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.337 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.337 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.337 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.337 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.337 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.337 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.338 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.338 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.338 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2023-07-21T22:38:48.338 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2023-07-21T22:38:48.338 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.338 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.338 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.339 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.339 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.339 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.339 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.339 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.339 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.339 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.339 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.340 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2023-07-21T22:38:48.340 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2023-07-21T22:38:48.340 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:48.340 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.340 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.340 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.340 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.341 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.341 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.341 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.341 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.341 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.341 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.341 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2023-07-21T22:38:48.342 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2023-07-21T22:38:48.342 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.342 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.342 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.342 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.342 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.342 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.342 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.343 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.343 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.343 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.343 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.343 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2023-07-21T22:38:48.343 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2023-07-21T22:38:48.344 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.344 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.344 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.344 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.344 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.344 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.344 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.344 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.345 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.345 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.345 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.345 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2023-07-21T22:38:48.345 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2023-07-21T22:38:48.345 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.345 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.346 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.346 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.346 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.346 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.346 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.346 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.346 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.346 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.347 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.347 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2023-07-21T22:38:48.347 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2023-07-21T22:38:48.347 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.347 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.347 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.347 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.348 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2023-07-21T22:38:48.348 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2023-07-21T22:38:48.348 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.348 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.348 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.348 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.348 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.349 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2023-07-21T22:38:48.349 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2023-07-21T22:38:48.349 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.349 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.349 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.349 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.349 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.349 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.350 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.350 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.350 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.350 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.350 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.350 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2023-07-21T22:38:48.350 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2023-07-21T22:38:48.351 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.351 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.351 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.351 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.351 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.351 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.351 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.352 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.352 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.352 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.352 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.352 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2023-07-21T22:38:48.352 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2023-07-21T22:38:48.352 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.353 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.353 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.353 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.353 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.353 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.353 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.353 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.353 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.354 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.354 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.354 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.354 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.354 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.354 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2023-07-21T22:38:48.355 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.355 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.355 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.355 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2023-07-21T22:38:48.355 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2023-07-21T22:38:48.355 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.355 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.355 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.356 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.356 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.356 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.356 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.356 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T22:38:48.356 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.356 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.356 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.357 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2023-07-21T22:38:48.357 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2023-07-21T22:38:48.357 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.357 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.357 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.357 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.357 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.358 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.358 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.358 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2023-07-21T22:38:48.358 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.358 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.358 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.358 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.359 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.359 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.359 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.359 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.359 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.359 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.359 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.359 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.360 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.360 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.360 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.360 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.360 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.360 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.360 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.361 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.361 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.361 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.361 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.361 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.361 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.361 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.362 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.362 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.362 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.362 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.362 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.362 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.362 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.362 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.363 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.363 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.363 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.363 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.363 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.363 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.363 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.364 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.364 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.364 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.364 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.364 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.364 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.364 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.364 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.365 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.365 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.365 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.365 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.365 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.365 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.365 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.366 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.366 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.366 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.366 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.366 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.366 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.366 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.367 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.367 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.367 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.367 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.367 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.367 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.367 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.367 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.368 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2023-07-21T22:38:48.368 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.368 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.368 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.368 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.369 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.369 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.369 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.369 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.369 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.369 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.369 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.369 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.370 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.370 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.370 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.370 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.370 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.370 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.370 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.370 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.371 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.371 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.371 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.371 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.371 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.371 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.372 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.372 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.372 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.372 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.372 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.372 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.372 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.372 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.373 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.373 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.373 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.373 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.373 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.373 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.373 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.374 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.374 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.374 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.374 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.374 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.374 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.374 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.374 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.375 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.375 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.375 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.375 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.375 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.375 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.375 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.376 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.376 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.376 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.376 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.376 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.376 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.376 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.377 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.377 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.377 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.377 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.377 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.377 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.377 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.378 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.378 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2023-07-21T22:38:48.378 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.378 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.378 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.378 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.378 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.379 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.379 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.379 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.379 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.379 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.379 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.379 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.380 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.380 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.380 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.380 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.380 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.380 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.380 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.380 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.381 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.381 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.381 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.381 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.381 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.381 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.381 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.382 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.382 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.382 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.382 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.382 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.382 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.382 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.382 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.383 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.383 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.383 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.383 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.383 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.383 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.383 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.384 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.384 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.384 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.384 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.384 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.384 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.384 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.385 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.385 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.385 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.385 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.385 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.385 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.385 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.386 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.386 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.386 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.386 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.386 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.386 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.386 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.386 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.387 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.387 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.387 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.387 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.387 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.387 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.387 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.388 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2023-07-21T22:38:48.388 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.388 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.388 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.388 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2023-07-21T22:38:48.388 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2023-07-21T22:38:48.388 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.388 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.389 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.389 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T22:38:48.389 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.389 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.389 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.389 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.389 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.389 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.390 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.390 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:48.390 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:48.390 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.390 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.390 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.390 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2023-07-21T22:38:48.391 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.391 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.391 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.391 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.391 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.391 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.391 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.392 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.392 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.392 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.392 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.392 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.392 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.392 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.393 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.393 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.393 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.393 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.393 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.393 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.393 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.393 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.394 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.394 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.394 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.394 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.394 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.394 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.394 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.395 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.395 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.395 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.395 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.395 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.395 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.395 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.395 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.396 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.396 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.396 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.396 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.396 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.396 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.396 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.397 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.397 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.397 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.397 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.397 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.397 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.397 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.398 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.398 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.398 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.398 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.398 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.398 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.398 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.398 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.399 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.399 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.399 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.399 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.399 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.399 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.399 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.400 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.400 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.400 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.400 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.400 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.400 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.400 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.400 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.401 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.401 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.401 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2023-07-21T22:38:48.401 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.401 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.401 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.402 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2023-07-21T22:38:48.402 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2023-07-21T22:38:48.402 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.402 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.402 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.402 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.402 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.402 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.403 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.403 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T22:38:48.403 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.403 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.403 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.403 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2023-07-21T22:38:48.403 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2023-07-21T22:38:48.404 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.404 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.404 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.404 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.404 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.404 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.404 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.405 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T22:38:48.405 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.405 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.405 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.405 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2023-07-21T22:38:48.405 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2023-07-21T22:38:48.405 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.406 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.406 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.406 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.406 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.406 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.406 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.406 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T22:38:48.406 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.407 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.407 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.407 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2023-07-21T22:38:48.407 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2023-07-21T22:38:48.407 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.407 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.407 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.408 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.408 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.408 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.408 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.408 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T22:38:48.408 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.408 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.408 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.409 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2023-07-21T22:38:48.409 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2023-07-21T22:38:48.409 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.409 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.409 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.409 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.410 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.410 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.410 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.410 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T22:38:48.410 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.410 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.410 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.411 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2023-07-21T22:38:48.411 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2023-07-21T22:38:48.411 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.411 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.411 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.411 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.411 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.411 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.412 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.412 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.412 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.412 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.412 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.412 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2023-07-21T22:38:48.412 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2023-07-21T22:38:48.413 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.413 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.413 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.413 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.413 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.413 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.413 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.413 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.414 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.414 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.414 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.414 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2023-07-21T22:38:48.414 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2023-07-21T22:38:48.414 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.414 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.415 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.415 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T22:38:48.415 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.415 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.415 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.415 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.415 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.415 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.416 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.416 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2023-07-21T22:38:48.416 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2023-07-21T22:38:48.416 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.416 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.416 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.416 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.417 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.417 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.417 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.417 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.417 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.417 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.417 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.418 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:48.418 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:48.418 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.418 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.418 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.418 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2023-07-21T22:38:48.418 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2023-07-21T22:38:48.419 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.419 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.419 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.419 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.419 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.419 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.419 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2023-07-21T22:38:48.420 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2023-07-21T22:38:48.420 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.420 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.420 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.420 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2023-07-21T22:38:48.420 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.420 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.420 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.421 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.421 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.421 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.421 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.421 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2023-07-21T22:38:48.421 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2023-07-21T22:38:48.421 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.422 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.422 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.422 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.422 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.422 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.422 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.422 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.422 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.423 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.423 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.423 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2023-07-21T22:38:48.423 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2023-07-21T22:38:48.423 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.423 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.423 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.424 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.424 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.424 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.424 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.424 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.424 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.424 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.425 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.425 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.425 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.425 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.425 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.425 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.425 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.426 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.426 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.426 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.426 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.426 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.426 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.426 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.426 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.427 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.427 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.427 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.427 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.427 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.427 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.427 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.428 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.428 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.428 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.428 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.428 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.428 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.428 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.429 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.429 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.429 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.429 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.429 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.429 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.429 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.430 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.430 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.430 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.430 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.430 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.430 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.430 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.430 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.431 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.431 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.431 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.431 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.431 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.431 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.431 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.431 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.432 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.432 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.432 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.432 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.432 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.432 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.432 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.433 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.433 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.433 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.433 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.433 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.433 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.433 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.433 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.434 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2023-07-21T22:38:48.434 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2023-07-21T22:38:48.434 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.435 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.435 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.436 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.436 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.436 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.436 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.436 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.436 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.436 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.437 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.437 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2023-07-21T22:38:48.437 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2023-07-21T22:38:48.437 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.437 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.437 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.438 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.438 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.438 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.438 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.438 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.438 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.439 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.439 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.439 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2023-07-21T22:38:48.439 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2023-07-21T22:38:48.439 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.439 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.440 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.440 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T22:38:48.440 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.440 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.440 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.440 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.440 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.440 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.441 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.441 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.441 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.441 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.441 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2023-07-21T22:38:48.442 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.442 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.442 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.442 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.442 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.442 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.442 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.443 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.443 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.443 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.443 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.443 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.443 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.443 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.443 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.444 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.444 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.444 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.444 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.444 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.444 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.444 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.445 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.445 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.445 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.445 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.445 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.445 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.445 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.446 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.446 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.446 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.446 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.446 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.446 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.446 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.447 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.447 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.447 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.447 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.447 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.447 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.447 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.448 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.448 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.448 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.448 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.448 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.448 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.448 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.448 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.449 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.449 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.449 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.449 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.449 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.449 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.450 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.450 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.450 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.450 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.450 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.450 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.450 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.450 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.451 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.451 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.451 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.451 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.451 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.451 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.451 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.452 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2023-07-21T22:38:48.452 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.452 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.452 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.452 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.452 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.453 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.453 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.453 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.453 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.453 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.453 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.453 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.453 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.454 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.454 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.454 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.454 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.454 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.454 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.454 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.455 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.455 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.455 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.455 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.455 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.455 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.455 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.456 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.456 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.456 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.456 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.456 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.456 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.456 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.457 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.457 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.457 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.457 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.457 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.457 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.457 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.458 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.458 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.458 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.458 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.458 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.458 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.458 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.459 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.459 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.459 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.459 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.459 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.459 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.459 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.459 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.460 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.460 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.460 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.460 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.460 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.460 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.460 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.461 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.461 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.461 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.461 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.461 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.461 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.462 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2023-07-21T22:38:48.462 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2023-07-21T22:38:48.462 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.462 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.462 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.462 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2023-07-21T22:38:48.462 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.462 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.463 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.463 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T22:38:48.463 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.463 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.463 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.463 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2023-07-21T22:38:48.463 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2023-07-21T22:38:48.464 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.464 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.464 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.464 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:48.464 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.464 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.464 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.465 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T22:38:48.465 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.465 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.465 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.465 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.465 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.465 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2023-07-21T22:38:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2023-07-21T22:38:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2023-07-21T22:38:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2023-07-21T22:38:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2023-07-21T22:38:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2023-07-21T22:38:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2023-07-21T22:38:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2023-07-21T22:38:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2023-07-21T22:38:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2023-07-21T22:38:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2023-07-21T22:38:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2023-07-21T22:38:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2023-07-21T22:38:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T22:38:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2023-07-21T22:38:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "modules": [ 2023-07-21T22:38:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2023-07-21T22:38:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "dashboard", 2023-07-21T22:38:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "iostat", 2023-07-21T22:38:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "nfs", 2023-07-21T22:38:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "restful" 2023-07-21T22:38:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2023-07-21T22:38:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2023-07-21T22:38:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:48.490 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.490 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.490 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.490 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T22:38:48.490 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.490 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.490 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2023-07-21T22:38:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2023-07-21T22:38:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2023-07-21T22:38:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2023-07-21T22:38:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2023-07-21T22:38:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2023-07-21T22:38:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2023-07-21T22:38:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2023-07-21T22:38:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2023-07-21T22:38:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2023-07-21T22:38:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2023-07-21T22:38:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2023-07-21T22:38:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2023-07-21T22:38:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2023-07-21T22:38:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2023-07-21T22:38:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2023-07-21T22:38:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2023-07-21T22:38:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2023-07-21T22:38:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2023-07-21T22:38:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2023-07-21T22:38:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2023-07-21T22:38:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2023-07-21T22:38:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T22:38:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2023-07-21T22:38:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2023-07-21T22:38:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2023-07-21T22:38:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2023-07-21T22:38:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2023-07-21T22:38:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2023-07-21T22:38:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T22:38:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2023-07-21T22:38:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2023-07-21T22:38:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2023-07-21T22:38:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T22:38:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:38:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2023-07-21T22:38:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2023-07-21T22:38:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:48.520 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.520 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.520 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:48.520 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2023-07-21T22:38:48.520 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2023-07-21T22:38:48.520 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.520 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T22:38:48.521 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:38:48.521 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.521 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.521 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2023-07-21T22:38:48.521 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2023-07-21T22:38:48.521 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:48.521 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.522 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.522 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2023-07-21T22:38:48.522 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2023-07-21T22:38:48.522 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2023-07-21T22:38:48.522 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.522 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T22:38:48.522 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.522 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.523 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.523 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2023-07-21T22:38:48.523 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2023-07-21T22:38:48.523 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.523 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.523 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.524 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T22:38:48.524 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.524 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.524 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.524 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T22:38:48.524 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T22:38:48.524 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.524 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.525 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2023-07-21T22:38:48.525 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2023-07-21T22:38:48.525 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:48.525 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.525 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.525 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2023-07-21T22:38:48.525 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2023-07-21T22:38:48.526 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2023-07-21T22:38:48.526 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.526 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2023-07-21T22:38:48.526 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T22:38:48.526 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.526 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.527 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2023-07-21T22:38:48.527 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2023-07-21T22:38:48.527 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.527 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.527 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.527 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2023-07-21T22:38:48.527 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.527 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.528 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.528 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T22:38:48.528 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:38:48.528 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.528 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.528 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2023-07-21T22:38:48.529 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2023-07-21T22:38:48.529 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:48.529 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.529 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.529 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:48.529 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2023-07-21T22:38:48.529 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2023-07-21T22:38:48.530 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.530 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T22:38:48.530 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:38:48.530 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.530 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.530 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.530 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.531 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.531 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.531 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.531 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.531 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.531 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.531 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.531 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.532 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.532 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.532 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.532 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.532 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.532 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.532 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.533 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.533 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.533 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.533 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.533 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.533 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.533 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.534 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.534 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.534 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.534 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.534 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.534 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.534 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.535 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.535 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.535 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.535 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.535 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.535 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.536 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.536 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.536 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.536 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.536 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.536 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.536 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.537 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.537 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.537 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.537 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.537 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.537 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.537 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.537 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.538 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.538 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.538 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.538 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.538 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.538 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.538 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.539 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.539 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.539 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.539 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.539 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.539 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.539 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.540 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2023-07-21T22:38:48.540 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2023-07-21T22:38:48.540 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:48.540 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.540 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.540 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:48.541 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.541 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.541 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.541 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T22:38:48.541 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.541 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.541 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.542 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2023-07-21T22:38:48.542 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2023-07-21T22:38:48.542 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.542 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.542 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.542 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2023-07-21T22:38:48.542 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.543 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.543 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.543 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2023-07-21T22:38:48.543 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2023-07-21T22:38:48.543 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap" 2023-07-21T22:38:48.543 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.543 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2023-07-21T22:38:48.544 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.544 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.544 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.544 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2023-07-21T22:38:48.544 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2023-07-21T22:38:48.544 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.544 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.545 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.545 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.545 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.545 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.545 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.545 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T22:38:48.545 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.546 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.546 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.546 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:48.546 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:48.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.546 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.546 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.547 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:48.547 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.547 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.547 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.547 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T22:38:48.547 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.547 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.547 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.548 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2023-07-21T22:38:48.548 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2023-07-21T22:38:48.548 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.548 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.548 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.549 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2023-07-21T22:38:48.549 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2023-07-21T22:38:48.549 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.549 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.549 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T22:38:48.549 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T22:38:48.549 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.550 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.550 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2023-07-21T22:38:48.550 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2023-07-21T22:38:48.550 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:48.550 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.551 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.551 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:48.551 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.551 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.551 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.552 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T22:38:48.552 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.552 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.552 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.552 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.552 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.553 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.553 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2023-07-21T22:38:48.553 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.553 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.553 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.553 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2023-07-21T22:38:48.554 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2023-07-21T22:38:48.554 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:48.554 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.554 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.554 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2023-07-21T22:38:48.554 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.554 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.554 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.555 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T22:38:48.555 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.555 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.555 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.555 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2023-07-21T22:38:48.555 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2023-07-21T22:38:48.556 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.556 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.556 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.556 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2023-07-21T22:38:48.556 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.556 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.556 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.556 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T22:38:48.557 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.557 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.557 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.557 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2023-07-21T22:38:48.558 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2023-07-21T22:38:48.558 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.558 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.558 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.558 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2023-07-21T22:38:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T22:38:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.559 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.560 INFO:teuthology.orchestra.run.smithi107.stdout: "alertmanager_web_password": { 2023-07-21T22:38:48.560 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alertmanager_web_password", 2023-07-21T22:38:48.560 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.560 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.560 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.560 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:48.560 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.561 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.561 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.561 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Alertmanager web password", 2023-07-21T22:38:48.561 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.561 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.561 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.562 INFO:teuthology.orchestra.run.smithi107.stdout: "alertmanager_web_user": { 2023-07-21T22:38:48.562 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alertmanager_web_user", 2023-07-21T22:38:48.562 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.562 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.562 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.562 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:48.562 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.562 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.563 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.563 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Alertmanager web user", 2023-07-21T22:38:48.563 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.563 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.563 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.563 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2023-07-21T22:38:48.563 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2023-07-21T22:38:48.564 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.564 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.564 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.564 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.564 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.564 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.564 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.564 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T22:38:48.565 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T22:38:48.565 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.565 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.565 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2023-07-21T22:38:48.565 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2023-07-21T22:38:48.565 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.566 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.566 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.566 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:48.566 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.566 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.566 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.566 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T22:38:48.566 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.567 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.567 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.567 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2023-07-21T22:38:48.567 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T22:38:48.567 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:48.567 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.568 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.568 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2023-07-21T22:38:48.568 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.568 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.568 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.568 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T22:38:48.568 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.568 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.569 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.569 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2023-07-21T22:38:48.569 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2023-07-21T22:38:48.569 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.569 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.569 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.569 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.570 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.570 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.570 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.570 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T22:38:48.570 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.570 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.570 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.571 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2023-07-21T22:38:48.571 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2023-07-21T22:38:48.571 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.571 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.571 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.571 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.571 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.572 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.572 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.572 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T22:38:48.572 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.572 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.572 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.572 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2023-07-21T22:38:48.573 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2023-07-21T22:38:48.573 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.573 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.573 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.573 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.573 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.573 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.573 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.574 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T22:38:48.574 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.574 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.574 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.574 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2023-07-21T22:38:48.574 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2023-07-21T22:38:48.575 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.575 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.575 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.575 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T22:38:48.575 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.575 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.575 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.575 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2023-07-21T22:38:48.576 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.576 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.576 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.576 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2023-07-21T22:38:48.576 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2023-07-21T22:38:48.576 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.576 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.577 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.577 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T22:38:48.577 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.577 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.577 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.577 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2023-07-21T22:38:48.577 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.578 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.578 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.578 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2023-07-21T22:38:48.578 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2023-07-21T22:38:48.578 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.578 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.579 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.579 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T22:38:48.579 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.579 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.579 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.579 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2023-07-21T22:38:48.579 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.579 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.580 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.580 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2023-07-21T22:38:48.580 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2023-07-21T22:38:48.580 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.580 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.580 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.581 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T22:38:48.581 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.581 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.581 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.581 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2023-07-21T22:38:48.581 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.581 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.581 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.582 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2023-07-21T22:38:48.582 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2023-07-21T22:38:48.582 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.582 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.582 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.582 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T22:38:48.583 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.583 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.583 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.583 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2023-07-21T22:38:48.583 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.583 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.583 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.584 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2023-07-21T22:38:48.584 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2023-07-21T22:38:48.584 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.584 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.584 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.584 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T22:38:48.584 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.585 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.585 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.585 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2023-07-21T22:38:48.585 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.585 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.585 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.586 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2023-07-21T22:38:48.586 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2023-07-21T22:38:48.586 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.586 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.586 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.586 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T22:38:48.586 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.587 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.587 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.587 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2023-07-21T22:38:48.587 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.587 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.587 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.587 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2023-07-21T22:38:48.588 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2023-07-21T22:38:48.588 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.588 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.588 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.588 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T22:38:48.588 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.588 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.588 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.589 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2023-07-21T22:38:48.589 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.589 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.589 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.589 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2023-07-21T22:38:48.589 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2023-07-21T22:38:48.590 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.590 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.590 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.590 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T22:38:48.590 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.590 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.590 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.591 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2023-07-21T22:38:48.591 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.591 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.591 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.591 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2023-07-21T22:38:48.591 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2023-07-21T22:38:48.592 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.592 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.592 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.592 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T22:38:48.592 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.592 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.592 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.593 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2023-07-21T22:38:48.593 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.593 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.593 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.593 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2023-07-21T22:38:48.593 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2023-07-21T22:38:48.593 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.594 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.594 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.594 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T22:38:48.594 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.594 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.594 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.594 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2023-07-21T22:38:48.594 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.595 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.595 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.595 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2023-07-21T22:38:48.595 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2023-07-21T22:38:48.595 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.595 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.596 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.596 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T22:38:48.596 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.596 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.596 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.596 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2023-07-21T22:38:48.596 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.597 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.597 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.597 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2023-07-21T22:38:48.597 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2023-07-21T22:38:48.597 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.597 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.597 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.598 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T22:38:48.598 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.598 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.598 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.598 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2023-07-21T22:38:48.598 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.598 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.598 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.599 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2023-07-21T22:38:48.599 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2023-07-21T22:38:48.599 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.599 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.599 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.599 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T22:38:48.600 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.600 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.600 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.600 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2023-07-21T22:38:48.600 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.600 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.600 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.601 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2023-07-21T22:38:48.601 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2023-07-21T22:38:48.601 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.601 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.601 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.601 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.601 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.602 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.602 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.602 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T22:38:48.602 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.602 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.602 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.602 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2023-07-21T22:38:48.603 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2023-07-21T22:38:48.603 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.603 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.603 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.603 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:48.603 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.603 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.604 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.604 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T22:38:48.604 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.604 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.604 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.604 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2023-07-21T22:38:48.604 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T22:38:48.605 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.605 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.605 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.605 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2023-07-21T22:38:48.605 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.605 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.605 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.606 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T22:38:48.606 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.606 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.606 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.606 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2023-07-21T22:38:48.606 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2023-07-21T22:38:48.606 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.607 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.607 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.607 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2023-07-21T22:38:48.607 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.607 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.607 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.607 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T22:38:48.607 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.608 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.608 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.608 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2023-07-21T22:38:48.608 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2023-07-21T22:38:48.608 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.608 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.609 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.609 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2023-07-21T22:38:48.609 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.609 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.609 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.609 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2023-07-21T22:38:48.609 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.609 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.610 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.610 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2023-07-21T22:38:48.610 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2023-07-21T22:38:48.610 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.610 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.610 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.611 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.611 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.611 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.611 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.611 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T22:38:48.611 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.611 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.611 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.612 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2023-07-21T22:38:48.612 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2023-07-21T22:38:48.612 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.612 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.612 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.612 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:48.612 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.613 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.613 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.613 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2023-07-21T22:38:48.613 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.613 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.613 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.614 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2023-07-21T22:38:48.614 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2023-07-21T22:38:48.614 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.614 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.614 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.614 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:48.614 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.614 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.615 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.615 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2023-07-21T22:38:48.615 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.615 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.615 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.615 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.615 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.616 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.616 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.616 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.616 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.616 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.616 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.616 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.617 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.617 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.617 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.617 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.617 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.617 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.617 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.617 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.618 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.618 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.618 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.618 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2023-07-21T22:38:48.618 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2023-07-21T22:38:48.618 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.618 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.619 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.619 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.619 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.619 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.619 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.619 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T22:38:48.619 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.620 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.620 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.620 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.620 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.620 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.620 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.621 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.621 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.621 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.621 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.621 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.621 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T22:38:48.621 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.621 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.622 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.622 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.622 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.622 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.622 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.622 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.623 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.623 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.623 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.623 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.623 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.623 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.623 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.623 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.624 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.624 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.624 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.624 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.624 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.624 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.624 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.625 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.625 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.625 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.625 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.625 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.625 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.625 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.626 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.626 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.626 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.626 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.626 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.626 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.626 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T22:38:48.627 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T22:38:48.627 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.627 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.627 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.627 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.627 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.628 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.628 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.628 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T22:38:48.628 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.628 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.628 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.629 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T22:38:48.629 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T22:38:48.629 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.629 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.629 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.630 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2023-07-21T22:38:48.630 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.630 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.630 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.630 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T22:38:48.630 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.631 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.631 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.631 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2023-07-21T22:38:48.631 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2023-07-21T22:38:48.632 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.632 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.632 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.632 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:48.632 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.633 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.633 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.633 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2023-07-21T22:38:48.633 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.633 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.633 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.634 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2023-07-21T22:38:48.634 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2023-07-21T22:38:48.634 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.634 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.634 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.634 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:48.634 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.634 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.635 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.635 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T22:38:48.635 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.635 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.635 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.635 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2023-07-21T22:38:48.635 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2023-07-21T22:38:48.636 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.636 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.636 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.636 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.636 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.636 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.636 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.637 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2023-07-21T22:38:48.637 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.637 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.637 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.637 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2023-07-21T22:38:48.637 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2023-07-21T22:38:48.637 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.638 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.638 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.638 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2023-07-21T22:38:48.638 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.638 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.638 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.638 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2023-07-21T22:38:48.639 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2023-07-21T22:38:48.639 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.639 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T22:38:48.639 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.639 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.639 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.639 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2023-07-21T22:38:48.640 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2023-07-21T22:38:48.640 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.640 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.640 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.640 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T22:38:48.640 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.640 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.641 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.641 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T22:38:48.641 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.641 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.641 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.641 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_web_password": { 2023-07-21T22:38:48.641 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_web_password", 2023-07-21T22:38:48.642 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.642 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.642 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.642 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:48.642 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.642 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.642 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.642 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus web password", 2023-07-21T22:38:48.643 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.643 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.643 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.643 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_web_user": { 2023-07-21T22:38:48.643 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_web_user", 2023-07-21T22:38:48.643 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.644 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.644 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.644 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:48.644 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.644 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.644 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.644 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus web user", 2023-07-21T22:38:48.644 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.645 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.645 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.645 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2023-07-21T22:38:48.645 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2023-07-21T22:38:48.645 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.645 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.645 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.646 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.646 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.646 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.646 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.646 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T22:38:48.646 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.646 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.647 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.647 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2023-07-21T22:38:48.647 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2023-07-21T22:38:48.647 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.647 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.647 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.647 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.648 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.648 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.648 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.648 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T22:38:48.648 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.648 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.648 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.649 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2023-07-21T22:38:48.649 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2023-07-21T22:38:48.649 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.649 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.649 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.649 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.649 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.650 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.650 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.650 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T22:38:48.650 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.650 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.650 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.651 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2023-07-21T22:38:48.651 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2023-07-21T22:38:48.651 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.651 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.651 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.651 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.651 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.651 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.652 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.652 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T22:38:48.652 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.652 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.652 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.652 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2023-07-21T22:38:48.653 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2023-07-21T22:38:48.653 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.653 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.653 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.653 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.653 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.653 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.654 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.654 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T22:38:48.654 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.654 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.654 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.654 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2023-07-21T22:38:48.654 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2023-07-21T22:38:48.655 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.655 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.655 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.655 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2023-07-21T22:38:48.655 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.655 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.655 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.655 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2023-07-21T22:38:48.656 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.656 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.656 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.656 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2023-07-21T22:38:48.656 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2023-07-21T22:38:48.656 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.657 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.657 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.657 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.657 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.657 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.657 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.657 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T22:38:48.658 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.658 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.658 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.658 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2023-07-21T22:38:48.658 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2023-07-21T22:38:48.658 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.658 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.659 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.659 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.659 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.659 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.659 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.659 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T22:38:48.659 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.660 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.660 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.660 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2023-07-21T22:38:48.660 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2023-07-21T22:38:48.660 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.660 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.661 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.661 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.661 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.661 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.661 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.661 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T22:38:48.661 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.661 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.662 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.662 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2023-07-21T22:38:48.662 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2023-07-21T22:38:48.662 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.662 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.662 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.663 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.663 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.663 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.663 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.663 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T22:38:48.663 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.663 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.663 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.664 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2023-07-21T22:38:48.664 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2023-07-21T22:38:48.664 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.664 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.664 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.664 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.664 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.665 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.665 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.665 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T22:38:48.665 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.665 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.665 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.666 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2023-07-21T22:38:48.666 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2023-07-21T22:38:48.666 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.666 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.666 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.666 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.666 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.666 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.667 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.667 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T22:38:48.667 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.667 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.667 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.667 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.667 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.668 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.668 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2023-07-21T22:38:48.668 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.668 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.668 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.668 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.668 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.669 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.669 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.669 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.669 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.669 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.669 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.669 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.669 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.670 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.670 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.670 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.670 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.670 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.670 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.670 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.670 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.671 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.671 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.671 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.671 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.671 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.671 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.672 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.672 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.672 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.672 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.672 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.672 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.672 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.672 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.673 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.673 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.673 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.673 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.673 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.673 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.674 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.674 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.674 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.674 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.674 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.674 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.674 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.674 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.675 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.675 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.675 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.675 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.675 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.675 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.675 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.676 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.676 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.676 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.676 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.676 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.676 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.676 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.677 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.677 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.677 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.677 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.677 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.677 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.677 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2023-07-21T22:38:48.678 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2023-07-21T22:38:48.678 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.678 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.678 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.678 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2023-07-21T22:38:48.678 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.678 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.678 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.679 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T22:38:48.679 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.679 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.679 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.679 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2023-07-21T22:38:48.679 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2023-07-21T22:38:48.680 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.680 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.680 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.680 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2023-07-21T22:38:48.680 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.680 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.680 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.680 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T22:38:48.681 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.681 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.681 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.681 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.681 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.681 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.682 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2023-07-21T22:38:48.682 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.682 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.682 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.682 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T22:38:48.682 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T22:38:48.682 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.682 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.683 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.683 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:48.683 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.683 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.683 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.683 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.683 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.683 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.684 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.684 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T22:38:48.684 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T22:38:48.684 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.684 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.684 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.685 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.685 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.685 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.685 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.685 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.685 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.685 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.685 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.686 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T22:38:48.686 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T22:38:48.686 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.686 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.686 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.686 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.686 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.687 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.687 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.687 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.687 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.687 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.687 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.687 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2023-07-21T22:38:48.688 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T22:38:48.688 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.688 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.688 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.688 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.688 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.688 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.689 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.689 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.689 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.689 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.689 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.689 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T22:38:48.690 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T22:38:48.690 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.690 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.690 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.690 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.690 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.690 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.690 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.691 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.691 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.691 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.691 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.691 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T22:38:48.691 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T22:38:48.691 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.692 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.692 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.692 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.692 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.692 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.692 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.692 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.693 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.693 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.693 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.693 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T22:38:48.693 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T22:38:48.693 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.693 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.694 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.694 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.694 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.694 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.694 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.694 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.694 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.694 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.695 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.695 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T22:38:48.695 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T22:38:48.695 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.695 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.695 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.695 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.696 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.696 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.696 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.696 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.696 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.696 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.696 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.697 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T22:38:48.697 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T22:38:48.697 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.697 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.697 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.697 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.697 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.698 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.698 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.698 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.698 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.698 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.698 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.699 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T22:38:48.699 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T22:38:48.699 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.699 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.699 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.699 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.699 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.699 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.700 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.700 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.700 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.700 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.700 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.700 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T22:38:48.700 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T22:38:48.701 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.701 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.701 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.701 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.701 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.701 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.701 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.702 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.702 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.702 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.702 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.702 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T22:38:48.702 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T22:38:48.702 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.703 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.703 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.703 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.703 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.703 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.703 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.703 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.703 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.704 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.704 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.704 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T22:38:48.704 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T22:38:48.704 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.704 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.704 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.705 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.705 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.705 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.705 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.705 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.705 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.705 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.706 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.706 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T22:38:48.706 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T22:38:48.706 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.706 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.706 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.706 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.707 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.707 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.707 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.707 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.707 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.707 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.707 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.708 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T22:38:48.708 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T22:38:48.708 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.708 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.708 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.708 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:48.708 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.709 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.709 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.709 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.709 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.709 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.709 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.709 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T22:38:48.710 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T22:38:48.710 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.710 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.710 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.710 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.710 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.710 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.710 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.711 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.711 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.711 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.711 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.711 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2023-07-21T22:38:48.711 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2023-07-21T22:38:48.712 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.712 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.712 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.712 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.712 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.712 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.712 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.712 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.713 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.713 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.713 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.713 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T22:38:48.713 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T22:38:48.713 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.714 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.714 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.714 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:48.714 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.714 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.714 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.714 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.714 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.715 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.715 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.715 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T22:38:48.715 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T22:38:48.715 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.715 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.715 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.716 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.716 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.716 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.716 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.716 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.716 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.716 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.716 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.717 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T22:38:48.717 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T22:38:48.717 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.717 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.717 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.717 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.718 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.718 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.718 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.718 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.718 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.718 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.718 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.719 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T22:38:48.719 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T22:38:48.719 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.719 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.719 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.719 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.719 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.720 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.720 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.720 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.720 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.720 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.720 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.720 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T22:38:48.721 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T22:38:48.721 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.721 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.721 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.721 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.721 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.721 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.722 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.722 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.722 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.722 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.722 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.722 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T22:38:48.722 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T22:38:48.723 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.723 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.723 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.723 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.723 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.723 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.723 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.723 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.724 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.724 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.724 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.724 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T22:38:48.724 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T22:38:48.724 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.725 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.725 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.725 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.725 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.725 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.725 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.725 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.725 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.726 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.726 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.726 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T22:38:48.726 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T22:38:48.726 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.726 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.726 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.727 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.727 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.727 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.727 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.727 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.727 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.727 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.728 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.728 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T22:38:48.728 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T22:38:48.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.728 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.728 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.728 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.729 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.729 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.729 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.729 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.729 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.729 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.729 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.730 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T22:38:48.730 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T22:38:48.730 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.730 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.730 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.730 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.730 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.731 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.731 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.731 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.731 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.731 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.731 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.731 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T22:38:48.732 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T22:38:48.732 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.732 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.732 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.732 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.732 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.732 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.733 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.733 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.733 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.733 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.733 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.733 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T22:38:48.733 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T22:38:48.734 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.734 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.734 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.734 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.734 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.734 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.734 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.734 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.735 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.735 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.735 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.735 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T22:38:48.735 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T22:38:48.735 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.736 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.736 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.736 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.736 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.736 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.736 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.736 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.736 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.737 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.737 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.737 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T22:38:48.737 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T22:38:48.737 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.737 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.738 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.738 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.738 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.738 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.738 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.738 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.738 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.738 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.739 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.739 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T22:38:48.739 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T22:38:48.739 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.739 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.739 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.739 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.740 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.740 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.740 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.740 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.740 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.740 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.740 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.741 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T22:38:48.741 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T22:38:48.741 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.741 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.741 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.741 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T22:38:48.741 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.742 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.742 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.742 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.742 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.742 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.742 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.743 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T22:38:48.743 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T22:38:48.743 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.743 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.743 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.743 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:48.743 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.744 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.744 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.744 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.744 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.744 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.744 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T22:38:48.745 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T22:38:48.745 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.745 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.745 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.745 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2023-07-21T22:38:48.745 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.745 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.745 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.746 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.746 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.746 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.746 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.746 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T22:38:48.746 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T22:38:48.747 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.747 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.747 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2023-07-21T22:38:48.747 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.747 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.747 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.747 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.748 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.748 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.748 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.748 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T22:38:48.748 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T22:38:48.748 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.749 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.749 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.749 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.749 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.749 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.749 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.749 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.749 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.750 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.750 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.750 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T22:38:48.750 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T22:38:48.750 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.750 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.751 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.751 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2023-07-21T22:38:48.751 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.751 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.751 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.751 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T22:38:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T22:38:48.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.753 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.753 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.753 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.753 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.753 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.753 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.754 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.754 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.754 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.754 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T22:38:48.754 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T22:38:48.754 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.755 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.755 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.755 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.755 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.755 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.755 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.755 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.755 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.756 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.756 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.756 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T22:38:48.756 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T22:38:48.756 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.756 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.757 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.757 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:48.757 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.757 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.757 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.757 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.757 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.758 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.758 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.758 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T22:38:48.758 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T22:38:48.758 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.758 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.759 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.759 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:48.759 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.759 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.759 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.759 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.759 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.760 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.760 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.760 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T22:38:48.760 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T22:38:48.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.761 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.761 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.761 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2023-07-21T22:38:48.761 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.761 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.761 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.761 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.762 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.762 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.762 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.762 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2023-07-21T22:38:48.762 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2023-07-21T22:38:48.762 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.763 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.763 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.763 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.763 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.763 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.763 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.763 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.763 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.764 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.764 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.764 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2023-07-21T22:38:48.764 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2023-07-21T22:38:48.764 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.764 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.765 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.765 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.765 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.765 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.765 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.765 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.765 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.765 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.766 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.766 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2023-07-21T22:38:48.766 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2023-07-21T22:38:48.766 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.766 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.766 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.766 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.767 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.767 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.767 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.767 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2023-07-21T22:38:48.767 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.767 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.767 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.768 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2023-07-21T22:38:48.768 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2023-07-21T22:38:48.768 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.768 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.768 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.768 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2023-07-21T22:38:48.769 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.769 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.769 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.769 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.769 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.769 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.770 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2023-07-21T22:38:48.770 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2023-07-21T22:38:48.770 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.770 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.770 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.770 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.770 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.771 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.771 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.771 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.771 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.771 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.771 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.772 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.772 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.772 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.772 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.772 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.773 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.773 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.773 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.773 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.773 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.773 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.773 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.773 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.774 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.774 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.774 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.774 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.774 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.774 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.774 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.775 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.775 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.775 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.775 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.775 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.775 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.776 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.776 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.776 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.776 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.776 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.776 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.776 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.777 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.777 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.777 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.777 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.777 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.777 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.777 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.778 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.778 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.778 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.778 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.778 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.778 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.778 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.778 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.779 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.779 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.779 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.779 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.779 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.779 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.780 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.780 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.780 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.780 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.780 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.780 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.780 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.781 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.781 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.781 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.781 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.781 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.781 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2023-07-21T22:38:48.781 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2023-07-21T22:38:48.782 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.782 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.782 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.782 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.782 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.782 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.782 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.782 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2023-07-21T22:38:48.783 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.783 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.783 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.783 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2023-07-21T22:38:48.783 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T22:38:48.783 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.783 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.784 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.784 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.784 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.784 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.784 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.784 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.784 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.785 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.785 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.785 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2023-07-21T22:38:48.785 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2023-07-21T22:38:48.785 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.785 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.785 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.786 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2023-07-21T22:38:48.786 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.786 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.786 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.786 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.786 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.786 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.786 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.787 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2023-07-21T22:38:48.787 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2023-07-21T22:38:48.787 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.787 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.787 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.787 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2023-07-21T22:38:48.788 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.788 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.788 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.788 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.788 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.788 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.788 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.789 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2023-07-21T22:38:48.789 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2023-07-21T22:38:48.789 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.789 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.789 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.789 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.789 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.790 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.790 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.790 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.790 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.790 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.790 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.790 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2023-07-21T22:38:48.791 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2023-07-21T22:38:48.791 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.791 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.791 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.791 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2023-07-21T22:38:48.791 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.791 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.792 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.792 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.792 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.792 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.792 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.792 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2023-07-21T22:38:48.792 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2023-07-21T22:38:48.793 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.793 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.793 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.793 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2023-07-21T22:38:48.793 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.793 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.793 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.794 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.794 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2023-07-21T22:38:48.794 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.794 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.794 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.794 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.794 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.795 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2023-07-21T22:38:48.795 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2023-07-21T22:38:48.795 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.795 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.795 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.795 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2023-07-21T22:38:48.795 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2023-07-21T22:38:48.795 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2023-07-21T22:38:48.796 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.796 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.796 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.796 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.796 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.796 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2023-07-21T22:38:48.797 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2023-07-21T22:38:48.797 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.797 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.797 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.797 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.797 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.797 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.797 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.798 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.798 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.798 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.798 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.798 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.798 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.799 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2023-07-21T22:38:48.799 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.799 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.799 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.799 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2023-07-21T22:38:48.799 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2023-07-21T22:38:48.799 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.800 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.800 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.800 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.800 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.800 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.800 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.800 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2023-07-21T22:38:48.801 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.801 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.801 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.801 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.801 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.801 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.801 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.802 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.802 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.802 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.802 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.802 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.802 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.802 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.802 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.803 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.803 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.803 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.803 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.803 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.803 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.803 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.804 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.804 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.804 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.804 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.804 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.804 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.804 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.805 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.805 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.805 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.805 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.805 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.805 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.806 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.806 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.806 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.806 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.806 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.806 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.806 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.807 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.807 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.807 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.807 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.807 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.807 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.807 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.808 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.808 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.808 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.808 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.808 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.808 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.808 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.809 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.809 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.809 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.809 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.809 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.809 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.810 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.810 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.810 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.810 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.810 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.810 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.810 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2023-07-21T22:38:48.811 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2023-07-21T22:38:48.811 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.811 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.811 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.811 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2023-07-21T22:38:48.811 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.811 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.812 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.812 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T22:38:48.812 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.812 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.812 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.812 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2023-07-21T22:38:48.812 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2023-07-21T22:38:48.813 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.813 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.813 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.813 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2023-07-21T22:38:48.813 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.813 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.813 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.814 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T22:38:48.814 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.814 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.814 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.814 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2023-07-21T22:38:48.814 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2023-07-21T22:38:48.814 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.815 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.815 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.815 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2023-07-21T22:38:48.815 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.815 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.815 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.815 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2023-07-21T22:38:48.815 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.816 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.816 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.816 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2023-07-21T22:38:48.816 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2023-07-21T22:38:48.816 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.816 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.817 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.817 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2023-07-21T22:38:48.817 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.817 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.817 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.817 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T22:38:48.817 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.817 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.818 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.818 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2023-07-21T22:38:48.818 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2023-07-21T22:38:48.818 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.818 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.818 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.819 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:48.819 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.819 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.819 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.819 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T22:38:48.819 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.819 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.819 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.820 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:48.820 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:48.820 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.820 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.820 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.820 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:48.821 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.821 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.821 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.821 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T22:38:48.821 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.821 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.822 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2023-07-21T22:38:48.822 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2023-07-21T22:38:48.822 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.822 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.822 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.822 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2023-07-21T22:38:48.822 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.822 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.823 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.823 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T22:38:48.823 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.823 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.823 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.823 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.823 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.824 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2023-07-21T22:38:48.824 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.824 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.824 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.824 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.824 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.825 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.825 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.825 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.825 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.825 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.825 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.825 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.826 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.826 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.826 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.826 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.826 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.826 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.826 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.827 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.827 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.827 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.827 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.827 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.827 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.827 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.828 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.828 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.828 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.828 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.828 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.828 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.828 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.829 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.829 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.829 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.829 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.829 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.829 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.829 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.830 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.830 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.830 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.830 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.830 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.830 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.830 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.831 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.831 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.831 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.831 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.831 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.831 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.831 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.832 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.832 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.832 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.832 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.832 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.832 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.832 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.833 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.833 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.833 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.833 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.833 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.833 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.833 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.834 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2023-07-21T22:38:48.834 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2023-07-21T22:38:48.834 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.834 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.834 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.834 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2023-07-21T22:38:48.834 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2023-07-21T22:38:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2023-07-21T22:38:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2023-07-21T22:38:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.837 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.837 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.837 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.837 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.837 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.837 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:48.837 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:48.838 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.838 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.838 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.838 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2023-07-21T22:38:48.838 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.838 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.838 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.839 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.839 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.839 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.839 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.839 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.839 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.840 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2023-07-21T22:38:48.840 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2023-07-21T22:38:48.840 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2023-07-21T22:38:48.840 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.840 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2023-07-21T22:38:48.840 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2023-07-21T22:38:48.840 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.841 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.841 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.841 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2023-07-21T22:38:48.841 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.841 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.841 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.841 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T22:38:48.841 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.842 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.842 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.842 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2023-07-21T22:38:48.842 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2023-07-21T22:38:48.842 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.842 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.842 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.843 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2023-07-21T22:38:48.843 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.843 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.843 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.843 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T22:38:48.843 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.843 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.844 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.844 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2023-07-21T22:38:48.844 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2023-07-21T22:38:48.844 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.844 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.844 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.845 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.845 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.845 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.845 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.845 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2023-07-21T22:38:48.845 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.845 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.845 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.846 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:48.846 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:48.846 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:48.846 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.846 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.846 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2023-07-21T22:38:48.846 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2023-07-21T22:38:48.847 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.847 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.847 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T22:38:48.847 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.847 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.847 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.847 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.848 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.848 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.848 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.848 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.848 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.848 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.848 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.849 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.849 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.849 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.849 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.849 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.849 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.849 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.850 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.850 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.850 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.850 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.850 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.850 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.850 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.851 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.851 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.851 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.851 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.851 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.851 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.851 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.851 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.852 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.852 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.852 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.852 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.852 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.852 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.853 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.853 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.853 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.853 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.853 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.853 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.853 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.853 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.854 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.854 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.854 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.854 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.854 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.854 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.854 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.854 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2023-07-21T22:38:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2023-07-21T22:38:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2023-07-21T22:38:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2023-07-21T22:38:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2023-07-21T22:38:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2023-07-21T22:38:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2023-07-21T22:38:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2023-07-21T22:38:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2023-07-21T22:38:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2023-07-21T22:38:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:38:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2023-07-21T22:38:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2023-07-21T22:38:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2023-07-21T22:38:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2023-07-21T22:38:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2023-07-21T22:38:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T22:38:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2023-07-21T22:38:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2023-07-21T22:38:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2023-07-21T22:38:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2023-07-21T22:38:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2023-07-21T22:38:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2023-07-21T22:38:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:38:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2023-07-21T22:38:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.870 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.870 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.870 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.870 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.870 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.870 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.870 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.871 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.871 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.871 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.871 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.871 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.871 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.871 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.871 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.872 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.872 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.872 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.872 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.872 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.872 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.872 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.873 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.873 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.873 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.873 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.873 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.873 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.874 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.874 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.874 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.874 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.874 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.874 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.874 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.874 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.875 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.875 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.875 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.875 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.875 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.875 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.875 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.875 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.876 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.876 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.876 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.876 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.876 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.876 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.876 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.877 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.877 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.877 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.877 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.877 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.877 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.877 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.878 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.878 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.878 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.878 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.878 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.878 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.879 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2023-07-21T22:38:48.879 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.879 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.879 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.879 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.879 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.879 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.880 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.880 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.880 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.880 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.880 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.880 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.880 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.880 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.881 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.881 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.881 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.881 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.881 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.881 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.881 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.882 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.882 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.882 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.882 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.882 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.882 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.882 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.883 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.883 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.883 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.883 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.883 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.883 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.883 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.884 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.884 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.884 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.884 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.884 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.884 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.885 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.885 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.885 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.885 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.885 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.885 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.885 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.885 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.886 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.886 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.886 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.886 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.886 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.886 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.887 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.887 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.887 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.887 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.887 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.887 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.887 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.887 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.888 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.888 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.888 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.888 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.888 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.888 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.888 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.889 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2023-07-21T22:38:48.889 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.889 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.889 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.889 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2023-07-21T22:38:48.889 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2023-07-21T22:38:48.889 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.890 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.890 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.890 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2023-07-21T22:38:48.890 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.890 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.890 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.890 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T22:38:48.890 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.891 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.891 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.891 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2023-07-21T22:38:48.891 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2023-07-21T22:38:48.891 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.891 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.892 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.892 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:48.892 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2023-07-21T22:38:48.892 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.892 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.892 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T22:38:48.892 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.893 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.893 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.893 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.893 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.893 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.893 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.893 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.894 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.894 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.894 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.894 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.894 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.894 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.894 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.894 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.895 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.895 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.895 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.895 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.895 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.895 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.895 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.896 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.896 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.896 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.896 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.896 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.896 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.896 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.897 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.897 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.897 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.897 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.897 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.897 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.897 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.898 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.898 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.898 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.898 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.898 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.898 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.898 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.899 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.899 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.899 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.899 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.899 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.899 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.899 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.899 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.900 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.900 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.900 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.900 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.900 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.900 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.900 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.901 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.901 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.901 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.901 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.901 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.901 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.901 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.902 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.902 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.902 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.902 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.902 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.902 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.902 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2023-07-21T22:38:48.903 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.903 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.903 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.903 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2023-07-21T22:38:48.903 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2023-07-21T22:38:48.903 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.903 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.903 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.904 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2023-07-21T22:38:48.904 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.904 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.904 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.904 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2023-07-21T22:38:48.904 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T22:38:48.904 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.905 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.905 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.905 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.905 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.905 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.905 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.905 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.906 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.906 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.906 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.906 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.906 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.906 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.906 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.906 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.907 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.907 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.907 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.907 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.907 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.907 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.907 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.908 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.908 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.908 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.908 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.908 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.908 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.908 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.909 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.909 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.909 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.909 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.909 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.909 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.909 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.910 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.910 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.910 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.910 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.910 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.910 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.910 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.911 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.911 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.911 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.911 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.911 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.911 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.911 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.911 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.912 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.912 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.912 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.912 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.912 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.912 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.912 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.913 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.913 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.913 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.913 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.913 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.913 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.913 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.914 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.914 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.914 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2023-07-21T22:38:48.914 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2023-07-21T22:38:48.914 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.914 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.914 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.915 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.915 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.915 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.915 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.915 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2023-07-21T22:38:48.915 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T22:38:48.915 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.915 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.916 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2023-07-21T22:38:48.916 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2023-07-21T22:38:48.916 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.916 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.916 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.916 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2023-07-21T22:38:48.917 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.917 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.917 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.917 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2023-07-21T22:38:48.917 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.917 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.917 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.918 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2023-07-21T22:38:48.918 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2023-07-21T22:38:48.918 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:48.918 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.918 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.918 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2023-07-21T22:38:48.918 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.918 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.919 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.919 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T22:38:48.919 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.919 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.919 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.919 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2023-07-21T22:38:48.920 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2023-07-21T22:38:48.920 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.920 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.920 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.920 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.920 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.920 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.921 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.921 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2023-07-21T22:38:48.921 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.921 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.921 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.921 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2023-07-21T22:38:48.921 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2023-07-21T22:38:48.922 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.922 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.922 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.922 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2023-07-21T22:38:48.922 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.922 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.922 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.923 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T22:38:48.923 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T22:38:48.923 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.923 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.923 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.923 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.924 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2023-07-21T22:38:48.924 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.924 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.924 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.924 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.924 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.924 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.925 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.925 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.925 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.925 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.925 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.925 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.925 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.925 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.926 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.926 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.926 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.926 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.926 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.926 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.926 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.926 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.927 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.927 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.927 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.927 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.927 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.927 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.928 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.928 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.928 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.928 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.928 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.928 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.928 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.928 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.929 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.929 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.929 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.929 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.929 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.929 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.929 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.930 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.930 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.930 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.930 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.930 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.930 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.930 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.931 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.931 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.931 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.931 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.931 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.931 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.931 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.932 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.932 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.932 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.932 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.932 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.932 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.932 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.932 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.933 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.933 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.933 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.933 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.933 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.933 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.934 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.934 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2023-07-21T22:38:48.934 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.934 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.934 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.934 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.934 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.934 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.935 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.935 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.935 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.935 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.935 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.935 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.935 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.935 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.936 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.936 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.936 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.936 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.936 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.936 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.936 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.937 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.937 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.937 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.937 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.937 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.937 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.937 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.938 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.938 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.938 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.938 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.938 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.938 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.938 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.939 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.939 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.939 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.939 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.939 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.939 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.939 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.940 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.940 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.940 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.940 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.940 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.940 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.940 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.941 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.941 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.941 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.941 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.941 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.941 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.941 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.942 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.942 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.942 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.942 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.942 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.942 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.942 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.942 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.943 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.943 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.943 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.943 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.943 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.943 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.943 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.944 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2023-07-21T22:38:48.944 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.944 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.944 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.944 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.945 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.945 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.945 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.945 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.945 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.945 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.945 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.946 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.946 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.946 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.946 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.946 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.946 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.946 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.946 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.947 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.947 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.947 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.947 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.947 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.947 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.948 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.948 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.948 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.948 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.948 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.948 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.948 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.948 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.949 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.949 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.949 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.949 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.949 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.949 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.950 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.950 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.950 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.950 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.950 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.950 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.950 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.950 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.951 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.951 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.951 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.951 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.951 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.951 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.951 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.951 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.952 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.952 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.952 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.952 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.952 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.952 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.953 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.953 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.953 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.953 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.953 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.953 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.953 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.953 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.954 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.954 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.954 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.954 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2023-07-21T22:38:48.954 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.954 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.955 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.955 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2023-07-21T22:38:48.955 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2023-07-21T22:38:48.955 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.955 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.955 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.955 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.955 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.956 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.956 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.956 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T22:38:48.956 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.956 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.956 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.957 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.957 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.957 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.957 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.957 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.957 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.957 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.957 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.958 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.958 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.958 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.958 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.958 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.958 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.958 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.958 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.959 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.959 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.959 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.959 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.959 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.959 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.960 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.960 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.960 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.960 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.960 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.960 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.960 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.960 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.961 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.961 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.961 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.961 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.961 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.961 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.962 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.962 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.962 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.962 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.962 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.962 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.962 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.962 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.963 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.963 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.963 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.963 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.963 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.963 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.963 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.963 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.964 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.964 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.964 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.964 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.964 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.964 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.965 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.965 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.965 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.965 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.965 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.965 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.965 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.965 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.966 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2023-07-21T22:38:48.966 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2023-07-21T22:38:48.966 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.966 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.966 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.966 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.966 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.967 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.967 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.967 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2023-07-21T22:38:48.967 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2023-07-21T22:38:48.967 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2023-07-21T22:38:48.967 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.967 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2023-07-21T22:38:48.968 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.968 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.968 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.968 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.968 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2023-07-21T22:38:48.969 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.969 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.969 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.969 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.969 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.969 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.969 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.970 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.970 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.970 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.970 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.970 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.970 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.970 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.970 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.971 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.971 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.971 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.971 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.971 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.971 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.971 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.971 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.972 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.972 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.972 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.972 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.972 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.972 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.973 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.973 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.973 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.973 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.973 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.973 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.973 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.974 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.974 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.974 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.974 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.974 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.974 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.974 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.974 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.975 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.975 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.975 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.975 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.975 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.975 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.975 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.976 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.976 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.976 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.976 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.976 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.976 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.976 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.977 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.977 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.977 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.977 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.977 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.977 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.978 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.978 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.978 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.978 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.978 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.978 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.979 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2023-07-21T22:38:48.979 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.979 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.979 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.979 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.979 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.979 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.979 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.980 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.980 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.980 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.980 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.980 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.980 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.980 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.980 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.981 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.981 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.981 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.981 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.981 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.981 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.981 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.982 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.982 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.982 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.982 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.982 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.982 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.982 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.983 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.983 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.983 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.983 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.983 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.983 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.983 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.984 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.984 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.984 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.984 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.984 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.984 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.984 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.985 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.985 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.985 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.985 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.985 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.985 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.985 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.985 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.986 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.986 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.986 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.986 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.986 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.986 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.986 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.987 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.987 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.987 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.987 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.987 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.987 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.987 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.988 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.988 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.988 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.988 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.988 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.988 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:48.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.988 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:48.989 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2023-07-21T22:38:48.989 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:48.989 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:48.989 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:48.989 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:48.989 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:48.989 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.989 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.990 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.990 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:48.990 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.990 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.990 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.990 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.990 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.991 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.991 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.991 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.991 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.991 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.991 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.991 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.991 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.992 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.992 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:48.992 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:48.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.992 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.992 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.993 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.993 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.993 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.993 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.993 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.993 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.993 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.993 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.994 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:48.994 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:48.994 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:48.994 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.994 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.994 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:48.994 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.995 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.995 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:48.995 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:48.995 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:48.995 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:48.995 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:48.995 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:48.995 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:48.996 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:48.996 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.996 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.996 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.996 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.996 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:48.997 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:48.997 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.997 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.997 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:48.997 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.997 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.997 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:48.997 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:48.998 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:48.998 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:48.998 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:48.998 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:48.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:48.998 INFO:teuthology.orchestra.run.smithi107.stdout: "noautoscale": { 2023-07-21T22:38:48.998 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "noautoscale", 2023-07-21T22:38:48.999 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:48.999 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:48.999 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:48.999 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:48.999 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:48.999 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.000 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.000 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "global autoscale flag", 2023-07-21T22:38:49.000 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T22:38:49.000 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.000 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.001 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:49.001 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:49.001 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:49.001 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.001 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.002 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:49.002 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.002 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.002 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.002 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.002 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.003 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.003 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.003 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2023-07-21T22:38:49.003 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2023-07-21T22:38:49.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:49.004 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.004 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2023-07-21T22:38:49.004 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2023-07-21T22:38:49.004 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.004 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.005 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2023-07-21T22:38:49.005 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T22:38:49.005 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.005 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.005 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.006 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.006 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2023-07-21T22:38:49.006 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.006 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.007 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.007 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2023-07-21T22:38:49.007 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2023-07-21T22:38:49.007 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.007 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.008 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.008 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.008 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.008 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.008 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.008 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T22:38:49.008 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.009 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.009 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.009 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2023-07-21T22:38:49.009 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2023-07-21T22:38:49.009 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.009 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.010 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.010 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:49.010 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.010 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.010 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.010 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.010 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.010 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.011 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.011 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.011 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.011 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.011 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.011 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.011 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.012 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.012 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.012 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.012 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.012 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.012 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.012 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.013 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.013 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.013 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.013 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.013 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.013 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.013 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.014 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.014 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.014 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.014 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.014 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.014 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.014 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.014 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.015 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.015 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.015 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.015 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.015 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.015 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.015 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.016 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.016 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.016 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.016 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.016 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.016 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.016 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.017 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.017 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.017 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.017 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.017 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.017 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.017 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.017 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.018 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.018 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.018 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.018 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.018 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.018 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.018 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.019 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.019 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.019 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.019 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.019 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.019 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.019 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.020 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.020 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.020 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2023-07-21T22:38:49.020 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2023-07-21T22:38:49.020 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:49.020 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.020 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.021 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2023-07-21T22:38:49.021 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.021 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.021 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.021 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2023-07-21T22:38:49.021 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.021 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.021 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.022 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2023-07-21T22:38:49.022 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2023-07-21T22:38:49.022 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:49.022 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.022 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.022 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2023-07-21T22:38:49.023 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.023 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.023 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.023 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2023-07-21T22:38:49.023 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.023 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.023 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.023 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.024 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.024 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.024 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2023-07-21T22:38:49.024 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.024 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.024 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.024 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2023-07-21T22:38:49.025 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2023-07-21T22:38:49.025 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.025 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.025 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.025 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:49.025 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.025 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.026 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.026 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.026 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.026 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.026 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.026 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2023-07-21T22:38:49.026 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2023-07-21T22:38:49.027 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.027 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.027 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.027 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:49.027 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.027 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.027 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.027 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T22:38:49.028 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T22:38:49.028 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.028 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.028 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.028 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.029 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.029 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.029 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.029 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.029 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.029 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.029 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.029 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.030 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.030 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.030 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.030 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.030 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.030 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.030 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.031 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.031 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.031 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.031 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.031 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.031 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.031 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.032 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.032 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.032 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.032 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.032 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.032 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.032 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.033 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.033 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.033 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.033 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.033 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.033 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.033 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.034 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.034 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.034 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.034 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.034 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.034 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.034 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.035 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.035 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.035 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.035 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.035 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.035 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.035 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.035 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.036 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.036 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.036 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.036 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.036 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.036 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.036 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.037 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.037 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.037 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.037 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.037 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.037 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.038 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2023-07-21T22:38:49.038 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2023-07-21T22:38:49.038 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.038 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.038 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.038 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.038 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.038 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.039 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.039 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.039 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.039 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.039 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.039 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T22:38:49.039 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T22:38:49.040 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:49.040 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.040 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2023-07-21T22:38:49.040 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.040 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.040 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.040 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.041 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.041 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.041 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.041 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2023-07-21T22:38:49.041 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2023-07-21T22:38:49.041 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:49.042 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.042 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.042 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2023-07-21T22:38:49.042 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.042 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.042 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.042 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.042 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.043 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.043 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.043 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2023-07-21T22:38:49.043 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2023-07-21T22:38:49.043 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.043 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.043 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.044 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2023-07-21T22:38:49.044 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.044 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.044 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.044 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T22:38:49.044 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.044 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.045 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.045 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2023-07-21T22:38:49.045 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2023-07-21T22:38:49.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:49.045 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.045 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2023-07-21T22:38:49.046 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.046 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.046 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.046 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T22:38:49.046 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.046 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.047 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.047 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2023-07-21T22:38:49.047 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2023-07-21T22:38:49.048 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.048 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.048 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.048 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2023-07-21T22:38:49.049 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.049 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.049 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.049 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.050 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.050 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.050 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.050 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2023-07-21T22:38:49.050 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2023-07-21T22:38:49.051 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.051 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.051 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.051 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2023-07-21T22:38:49.051 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.051 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.051 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.052 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2023-07-21T22:38:49.052 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2023-07-21T22:38:49.052 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.052 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.052 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.052 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.052 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.053 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2023-07-21T22:38:49.053 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2023-07-21T22:38:49.053 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:49.053 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.053 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.053 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2023-07-21T22:38:49.053 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2023-07-21T22:38:49.054 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2023-07-21T22:38:49.054 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.054 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.054 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.054 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.054 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.054 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.054 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.055 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.055 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2023-07-21T22:38:49.055 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.055 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.055 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.055 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.056 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.056 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.056 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.056 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.056 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.056 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.056 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.057 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.057 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.057 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.057 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.057 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.057 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.057 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.058 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.058 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.058 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.058 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.058 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.058 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.058 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.059 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.059 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.059 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.059 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.059 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.059 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.059 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.059 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.060 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.060 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.060 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.060 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.060 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.061 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.061 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.061 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.061 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.061 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.061 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.061 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.062 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.062 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.062 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.062 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.062 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.062 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.062 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.062 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.063 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.063 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.063 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.063 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.063 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.063 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.063 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.064 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.064 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.064 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.064 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.064 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.064 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.065 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2023-07-21T22:38:49.065 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2023-07-21T22:38:49.065 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:49.065 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.065 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.065 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2023-07-21T22:38:49.065 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.066 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.066 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.066 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.066 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.066 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.066 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.067 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2023-07-21T22:38:49.067 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T22:38:49.067 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.067 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.067 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.067 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.067 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.067 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.068 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.068 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.068 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.068 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.068 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.068 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2023-07-21T22:38:49.068 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2023-07-21T22:38:49.069 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.069 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.069 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.069 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.069 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.069 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.069 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.070 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.070 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.070 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.070 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.070 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.070 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.071 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2023-07-21T22:38:49.071 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.071 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.071 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.071 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2023-07-21T22:38:49.071 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2023-07-21T22:38:49.071 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.071 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.072 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.072 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:49.072 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.072 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.072 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.072 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.072 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.073 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.073 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.073 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2023-07-21T22:38:49.073 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2023-07-21T22:38:49.073 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.073 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.073 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.074 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.074 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.074 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.074 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.074 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.074 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.074 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.074 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.075 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.075 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.075 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.075 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.075 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.075 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.076 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.076 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.076 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.076 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.076 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.076 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.076 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.076 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.077 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.077 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.077 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.077 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.077 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.077 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.077 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.078 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.078 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.078 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.078 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.078 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.078 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.078 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.079 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.079 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.079 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.079 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.079 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.079 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.079 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.080 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.080 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.080 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.080 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.080 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.080 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.080 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.080 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.081 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.081 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.081 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.081 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.081 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.081 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.081 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.082 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.082 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.082 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.082 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.082 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.082 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.082 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.083 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.083 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.083 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.083 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.083 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.083 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.083 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.083 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.084 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.084 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2023-07-21T22:38:49.084 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2023-07-21T22:38:49.084 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.084 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.084 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.085 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.085 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.085 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.085 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.085 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.085 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.085 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.085 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.086 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2023-07-21T22:38:49.086 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2023-07-21T22:38:49.086 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.086 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.086 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.086 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.086 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.087 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.087 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.087 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.087 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.087 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.087 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.087 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.087 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.088 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2023-07-21T22:38:49.088 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.088 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.088 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.088 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.089 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.089 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.089 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.089 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.089 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.089 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.089 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.090 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.090 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.090 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.090 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.090 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.090 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.090 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.090 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.091 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.091 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.091 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.091 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.091 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.091 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.091 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.092 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.092 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.092 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.092 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.092 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.092 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.092 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.092 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.093 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.094 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.094 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.094 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.094 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.094 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.094 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.094 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.094 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.095 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.095 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.095 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.095 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.095 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.095 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.095 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.095 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.096 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.096 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.096 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.096 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.096 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.096 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.097 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.097 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.097 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.097 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.097 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.097 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.097 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.097 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.098 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.098 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2023-07-21T22:38:49.098 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.098 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.098 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.098 INFO:teuthology.orchestra.run.smithi107.stdout: "drive_group_interval": { 2023-07-21T22:38:49.099 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "drive_group_interval", 2023-07-21T22:38:49.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:49.099 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.099 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.099 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300.0", 2023-07-21T22:38:49.099 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.099 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.099 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.100 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T22:38:49.100 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.100 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.100 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.100 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.100 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.101 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.101 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.101 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.101 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.101 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.101 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.101 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.101 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.102 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.102 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.102 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.102 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.102 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.102 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.102 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.102 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.103 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.103 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.103 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.103 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.103 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.104 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.104 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.104 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.105 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.105 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.105 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.105 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.106 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.106 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.106 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.106 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.106 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.107 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.107 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.107 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.107 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.107 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.107 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.107 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.107 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.108 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.108 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.108 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.108 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.108 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.108 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.108 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.109 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.109 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.109 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.109 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.109 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.109 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.110 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.110 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.110 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.110 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.110 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.110 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.110 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.110 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.111 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.111 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.111 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2023-07-21T22:38:49.111 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2023-07-21T22:38:49.111 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.111 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.111 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.112 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2023-07-21T22:38:49.112 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.112 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.112 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.112 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T22:38:49.112 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.112 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.113 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.113 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.113 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.113 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.113 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2023-07-21T22:38:49.113 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.114 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.114 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.114 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.114 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.114 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.114 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.114 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.114 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.115 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.115 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.115 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.115 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.115 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.115 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.115 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.115 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.116 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.116 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.116 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.116 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.116 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.116 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.117 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.117 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.117 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.117 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.117 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.117 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.117 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.118 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.118 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.118 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.118 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.118 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.118 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.118 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.119 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.119 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.119 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.119 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.119 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.119 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.120 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.120 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.120 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.120 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.120 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.120 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.120 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.120 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.121 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.121 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.121 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.121 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.121 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.121 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.122 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.122 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.122 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.122 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.122 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.122 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.122 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.122 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.123 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.123 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.123 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.123 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2023-07-21T22:38:49.123 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2023-07-21T22:38:49.123 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.124 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.124 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.124 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.124 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.124 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.124 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.124 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.124 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.125 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.125 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.125 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2023-07-21T22:38:49.125 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2023-07-21T22:38:49.125 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.125 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.126 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.126 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2023-07-21T22:38:49.126 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.126 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.126 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.126 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.126 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.126 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.127 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.127 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2023-07-21T22:38:49.127 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2023-07-21T22:38:49.127 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.127 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.127 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.127 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.128 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.128 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.128 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.128 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.128 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.128 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.128 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.129 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2023-07-21T22:38:49.129 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2023-07-21T22:38:49.129 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:49.129 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.129 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.129 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.129 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.130 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.130 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.130 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.130 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.130 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.130 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.131 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2023-07-21T22:38:49.131 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2023-07-21T22:38:49.131 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2023-07-21T22:38:49.131 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.131 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.131 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.131 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.131 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.132 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.132 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.132 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.132 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.132 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.132 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2023-07-21T22:38:49.132 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2023-07-21T22:38:49.133 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.133 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.133 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.133 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.133 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.133 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.133 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.134 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.134 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.134 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.134 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.134 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2023-07-21T22:38:49.134 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2023-07-21T22:38:49.134 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.135 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.135 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.135 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.135 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.135 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.135 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.135 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.135 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.136 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.136 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.136 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2023-07-21T22:38:49.136 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2023-07-21T22:38:49.136 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.136 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.136 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.137 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:49.137 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.137 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.137 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.137 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.137 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.137 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.137 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.138 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2023-07-21T22:38:49.138 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2023-07-21T22:38:49.138 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:49.138 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.138 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.138 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.139 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2023-07-21T22:38:49.139 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2023-07-21T22:38:49.139 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.139 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.139 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.139 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.139 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.140 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2023-07-21T22:38:49.140 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2023-07-21T22:38:49.140 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.140 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.140 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.140 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.140 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.140 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.141 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.141 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.141 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.141 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.141 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.141 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2023-07-21T22:38:49.142 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2023-07-21T22:38:49.142 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.142 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.142 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.142 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.142 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.142 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.142 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.143 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.143 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.143 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.143 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.143 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2023-07-21T22:38:49.143 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2023-07-21T22:38:49.143 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.144 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.144 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.144 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.144 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.144 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.144 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.144 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.145 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.145 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.145 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.145 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.145 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.145 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2023-07-21T22:38:49.146 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.146 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.146 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.146 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2023-07-21T22:38:49.146 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2023-07-21T22:38:49.146 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.146 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.147 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.147 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.147 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.147 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.147 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.147 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T22:38:49.147 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.147 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.148 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.148 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2023-07-21T22:38:49.148 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2023-07-21T22:38:49.148 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.148 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.148 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.149 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.149 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.149 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.149 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.149 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2023-07-21T22:38:49.149 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.149 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.149 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.150 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.150 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.150 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.150 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.150 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.150 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.151 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.151 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.151 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.151 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.151 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.151 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.151 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.151 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.152 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.152 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.152 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.152 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.152 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.152 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.152 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.153 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.153 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.153 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.153 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.153 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.153 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.153 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.153 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.154 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.154 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.154 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.154 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.154 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.154 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.154 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.155 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.155 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.155 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.155 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.155 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.155 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.155 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.157 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.157 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.157 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.157 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.157 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.157 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.157 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.158 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.158 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.158 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.158 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.158 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.158 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.158 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.158 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.159 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.159 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.159 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.159 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2023-07-21T22:38:49.159 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.159 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.160 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.160 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.160 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.160 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.160 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.160 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.160 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.160 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.161 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.161 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.161 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.161 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.161 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.161 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.161 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.161 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.162 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.162 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.162 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.162 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.162 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.162 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.163 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.163 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.163 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.163 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.163 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.163 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.163 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.163 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.164 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.164 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.164 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.164 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.164 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.164 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.164 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.165 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.165 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.165 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.165 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.165 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.165 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.165 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.165 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.166 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.166 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.166 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.166 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.166 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.166 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.166 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.167 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.167 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.167 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.167 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.167 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.167 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.167 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.168 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.168 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.168 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.168 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.168 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.168 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.168 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.168 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.169 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.169 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.169 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2023-07-21T22:38:49.169 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.169 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.169 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.170 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.170 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.170 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.170 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.170 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.170 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.170 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.171 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.171 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.171 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.171 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.171 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.171 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.171 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.171 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.172 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.172 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.172 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.172 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.172 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.172 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.173 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.173 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.173 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.173 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.173 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.173 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.173 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.173 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.174 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.174 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.174 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.174 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.174 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.174 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.174 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.175 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.175 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.175 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.175 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.175 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.175 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.175 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.176 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.176 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.176 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.176 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.176 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.176 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.176 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.176 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.177 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.177 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.177 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.177 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.177 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.177 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.177 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.178 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.178 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.178 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.178 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.178 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.178 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.178 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.179 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.179 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.179 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.179 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.179 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2023-07-21T22:38:49.179 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.180 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.180 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.180 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2023-07-21T22:38:49.180 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2023-07-21T22:38:49.180 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.180 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.180 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.180 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T22:38:49.181 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.181 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.181 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.181 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.181 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.181 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.181 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.182 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:49.182 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:49.182 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:49.182 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.182 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.182 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2023-07-21T22:38:49.182 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.183 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.183 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.183 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.183 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.183 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.183 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.183 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.184 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.184 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.184 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.184 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.184 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.184 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.184 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.184 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.185 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.185 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.185 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.185 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.185 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.185 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.185 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.186 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.186 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.186 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.186 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.186 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.186 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.186 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.186 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.187 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.187 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.187 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.187 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.187 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.187 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.187 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.187 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.188 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.188 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.188 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.188 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.188 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.188 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.188 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.189 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.189 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.189 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.189 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.189 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.189 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.189 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.190 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.190 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.190 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.190 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.190 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.190 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.190 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.190 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.191 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.191 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.191 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.191 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.191 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.191 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.191 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.191 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.192 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.192 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.192 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.192 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.192 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.192 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.192 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.192 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.193 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.193 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2023-07-21T22:38:49.193 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.193 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.193 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.193 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2023-07-21T22:38:49.194 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2023-07-21T22:38:49.194 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.194 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.194 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.194 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:49.194 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.194 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.194 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.195 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T22:38:49.195 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.195 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.195 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.195 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2023-07-21T22:38:49.195 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2023-07-21T22:38:49.195 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.196 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.196 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.196 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:49.196 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.196 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.196 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.196 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T22:38:49.196 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.197 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.197 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.197 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2023-07-21T22:38:49.197 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2023-07-21T22:38:49.197 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.197 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.198 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.198 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2023-07-21T22:38:49.198 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.198 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.198 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.198 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T22:38:49.198 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.198 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.199 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.199 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2023-07-21T22:38:49.199 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2023-07-21T22:38:49.199 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.199 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.199 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.199 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.200 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.200 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.200 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.200 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T22:38:49.200 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.200 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.200 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.201 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2023-07-21T22:38:49.201 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2023-07-21T22:38:49.201 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.201 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.201 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.201 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.202 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.202 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.202 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.202 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T22:38:49.202 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.202 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.202 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.203 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2023-07-21T22:38:49.203 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2023-07-21T22:38:49.203 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.203 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.203 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.203 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.203 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.203 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.204 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.204 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.204 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.204 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.204 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.204 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2023-07-21T22:38:49.205 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2023-07-21T22:38:49.205 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.205 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.205 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.205 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.205 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.205 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.205 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.206 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.206 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.206 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.206 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.206 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2023-07-21T22:38:49.206 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2023-07-21T22:38:49.206 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.207 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.207 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.207 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T22:38:49.207 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.207 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.207 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.207 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.208 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.208 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.208 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.208 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2023-07-21T22:38:49.208 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2023-07-21T22:38:49.208 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.208 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.209 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.209 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.209 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.209 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.209 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.209 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.209 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.210 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.210 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.210 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:49.210 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:49.210 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:49.210 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.210 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.211 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2023-07-21T22:38:49.211 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2023-07-21T22:38:49.211 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.211 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.211 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.211 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.211 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.212 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2023-07-21T22:38:49.212 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2023-07-21T22:38:49.212 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:49.212 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.212 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.212 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2023-07-21T22:38:49.213 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.213 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.213 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.213 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.213 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.213 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.213 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.214 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2023-07-21T22:38:49.214 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2023-07-21T22:38:49.214 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.214 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.214 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.214 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.214 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.214 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.215 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.215 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.215 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.215 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.215 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.215 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2023-07-21T22:38:49.215 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2023-07-21T22:38:49.216 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.216 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.216 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.216 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.216 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.216 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.216 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.217 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.217 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.217 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.217 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.217 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.217 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.217 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.218 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.218 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.218 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.218 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.218 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.218 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.218 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.219 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.219 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.219 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.219 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.219 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.219 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.219 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.219 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.220 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.220 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.220 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.220 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.220 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.220 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.221 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.221 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.221 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.221 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.221 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.221 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.221 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.221 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.222 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.222 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.222 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.222 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.222 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.222 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.223 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.223 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.223 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.223 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.223 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.223 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.223 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.223 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.224 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.224 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.224 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.224 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.224 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.224 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.224 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.225 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.225 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.225 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.225 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.225 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.225 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.225 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.226 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.226 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.226 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.226 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.226 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.226 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.226 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2023-07-21T22:38:49.227 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2023-07-21T22:38:49.227 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.227 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.227 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.227 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.227 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.227 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.227 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.228 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.228 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.228 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.228 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.228 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2023-07-21T22:38:49.228 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2023-07-21T22:38:49.229 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.229 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.229 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.229 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.229 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.229 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.229 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.230 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.230 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.230 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.230 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.230 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2023-07-21T22:38:49.230 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2023-07-21T22:38:49.231 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.231 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.231 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.231 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T22:38:49.231 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.231 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.231 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.231 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.232 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.232 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.232 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.232 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.232 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.232 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.233 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2023-07-21T22:38:49.233 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.233 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.233 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.233 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.233 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.233 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.233 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.234 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.234 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.234 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.234 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.234 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.234 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.235 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.235 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.235 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.235 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.235 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.235 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.235 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.235 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.236 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.236 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.236 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.236 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.236 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.236 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.237 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.237 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.237 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.237 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.237 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.237 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.237 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.238 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.238 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.238 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.239 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.239 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.239 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.239 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.239 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.239 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.239 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.240 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.240 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.240 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.240 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.240 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.240 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.240 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.240 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.241 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.241 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.241 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.241 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.241 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.241 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.242 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.242 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.242 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.242 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.242 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.242 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.242 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.242 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.243 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.243 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.243 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.243 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.243 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2023-07-21T22:38:49.243 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.243 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.244 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.244 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.244 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.244 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.244 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.244 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.244 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.244 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.245 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.245 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.245 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.245 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.245 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.245 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.245 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.246 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.246 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.246 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.246 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.246 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.246 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.246 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.247 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.247 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.247 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.247 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.247 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.247 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.247 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.247 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.248 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.248 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.248 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.248 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.248 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.248 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.248 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.249 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.249 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.249 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.249 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.249 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.249 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.249 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.249 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.250 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.250 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.250 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.250 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.250 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.250 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.250 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.250 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.251 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.251 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.251 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.251 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.251 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.251 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.251 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.252 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.252 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.252 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.252 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.252 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.252 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.252 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.253 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2023-07-21T22:38:49.253 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2023-07-21T22:38:49.253 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:49.253 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.253 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.253 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2023-07-21T22:38:49.253 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.254 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.254 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.254 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T22:38:49.254 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.254 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.254 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.254 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2023-07-21T22:38:49.255 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2023-07-21T22:38:49.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:49.255 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.255 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.255 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2023-07-21T22:38:49.255 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.255 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.255 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.256 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T22:38:49.256 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.256 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.256 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.257 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2023-07-21T22:38:49.257 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2023-07-21T22:38:49.257 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2023-07-21T22:38:49.257 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2023-07-21T22:38:49.257 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2023-07-21T22:38:49.257 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2023-07-21T22:38:49.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2023-07-21T22:38:49.258 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.258 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.258 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2023-07-21T22:38:49.258 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.258 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.258 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.258 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.258 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.259 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.259 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.259 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2023-07-21T22:38:49.259 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2023-07-21T22:38:49.259 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.259 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.259 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.260 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.260 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.260 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.260 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.260 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.260 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.260 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.261 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.261 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2023-07-21T22:38:49.261 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2023-07-21T22:38:49.261 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2023-07-21T22:38:49.261 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.261 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.261 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2023-07-21T22:38:49.262 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.262 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.262 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.262 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.262 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.262 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.262 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.263 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2023-07-21T22:38:49.263 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2023-07-21T22:38:49.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.263 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.263 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.263 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.263 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.263 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.264 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.264 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.264 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.264 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.264 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.264 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.264 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.264 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.265 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.265 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.265 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.265 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.265 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2023-07-21T22:38:49.265 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2023-07-21T22:38:49.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.266 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.266 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.266 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.266 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.266 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.266 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.266 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.266 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.267 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.267 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.267 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2023-07-21T22:38:49.267 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2023-07-21T22:38:49.267 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.267 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.268 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.268 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2023-07-21T22:38:49.268 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.268 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.268 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2023-07-21T22:38:49.268 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2023-07-21T22:38:49.268 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2023-07-21T22:38:49.268 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2023-07-21T22:38:49.269 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2023-07-21T22:38:49.269 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2023-07-21T22:38:49.269 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2023-07-21T22:38:49.269 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.269 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.269 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.269 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.269 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.270 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2023-07-21T22:38:49.270 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2023-07-21T22:38:49.270 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2023-07-21T22:38:49.270 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.270 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2023-07-21T22:38:49.270 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2023-07-21T22:38:49.270 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.271 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.271 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.271 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.271 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.271 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.271 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.271 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2023-07-21T22:38:49.272 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2023-07-21T22:38:49.272 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.272 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.272 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.272 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2023-07-21T22:38:49.272 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.272 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.273 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.273 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.273 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.273 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.273 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.273 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2023-07-21T22:38:49.273 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2023-07-21T22:38:49.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2023-07-21T22:38:49.274 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.274 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.274 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2023-07-21T22:38:49.274 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.274 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.274 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.274 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.275 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.275 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.275 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.275 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2023-07-21T22:38:49.275 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2023-07-21T22:38:49.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2023-07-21T22:38:49.275 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2023-07-21T22:38:49.276 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2023-07-21T22:38:49.276 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T22:38:49.276 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2023-07-21T22:38:49.276 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2023-07-21T22:38:49.276 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2023-07-21T22:38:49.276 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2023-07-21T22:38:49.276 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2023-07-21T22:38:49.277 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2023-07-21T22:38:49.277 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2023-07-21T22:38:49.277 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.277 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.277 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.277 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.277 INFO:teuthology.orchestra.run.smithi107.stdout: "services": { 2023-07-21T22:38:49.277 INFO:teuthology.orchestra.run.smithi107.stdout: "dashboard": "https://172.21.15.107:8443/" 2023-07-21T22:38:49.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.278 INFO:teuthology.orchestra.run.smithi107.stdout: "always_on_modules": { 2023-07-21T22:38:49.278 INFO:teuthology.orchestra.run.smithi107.stdout: "octopus": [ 2023-07-21T22:38:49.278 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2023-07-21T22:38:49.278 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2023-07-21T22:38:49.278 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2023-07-21T22:38:49.278 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2023-07-21T22:38:49.278 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2023-07-21T22:38:49.279 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2023-07-21T22:38:49.279 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2023-07-21T22:38:49.279 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2023-07-21T22:38:49.279 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2023-07-21T22:38:49.279 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2023-07-21T22:38:49.279 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.279 INFO:teuthology.orchestra.run.smithi107.stdout: "pacific": [ 2023-07-21T22:38:49.279 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2023-07-21T22:38:49.280 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2023-07-21T22:38:49.280 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2023-07-21T22:38:49.280 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2023-07-21T22:38:49.280 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2023-07-21T22:38:49.280 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2023-07-21T22:38:49.280 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2023-07-21T22:38:49.280 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2023-07-21T22:38:49.280 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2023-07-21T22:38:49.281 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2023-07-21T22:38:49.281 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.281 INFO:teuthology.orchestra.run.smithi107.stdout: "quincy": [ 2023-07-21T22:38:49.281 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2023-07-21T22:38:49.281 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2023-07-21T22:38:49.281 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2023-07-21T22:38:49.281 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2023-07-21T22:38:49.282 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2023-07-21T22:38:49.282 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2023-07-21T22:38:49.282 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2023-07-21T22:38:49.282 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2023-07-21T22:38:49.282 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2023-07-21T22:38:49.282 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2023-07-21T22:38:49.282 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2023-07-21T22:38:49.282 INFO:teuthology.orchestra.run.smithi107.stdout: "reef": [ 2023-07-21T22:38:49.283 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2023-07-21T22:38:49.283 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2023-07-21T22:38:49.283 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2023-07-21T22:38:49.283 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2023-07-21T22:38:49.283 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2023-07-21T22:38:49.283 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2023-07-21T22:38:49.283 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2023-07-21T22:38:49.283 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2023-07-21T22:38:49.284 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2023-07-21T22:38:49.284 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2023-07-21T22:38:49.284 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2023-07-21T22:38:49.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.284 INFO:teuthology.orchestra.run.smithi107.stdout: "last_failure_osd_epoch": 3, 2023-07-21T22:38:49.284 INFO:teuthology.orchestra.run.smithi107.stdout: "active_clients": [ 2023-07-21T22:38:49.284 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.284 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2023-07-21T22:38:49.285 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2023-07-21T22:38:49.285 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2023-07-21T22:38:49.285 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2023-07-21T22:38:49.285 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2021632209 2023-07-21T22:38:49.285 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.285 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2023-07-21T22:38:49.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.286 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "libcephsqlite", 2023-07-21T22:38:49.286 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2023-07-21T22:38:49.286 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.286 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2023-07-21T22:38:49.286 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2023-07-21T22:38:49.286 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1251147271 2023-07-21T22:38:49.286 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.287 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2023-07-21T22:38:49.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.287 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2023-07-21T22:38:49.287 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2023-07-21T22:38:49.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2023-07-21T22:38:49.287 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2023-07-21T22:38:49.288 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 322944938 2023-07-21T22:38:49.288 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.288 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2023-07-21T22:38:49.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:49.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.288 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2023-07-21T22:38:49.288 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2023-07-21T22:38:49.288 INFO:teuthology.orchestra.run.smithi107.stdout: { 2023-07-21T22:38:49.289 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2023-07-21T22:38:49.289 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2023-07-21T22:38:49.289 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 846018660 2023-07-21T22:38:49.289 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.289 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2023-07-21T22:38:49.289 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:49.289 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2023-07-21T22:38:49.290 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:38:49.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.811+0000 7ff1331e5700 1 -- 172.21.15.107:0/3325318892 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff11c0a11a0 msgr2=0x7ff11c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:49.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.811+0000 7ff1331e5700 1 --2- 172.21.15.107:0/3325318892 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff11c0a11a0 0x7ff11c0a3650 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff118005fd0 tx=0x7ff118005d00 comp rx=0 tx=0).stop 2023-07-21T22:38:49.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.811+0000 7ff1331e5700 1 -- 172.21.15.107:0/3325318892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff12c0fea00 msgr2=0x7ff12c194630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:49.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.811+0000 7ff1331e5700 1 --2- 172.21.15.107:0/3325318892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff12c0fea00 0x7ff12c194630 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff1200059d0 tx=0x7ff120005ef0 comp rx=0 tx=0).stop 2023-07-21T22:38:49.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.811+0000 7ff1331e5700 1 -- 172.21.15.107:0/3325318892 shutdown_connections 2023-07-21T22:38:49.291 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.811+0000 7ff1331e5700 1 --2- 172.21.15.107:0/3325318892 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7ff11c0a11a0 0x7ff11c0a3650 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:49.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.811+0000 7ff1331e5700 1 --2- 172.21.15.107:0/3325318892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12c100580 0x7ff12c19bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:49.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.811+0000 7ff1331e5700 1 --2- 172.21.15.107:0/3325318892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff12c0fea00 0x7ff12c194630 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:49.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.811+0000 7ff1331e5700 1 --2- 172.21.15.107:0/3325318892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff12c0ffbf0 0x7ff12c194b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:49.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.811+0000 7ff1331e5700 1 -- 172.21.15.107:0/3325318892 >> 172.21.15.107:0/3325318892 conn(0x7ff12c0fa020 msgr2=0x7ff12c0fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:49.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.811+0000 7ff1331e5700 1 -- 172.21.15.107:0/3325318892 shutdown_connections 2023-07-21T22:38:49.292 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:38:46.811+0000 7ff1331e5700 1 -- 172.21.15.107:0/3325318892 wait complete. 2023-07-21T22:38:49.293 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:46 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/1156548886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:38:49.293 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:46 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/4284927935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:38:49.293 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:46 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/4267933365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:38:49.293 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:46 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/1156548886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:38:49.294 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:46 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/4284927935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:38:49.294 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:46 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/4267933365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:38:49.294 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:47 smithi132 ceph-mon[97863]: pgmap v585: 97 pgs: 97 active+clean; 784 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 346 op/s 2023-07-21T22:38:49.294 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:47 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3325318892' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T22:38:49.294 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:46 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/1156548886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:38:49.294 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:46 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/4284927935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:38:49.294 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:46 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/4267933365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:38:49.295 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:47 smithi190 ceph-mon[98334]: pgmap v585: 97 pgs: 97 active+clean; 784 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 346 op/s 2023-07-21T22:38:49.295 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:47 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3325318892' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T22:38:49.301 DEBUG:tasks.check_counter:Getting stats from a 2023-07-21T22:38:49.302 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph daemon mds.a perf dump 2023-07-21T22:38:49.302 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:47 smithi107 ceph-mon[96543]: pgmap v585: 97 pgs: 97 active+clean; 784 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 346 op/s 2023-07-21T22:38:49.303 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:47 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3325318892' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T22:38:49.876 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:38:50.071 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:49 smithi107 ceph-mon[96543]: pgmap v586: 97 pgs: 97 active+clean; 767 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 395 op/s 2023-07-21T22:38:50.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:49 smithi190 ceph-mon[98334]: pgmap v586: 97 pgs: 97 active+clean; 767 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 395 op/s 2023-07-21T22:38:50.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:49 smithi132 ceph-mon[97863]: pgmap v586: 97 pgs: 97 active+clean; 767 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 395 op/s 2023-07-21T22:38:51.107 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:38:51.107 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:38:51.107 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1242, 2023-07-21T22:38:51.107 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1208, 2023-07-21T22:38:51.108 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 60076872, 2023-07-21T22:38:51.108 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 421489, 2023-07-21T22:38:51.108 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 7, 2023-07-21T22:38:51.108 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2023-07-21T22:38:51.108 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.178974777, 2023-07-21T22:38:51.108 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.042128205, 2023-07-21T22:38:51.108 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.077848871, 2023-07-21T22:38:51.108 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.049955624, 2023-07-21T22:38:51.109 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:38:51.109 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1208, 2023-07-21T22:38:51.109 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.386955740, 2023-07-21T22:38:51.109 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000320327 2023-07-21T22:38:51.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.109 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:38:51.109 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.110 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.110 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.110 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 657920, 2023-07-21T22:38:51.110 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 105824 2023-07-21T22:38:51.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.110 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:38:51.110 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 640, 2023-07-21T22:38:51.111 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 778, 2023-07-21T22:38:51.111 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 113457837, 2023-07-21T22:38:51.111 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 514045, 2023-07-21T22:38:51.111 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 7, 2023-07-21T22:38:51.111 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 6, 2023-07-21T22:38:51.111 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.135080714, 2023-07-21T22:38:51.111 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.021020736, 2023-07-21T22:38:51.111 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.087195512, 2023-07-21T22:38:51.112 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.021070029, 2023-07-21T22:38:51.112 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:38:51.112 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 776, 2023-07-21T22:38:51.112 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.181407653, 2023-07-21T22:38:51.112 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000233772 2023-07-21T22:38:51.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.112 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:38:51.113 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.113 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.113 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.113 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-21T22:38:51.113 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 315248 2023-07-21T22:38:51.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.113 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:38:51.114 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 836, 2023-07-21T22:38:51.114 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 838, 2023-07-21T22:38:51.114 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 155428234, 2023-07-21T22:38:51.114 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 259997, 2023-07-21T22:38:51.114 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2023-07-21T22:38:51.114 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2023-07-21T22:38:51.114 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.170030071, 2023-07-21T22:38:51.114 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.021200217, 2023-07-21T22:38:51.115 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.109144342, 2023-07-21T22:38:51.115 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.033184081, 2023-07-21T22:38:51.115 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:38:51.115 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 836, 2023-07-21T22:38:51.115 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.232140935, 2023-07-21T22:38:51.115 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000277680 2023-07-21T22:38:51.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.115 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:38:51.116 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.116 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.116 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.116 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:38:51.116 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:38:51.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.117 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2023-07-21T22:38:51.117 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2023-07-21T22:38:51.117 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2023-07-21T22:38:51.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.117 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2023-07-21T22:38:51.117 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2023-07-21T22:38:51.117 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2023-07-21T22:38:51.117 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 4017, 2023-07-21T22:38:51.118 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.157283250, 2023-07-21T22:38:51.118 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000039154 2023-07-21T22:38:51.118 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:51.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.118 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2023-07-21T22:38:51.118 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2023-07-21T22:38:51.118 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2023-07-21T22:38:51.118 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.119 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.119 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.119 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:51.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.119 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2023-07-21T22:38:51.119 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 0, 2023-07-21T22:38:51.119 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 0, 2023-07-21T22:38:51.119 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2023-07-21T22:38:51.120 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.120 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.120 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.120 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2023-07-21T22:38:51.120 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2023-07-21T22:38:51.120 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2023-07-21T22:38:51.120 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2023-07-21T22:38:51.121 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 0, 2023-07-21T22:38:51.121 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2023-07-21T22:38:51.121 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2023-07-21T22:38:51.121 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 11367, 2023-07-21T22:38:51.121 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 0, 2023-07-21T22:38:51.121 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 0, 2023-07-21T22:38:51.121 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 11367, 2023-07-21T22:38:51.121 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 11367, 2023-07-21T22:38:51.122 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 3319, 2023-07-21T22:38:51.122 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 0, 2023-07-21T22:38:51.122 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 0, 2023-07-21T22:38:51.122 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 1, 2023-07-21T22:38:51.122 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 0, 2023-07-21T22:38:51.122 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 0, 2023-07-21T22:38:51.122 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2023-07-21T22:38:51.122 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2023-07-21T22:38:51.123 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:38:51.123 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2023-07-21T22:38:51.123 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2023-07-21T22:38:51.123 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2023-07-21T22:38:51.123 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2023-07-21T22:38:51.123 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2023-07-21T22:38:51.123 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2023-07-21T22:38:51.124 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2023-07-21T22:38:51.124 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2023-07-21T22:38:51.124 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2023-07-21T22:38:51.124 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:38:51.124 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2023-07-21T22:38:51.124 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 0, 2023-07-21T22:38:51.124 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 0, 2023-07-21T22:38:51.124 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2023-07-21T22:38:51.125 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:38:51.125 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2023-07-21T22:38:51.125 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:38:51.125 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:38:51.125 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:38:51.125 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:38:51.125 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:38:51.125 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2023-07-21T22:38:51.126 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2023-07-21T22:38:51.126 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T22:38:51.126 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2023-07-21T22:38:51.126 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:38:51.126 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:38:51.126 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:38:51.126 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2023-07-21T22:38:51.126 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2023-07-21T22:38:51.127 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2023-07-21T22:38:51.127 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2023-07-21T22:38:51.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.127 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2023-07-21T22:38:51.127 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2023-07-21T22:38:51.127 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2023-07-21T22:38:51.127 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2023-07-21T22:38:51.127 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2023-07-21T22:38:51.128 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2023-07-21T22:38:51.128 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2023-07-21T22:38:51.128 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2023-07-21T22:38:51.128 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:38:51.128 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2023-07-21T22:38:51.128 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2023-07-21T22:38:51.128 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2023-07-21T22:38:51.128 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2023-07-21T22:38:51.129 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2023-07-21T22:38:51.129 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:38:51.129 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:38:51.129 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2023-07-21T22:38:51.129 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2023-07-21T22:38:51.129 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:38:51.129 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2023-07-21T22:38:51.129 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2023-07-21T22:38:51.130 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2023-07-21T22:38:51.130 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2023-07-21T22:38:51.130 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2023-07-21T22:38:51.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.130 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2023-07-21T22:38:51.130 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 0, 2023-07-21T22:38:51.130 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2023-07-21T22:38:51.130 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2023-07-21T22:38:51.131 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 0, 2023-07-21T22:38:51.131 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2023-07-21T22:38:51.131 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2023-07-21T22:38:51.131 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 0, 2023-07-21T22:38:51.131 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2023-07-21T22:38:51.131 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2023-07-21T22:38:51.131 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 134, 2023-07-21T22:38:51.131 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2023-07-21T22:38:51.132 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2023-07-21T22:38:51.132 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 82265413, 2023-07-21T22:38:51.132 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 0, 2023-07-21T22:38:51.132 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 331733880, 2023-07-21T22:38:51.132 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2023-07-21T22:38:51.132 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.132 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.133 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.133 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 135135 2023-07-21T22:38:51.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.133 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2023-07-21T22:38:51.133 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 31, 2023-07-21T22:38:51.133 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 40111, 2023-07-21T22:38:51.133 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 40080, 2023-07-21T22:38:51.134 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 40, 2023-07-21T22:38:51.134 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 1740, 2023-07-21T22:38:51.134 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 1700, 2023-07-21T22:38:51.134 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 11367, 2023-07-21T22:38:51.134 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 80782, 2023-07-21T22:38:51.134 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 69415, 2023-07-21T22:38:51.134 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 0, 2023-07-21T22:38:51.134 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 0, 2023-07-21T22:38:51.135 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2023-07-21T22:38:51.135 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 89700, 2023-07-21T22:38:51.135 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 207156 2023-07-21T22:38:51.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.135 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2023-07-21T22:38:51.135 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 0, 2023-07-21T22:38:51.135 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2023-07-21T22:38:51.135 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 0, 2023-07-21T22:38:51.136 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 0, 2023-07-21T22:38:51.136 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2023-07-21T22:38:51.136 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2023-07-21T22:38:51.136 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.136 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.136 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.136 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2023-07-21T22:38:51.137 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.137 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.137 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.137 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2023-07-21T22:38:51.137 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.137 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.137 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.138 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2023-07-21T22:38:51.138 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.138 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.138 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.138 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2023-07-21T22:38:51.138 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.139 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.139 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.139 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2023-07-21T22:38:51.139 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.139 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.139 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.140 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2023-07-21T22:38:51.140 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.140 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.140 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.140 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2023-07-21T22:38:51.140 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.141 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.141 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.141 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2023-07-21T22:38:51.141 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.141 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.141 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.142 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2023-07-21T22:38:51.142 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.142 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.142 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.142 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2023-07-21T22:38:51.142 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.142 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.143 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.143 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2023-07-21T22:38:51.143 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.143 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.143 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.143 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2023-07-21T22:38:51.144 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.144 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.144 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.144 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2023-07-21T22:38:51.144 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.144 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.144 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.145 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2023-07-21T22:38:51.145 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.145 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.145 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.145 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2023-07-21T22:38:51.145 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.146 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.146 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.146 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2023-07-21T22:38:51.146 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.146 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.146 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.147 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2023-07-21T22:38:51.147 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.147 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.147 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.147 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2023-07-21T22:38:51.147 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.148 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.148 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.148 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2023-07-21T22:38:51.148 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.148 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.148 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.149 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2023-07-21T22:38:51.149 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.149 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.149 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.149 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2023-07-21T22:38:51.149 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.149 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.150 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.150 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2023-07-21T22:38:51.150 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.150 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.150 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.150 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2023-07-21T22:38:51.151 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.151 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.151 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.151 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2023-07-21T22:38:51.151 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.151 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.151 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.152 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2023-07-21T22:38:51.152 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.152 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.152 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.152 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2023-07-21T22:38:51.152 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.153 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.153 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.153 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2023-07-21T22:38:51.153 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.153 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.153 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.154 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2023-07-21T22:38:51.154 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.154 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.154 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.154 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:38:51.154 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:38:51.155 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2023-07-21T22:38:51.155 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.155 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.155 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.155 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:51.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.155 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2023-07-21T22:38:51.155 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2023-07-21T22:38:51.156 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 2, 2023-07-21T22:38:51.156 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 1, 2023-07-21T22:38:51.156 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 2, 2023-07-21T22:38:51.156 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2023-07-21T22:38:51.156 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 0, 2023-07-21T22:38:51.156 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2023-07-21T22:38:51.156 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 3565 2023-07-21T22:38:51.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.157 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2023-07-21T22:38:51.157 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:38:51.157 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2023-07-21T22:38:51.157 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:38:51.157 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:38:51.157 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:38:51.157 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:38:51.157 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:38:51.158 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:38:51.158 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:38:51.158 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:38:51.158 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:38:51.158 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:38:51.158 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:38:51.158 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:38:51.158 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:38:51.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2023-07-21T22:38:51.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:38:51.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2023-07-21T22:38:51.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:38:51.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:38:51.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:38:51.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:38:51.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:38:51.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:38:51.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:38:51.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2023-07-21T22:38:51.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:38:51.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:38:51.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:38:51.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2023-07-21T22:38:51.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2023-07-21T22:38:51.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2023-07-21T22:38:51.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:38:51.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:38:51.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:38:51.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:38:51.161 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 239293, 2023-07-21T22:38:51.161 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 96, 2023-07-21T22:38:51.162 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:38:51.162 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2023-07-21T22:38:51.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2023-07-21T22:38:51.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2023-07-21T22:38:51.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:38:51.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2023-07-21T22:38:51.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:38:51.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2023-07-21T22:38:51.163 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 7720, 2023-07-21T22:38:51.163 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 162, 2023-07-21T22:38:51.163 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:38:51.163 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:38:51.163 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2023-07-21T22:38:51.163 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2023-07-21T22:38:51.163 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 8125333, 2023-07-21T22:38:51.163 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 48820, 2023-07-21T22:38:51.164 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2023-07-21T22:38:51.164 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2023-07-21T22:38:51.164 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2023-07-21T22:38:51.164 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2023-07-21T22:38:51.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.164 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2023-07-21T22:38:51.164 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2023-07-21T22:38:51.164 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2023-07-21T22:38:51.165 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 2490, 2023-07-21T22:38:51.165 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 21, 2023-07-21T22:38:51.165 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 3, 2023-07-21T22:38:51.165 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 2490, 2023-07-21T22:38:51.165 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2023-07-21T22:38:51.165 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2490, 2023-07-21T22:38:51.165 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 13.956771818, 2023-07-21T22:38:51.165 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.005605129 2023-07-21T22:38:51.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.166 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2023-07-21T22:38:51.166 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2023-07-21T22:38:51.166 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2490, 2023-07-21T22:38:51.166 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 2491 2023-07-21T22:38:51.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.166 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 2490, 2023-07-21T22:38:51.166 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 2490, 2023-07-21T22:38:51.167 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 0, 2023-07-21T22:38:51.167 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2023-07-21T22:38:51.167 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2023-07-21T22:38:51.167 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 1403, 2023-07-21T22:38:51.167 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 0, 2023-07-21T22:38:51.167 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 1085, 2023-07-21T22:38:51.167 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 0, 2023-07-21T22:38:51.167 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 0, 2023-07-21T22:38:51.168 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2023-07-21T22:38:51.168 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2023-07-21T22:38:51.168 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2023-07-21T22:38:51.168 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2023-07-21T22:38:51.168 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 0, 2023-07-21T22:38:51.168 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2023-07-21T22:38:51.168 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2023-07-21T22:38:51.169 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2023-07-21T22:38:51.169 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 1, 2023-07-21T22:38:51.169 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 0, 2023-07-21T22:38:51.169 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:38:51.169 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2023-07-21T22:38:51.169 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:38:51.169 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2023-07-21T22:38:51.169 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2023-07-21T22:38:51.170 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2023-07-21T22:38:51.170 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:38:51.170 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2023-07-21T22:38:51.170 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:38:51.170 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2023-07-21T22:38:51.170 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2023-07-21T22:38:51.170 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2023-07-21T22:38:51.170 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2023-07-21T22:38:51.171 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2023-07-21T22:38:51.171 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2023-07-21T22:38:51.171 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2023-07-21T22:38:51.171 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2023-07-21T22:38:51.171 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2023-07-21T22:38:51.171 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2023-07-21T22:38:51.171 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2023-07-21T22:38:51.171 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2023-07-21T22:38:51.172 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2023-07-21T22:38:51.172 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2023-07-21T22:38:51.172 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2023-07-21T22:38:51.172 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2023-07-21T22:38:51.172 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2023-07-21T22:38:51.172 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 85, 2023-07-21T22:38:51.172 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2023-07-21T22:38:51.172 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 3, 2023-07-21T22:38:51.173 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 11, 2023-07-21T22:38:51.173 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 11, 2023-07-21T22:38:51.173 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2023-07-21T22:38:51.173 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2023-07-21T22:38:51.173 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 0, 2023-07-21T22:38:51.173 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 2, 2023-07-21T22:38:51.173 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 0 2023-07-21T22:38:51.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.174 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2023-07-21T22:38:51.174 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 0, 2023-07-21T22:38:51.174 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:38:51.174 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2023-07-21T22:38:51.174 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2023-07-21T22:38:51.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.174 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2023-07-21T22:38:51.174 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2023-07-21T22:38:51.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.175 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2023-07-21T22:38:51.175 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2023-07-21T22:38:51.175 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T22:38:51.175 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2023-07-21T22:38:51.175 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2023-07-21T22:38:51.175 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2023-07-21T22:38:51.175 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2023-07-21T22:38:51.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.176 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:38:51.176 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2023-07-21T22:38:51.176 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2023-07-21T22:38:51.176 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2023-07-21T22:38:51.176 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 2714, 2023-07-21T22:38:51.176 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 328704377, 2023-07-21T22:38:51.177 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2023-07-21T22:38:51.177 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 2714, 2023-07-21T22:38:51.177 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2023-07-21T22:38:51.177 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2023-07-21T22:38:51.177 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 2714, 2023-07-21T22:38:51.177 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 328704377, 2023-07-21T22:38:51.177 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2023-07-21T22:38:51.177 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.178 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.178 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.178 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:51.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.178 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2023-07-21T22:38:51.178 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2023-07-21T22:38:51.178 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2023-07-21T22:38:51.178 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2023-07-21T22:38:51.179 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2023-07-21T22:38:51.179 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2023-07-21T22:38:51.179 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2023-07-21T22:38:51.179 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2023-07-21T22:38:51.179 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 2490, 2023-07-21T22:38:51.179 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 327617903, 2023-07-21T22:38:51.179 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 2490, 2023-07-21T22:38:51.179 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 327617903, 2023-07-21T22:38:51.180 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2023-07-21T22:38:51.180 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.180 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.180 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.180 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:51.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.180 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2023-07-21T22:38:51.180 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2023-07-21T22:38:51.181 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2023-07-21T22:38:51.181 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2023-07-21T22:38:51.181 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2023-07-21T22:38:51.181 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2023-07-21T22:38:51.181 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2023-07-21T22:38:51.181 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2023-07-21T22:38:51.181 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 2490, 2023-07-21T22:38:51.181 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 2490, 2023-07-21T22:38:51.182 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 2490, 2023-07-21T22:38:51.182 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 2490, 2023-07-21T22:38:51.182 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2023-07-21T22:38:51.182 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.182 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.182 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.182 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:51.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.183 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:38:51.183 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2023-07-21T22:38:51.183 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2023-07-21T22:38:51.183 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2023-07-21T22:38:51.183 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2023-07-21T22:38:51.183 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2023-07-21T22:38:51.183 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2023-07-21T22:38:51.183 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2023-07-21T22:38:51.184 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2023-07-21T22:38:51.184 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2023-07-21T22:38:51.184 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2023-07-21T22:38:51.184 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2023-07-21T22:38:51.184 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2023-07-21T22:38:51.184 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.184 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.185 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.185 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:51.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:51.185 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x55a2f4654460": { 2023-07-21T22:38:51.185 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2023-07-21T22:38:51.185 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2023-07-21T22:38:51.185 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2023-07-21T22:38:51.185 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2023-07-21T22:38:51.186 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2023-07-21T22:38:51.186 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2023-07-21T22:38:51.186 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2023-07-21T22:38:51.186 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2023-07-21T22:38:51.186 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2023-07-21T22:38:51.186 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2023-07-21T22:38:51.186 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2023-07-21T22:38:51.186 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2023-07-21T22:38:51.187 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:51.187 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:51.187 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:51.187 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:51.187 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:51.187 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:38:51.437 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-07-21T22:38:51.437 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-07-21T22:38:51.437 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2023-07-21T22:38:51.438 DEBUG:tasks.check_counter:Getting stats from d 2023-07-21T22:38:51.438 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph daemon mds.d perf dump 2023-07-21T22:38:51.993 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:38:52.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:51 smithi190 ceph-mon[98334]: pgmap v587: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 970 KiB/s wr, 300 op/s 2023-07-21T22:38:52.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:51 smithi132 ceph-mon[97863]: pgmap v587: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 970 KiB/s wr, 300 op/s 2023-07-21T22:38:52.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:51 smithi107 ceph-mon[96543]: pgmap v587: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 970 KiB/s wr, 300 op/s 2023-07-21T22:38:53.390 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:38:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:38:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 190, 2023-07-21T22:38:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 187, 2023-07-21T22:38:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 463200, 2023-07-21T22:38:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 106384, 2023-07-21T22:38:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2023-07-21T22:38:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2023-07-21T22:38:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.027695897, 2023-07-21T22:38:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.010892556, 2023-07-21T22:38:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.011173077, 2023-07-21T22:38:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.004664790, 2023-07-21T22:38:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:38:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 186, 2023-07-21T22:38:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.045097230, 2023-07-21T22:38:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000242458 2023-07-21T22:38:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:38:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 463200, 2023-07-21T22:38:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 106384 2023-07-21T22:38:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:38:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 5, 2023-07-21T22:38:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 144, 2023-07-21T22:38:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 3104, 2023-07-21T22:38:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 101008, 2023-07-21T22:38:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 3, 2023-07-21T22:38:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2023-07-21T22:38:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.010590200, 2023-07-21T22:38:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.007920576, 2023-07-21T22:38:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.001506850, 2023-07-21T22:38:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:38:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:38:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 143, 2023-07-21T22:38:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.021464721, 2023-07-21T22:38:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000150102 2023-07-21T22:38:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:38:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-07-21T22:38:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 101008 2023-07-21T22:38:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:38:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 0, 2023-07-21T22:38:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2, 2023-07-21T22:38:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 0, 2023-07-21T22:38:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 0, 2023-07-21T22:38:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2023-07-21T22:38:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2023-07-21T22:38:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.000388646, 2023-07-21T22:38:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000000000, 2023-07-21T22:38:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.000313559, 2023-07-21T22:38:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:38:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:38:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:38:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:38:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:38:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2023-07-21T22:38:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2023-07-21T22:38:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2023-07-21T22:38:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2023-07-21T22:38:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:38:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2023-07-21T22:38:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:38:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:38:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:38:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:38:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:38:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:38:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:38:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:38:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:38:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:38:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:38:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:38:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:38:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2023-07-21T22:38:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:38:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2023-07-21T22:38:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:38:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:38:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:38:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:38:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:38:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:38:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:38:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2023-07-21T22:38:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:38:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:38:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:38:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2023-07-21T22:38:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2023-07-21T22:38:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2023-07-21T22:38:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:38:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:38:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:38:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:38:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 148967, 2023-07-21T22:38:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 29, 2023-07-21T22:38:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:38:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2023-07-21T22:38:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2023-07-21T22:38:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2023-07-21T22:38:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:38:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2023-07-21T22:38:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:38:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2023-07-21T22:38:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2023-07-21T22:38:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2023-07-21T22:38:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:38:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:38:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2023-07-21T22:38:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2023-07-21T22:38:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2023-07-21T22:38:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2023-07-21T22:38:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2023-07-21T22:38:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2023-07-21T22:38:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2023-07-21T22:38:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2023-07-21T22:38:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:38:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2023-07-21T22:38:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2023-07-21T22:38:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2023-07-21T22:38:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 191, 2023-07-21T22:38:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 436347, 2023-07-21T22:38:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2023-07-21T22:38:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 191, 2023-07-21T22:38:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2023-07-21T22:38:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2023-07-21T22:38:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 191, 2023-07-21T22:38:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 436347, 2023-07-21T22:38:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2023-07-21T22:38:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:53.411 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:38:53.850 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:53 smithi107 ceph-mon[96543]: pgmap v588: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 977 KiB/s wr, 292 op/s 2023-07-21T22:38:53.851 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-07-21T22:38:53.851 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-07-21T22:38:53.851 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-07-21T22:38:53.852 DEBUG:tasks.check_counter:Getting stats from g 2023-07-21T22:38:53.852 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph daemon mds.g perf dump 2023-07-21T22:38:54.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:53 smithi190 ceph-mon[98334]: pgmap v588: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 977 KiB/s wr, 292 op/s 2023-07-21T22:38:54.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:53 smithi132 ceph-mon[97863]: pgmap v588: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 977 KiB/s wr, 292 op/s 2023-07-21T22:38:54.407 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:38:55.625 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:38:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:38:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1171, 2023-07-21T22:38:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1137, 2023-07-21T22:38:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 34508960, 2023-07-21T22:38:55.627 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 399970, 2023-07-21T22:38:55.627 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 7, 2023-07-21T22:38:55.627 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2023-07-21T22:38:55.627 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.148614114, 2023-07-21T22:38:55.628 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.045614979, 2023-07-21T22:38:55.628 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.058205936, 2023-07-21T22:38:55.628 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.037720296, 2023-07-21T22:38:55.629 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:38:55.629 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1137, 2023-07-21T22:38:55.629 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.420798811, 2023-07-21T22:38:55.629 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000370095 2023-07-21T22:38:55.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.630 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:38:55.630 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.630 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.630 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.630 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 658464, 2023-07-21T22:38:55.631 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 107008 2023-07-21T22:38:55.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.631 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:38:55.631 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 744, 2023-07-21T22:38:55.631 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 883, 2023-07-21T22:38:55.631 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 46327498, 2023-07-21T22:38:55.631 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 546554, 2023-07-21T22:38:55.632 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2023-07-21T22:38:55.632 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2023-07-21T22:38:55.632 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.103513264, 2023-07-21T22:38:55.632 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.022765145, 2023-07-21T22:38:55.632 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.051941447, 2023-07-21T22:38:55.632 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.022854963, 2023-07-21T22:38:55.632 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:38:55.633 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 881, 2023-07-21T22:38:55.633 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.234087235, 2023-07-21T22:38:55.633 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000265706 2023-07-21T22:38:55.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.633 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:38:55.633 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.633 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.634 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.634 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-21T22:38:55.634 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 315424 2023-07-21T22:38:55.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.634 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:38:55.634 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 454, 2023-07-21T22:38:55.635 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 456, 2023-07-21T22:38:55.635 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 42577892, 2023-07-21T22:38:55.635 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 141206, 2023-07-21T22:38:55.635 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2023-07-21T22:38:55.635 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2023-07-21T22:38:55.635 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.070004716, 2023-07-21T22:38:55.635 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.010039322, 2023-07-21T22:38:55.636 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.041689174, 2023-07-21T22:38:55.636 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.014944577, 2023-07-21T22:38:55.636 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:38:55.636 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 454, 2023-07-21T22:38:55.636 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.157741876, 2023-07-21T22:38:55.636 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000347449 2023-07-21T22:38:55.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.636 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:38:55.637 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.637 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.637 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.637 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:38:55.637 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:38:55.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.638 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2023-07-21T22:38:55.638 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2023-07-21T22:38:55.638 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2023-07-21T22:38:55.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.638 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2023-07-21T22:38:55.638 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2023-07-21T22:38:55.638 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2023-07-21T22:38:55.639 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3689, 2023-07-21T22:38:55.639 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.109357476, 2023-07-21T22:38:55.639 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000029644 2023-07-21T22:38:55.639 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:55.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.639 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2023-07-21T22:38:55.639 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2023-07-21T22:38:55.640 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2023-07-21T22:38:55.640 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.640 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.640 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.640 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:55.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.640 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2023-07-21T22:38:55.641 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 0, 2023-07-21T22:38:55.641 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 0, 2023-07-21T22:38:55.641 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2023-07-21T22:38:55.641 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.641 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.641 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.641 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2023-07-21T22:38:55.642 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2023-07-21T22:38:55.642 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2023-07-21T22:38:55.642 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2023-07-21T22:38:55.642 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 0, 2023-07-21T22:38:55.642 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2023-07-21T22:38:55.642 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2023-07-21T22:38:55.642 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 10660, 2023-07-21T22:38:55.643 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 0, 2023-07-21T22:38:55.643 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 0, 2023-07-21T22:38:55.643 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 10660, 2023-07-21T22:38:55.643 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 10660, 2023-07-21T22:38:55.643 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 1124, 2023-07-21T22:38:55.643 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 0, 2023-07-21T22:38:55.643 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 0, 2023-07-21T22:38:55.643 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 1, 2023-07-21T22:38:55.644 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 0, 2023-07-21T22:38:55.644 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 0, 2023-07-21T22:38:55.644 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2023-07-21T22:38:55.644 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2023-07-21T22:38:55.644 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:38:55.644 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2023-07-21T22:38:55.644 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2023-07-21T22:38:55.645 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2023-07-21T22:38:55.645 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2023-07-21T22:38:55.645 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2023-07-21T22:38:55.645 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2023-07-21T22:38:55.645 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2023-07-21T22:38:55.645 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2023-07-21T22:38:55.646 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2023-07-21T22:38:55.646 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:38:55.646 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2023-07-21T22:38:55.647 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 0, 2023-07-21T22:38:55.647 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 0, 2023-07-21T22:38:55.647 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2023-07-21T22:38:55.647 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:38:55.648 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2023-07-21T22:38:55.648 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:38:55.648 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:38:55.648 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:38:55.649 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:38:55.649 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:38:55.649 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2023-07-21T22:38:55.650 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2023-07-21T22:38:55.650 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T22:38:55.650 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2023-07-21T22:38:55.651 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:38:55.651 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:38:55.651 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:38:55.651 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2023-07-21T22:38:55.652 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2023-07-21T22:38:55.652 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2023-07-21T22:38:55.652 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2023-07-21T22:38:55.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.653 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2023-07-21T22:38:55.653 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2023-07-21T22:38:55.653 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2023-07-21T22:38:55.654 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2023-07-21T22:38:55.654 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2023-07-21T22:38:55.654 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2023-07-21T22:38:55.654 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2023-07-21T22:38:55.654 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2023-07-21T22:38:55.655 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:38:55.655 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2023-07-21T22:38:55.655 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2023-07-21T22:38:55.655 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2023-07-21T22:38:55.655 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2023-07-21T22:38:55.656 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2023-07-21T22:38:55.656 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:38:55.656 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:38:55.656 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2023-07-21T22:38:55.656 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2023-07-21T22:38:55.656 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:38:55.656 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2023-07-21T22:38:55.657 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2023-07-21T22:38:55.657 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2023-07-21T22:38:55.657 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2023-07-21T22:38:55.657 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2023-07-21T22:38:55.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.657 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2023-07-21T22:38:55.657 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 0, 2023-07-21T22:38:55.658 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2023-07-21T22:38:55.658 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2023-07-21T22:38:55.658 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 0, 2023-07-21T22:38:55.658 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2023-07-21T22:38:55.658 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2023-07-21T22:38:55.658 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 0, 2023-07-21T22:38:55.658 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2023-07-21T22:38:55.658 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2023-07-21T22:38:55.659 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 75, 2023-07-21T22:38:55.659 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2023-07-21T22:38:55.659 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2023-07-21T22:38:55.659 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2023-07-21T22:38:55.659 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 0, 2023-07-21T22:38:55.659 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 126343926, 2023-07-21T22:38:55.659 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2023-07-21T22:38:55.659 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.660 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.660 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.660 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 59671 2023-07-21T22:38:55.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.660 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2023-07-21T22:38:55.660 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 13, 2023-07-21T22:38:55.661 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 11898, 2023-07-21T22:38:55.661 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 11885, 2023-07-21T22:38:55.661 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 11, 2023-07-21T22:38:55.661 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 516, 2023-07-21T22:38:55.661 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 505, 2023-07-21T22:38:55.661 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 10660, 2023-07-21T22:38:55.661 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 29076, 2023-07-21T22:38:55.661 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 18416, 2023-07-21T22:38:55.662 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 0, 2023-07-21T22:38:55.662 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 0, 2023-07-21T22:38:55.662 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2023-07-21T22:38:55.662 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 70960, 2023-07-21T22:38:55.662 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 207156 2023-07-21T22:38:55.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.662 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2023-07-21T22:38:55.662 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 0, 2023-07-21T22:38:55.663 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2023-07-21T22:38:55.663 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 0, 2023-07-21T22:38:55.663 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 0, 2023-07-21T22:38:55.663 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2023-07-21T22:38:55.663 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2023-07-21T22:38:55.663 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.663 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.663 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.664 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2023-07-21T22:38:55.664 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.664 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.664 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.664 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2023-07-21T22:38:55.665 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.665 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.665 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.665 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2023-07-21T22:38:55.665 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.665 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.665 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.666 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2023-07-21T22:38:55.666 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.666 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.666 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.666 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2023-07-21T22:38:55.666 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.667 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.667 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.667 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2023-07-21T22:38:55.667 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.667 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.667 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.668 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2023-07-21T22:38:55.668 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.668 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.668 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.668 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2023-07-21T22:38:55.668 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.669 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.669 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.669 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2023-07-21T22:38:55.669 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.669 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.669 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.670 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2023-07-21T22:38:55.670 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.670 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.670 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.670 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2023-07-21T22:38:55.670 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.670 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.671 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.671 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2023-07-21T22:38:55.671 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.671 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.671 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.671 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2023-07-21T22:38:55.672 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.672 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.672 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.672 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2023-07-21T22:38:55.672 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.672 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.673 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.673 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2023-07-21T22:38:55.673 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.673 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.673 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.673 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2023-07-21T22:38:55.674 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.674 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.674 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.674 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2023-07-21T22:38:55.674 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.674 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.674 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.675 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2023-07-21T22:38:55.675 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.675 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.675 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.675 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2023-07-21T22:38:55.675 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.676 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.676 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.676 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2023-07-21T22:38:55.676 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.676 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.676 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.677 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2023-07-21T22:38:55.677 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.677 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.677 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.677 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2023-07-21T22:38:55.677 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.678 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.678 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.678 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2023-07-21T22:38:55.678 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.678 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.678 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.679 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2023-07-21T22:38:55.679 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.679 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.679 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.679 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2023-07-21T22:38:55.679 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.679 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.680 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.680 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2023-07-21T22:38:55.680 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.680 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.680 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.681 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2023-07-21T22:38:55.681 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.681 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.681 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.681 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2023-07-21T22:38:55.681 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.681 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.682 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.682 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:38:55.682 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:38:55.682 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2023-07-21T22:38:55.682 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.682 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.682 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.683 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:55.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.683 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2023-07-21T22:38:55.683 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2023-07-21T22:38:55.683 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 1, 2023-07-21T22:38:55.683 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 0, 2023-07-21T22:38:55.683 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 1, 2023-07-21T22:38:55.684 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2023-07-21T22:38:55.684 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 0, 2023-07-21T22:38:55.684 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2023-07-21T22:38:55.684 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 2115 2023-07-21T22:38:55.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.684 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2023-07-21T22:38:55.684 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:38:55.684 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2023-07-21T22:38:55.685 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:38:55.685 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:38:55.685 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:38:55.685 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:38:55.685 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:38:55.685 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:38:55.685 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:38:55.685 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:38:55.686 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:38:55.686 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:38:55.686 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:38:55.686 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:38:55.686 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:38:55.686 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2023-07-21T22:38:55.686 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:38:55.686 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2023-07-21T22:38:55.687 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:38:55.687 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:38:55.687 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:38:55.687 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:38:55.687 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:38:55.687 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:38:55.687 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:38:55.687 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2023-07-21T22:38:55.688 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:38:55.688 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:38:55.688 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:38:55.688 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2023-07-21T22:38:55.688 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2023-07-21T22:38:55.688 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2023-07-21T22:38:55.688 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:38:55.689 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:38:55.689 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:38:55.689 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:38:55.689 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 231391, 2023-07-21T22:38:55.689 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 75, 2023-07-21T22:38:55.689 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:38:55.689 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2023-07-21T22:38:55.689 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2023-07-21T22:38:55.690 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2023-07-21T22:38:55.690 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:38:55.690 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2023-07-21T22:38:55.690 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:38:55.690 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2023-07-21T22:38:55.690 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 7720, 2023-07-21T22:38:55.690 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 162, 2023-07-21T22:38:55.690 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:38:55.691 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:38:55.691 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2023-07-21T22:38:55.691 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2023-07-21T22:38:55.691 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 7547400, 2023-07-21T22:38:55.691 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 42768, 2023-07-21T22:38:55.691 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2023-07-21T22:38:55.691 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2023-07-21T22:38:55.691 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2023-07-21T22:38:55.692 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2023-07-21T22:38:55.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.692 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2023-07-21T22:38:55.692 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2023-07-21T22:38:55.692 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2023-07-21T22:38:55.692 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 2139, 2023-07-21T22:38:55.692 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 21, 2023-07-21T22:38:55.692 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 3, 2023-07-21T22:38:55.693 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 2139, 2023-07-21T22:38:55.693 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2023-07-21T22:38:55.693 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2139, 2023-07-21T22:38:55.693 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 10.090835091, 2023-07-21T22:38:55.693 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.004717547 2023-07-21T22:38:55.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.693 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2023-07-21T22:38:55.693 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2023-07-21T22:38:55.694 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2139, 2023-07-21T22:38:55.694 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 2140 2023-07-21T22:38:55.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.694 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 2139, 2023-07-21T22:38:55.694 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 2139, 2023-07-21T22:38:55.694 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 0, 2023-07-21T22:38:55.694 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2023-07-21T22:38:55.695 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2023-07-21T22:38:55.695 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 1157, 2023-07-21T22:38:55.695 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 0, 2023-07-21T22:38:55.695 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 980, 2023-07-21T22:38:55.695 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 0, 2023-07-21T22:38:55.695 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 0, 2023-07-21T22:38:55.695 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2023-07-21T22:38:55.695 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2023-07-21T22:38:55.696 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2023-07-21T22:38:55.696 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2023-07-21T22:38:55.696 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 0, 2023-07-21T22:38:55.696 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2023-07-21T22:38:55.696 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2023-07-21T22:38:55.696 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2023-07-21T22:38:55.696 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 1, 2023-07-21T22:38:55.696 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 0, 2023-07-21T22:38:55.697 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:38:55.697 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2023-07-21T22:38:55.697 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:38:55.697 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2023-07-21T22:38:55.697 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2023-07-21T22:38:55.697 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2023-07-21T22:38:55.697 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:38:55.697 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2023-07-21T22:38:55.698 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:38:55.698 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2023-07-21T22:38:55.698 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2023-07-21T22:38:55.698 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2023-07-21T22:38:55.698 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2023-07-21T22:38:55.698 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2023-07-21T22:38:55.698 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2023-07-21T22:38:55.698 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2023-07-21T22:38:55.699 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2023-07-21T22:38:55.699 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2023-07-21T22:38:55.699 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2023-07-21T22:38:55.699 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2023-07-21T22:38:55.699 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2023-07-21T22:38:55.699 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2023-07-21T22:38:55.699 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2023-07-21T22:38:55.699 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2023-07-21T22:38:55.700 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2023-07-21T22:38:55.700 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2023-07-21T22:38:55.700 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 85, 2023-07-21T22:38:55.700 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2023-07-21T22:38:55.700 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 3, 2023-07-21T22:38:55.700 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 10, 2023-07-21T22:38:55.700 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 10, 2023-07-21T22:38:55.701 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2023-07-21T22:38:55.701 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2023-07-21T22:38:55.701 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 0, 2023-07-21T22:38:55.701 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 2, 2023-07-21T22:38:55.701 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 0 2023-07-21T22:38:55.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.701 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2023-07-21T22:38:55.701 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 0, 2023-07-21T22:38:55.702 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:38:55.702 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2023-07-21T22:38:55.702 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2023-07-21T22:38:55.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.702 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2023-07-21T22:38:55.702 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2023-07-21T22:38:55.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.702 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2023-07-21T22:38:55.703 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2023-07-21T22:38:55.703 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T22:38:55.703 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2023-07-21T22:38:55.703 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2023-07-21T22:38:55.703 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2023-07-21T22:38:55.703 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2023-07-21T22:38:55.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.703 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:38:55.704 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2023-07-21T22:38:55.704 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2023-07-21T22:38:55.704 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2023-07-21T22:38:55.704 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 2365, 2023-07-21T22:38:55.704 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 123187090, 2023-07-21T22:38:55.704 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2023-07-21T22:38:55.704 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 2365, 2023-07-21T22:38:55.705 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2023-07-21T22:38:55.705 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2023-07-21T22:38:55.705 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 2365, 2023-07-21T22:38:55.705 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 123187090, 2023-07-21T22:38:55.705 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2023-07-21T22:38:55.705 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.705 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.705 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.706 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:55.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.706 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2023-07-21T22:38:55.706 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2023-07-21T22:38:55.706 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2023-07-21T22:38:55.706 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2023-07-21T22:38:55.706 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2023-07-21T22:38:55.706 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2023-07-21T22:38:55.707 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2023-07-21T22:38:55.707 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2023-07-21T22:38:55.707 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 2139, 2023-07-21T22:38:55.707 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 122156870, 2023-07-21T22:38:55.707 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 2139, 2023-07-21T22:38:55.707 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 122156870, 2023-07-21T22:38:55.707 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2023-07-21T22:38:55.707 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.708 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.708 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.708 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:55.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.708 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2023-07-21T22:38:55.708 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2023-07-21T22:38:55.708 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2023-07-21T22:38:55.708 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2023-07-21T22:38:55.709 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2023-07-21T22:38:55.709 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2023-07-21T22:38:55.709 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2023-07-21T22:38:55.709 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2023-07-21T22:38:55.709 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 2139, 2023-07-21T22:38:55.710 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 2139, 2023-07-21T22:38:55.710 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 2139, 2023-07-21T22:38:55.710 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 2139, 2023-07-21T22:38:55.710 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2023-07-21T22:38:55.710 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.710 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.710 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.710 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:55.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.711 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:38:55.711 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2023-07-21T22:38:55.711 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2023-07-21T22:38:55.711 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2023-07-21T22:38:55.711 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2023-07-21T22:38:55.711 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2023-07-21T22:38:55.711 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2023-07-21T22:38:55.712 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2023-07-21T22:38:55.712 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2023-07-21T22:38:55.712 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2023-07-21T22:38:55.712 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2023-07-21T22:38:55.712 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2023-07-21T22:38:55.712 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2023-07-21T22:38:55.712 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.712 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.713 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.713 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:55.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:55.713 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x555aa6a8c460": { 2023-07-21T22:38:55.713 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2023-07-21T22:38:55.713 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2023-07-21T22:38:55.713 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2023-07-21T22:38:55.714 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2023-07-21T22:38:55.714 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2023-07-21T22:38:55.714 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2023-07-21T22:38:55.714 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2023-07-21T22:38:55.714 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2023-07-21T22:38:55.714 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2023-07-21T22:38:55.714 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2023-07-21T22:38:55.714 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2023-07-21T22:38:55.715 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2023-07-21T22:38:55.715 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:55.715 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:55.715 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:55.715 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:55.715 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:55.715 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:38:55.934 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:55 smithi107 ceph-mon[96543]: pgmap v589: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 958 KiB/s wr, 204 op/s 2023-07-21T22:38:56.053 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-07-21T22:38:56.053 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-07-21T22:38:56.054 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2023-07-21T22:38:56.054 DEBUG:tasks.check_counter:Getting stats from j 2023-07-21T22:38:56.054 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph daemon mds.j perf dump 2023-07-21T22:38:56.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:55 smithi190 ceph-mon[98334]: pgmap v589: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 958 KiB/s wr, 204 op/s 2023-07-21T22:38:56.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:55 smithi132 ceph-mon[97863]: pgmap v589: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 958 KiB/s wr, 204 op/s 2023-07-21T22:38:56.605 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:38:57.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:57 smithi107 ceph-mon[96543]: pgmap v590: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 288 KiB/s wr, 135 op/s 2023-07-21T22:38:58.015 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-21T22:38:58.015 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:38:58.015 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 0, 2023-07-21T22:38:58.015 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2, 2023-07-21T22:38:58.016 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 0, 2023-07-21T22:38:58.016 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 0, 2023-07-21T22:38:58.016 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2023-07-21T22:38:58.016 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2023-07-21T22:38:58.016 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.001165417, 2023-07-21T22:38:58.016 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000000000, 2023-07-21T22:38:58.016 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.000205879, 2023-07-21T22:38:58.017 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:38:58.017 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:38:58.017 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:58.017 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:58.017 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:58.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:58.017 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:38:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:38:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:38:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:38:58.019 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 196, 2023-07-21T22:38:58.019 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 191, 2023-07-21T22:38:58.019 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 466688, 2023-07-21T22:38:58.019 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 108016, 2023-07-21T22:38:58.019 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2023-07-21T22:38:58.019 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2023-07-21T22:38:58.019 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.029107734, 2023-07-21T22:38:58.019 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.011442825, 2023-07-21T22:38:58.020 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.012987820, 2023-07-21T22:38:58.020 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.004892749, 2023-07-21T22:38:58.020 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:38:58.020 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 191, 2023-07-21T22:38:58.020 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.070810148, 2023-07-21T22:38:58.020 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000370733 2023-07-21T22:38:58.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:58.021 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:38:58.021 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:58.021 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:58.021 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:58.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:58.021 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 466688, 2023-07-21T22:38:58.021 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 108016 2023-07-21T22:38:58.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:58.022 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:38:58.022 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1, 2023-07-21T22:38:58.022 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 144, 2023-07-21T22:38:58.022 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 160, 2023-07-21T22:38:58.022 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 100912, 2023-07-21T22:38:58.022 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 3, 2023-07-21T22:38:58.022 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2023-07-21T22:38:58.022 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.010752054, 2023-07-21T22:38:58.023 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.008341856, 2023-07-21T22:38:58.023 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.001087686, 2023-07-21T22:38:58.023 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:38:58.023 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:38:58.023 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 142, 2023-07-21T22:38:58.023 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.010011665, 2023-07-21T22:38:58.023 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000070504 2023-07-21T22:38:58.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:58.024 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:38:58.024 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:58.024 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:58.024 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:58.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:58.024 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-21T22:38:58.024 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 100912 2023-07-21T22:38:58.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:58.025 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2023-07-21T22:38:58.025 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2023-07-21T22:38:58.025 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2023-07-21T22:38:58.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:58.025 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2023-07-21T22:38:58.025 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:38:58.025 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2023-07-21T22:38:58.026 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:38:58.026 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:38:58.026 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:38:58.026 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:38:58.026 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:38:58.026 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:38:58.026 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:38:58.026 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:38:58.027 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:38:58.027 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:38:58.027 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:38:58.027 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:38:58.027 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:38:58.027 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2023-07-21T22:38:58.027 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:38:58.028 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2023-07-21T22:38:58.028 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:38:58.028 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:38:58.028 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:38:58.028 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:38:58.028 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:38:58.028 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:38:58.028 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:38:58.029 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2023-07-21T22:38:58.029 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:38:58.029 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:38:58.029 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:38:58.029 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2023-07-21T22:38:58.029 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2023-07-21T22:38:58.029 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2023-07-21T22:38:58.029 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:38:58.030 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:38:58.030 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:38:58.030 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:38:58.030 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 149007, 2023-07-21T22:38:58.030 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 30, 2023-07-21T22:38:58.030 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:38:58.030 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2023-07-21T22:38:58.030 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2023-07-21T22:38:58.031 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2023-07-21T22:38:58.031 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:38:58.031 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2023-07-21T22:38:58.031 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:38:58.031 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2023-07-21T22:38:58.031 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2023-07-21T22:38:58.031 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2023-07-21T22:38:58.032 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:38:58.032 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:38:58.032 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2023-07-21T22:38:58.032 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2023-07-21T22:38:58.032 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2023-07-21T22:38:58.032 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2023-07-21T22:38:58.033 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2023-07-21T22:38:58.033 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2023-07-21T22:38:58.033 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2023-07-21T22:38:58.033 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2023-07-21T22:38:58.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-21T22:38:58.033 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:38:58.033 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2023-07-21T22:38:58.034 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2023-07-21T22:38:58.034 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2023-07-21T22:38:58.034 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 193, 2023-07-21T22:38:58.034 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 436607, 2023-07-21T22:38:58.034 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2023-07-21T22:38:58.034 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 193, 2023-07-21T22:38:58.034 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2023-07-21T22:38:58.034 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2023-07-21T22:38:58.035 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 193, 2023-07-21T22:38:58.035 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 436607, 2023-07-21T22:38:58.035 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2023-07-21T22:38:58.035 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2023-07-21T22:38:58.035 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2023-07-21T22:38:58.035 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2023-07-21T22:38:58.035 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:58.035 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-21T22:38:58.036 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-21T22:38:58.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:57 smithi190 ceph-mon[98334]: pgmap v590: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 288 KiB/s wr, 135 op/s 2023-07-21T22:38:58.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:57 smithi132 ceph-mon[97863]: pgmap v590: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 288 KiB/s wr, 135 op/s 2023-07-21T22:38:58.350 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-07-21T22:38:58.351 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-07-21T22:38:58.351 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-07-21T22:38:58.351 DEBUG:tasks.check_counter:Getting stats from b 2023-07-21T22:38:58.351 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph daemon mds.b perf dump 2023-07-21T22:38:59.009 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:38:59.879 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:38:59 smithi132 ceph-mon[97863]: pgmap v591: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 98 KiB/s wr, 91 op/s 2023-07-21T22:39:00.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:38:59 smithi190 ceph-mon[98334]: pgmap v591: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 98 KiB/s wr, 91 op/s 2023-07-21T22:39:00.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:38:59 smithi107 ceph-mon[96543]: pgmap v591: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 98 KiB/s wr, 91 op/s 2023-07-21T22:39:00.472 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2023-07-21T22:39:00.472 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:39:00.472 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 364373, 2023-07-21T22:39:00.473 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 368099, 2023-07-21T22:39:00.473 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 68462544, 2023-07-21T22:39:00.473 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 121170922, 2023-07-21T22:39:00.473 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 11, 2023-07-21T22:39:00.473 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 5, 2023-07-21T22:39:00.473 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 15.688963289, 2023-07-21T22:39:00.473 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 6.757684408, 2023-07-21T22:39:00.474 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 10.610503478, 2023-07-21T22:39:00.474 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.441607907, 2023-07-21T22:39:00.474 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:00.474 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 368097, 2023-07-21T22:39:00.474 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 26.393856269, 2023-07-21T22:39:00.474 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000071703 2023-07-21T22:39:00.474 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.474 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:00.475 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 710252, 2023-07-21T22:39:00.475 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.079371872, 2023-07-21T22:39:00.475 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000111 2023-07-21T22:39:00.475 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.475 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 699168, 2023-07-21T22:39:00.475 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 119072 2023-07-21T22:39:00.475 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.475 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:39:00.476 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 1226157, 2023-07-21T22:39:00.476 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 1228707, 2023-07-21T22:39:00.476 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 202209043, 2023-07-21T22:39:00.476 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 244200172, 2023-07-21T22:39:00.476 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 142, 2023-07-21T22:39:00.476 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 7, 2023-07-21T22:39:00.477 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 35.076091112, 2023-07-21T22:39:00.477 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 13.704880516, 2023-07-21T22:39:00.477 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 23.793980854, 2023-07-21T22:39:00.478 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.502471620, 2023-07-21T22:39:00.478 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:00.478 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1228707, 2023-07-21T22:39:00.478 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 51.875692619, 2023-07-21T22:39:00.479 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000042219 2023-07-21T22:39:00.479 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.479 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:00.480 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2357604, 2023-07-21T22:39:00.480 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.260936335, 2023-07-21T22:39:00.480 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000110 2023-07-21T22:39:00.481 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.481 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 7952, 2023-07-21T22:39:00.482 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 327216 2023-07-21T22:39:00.482 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.482 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:39:00.482 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 140200, 2023-07-21T22:39:00.483 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 204595, 2023-07-21T22:39:00.483 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 43672143, 2023-07-21T22:39:00.483 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 153694097, 2023-07-21T22:39:00.483 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 15, 2023-07-21T22:39:00.484 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 7, 2023-07-21T22:39:00.484 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 5.613896716, 2023-07-21T22:39:00.484 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 2.699607539, 2023-07-21T22:39:00.484 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 4.692227320, 2023-07-21T22:39:00.485 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.474377128, 2023-07-21T22:39:00.485 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:00.485 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 204593, 2023-07-21T22:39:00.485 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 54.951998894, 2023-07-21T22:39:00.486 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000268591 2023-07-21T22:39:00.486 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.486 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:00.486 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 116465, 2023-07-21T22:39:00.487 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.029662527, 2023-07-21T22:39:00.487 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000254 2023-07-21T22:39:00.487 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.487 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 50592, 2023-07-21T22:39:00.487 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 29504 2023-07-21T22:39:00.487 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.487 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2023-07-21T22:39:00.488 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 1, 2023-07-21T22:39:00.488 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2023-07-21T22:39:00.488 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.488 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-MDSRank": { 2023-07-21T22:39:00.488 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-07-21T22:39:00.488 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-07-21T22:39:00.488 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 6812, 2023-07-21T22:39:00.489 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 36.419815893, 2023-07-21T22:39:00.489 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.005346420 2023-07-21T22:39:00.489 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:00.489 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.489 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-PurgeQueue": { 2023-07-21T22:39:00.489 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-07-21T22:39:00.489 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-07-21T22:39:00.490 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 21993, 2023-07-21T22:39:00.490 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 4.387299254, 2023-07-21T22:39:00.490 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000199486 2023-07-21T22:39:00.490 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:00.490 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.490 INFO:teuthology.orchestra.run.smithi132.stdout: "mds": { 2023-07-21T22:39:00.490 INFO:teuthology.orchestra.run.smithi132.stdout: "request": 75591, 2023-07-21T22:39:00.490 INFO:teuthology.orchestra.run.smithi132.stdout: "reply": 75047, 2023-07-21T22:39:00.491 INFO:teuthology.orchestra.run.smithi132.stdout: "reply_latency": { 2023-07-21T22:39:00.491 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 75047, 2023-07-21T22:39:00.491 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 2014.855659246, 2023-07-21T22:39:00.491 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.026847917 2023-07-21T22:39:00.491 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.491 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_reply": 0, 2023-07-21T22:39:00.491 INFO:teuthology.orchestra.run.smithi132.stdout: "forward": 544, 2023-07-21T22:39:00.492 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_complete": 52, 2023-07-21T22:39:00.492 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_keys": 0, 2023-07-21T22:39:00.492 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_commit": 3503, 2023-07-21T22:39:00.492 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_split": 41, 2023-07-21T22:39:00.492 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_merge": 0, 2023-07-21T22:39:00.492 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes": 40, 2023-07-21T22:39:00.493 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_top": 17, 2023-07-21T22:39:00.493 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_bottom": 13, 2023-07-21T22:39:00.493 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pin_tail": 10, 2023-07-21T22:39:00.493 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pinned": 15, 2023-07-21T22:39:00.493 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_expired": 8170, 2023-07-21T22:39:00.493 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_with_caps": 1, 2023-07-21T22:39:00.493 INFO:teuthology.orchestra.run.smithi132.stdout: "caps": 1, 2023-07-21T22:39:00.494 INFO:teuthology.orchestra.run.smithi132.stdout: "subtrees": 6, 2023-07-21T22:39:00.494 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse": 133012, 2023-07-21T22:39:00.494 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_hit": 109746, 2023-07-21T22:39:00.494 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_forward": 429, 2023-07-21T22:39:00.494 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_discover": 293, 2023-07-21T22:39:00.494 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:39:00.494 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_remote_ino": 0, 2023-07-21T22:39:00.494 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_lock": 987, 2023-07-21T22:39:00.495 INFO:teuthology.orchestra.run.smithi132.stdout: "load_cent": 344, 2023-07-21T22:39:00.495 INFO:teuthology.orchestra.run.smithi132.stdout: "q": 0, 2023-07-21T22:39:00.495 INFO:teuthology.orchestra.run.smithi132.stdout: "exported": 218, 2023-07-21T22:39:00.495 INFO:teuthology.orchestra.run.smithi132.stdout: "exported_inodes": 5187, 2023-07-21T22:39:00.495 INFO:teuthology.orchestra.run.smithi132.stdout: "imported": 190, 2023-07-21T22:39:00.495 INFO:teuthology.orchestra.run.smithi132.stdout: "imported_inodes": 11917, 2023-07-21T22:39:00.495 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_dir_fetch": 0, 2023-07-21T22:39:00.495 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_backtrace_fetch": 331, 2023-07-21T22:39:00.496 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_peer_discover": 0, 2023-07-21T22:39:00.496 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rfiles": 4, 2023-07-21T22:39:00.496 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rbytes": 246, 2023-07-21T22:39:00.496 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rsnaps": 0, 2023-07-21T22:39:00.496 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_fetch": 331, 2023-07-21T22:39:00.496 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_set_tag": 0, 2023-07-21T22:39:00.496 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:39:00.497 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:39:00.497 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_inodes": 30, 2023-07-21T22:39:00.497 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_base_inodes": 4, 2023-07-21T22:39:00.497 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dirfrag_rstats": 26, 2023-07-21T22:39:00.497 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_file_inodes": 301, 2023-07-21T22:39:00.497 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_inode_file_caps": 155, 2023-07-21T22:39:00.497 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_revoke": 9096, 2023-07-21T22:39:00.498 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_grant": 21851, 2023-07-21T22:39:00.498 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:39:00.498 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:39:00.498 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:39:00.498 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps": 14225, 2023-07-21T22:39:00.498 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps_dirty": 8533, 2023-07-21T22:39:00.498 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_cap_release": 375, 2023-07-21T22:39:00.498 INFO:teuthology.orchestra.run.smithi132.stdout: "process_request_cap_release": 26725 2023-07-21T22:39:00.499 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.499 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_cache": { 2023-07-21T22:39:00.499 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update": 3, 2023-07-21T22:39:00.499 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update_receipt": 9, 2023-07-21T22:39:00.499 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_try_discover": 6, 2023-07-21T22:39:00.499 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_send_discover": 391, 2023-07-21T22:39:00.499 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_handle_discover": 594, 2023-07-21T22:39:00.499 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays": 0, 2023-07-21T22:39:00.500 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_delayed": 0, 2023-07-21T22:39:00.500 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:39:00.500 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_created": 12024, 2023-07-21T22:39:00.500 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_enqueued": 12024, 2023-07-21T22:39:00.500 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_reintegrated": 0, 2023-07-21T22:39:00.500 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_migrated": 0, 2023-07-21T22:39:00.500 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_processing": 0, 2023-07-21T22:39:00.501 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:39:00.501 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:39:00.501 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_started": 0, 2023-07-21T22:39:00.501 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_completed": 0, 2023-07-21T22:39:00.501 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_enqueue_scrub": 4, 2023-07-21T22:39:00.501 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_exportdir": 218, 2023-07-21T22:39:00.501 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_flush": 0, 2023-07-21T22:39:00.501 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragmentdir": 42, 2023-07-21T22:39:00.502 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragstats": 0, 2023-07-21T22:39:00.502 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_inodestats": 0 2023-07-21T22:39:00.502 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.502 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_log": { 2023-07-21T22:39:00.502 INFO:teuthology.orchestra.run.smithi132.stdout: "evadd": 57816, 2023-07-21T22:39:00.502 INFO:teuthology.orchestra.run.smithi132.stdout: "evex": 57810, 2023-07-21T22:39:00.502 INFO:teuthology.orchestra.run.smithi132.stdout: "evtrm": 57810, 2023-07-21T22:39:00.502 INFO:teuthology.orchestra.run.smithi132.stdout: "ev": 6, 2023-07-21T22:39:00.503 INFO:teuthology.orchestra.run.smithi132.stdout: "evexg": 0, 2023-07-21T22:39:00.503 INFO:teuthology.orchestra.run.smithi132.stdout: "evexd": 10, 2023-07-21T22:39:00.503 INFO:teuthology.orchestra.run.smithi132.stdout: "segadd": 102, 2023-07-21T22:39:00.503 INFO:teuthology.orchestra.run.smithi132.stdout: "segex": 101, 2023-07-21T22:39:00.503 INFO:teuthology.orchestra.run.smithi132.stdout: "segtrm": 101, 2023-07-21T22:39:00.503 INFO:teuthology.orchestra.run.smithi132.stdout: "seg": 2, 2023-07-21T22:39:00.503 INFO:teuthology.orchestra.run.smithi132.stdout: "segexg": 0, 2023-07-21T22:39:00.503 INFO:teuthology.orchestra.run.smithi132.stdout: "segexd": 1, 2023-07-21T22:39:00.504 INFO:teuthology.orchestra.run.smithi132.stdout: "expos": 133994271, 2023-07-21T22:39:00.504 INFO:teuthology.orchestra.run.smithi132.stdout: "wrpos": 134012917, 2023-07-21T22:39:00.504 INFO:teuthology.orchestra.run.smithi132.stdout: "rdpos": 0, 2023-07-21T22:39:00.504 INFO:teuthology.orchestra.run.smithi132.stdout: "jlat": { 2023-07-21T22:39:00.504 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2562, 2023-07-21T22:39:00.504 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 70.006664316, 2023-07-21T22:39:00.504 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.027325005 2023-07-21T22:39:00.505 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.505 INFO:teuthology.orchestra.run.smithi132.stdout: "replayed": 0 2023-07-21T22:39:00.505 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.505 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_mem": { 2023-07-21T22:39:00.505 INFO:teuthology.orchestra.run.smithi132.stdout: "ino": 46, 2023-07-21T22:39:00.505 INFO:teuthology.orchestra.run.smithi132.stdout: "ino+": 15606, 2023-07-21T22:39:00.505 INFO:teuthology.orchestra.run.smithi132.stdout: "ino-": 15560, 2023-07-21T22:39:00.505 INFO:teuthology.orchestra.run.smithi132.stdout: "dir": 45, 2023-07-21T22:39:00.506 INFO:teuthology.orchestra.run.smithi132.stdout: "dir+": 737, 2023-07-21T22:39:00.506 INFO:teuthology.orchestra.run.smithi132.stdout: "dir-": 692, 2023-07-21T22:39:00.506 INFO:teuthology.orchestra.run.smithi132.stdout: "dn": 40, 2023-07-21T22:39:00.506 INFO:teuthology.orchestra.run.smithi132.stdout: "dn+": 38843, 2023-07-21T22:39:00.506 INFO:teuthology.orchestra.run.smithi132.stdout: "dn-": 38803, 2023-07-21T22:39:00.506 INFO:teuthology.orchestra.run.smithi132.stdout: "cap": 1, 2023-07-21T22:39:00.506 INFO:teuthology.orchestra.run.smithi132.stdout: "cap+": 17886, 2023-07-21T22:39:00.506 INFO:teuthology.orchestra.run.smithi132.stdout: "cap-": 17885, 2023-07-21T22:39:00.507 INFO:teuthology.orchestra.run.smithi132.stdout: "rss": 104288, 2023-07-21T22:39:00.507 INFO:teuthology.orchestra.run.smithi132.stdout: "heap": 207156 2023-07-21T22:39:00.507 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.507 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_server": { 2023-07-21T22:39:00.509 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_client_request": 89730, 2023-07-21T22:39:00.509 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_server_request": 1736, 2023-07-21T22:39:00.509 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_request": 75591, 2023-07-21T22:39:00.509 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_session": 242, 2023-07-21T22:39:00.509 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_peer_request": 3520, 2023-07-21T22:39:00.509 INFO:teuthology.orchestra.run.smithi132.stdout: "req_create_latency": { 2023-07-21T22:39:00.509 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 8317, 2023-07-21T22:39:00.510 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 26.488605064, 2023-07-21T22:39:00.510 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.003184874 2023-07-21T22:39:00.510 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.510 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getattr_latency": { 2023-07-21T22:39:00.510 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1659, 2023-07-21T22:39:00.510 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 37.386441993, 2023-07-21T22:39:00.510 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.022535528 2023-07-21T22:39:00.510 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.511 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getfilelock_latency": { 2023-07-21T22:39:00.511 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.511 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.511 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.511 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.511 INFO:teuthology.orchestra.run.smithi132.stdout: "req_link_latency": { 2023-07-21T22:39:00.511 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.511 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.512 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.512 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.512 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookup_latency": { 2023-07-21T22:39:00.512 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 42999, 2023-07-21T22:39:00.512 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1690.002122332, 2023-07-21T22:39:00.512 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.039303288 2023-07-21T22:39:00.512 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.513 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookuphash_latency": { 2023-07-21T22:39:00.513 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.513 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.513 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.513 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.513 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupino_latency": { 2023-07-21T22:39:00.513 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.514 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.514 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.514 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.514 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupname_latency": { 2023-07-21T22:39:00.514 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.514 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.514 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.514 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.515 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupparent_latency": { 2023-07-21T22:39:00.515 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.515 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.515 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.515 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.515 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupsnap_latency": { 2023-07-21T22:39:00.515 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.515 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.516 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.516 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.516 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lssnap_latency": { 2023-07-21T22:39:00.516 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.516 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.516 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.516 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.517 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mkdir_latency": { 2023-07-21T22:39:00.517 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 47, 2023-07-21T22:39:00.517 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1.827601118, 2023-07-21T22:39:00.517 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.038885130 2023-07-21T22:39:00.517 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.517 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mknod_latency": { 2023-07-21T22:39:00.517 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.517 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.518 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.518 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.518 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mksnap_latency": { 2023-07-21T22:39:00.518 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.518 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.518 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.519 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.519 INFO:teuthology.orchestra.run.smithi132.stdout: "req_open_latency": { 2023-07-21T22:39:00.519 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3087, 2023-07-21T22:39:00.519 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 7.243291150, 2023-07-21T22:39:00.519 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.002346385 2023-07-21T22:39:00.520 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.520 INFO:teuthology.orchestra.run.smithi132.stdout: "req_readdir_latency": { 2023-07-21T22:39:00.520 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 441, 2023-07-21T22:39:00.520 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.803796468, 2023-07-21T22:39:00.520 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.001822667 2023-07-21T22:39:00.520 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.520 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rename_latency": { 2023-07-21T22:39:00.521 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 8353, 2023-07-21T22:39:00.521 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 229.425874625, 2023-07-21T22:39:00.521 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.027466284 2023-07-21T22:39:00.521 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.521 INFO:teuthology.orchestra.run.smithi132.stdout: "req_renamesnap_latency": { 2023-07-21T22:39:00.521 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.521 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.521 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.522 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.522 INFO:teuthology.orchestra.run.smithi132.stdout: "req_snapdiff_latency": { 2023-07-21T22:39:00.522 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.522 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.522 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.522 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.522 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmdir_latency": { 2023-07-21T22:39:00.523 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 57, 2023-07-21T22:39:00.523 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 6.133835035, 2023-07-21T22:39:00.523 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.107611140 2023-07-21T22:39:00.523 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.523 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmsnap_latency": { 2023-07-21T22:39:00.523 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.523 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.523 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.524 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.524 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmxattr_latency": { 2023-07-21T22:39:00.524 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.524 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.524 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.524 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.524 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setattr_latency": { 2023-07-21T22:39:00.525 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1, 2023-07-21T22:39:00.525 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.018304912, 2023-07-21T22:39:00.525 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.018304912 2023-07-21T22:39:00.525 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.525 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setdirlayout_latency": { 2023-07-21T22:39:00.525 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.525 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.525 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.526 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.526 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setfilelock_latency": { 2023-07-21T22:39:00.526 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.526 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.526 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.526 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.526 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setlayout_latency": { 2023-07-21T22:39:00.526 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.527 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.527 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.527 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.527 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setxattr_latency": { 2023-07-21T22:39:00.527 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 6, 2023-07-21T22:39:00.527 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.018615842, 2023-07-21T22:39:00.527 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.003102640 2023-07-21T22:39:00.527 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.528 INFO:teuthology.orchestra.run.smithi132.stdout: "req_symlink_latency": { 2023-07-21T22:39:00.528 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.528 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.528 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.528 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.528 INFO:teuthology.orchestra.run.smithi132.stdout: "req_unlink_latency": { 2023-07-21T22:39:00.528 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 10080, 2023-07-21T22:39:00.529 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 15.507170707, 2023-07-21T22:39:00.529 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.001538409 2023-07-21T22:39:00.529 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.529 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:39:00.531 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:39:00.531 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getvxattr_latency": { 2023-07-21T22:39:00.531 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.531 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.532 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.532 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:00.532 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.532 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_sessions": { 2023-07-21T22:39:00.532 INFO:teuthology.orchestra.run.smithi132.stdout: "session_count": 1, 2023-07-21T22:39:00.532 INFO:teuthology.orchestra.run.smithi132.stdout: "session_add": 2, 2023-07-21T22:39:00.532 INFO:teuthology.orchestra.run.smithi132.stdout: "session_remove": 1, 2023-07-21T22:39:00.533 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_open": 1, 2023-07-21T22:39:00.533 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_stale": 0, 2023-07-21T22:39:00.533 INFO:teuthology.orchestra.run.smithi132.stdout: "total_load": 5249, 2023-07-21T22:39:00.533 INFO:teuthology.orchestra.run.smithi132.stdout: "average_load": 5249, 2023-07-21T22:39:00.533 INFO:teuthology.orchestra.run.smithi132.stdout: "avg_session_uptime": 592 2023-07-21T22:39:00.533 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.533 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2023-07-21T22:39:00.533 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 360, 2023-07-21T22:39:00.534 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 360, 2023-07-21T22:39:00.534 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:39:00.534 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:39:00.534 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:39:00.534 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:39:00.534 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:39:00.534 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:39:00.535 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:39:00.535 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:39:00.535 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:39:00.535 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:39:00.535 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:39:00.535 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:39:00.535 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:39:00.535 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2023-07-21T22:39:00.536 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:39:00.536 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2023-07-21T22:39:00.536 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:39:00.536 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:39:00.536 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:39:00.536 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:39:00.536 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:39:00.537 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:39:00.537 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:39:00.537 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2023-07-21T22:39:00.537 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:39:00.537 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:39:00.537 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:39:00.537 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2023-07-21T22:39:00.537 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2023-07-21T22:39:00.538 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2023-07-21T22:39:00.538 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:39:00.538 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:39:00.538 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:39:00.538 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:39:00.538 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 314802, 2023-07-21T22:39:00.538 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 128, 2023-07-21T22:39:00.539 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:39:00.539 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2023-07-21T22:39:00.539 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2023-07-21T22:39:00.539 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2023-07-21T22:39:00.539 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:39:00.539 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2023-07-21T22:39:00.539 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:39:00.539 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2023-07-21T22:39:00.540 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 7720, 2023-07-21T22:39:00.540 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 162, 2023-07-21T22:39:00.540 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:39:00.540 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:39:00.540 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2023-07-21T22:39:00.540 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2023-07-21T22:39:00.540 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 191093, 2023-07-21T22:39:00.540 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 931, 2023-07-21T22:39:00.541 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2023-07-21T22:39:00.541 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2023-07-21T22:39:00.541 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2023-07-21T22:39:00.541 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2023-07-21T22:39:00.541 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.541 INFO:teuthology.orchestra.run.smithi132.stdout: "objecter": { 2023-07-21T22:39:00.541 INFO:teuthology.orchestra.run.smithi132.stdout: "op_active": 0, 2023-07-21T22:39:00.542 INFO:teuthology.orchestra.run.smithi132.stdout: "op_laggy": 0, 2023-07-21T22:39:00.542 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send": 36726, 2023-07-21T22:39:00.542 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send_bytes": 157793783, 2023-07-21T22:39:00.542 INFO:teuthology.orchestra.run.smithi132.stdout: "op_resend": 37, 2023-07-21T22:39:00.542 INFO:teuthology.orchestra.run.smithi132.stdout: "op_reply": 36726, 2023-07-21T22:39:00.542 INFO:teuthology.orchestra.run.smithi132.stdout: "op_latency": { 2023-07-21T22:39:00.542 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 36726, 2023-07-21T22:39:00.542 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 4569.136076478, 2023-07-21T22:39:00.543 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.124411481 2023-07-21T22:39:00.543 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.543 INFO:teuthology.orchestra.run.smithi132.stdout: "op_inflight": 0, 2023-07-21T22:39:00.543 INFO:teuthology.orchestra.run.smithi132.stdout: "oplen_avg": { 2023-07-21T22:39:00.543 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 36726, 2023-07-21T22:39:00.543 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 75391 2023-07-21T22:39:00.543 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.543 INFO:teuthology.orchestra.run.smithi132.stdout: "op": 36726, 2023-07-21T22:39:00.544 INFO:teuthology.orchestra.run.smithi132.stdout: "op_r": 919, 2023-07-21T22:39:00.544 INFO:teuthology.orchestra.run.smithi132.stdout: "op_w": 35807, 2023-07-21T22:39:00.544 INFO:teuthology.orchestra.run.smithi132.stdout: "op_rmw": 0, 2023-07-21T22:39:00.544 INFO:teuthology.orchestra.run.smithi132.stdout: "op_pg": 0, 2023-07-21T22:39:00.544 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_stat": 3097, 2023-07-21T22:39:00.544 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_create": 15778, 2023-07-21T22:39:00.544 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_read": 439, 2023-07-21T22:39:00.545 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_write": 3000, 2023-07-21T22:39:00.545 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writefull": 385, 2023-07-21T22:39:00.545 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writesame": 0, 2023-07-21T22:39:00.545 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_append": 0, 2023-07-21T22:39:00.545 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_zero": 0, 2023-07-21T22:39:00.545 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_truncate": 41, 2023-07-21T22:39:00.545 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_delete": 12940, 2023-07-21T22:39:00.546 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_mapext": 0, 2023-07-21T22:39:00.546 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_sparse_read": 0, 2023-07-21T22:39:00.546 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_clonerange": 0, 2023-07-21T22:39:00.546 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_getxattr": 335, 2023-07-21T22:39:00.546 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_setxattr": 31556, 2023-07-21T22:39:00.546 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:39:00.546 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_rmxattr": 0, 2023-07-21T22:39:00.546 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:39:00.547 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_call": 0, 2023-07-21T22:39:00.547 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_watch": 0, 2023-07-21T22:39:00.547 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_notify": 0, 2023-07-21T22:39:00.547 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:39:00.547 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls": 0, 2023-07-21T22:39:00.547 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:39:00.547 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_other": 0, 2023-07-21T22:39:00.548 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_active": 0, 2023-07-21T22:39:00.548 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_send": 0, 2023-07-21T22:39:00.548 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_resend": 0, 2023-07-21T22:39:00.548 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_ping": 0, 2023-07-21T22:39:00.548 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_active": 0, 2023-07-21T22:39:00.548 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_send": 0, 2023-07-21T22:39:00.548 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_resend": 0, 2023-07-21T22:39:00.548 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_active": 0, 2023-07-21T22:39:00.549 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_send": 0, 2023-07-21T22:39:00.549 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_resend": 0, 2023-07-21T22:39:00.549 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_active": 0, 2023-07-21T22:39:00.549 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_send": 0, 2023-07-21T22:39:00.549 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_resend": 0, 2023-07-21T22:39:00.549 INFO:teuthology.orchestra.run.smithi132.stdout: "command_active": 0, 2023-07-21T22:39:00.549 INFO:teuthology.orchestra.run.smithi132.stdout: "command_send": 0, 2023-07-21T22:39:00.550 INFO:teuthology.orchestra.run.smithi132.stdout: "command_resend": 0, 2023-07-21T22:39:00.550 INFO:teuthology.orchestra.run.smithi132.stdout: "map_epoch": 85, 2023-07-21T22:39:00.550 INFO:teuthology.orchestra.run.smithi132.stdout: "map_full": 0, 2023-07-21T22:39:00.550 INFO:teuthology.orchestra.run.smithi132.stdout: "map_inc": 4, 2023-07-21T22:39:00.550 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_sessions": 12, 2023-07-21T22:39:00.550 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_open": 12, 2023-07-21T22:39:00.550 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_close": 0, 2023-07-21T22:39:00.550 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_laggy": 0, 2023-07-21T22:39:00.551 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_wr": 6342, 2023-07-21T22:39:00.551 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_rd": 197, 2023-07-21T22:39:00.551 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_del": 1281 2023-07-21T22:39:00.551 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.551 INFO:teuthology.orchestra.run.smithi132.stdout: "oft": { 2023-07-21T22:39:00.551 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_objs": 1, 2023-07-21T22:39:00.551 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:39:00.552 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_updates": 0, 2023-07-21T22:39:00.552 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_removes": 0 2023-07-21T22:39:00.552 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.552 INFO:teuthology.orchestra.run.smithi132.stdout: "osd-slow-ops": { 2023-07-21T22:39:00.552 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_ops_count": 76151 2023-07-21T22:39:00.552 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.552 INFO:teuthology.orchestra.run.smithi132.stdout: "purge_queue": { 2023-07-21T22:39:00.553 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops": 0, 2023-07-21T22:39:00.553 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops_high_water": 14, 2023-07-21T22:39:00.553 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing": 0, 2023-07-21T22:39:00.553 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_high_water": 6, 2023-07-21T22:39:00.553 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed": 12024, 2023-07-21T22:39:00.553 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_item_in_journal": 0 2023-07-21T22:39:00.553 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.553 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:39:00.554 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:00.554 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-07-21T22:39:00.554 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:00.554 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 1730726, 2023-07-21T22:39:00.554 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 160473817, 2023-07-21T22:39:00.554 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:00.554 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 1730726, 2023-07-21T22:39:00.554 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-07-21T22:39:00.555 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-07-21T22:39:00.555 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 1730726, 2023-07-21T22:39:00.555 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 160473817, 2023-07-21T22:39:00.555 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:00.555 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.555 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.555 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.556 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:00.556 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.556 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_bytes": { 2023-07-21T22:39:00.556 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:00.556 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-07-21T22:39:00.556 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:00.556 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-07-21T22:39:00.556 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-07-21T22:39:00.557 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:00.557 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:00.557 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 36726, 2023-07-21T22:39:00.557 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 159004475, 2023-07-21T22:39:00.557 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 36726, 2023-07-21T22:39:00.557 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 159004475, 2023-07-21T22:39:00.557 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:00.558 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.558 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.558 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.558 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:00.558 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.558 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_ops": { 2023-07-21T22:39:00.558 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:00.558 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 1024, 2023-07-21T22:39:00.559 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:00.559 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-07-21T22:39:00.559 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-07-21T22:39:00.559 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:00.559 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:00.559 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 36726, 2023-07-21T22:39:00.559 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 36726, 2023-07-21T22:39:00.559 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 36726, 2023-07-21T22:39:00.560 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 36726, 2023-07-21T22:39:00.560 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:00.560 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.560 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.560 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.560 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:00.560 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.560 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:39:00.561 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:00.561 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-07-21T22:39:00.561 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:00.561 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 12024, 2023-07-21T22:39:00.561 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 1214768, 2023-07-21T22:39:00.561 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:00.561 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 12024, 2023-07-21T22:39:00.561 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-07-21T22:39:00.562 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-07-21T22:39:00.562 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 438, 2023-07-21T22:39:00.562 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1214768, 2023-07-21T22:39:00.562 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:00.562 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.562 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.562 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.562 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:00.563 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:00.563 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle-0x562a5716c0a0": { 2023-07-21T22:39:00.563 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:00.563 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-07-21T22:39:00.563 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:00.563 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 57816, 2023-07-21T22:39:00.563 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 129818613, 2023-07-21T22:39:00.563 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:00.564 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 57816, 2023-07-21T22:39:00.564 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-07-21T22:39:00.564 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-07-21T22:39:00.564 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 2562, 2023-07-21T22:39:00.564 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 129818613, 2023-07-21T22:39:00.564 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:00.564 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:00.565 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:00.565 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:00.565 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:00.565 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:00.565 INFO:teuthology.orchestra.run.smithi132.stdout:} 2023-07-21T22:39:00.852 INFO:tasks.check_counter:Daemon mds.b mds.exported=218 2023-07-21T22:39:00.852 INFO:tasks.check_counter:Daemon mds.b mds.imported=190 2023-07-21T22:39:00.853 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=41 2023-07-21T22:39:00.853 DEBUG:tasks.check_counter:Getting stats from e 2023-07-21T22:39:00.853 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph daemon mds.e perf dump 2023-07-21T22:39:01.404 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:39:01.897 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:01 smithi132 ceph-mon[97863]: pgmap v592: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 17 op/s 2023-07-21T22:39:02.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:01 smithi190 ceph-mon[98334]: pgmap v592: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 17 op/s 2023-07-21T22:39:02.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:01 smithi107 ceph-mon[96543]: pgmap v592: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 17 op/s 2023-07-21T22:39:02.624 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2023-07-21T22:39:02.625 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:39:02.625 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 176281, 2023-07-21T22:39:02.625 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 168755, 2023-07-21T22:39:02.625 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 44119955, 2023-07-21T22:39:02.625 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 150615659, 2023-07-21T22:39:02.625 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 11, 2023-07-21T22:39:02.625 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 5, 2023-07-21T22:39:02.626 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 6.946437254, 2023-07-21T22:39:02.626 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 2.871424375, 2023-07-21T22:39:02.626 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 4.414188628, 2023-07-21T22:39:02.626 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.322127217, 2023-07-21T22:39:02.626 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:02.626 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 168752, 2023-07-21T22:39:02.626 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 4.699113251, 2023-07-21T22:39:02.627 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000027846 2023-07-21T22:39:02.627 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.627 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:02.627 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 329626, 2023-07-21T22:39:02.627 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.039325610, 2023-07-21T22:39:02.627 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000119 2023-07-21T22:39:02.627 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.628 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 679456, 2023-07-21T22:39:02.628 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 118656 2023-07-21T22:39:02.628 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.628 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:39:02.628 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 435862, 2023-07-21T22:39:02.628 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 434963, 2023-07-21T22:39:02.628 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 78205184, 2023-07-21T22:39:02.629 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 199813737, 2023-07-21T22:39:02.629 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 9, 2023-07-21T22:39:02.629 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 5, 2023-07-21T22:39:02.629 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 17.585678861, 2023-07-21T22:39:02.629 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 7.678962057, 2023-07-21T22:39:02.629 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 11.856881059, 2023-07-21T22:39:02.629 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.323300279, 2023-07-21T22:39:02.629 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:02.630 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 434965, 2023-07-21T22:39:02.630 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 9.327343313, 2023-07-21T22:39:02.630 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000021443 2023-07-21T22:39:02.630 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.630 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:02.630 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 853325, 2023-07-21T22:39:02.630 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.090563707, 2023-07-21T22:39:02.631 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000106 2023-07-21T22:39:02.631 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.631 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-21T22:39:02.631 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 322448 2023-07-21T22:39:02.631 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.631 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:39:02.631 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 220278, 2023-07-21T22:39:02.631 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 381223, 2023-07-21T22:39:02.632 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 66800550, 2023-07-21T22:39:02.632 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 279951283, 2023-07-21T22:39:02.632 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 23, 2023-07-21T22:39:02.632 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 10, 2023-07-21T22:39:02.632 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 8.907540462, 2023-07-21T22:39:02.632 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 4.727124968, 2023-07-21T22:39:02.632 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 11.878064406, 2023-07-21T22:39:02.633 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.284173942, 2023-07-21T22:39:02.633 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:02.633 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 381220, 2023-07-21T22:39:02.633 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 29.237895424, 2023-07-21T22:39:02.633 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000076695 2023-07-21T22:39:02.633 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.633 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:02.633 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 236349, 2023-07-21T22:39:02.634 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.053300408, 2023-07-21T22:39:02.634 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000225 2023-07-21T22:39:02.634 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.634 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 4816, 2023-07-21T22:39:02.634 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 1984 2023-07-21T22:39:02.634 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.634 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2023-07-21T22:39:02.634 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 1, 2023-07-21T22:39:02.635 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2023-07-21T22:39:02.635 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.635 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-MDSRank": { 2023-07-21T22:39:02.635 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-07-21T22:39:02.635 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-07-21T22:39:02.635 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 10925, 2023-07-21T22:39:02.635 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 93.628306251, 2023-07-21T22:39:02.635 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.008570096 2023-07-21T22:39:02.636 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:02.636 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.636 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-PurgeQueue": { 2023-07-21T22:39:02.636 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-07-21T22:39:02.636 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-07-21T22:39:02.636 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 24499, 2023-07-21T22:39:02.636 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 5.598688948, 2023-07-21T22:39:02.637 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000228527 2023-07-21T22:39:02.637 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:02.637 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.637 INFO:teuthology.orchestra.run.smithi132.stdout: "mds": { 2023-07-21T22:39:02.637 INFO:teuthology.orchestra.run.smithi132.stdout: "request": 129770, 2023-07-21T22:39:02.637 INFO:teuthology.orchestra.run.smithi132.stdout: "reply": 117203, 2023-07-21T22:39:02.637 INFO:teuthology.orchestra.run.smithi132.stdout: "reply_latency": { 2023-07-21T22:39:02.637 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 117203, 2023-07-21T22:39:02.638 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 2614.638633484, 2023-07-21T22:39:02.638 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.022308632 2023-07-21T22:39:02.638 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.638 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_reply": 0, 2023-07-21T22:39:02.638 INFO:teuthology.orchestra.run.smithi132.stdout: "forward": 12567, 2023-07-21T22:39:02.638 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_complete": 13, 2023-07-21T22:39:02.638 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_keys": 0, 2023-07-21T22:39:02.638 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_commit": 1027, 2023-07-21T22:39:02.639 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_split": 111, 2023-07-21T22:39:02.639 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_merge": 0, 2023-07-21T22:39:02.639 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes": 75, 2023-07-21T22:39:02.639 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_top": 39, 2023-07-21T22:39:02.639 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_bottom": 22, 2023-07-21T22:39:02.639 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pin_tail": 14, 2023-07-21T22:39:02.639 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pinned": 18, 2023-07-21T22:39:02.639 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_expired": 60615, 2023-07-21T22:39:02.640 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_with_caps": 2, 2023-07-21T22:39:02.640 INFO:teuthology.orchestra.run.smithi132.stdout: "caps": 2, 2023-07-21T22:39:02.640 INFO:teuthology.orchestra.run.smithi132.stdout: "subtrees": 4, 2023-07-21T22:39:02.640 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse": 206098, 2023-07-21T22:39:02.640 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_hit": 172895, 2023-07-21T22:39:02.640 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_forward": 12430, 2023-07-21T22:39:02.640 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_discover": 824, 2023-07-21T22:39:02.641 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_dir_fetch": 1, 2023-07-21T22:39:02.641 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_remote_ino": 0, 2023-07-21T22:39:02.641 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_lock": 561, 2023-07-21T22:39:02.641 INFO:teuthology.orchestra.run.smithi132.stdout: "load_cent": 449, 2023-07-21T22:39:02.641 INFO:teuthology.orchestra.run.smithi132.stdout: "q": 0, 2023-07-21T22:39:02.641 INFO:teuthology.orchestra.run.smithi132.stdout: "exported": 267, 2023-07-21T22:39:02.641 INFO:teuthology.orchestra.run.smithi132.stdout: "exported_inodes": 47829, 2023-07-21T22:39:02.641 INFO:teuthology.orchestra.run.smithi132.stdout: "imported": 256, 2023-07-21T22:39:02.642 INFO:teuthology.orchestra.run.smithi132.stdout: "imported_inodes": 16813, 2023-07-21T22:39:02.642 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_dir_fetch": 0, 2023-07-21T22:39:02.642 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_backtrace_fetch": 4143, 2023-07-21T22:39:02.642 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_peer_discover": 0, 2023-07-21T22:39:02.642 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rfiles": 5452, 2023-07-21T22:39:02.642 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rbytes": 102857566, 2023-07-21T22:39:02.642 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rsnaps": 0, 2023-07-21T22:39:02.642 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_fetch": 4143, 2023-07-21T22:39:02.643 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_set_tag": 0, 2023-07-21T22:39:02.643 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:39:02.643 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:39:02.643 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_inodes": 47, 2023-07-21T22:39:02.643 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:39:02.643 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dirfrag_rstats": 47, 2023-07-21T22:39:02.643 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_file_inodes": 4096, 2023-07-21T22:39:02.643 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_inode_file_caps": 175, 2023-07-21T22:39:02.644 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_revoke": 32825, 2023-07-21T22:39:02.644 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_grant": 48831, 2023-07-21T22:39:02.644 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_trunc": 6, 2023-07-21T22:39:02.644 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:39:02.644 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:39:02.644 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps": 49869, 2023-07-21T22:39:02.644 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps_dirty": 29890, 2023-07-21T22:39:02.645 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_cap_release": 819, 2023-07-21T22:39:02.645 INFO:teuthology.orchestra.run.smithi132.stdout: "process_request_cap_release": 47471 2023-07-21T22:39:02.645 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.645 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_cache": { 2023-07-21T22:39:02.645 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update": 3, 2023-07-21T22:39:02.645 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update_receipt": 0, 2023-07-21T22:39:02.645 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_try_discover": 0, 2023-07-21T22:39:02.645 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_send_discover": 861, 2023-07-21T22:39:02.646 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_handle_discover": 1010, 2023-07-21T22:39:02.646 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays": 4, 2023-07-21T22:39:02.646 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_delayed": 0, 2023-07-21T22:39:02.646 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:39:02.646 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_created": 12982, 2023-07-21T22:39:02.646 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_enqueued": 12978, 2023-07-21T22:39:02.646 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_reintegrated": 0, 2023-07-21T22:39:02.647 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_migrated": 0, 2023-07-21T22:39:02.647 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_processing": 0, 2023-07-21T22:39:02.647 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:39:02.647 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:39:02.647 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_started": 0, 2023-07-21T22:39:02.647 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_completed": 0, 2023-07-21T22:39:02.647 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:39:02.647 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_exportdir": 267, 2023-07-21T22:39:02.648 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_flush": 0, 2023-07-21T22:39:02.648 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragmentdir": 111, 2023-07-21T22:39:02.648 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragstats": 0, 2023-07-21T22:39:02.648 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_inodestats": 0 2023-07-21T22:39:02.648 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.648 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_log": { 2023-07-21T22:39:02.648 INFO:teuthology.orchestra.run.smithi132.stdout: "evadd": 135150, 2023-07-21T22:39:02.648 INFO:teuthology.orchestra.run.smithi132.stdout: "evex": 32218, 2023-07-21T22:39:02.649 INFO:teuthology.orchestra.run.smithi132.stdout: "evtrm": 32218, 2023-07-21T22:39:02.649 INFO:teuthology.orchestra.run.smithi132.stdout: "ev": 102932, 2023-07-21T22:39:02.649 INFO:teuthology.orchestra.run.smithi132.stdout: "evexg": 0, 2023-07-21T22:39:02.649 INFO:teuthology.orchestra.run.smithi132.stdout: "evexd": 5073, 2023-07-21T22:39:02.649 INFO:teuthology.orchestra.run.smithi132.stdout: "segadd": 166, 2023-07-21T22:39:02.649 INFO:teuthology.orchestra.run.smithi132.stdout: "segex": 38, 2023-07-21T22:39:02.649 INFO:teuthology.orchestra.run.smithi132.stdout: "segtrm": 38, 2023-07-21T22:39:02.649 INFO:teuthology.orchestra.run.smithi132.stdout: "seg": 134, 2023-07-21T22:39:02.650 INFO:teuthology.orchestra.run.smithi132.stdout: "segexg": 0, 2023-07-21T22:39:02.650 INFO:teuthology.orchestra.run.smithi132.stdout: "segexd": 6, 2023-07-21T22:39:02.650 INFO:teuthology.orchestra.run.smithi132.stdout: "expos": 82265413, 2023-07-21T22:39:02.650 INFO:teuthology.orchestra.run.smithi132.stdout: "wrpos": 331747429, 2023-07-21T22:39:02.650 INFO:teuthology.orchestra.run.smithi132.stdout: "rdpos": 0, 2023-07-21T22:39:02.650 INFO:teuthology.orchestra.run.smithi132.stdout: "jlat": { 2023-07-21T22:39:02.650 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 4067, 2023-07-21T22:39:02.650 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 237.382234102, 2023-07-21T22:39:02.651 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.058367896 2023-07-21T22:39:02.651 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.651 INFO:teuthology.orchestra.run.smithi132.stdout: "replayed": 0 2023-07-21T22:39:02.651 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.651 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_mem": { 2023-07-21T22:39:02.651 INFO:teuthology.orchestra.run.smithi132.stdout: "ino": 37, 2023-07-21T22:39:02.651 INFO:teuthology.orchestra.run.smithi132.stdout: "ino+": 40682, 2023-07-21T22:39:02.652 INFO:teuthology.orchestra.run.smithi132.stdout: "ino-": 40645, 2023-07-21T22:39:02.652 INFO:teuthology.orchestra.run.smithi132.stdout: "dir": 55, 2023-07-21T22:39:02.652 INFO:teuthology.orchestra.run.smithi132.stdout: "dir+": 2039, 2023-07-21T22:39:02.652 INFO:teuthology.orchestra.run.smithi132.stdout: "dir-": 1984, 2023-07-21T22:39:02.652 INFO:teuthology.orchestra.run.smithi132.stdout: "dn": 76, 2023-07-21T22:39:02.652 INFO:teuthology.orchestra.run.smithi132.stdout: "dn+": 87016, 2023-07-21T22:39:02.652 INFO:teuthology.orchestra.run.smithi132.stdout: "dn-": 86940, 2023-07-21T22:39:02.652 INFO:teuthology.orchestra.run.smithi132.stdout: "cap": 2, 2023-07-21T22:39:02.653 INFO:teuthology.orchestra.run.smithi132.stdout: "cap+": 40617, 2023-07-21T22:39:02.653 INFO:teuthology.orchestra.run.smithi132.stdout: "cap-": 40615, 2023-07-21T22:39:02.653 INFO:teuthology.orchestra.run.smithi132.stdout: "rss": 123708, 2023-07-21T22:39:02.653 INFO:teuthology.orchestra.run.smithi132.stdout: "heap": 198964 2023-07-21T22:39:02.653 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.653 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_server": { 2023-07-21T22:39:02.653 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_client_request": 164374, 2023-07-21T22:39:02.653 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_server_request": 3857, 2023-07-21T22:39:02.654 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_request": 129770, 2023-07-21T22:39:02.654 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_session": 393, 2023-07-21T22:39:02.654 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_peer_request": 6243, 2023-07-21T22:39:02.654 INFO:teuthology.orchestra.run.smithi132.stdout: "req_create_latency": { 2023-07-21T22:39:02.654 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 30564, 2023-07-21T22:39:02.654 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 217.396645104, 2023-07-21T22:39:02.654 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.007112833 2023-07-21T22:39:02.654 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.655 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getattr_latency": { 2023-07-21T22:39:02.655 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1918, 2023-07-21T22:39:02.655 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 359.315870356, 2023-07-21T22:39:02.655 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.187338827 2023-07-21T22:39:02.655 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.655 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getfilelock_latency": { 2023-07-21T22:39:02.655 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.655 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: "req_link_latency": { 2023-07-21T22:39:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.656 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookup_latency": { 2023-07-21T22:39:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 37911, 2023-07-21T22:39:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1335.084551661, 2023-07-21T22:39:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.035216284 2023-07-21T22:39:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookuphash_latency": { 2023-07-21T22:39:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.657 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupino_latency": { 2023-07-21T22:39:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1, 2023-07-21T22:39:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000133081, 2023-07-21T22:39:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000133081 2023-07-21T22:39:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.658 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupname_latency": { 2023-07-21T22:39:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupparent_latency": { 2023-07-21T22:39:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupsnap_latency": { 2023-07-21T22:39:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lssnap_latency": { 2023-07-21T22:39:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mkdir_latency": { 2023-07-21T22:39:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 209, 2023-07-21T22:39:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 4.974395108, 2023-07-21T22:39:02.661 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.023800933 2023-07-21T22:39:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mknod_latency": { 2023-07-21T22:39:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mksnap_latency": { 2023-07-21T22:39:02.662 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "req_open_latency": { 2023-07-21T22:39:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 7943, 2023-07-21T22:39:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 52.980951862, 2023-07-21T22:39:02.663 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.006670143 2023-07-21T22:39:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "req_readdir_latency": { 2023-07-21T22:39:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 429, 2023-07-21T22:39:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1.036452188, 2023-07-21T22:39:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.002415972 2023-07-21T22:39:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rename_latency": { 2023-07-21T22:39:02.664 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 30182, 2023-07-21T22:39:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 624.811365756, 2023-07-21T22:39:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.020701456 2023-07-21T22:39:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "req_renamesnap_latency": { 2023-07-21T22:39:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.665 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "req_snapdiff_latency": { 2023-07-21T22:39:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmdir_latency": { 2023-07-21T22:39:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 88, 2023-07-21T22:39:02.666 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 7.672769824, 2023-07-21T22:39:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.087190566 2023-07-21T22:39:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmsnap_latency": { 2023-07-21T22:39:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.667 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmxattr_latency": { 2023-07-21T22:39:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setattr_latency": { 2023-07-21T22:39:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 6, 2023-07-21T22:39:02.668 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.125294330, 2023-07-21T22:39:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.020882388 2023-07-21T22:39:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setdirlayout_latency": { 2023-07-21T22:39:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.669 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setfilelock_latency": { 2023-07-21T22:39:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setlayout_latency": { 2023-07-21T22:39:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.670 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setxattr_latency": { 2023-07-21T22:39:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.671 INFO:teuthology.orchestra.run.smithi132.stdout: "req_symlink_latency": { 2023-07-21T22:39:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3, 2023-07-21T22:39:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.002320561, 2023-07-21T22:39:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000773520 2023-07-21T22:39:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "req_unlink_latency": { 2023-07-21T22:39:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 7949, 2023-07-21T22:39:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 11.237883653, 2023-07-21T22:39:02.672 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.001413748 2023-07-21T22:39:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:39:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:39:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getvxattr_latency": { 2023-07-21T22:39:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.673 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_sessions": { 2023-07-21T22:39:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "session_count": 1, 2023-07-21T22:39:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "session_add": 2, 2023-07-21T22:39:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "session_remove": 1, 2023-07-21T22:39:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_open": 1, 2023-07-21T22:39:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_stale": 0, 2023-07-21T22:39:02.674 INFO:teuthology.orchestra.run.smithi132.stdout: "total_load": 7145, 2023-07-21T22:39:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "average_load": 7145, 2023-07-21T22:39:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "avg_session_uptime": 590 2023-07-21T22:39:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2023-07-21T22:39:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:39:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2023-07-21T22:39:02.675 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:39:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:39:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:39:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:39:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:39:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:39:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:39:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:39:02.676 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:39:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:39:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:39:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:39:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:39:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2023-07-21T22:39:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:39:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2023-07-21T22:39:02.677 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:39:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:39:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:39:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:39:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:39:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:39:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:39:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2023-07-21T22:39:02.678 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:39:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:39:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:39:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2023-07-21T22:39:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2023-07-21T22:39:02.679 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2023-07-21T22:39:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:39:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:39:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:39:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:39:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 300599, 2023-07-21T22:39:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 116, 2023-07-21T22:39:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:39:02.680 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2023-07-21T22:39:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2023-07-21T22:39:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2023-07-21T22:39:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:39:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2023-07-21T22:39:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:39:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2023-07-21T22:39:02.681 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 7720, 2023-07-21T22:39:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 162, 2023-07-21T22:39:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:39:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:39:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2023-07-21T22:39:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2023-07-21T22:39:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 215759, 2023-07-21T22:39:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 1850, 2023-07-21T22:39:02.682 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2023-07-21T22:39:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2023-07-21T22:39:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2023-07-21T22:39:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2023-07-21T22:39:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "objecter": { 2023-07-21T22:39:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "op_active": 0, 2023-07-21T22:39:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "op_laggy": 0, 2023-07-21T22:39:02.683 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send": 24916, 2023-07-21T22:39:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send_bytes": 349906586, 2023-07-21T22:39:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "op_resend": 33, 2023-07-21T22:39:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "op_reply": 24916, 2023-07-21T22:39:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "op_latency": { 2023-07-21T22:39:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 24916, 2023-07-21T22:39:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 280.329900223, 2023-07-21T22:39:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.011250999 2023-07-21T22:39:02.684 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "op_inflight": 0, 2023-07-21T22:39:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "oplen_avg": { 2023-07-21T22:39:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 24916, 2023-07-21T22:39:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 27155 2023-07-21T22:39:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "op": 24916, 2023-07-21T22:39:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "op_r": 4734, 2023-07-21T22:39:02.685 INFO:teuthology.orchestra.run.smithi132.stdout: "op_w": 20182, 2023-07-21T22:39:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "op_rmw": 0, 2023-07-21T22:39:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "op_pg": 0, 2023-07-21T22:39:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_stat": 11, 2023-07-21T22:39:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_create": 446, 2023-07-21T22:39:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_read": 550, 2023-07-21T22:39:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_write": 4622, 2023-07-21T22:39:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writefull": 322, 2023-07-21T22:39:02.686 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writesame": 0, 2023-07-21T22:39:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_append": 0, 2023-07-21T22:39:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_zero": 0, 2023-07-21T22:39:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_truncate": 111, 2023-07-21T22:39:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_delete": 13550, 2023-07-21T22:39:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_mapext": 0, 2023-07-21T22:39:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_sparse_read": 0, 2023-07-21T22:39:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_clonerange": 0, 2023-07-21T22:39:02.687 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_getxattr": 4144, 2023-07-21T22:39:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_setxattr": 892, 2023-07-21T22:39:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:39:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_rmxattr": 0, 2023-07-21T22:39:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:39:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_call": 0, 2023-07-21T22:39:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_watch": 0, 2023-07-21T22:39:02.688 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_notify": 0, 2023-07-21T22:39:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:39:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls": 0, 2023-07-21T22:39:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:39:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_other": 6, 2023-07-21T22:39:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_active": 0, 2023-07-21T22:39:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_send": 0, 2023-07-21T22:39:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_resend": 0, 2023-07-21T22:39:02.689 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_ping": 0, 2023-07-21T22:39:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_active": 0, 2023-07-21T22:39:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_send": 0, 2023-07-21T22:39:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_resend": 0, 2023-07-21T22:39:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_active": 0, 2023-07-21T22:39:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_send": 0, 2023-07-21T22:39:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_resend": 0, 2023-07-21T22:39:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_active": 0, 2023-07-21T22:39:02.690 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_send": 0, 2023-07-21T22:39:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_resend": 0, 2023-07-21T22:39:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "command_active": 0, 2023-07-21T22:39:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "command_send": 0, 2023-07-21T22:39:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "command_resend": 0, 2023-07-21T22:39:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "map_epoch": 85, 2023-07-21T22:39:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "map_full": 0, 2023-07-21T22:39:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "map_inc": 3, 2023-07-21T22:39:02.691 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_sessions": 12, 2023-07-21T22:39:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_open": 12, 2023-07-21T22:39:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_close": 0, 2023-07-21T22:39:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_laggy": 0, 2023-07-21T22:39:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_wr": 2232, 2023-07-21T22:39:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_rd": 54, 2023-07-21T22:39:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_del": 215 2023-07-21T22:39:02.692 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "oft": { 2023-07-21T22:39:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_objs": 1, 2023-07-21T22:39:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:39:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_updates": 0, 2023-07-21T22:39:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_removes": 0 2023-07-21T22:39:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "osd-slow-ops": { 2023-07-21T22:39:02.693 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_ops_count": 131714 2023-07-21T22:39:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "purge_queue": { 2023-07-21T22:39:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops": 0, 2023-07-21T22:39:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops_high_water": 14, 2023-07-21T22:39:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing": 0, 2023-07-21T22:39:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_high_water": 6, 2023-07-21T22:39:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed": 12978, 2023-07-21T22:39:02.694 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_item_in_journal": 0 2023-07-21T22:39:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:39:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-07-21T22:39:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 832417, 2023-07-21T22:39:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 117547838, 2023-07-21T22:39:02.695 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 832417, 2023-07-21T22:39:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-07-21T22:39:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-07-21T22:39:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 832417, 2023-07-21T22:39:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 117547838, 2023-07-21T22:39:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:02.696 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_bytes": { 2023-07-21T22:39:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-07-21T22:39:02.697 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-07-21T22:39:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-07-21T22:39:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 24916, 2023-07-21T22:39:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 351216641, 2023-07-21T22:39:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 24916, 2023-07-21T22:39:02.698 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 351216641, 2023-07-21T22:39:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_ops": { 2023-07-21T22:39:02.699 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 1024, 2023-07-21T22:39:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-07-21T22:39:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-07-21T22:39:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 24916, 2023-07-21T22:39:02.700 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 24916, 2023-07-21T22:39:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 24916, 2023-07-21T22:39:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 24916, 2023-07-21T22:39:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:02.701 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:39:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-07-21T22:39:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 12978, 2023-07-21T22:39:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 1311242, 2023-07-21T22:39:02.702 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 12978, 2023-07-21T22:39:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-07-21T22:39:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-07-21T22:39:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 555, 2023-07-21T22:39:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1311242, 2023-07-21T22:39:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.703 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle-0x55ef3b4040a0": { 2023-07-21T22:39:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-07-21T22:39:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:02.704 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 135150, 2023-07-21T22:39:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 327553125, 2023-07-21T22:39:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 135150, 2023-07-21T22:39:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-07-21T22:39:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-07-21T22:39:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 4067, 2023-07-21T22:39:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 327553125, 2023-07-21T22:39:02.705 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:02.706 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:02.706 INFO:teuthology.orchestra.run.smithi132.stdout:} 2023-07-21T22:39:03.062 INFO:tasks.check_counter:Daemon mds.e mds.exported=267 2023-07-21T22:39:03.063 INFO:tasks.check_counter:Daemon mds.e mds.imported=256 2023-07-21T22:39:03.063 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=111 2023-07-21T22:39:03.063 DEBUG:tasks.check_counter:Getting stats from h 2023-07-21T22:39:03.063 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph daemon mds.h perf dump 2023-07-21T22:39:03.618 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:39:04.015 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:03 smithi132 ceph-mon[97863]: pgmap v593: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 KiB/s wr, 19 op/s 2023-07-21T22:39:04.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:03 smithi190 ceph-mon[98334]: pgmap v593: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 KiB/s wr, 19 op/s 2023-07-21T22:39:04.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:03 smithi107 ceph-mon[96543]: pgmap v593: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 KiB/s wr, 19 op/s 2023-07-21T22:39:04.850 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2023-07-21T22:39:04.850 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:39:04.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 1368, 2023-07-21T22:39:04.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 1343, 2023-07-21T22:39:04.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 54398116, 2023-07-21T22:39:04.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 449815, 2023-07-21T22:39:04.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 7, 2023-07-21T22:39:04.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2023-07-21T22:39:04.851 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.124717901, 2023-07-21T22:39:04.852 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.030967722, 2023-07-21T22:39:04.852 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.054759227, 2023-07-21T22:39:04.852 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.033074481, 2023-07-21T22:39:04.852 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:04.852 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1343, 2023-07-21T22:39:04.852 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.069195750, 2023-07-21T22:39:04.852 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000051523 2023-07-21T22:39:04.853 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:04.853 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.853 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.853 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.853 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 332144, 2023-07-21T22:39:04.853 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 75056 2023-07-21T22:39:04.854 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.854 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:39:04.854 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 257, 2023-07-21T22:39:04.854 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 355, 2023-07-21T22:39:04.854 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 37815997, 2023-07-21T22:39:04.854 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 330466, 2023-07-21T22:39:04.854 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 6, 2023-07-21T22:39:04.855 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 5, 2023-07-21T22:39:04.855 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.039760090, 2023-07-21T22:39:04.855 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.008114515, 2023-07-21T22:39:04.855 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.022893070, 2023-07-21T22:39:04.855 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.006563163, 2023-07-21T22:39:04.855 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:04.855 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 353, 2023-07-21T22:39:04.855 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.028997190, 2023-07-21T22:39:04.856 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000082145 2023-07-21T22:39:04.856 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.856 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:04.856 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.856 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.856 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.856 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.856 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-21T22:39:04.857 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 250784 2023-07-21T22:39:04.857 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.857 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:39:04.857 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 310, 2023-07-21T22:39:04.857 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 312, 2023-07-21T22:39:04.857 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 37837511, 2023-07-21T22:39:04.857 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 96417, 2023-07-21T22:39:04.858 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 6, 2023-07-21T22:39:04.858 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 5, 2023-07-21T22:39:04.858 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.044034890, 2023-07-21T22:39:04.858 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.004909187, 2023-07-21T22:39:04.858 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.029488810, 2023-07-21T22:39:04.858 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.007744631, 2023-07-21T22:39:04.858 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:04.858 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 310, 2023-07-21T22:39:04.859 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.026447170, 2023-07-21T22:39:04.859 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000085313 2023-07-21T22:39:04.859 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.859 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:04.859 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.859 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.859 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.859 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.860 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:39:04.860 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:39:04.860 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.860 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2023-07-21T22:39:04.860 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 1, 2023-07-21T22:39:04.860 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2023-07-21T22:39:04.860 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.860 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-MDSRank": { 2023-07-21T22:39:04.861 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-07-21T22:39:04.861 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-07-21T22:39:04.861 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2979, 2023-07-21T22:39:04.861 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.470263135, 2023-07-21T22:39:04.861 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000157859 2023-07-21T22:39:04.861 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:04.861 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.862 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-PurgeQueue": { 2023-07-21T22:39:04.862 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-07-21T22:39:04.862 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-07-21T22:39:04.862 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.862 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.862 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.862 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:04.863 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.863 INFO:teuthology.orchestra.run.smithi132.stdout: "mds": { 2023-07-21T22:39:04.863 INFO:teuthology.orchestra.run.smithi132.stdout: "request": 0, 2023-07-21T22:39:04.863 INFO:teuthology.orchestra.run.smithi132.stdout: "reply": 0, 2023-07-21T22:39:04.863 INFO:teuthology.orchestra.run.smithi132.stdout: "reply_latency": { 2023-07-21T22:39:04.863 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.863 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.864 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.864 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.864 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_reply": 0, 2023-07-21T22:39:04.864 INFO:teuthology.orchestra.run.smithi132.stdout: "forward": 0, 2023-07-21T22:39:04.864 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_complete": 0, 2023-07-21T22:39:04.864 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_keys": 0, 2023-07-21T22:39:04.864 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_commit": 0, 2023-07-21T22:39:04.864 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_split": 0, 2023-07-21T22:39:04.865 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_merge": 0, 2023-07-21T22:39:04.865 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes": 70, 2023-07-21T22:39:04.865 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_top": 42, 2023-07-21T22:39:04.865 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_bottom": 23, 2023-07-21T22:39:04.865 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pin_tail": 5, 2023-07-21T22:39:04.865 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pinned": 9, 2023-07-21T22:39:04.865 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_expired": 12814, 2023-07-21T22:39:04.865 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_with_caps": 0, 2023-07-21T22:39:04.866 INFO:teuthology.orchestra.run.smithi132.stdout: "caps": 0, 2023-07-21T22:39:04.866 INFO:teuthology.orchestra.run.smithi132.stdout: "subtrees": 2, 2023-07-21T22:39:04.866 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse": 0, 2023-07-21T22:39:04.866 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_hit": 0, 2023-07-21T22:39:04.866 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_forward": 0, 2023-07-21T22:39:04.866 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_discover": 0, 2023-07-21T22:39:04.866 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:39:04.866 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_remote_ino": 0, 2023-07-21T22:39:04.867 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_lock": 0, 2023-07-21T22:39:04.867 INFO:teuthology.orchestra.run.smithi132.stdout: "load_cent": 0, 2023-07-21T22:39:04.867 INFO:teuthology.orchestra.run.smithi132.stdout: "q": 0, 2023-07-21T22:39:04.867 INFO:teuthology.orchestra.run.smithi132.stdout: "exported": 0, 2023-07-21T22:39:04.867 INFO:teuthology.orchestra.run.smithi132.stdout: "exported_inodes": 0, 2023-07-21T22:39:04.867 INFO:teuthology.orchestra.run.smithi132.stdout: "imported": 0, 2023-07-21T22:39:04.867 INFO:teuthology.orchestra.run.smithi132.stdout: "imported_inodes": 0, 2023-07-21T22:39:04.868 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_dir_fetch": 0, 2023-07-21T22:39:04.868 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:39:04.868 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_peer_discover": 0, 2023-07-21T22:39:04.868 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rfiles": 4, 2023-07-21T22:39:04.868 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rbytes": 246, 2023-07-21T22:39:04.868 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rsnaps": 0, 2023-07-21T22:39:04.868 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:39:04.869 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_set_tag": 0, 2023-07-21T22:39:04.869 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:39:04.869 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:39:04.869 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:39:04.869 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:39:04.869 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:39:04.869 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_file_inodes": 0, 2023-07-21T22:39:04.869 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_inode_file_caps": 0, 2023-07-21T22:39:04.870 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T22:39:04.870 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_grant": 0, 2023-07-21T22:39:04.870 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:39:04.870 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:39:04.870 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:39:04.870 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps": 0, 2023-07-21T22:39:04.870 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps_dirty": 0, 2023-07-21T22:39:04.870 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_cap_release": 0, 2023-07-21T22:39:04.871 INFO:teuthology.orchestra.run.smithi132.stdout: "process_request_cap_release": 0 2023-07-21T22:39:04.871 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.871 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_cache": { 2023-07-21T22:39:04.871 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update": 0, 2023-07-21T22:39:04.871 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update_receipt": 0, 2023-07-21T22:39:04.871 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_try_discover": 0, 2023-07-21T22:39:04.871 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_send_discover": 0, 2023-07-21T22:39:04.871 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_handle_discover": 0, 2023-07-21T22:39:04.872 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays": 0, 2023-07-21T22:39:04.872 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_delayed": 0, 2023-07-21T22:39:04.872 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:39:04.872 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_created": 0, 2023-07-21T22:39:04.872 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_enqueued": 0, 2023-07-21T22:39:04.872 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_reintegrated": 0, 2023-07-21T22:39:04.872 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_migrated": 0, 2023-07-21T22:39:04.873 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_processing": 0, 2023-07-21T22:39:04.873 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:39:04.873 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:39:04.873 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_started": 0, 2023-07-21T22:39:04.873 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_completed": 0, 2023-07-21T22:39:04.873 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:39:04.873 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_exportdir": 0, 2023-07-21T22:39:04.873 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_flush": 0, 2023-07-21T22:39:04.874 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragmentdir": 0, 2023-07-21T22:39:04.874 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragstats": 0, 2023-07-21T22:39:04.874 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_inodestats": 0 2023-07-21T22:39:04.874 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.874 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_log": { 2023-07-21T22:39:04.874 INFO:teuthology.orchestra.run.smithi132.stdout: "evadd": 0, 2023-07-21T22:39:04.874 INFO:teuthology.orchestra.run.smithi132.stdout: "evex": 0, 2023-07-21T22:39:04.874 INFO:teuthology.orchestra.run.smithi132.stdout: "evtrm": 0, 2023-07-21T22:39:04.875 INFO:teuthology.orchestra.run.smithi132.stdout: "ev": 0, 2023-07-21T22:39:04.875 INFO:teuthology.orchestra.run.smithi132.stdout: "evexg": 0, 2023-07-21T22:39:04.875 INFO:teuthology.orchestra.run.smithi132.stdout: "evexd": 0, 2023-07-21T22:39:04.875 INFO:teuthology.orchestra.run.smithi132.stdout: "segadd": 0, 2023-07-21T22:39:04.875 INFO:teuthology.orchestra.run.smithi132.stdout: "segex": 0, 2023-07-21T22:39:04.875 INFO:teuthology.orchestra.run.smithi132.stdout: "segtrm": 0, 2023-07-21T22:39:04.875 INFO:teuthology.orchestra.run.smithi132.stdout: "seg": 2, 2023-07-21T22:39:04.875 INFO:teuthology.orchestra.run.smithi132.stdout: "segexg": 0, 2023-07-21T22:39:04.876 INFO:teuthology.orchestra.run.smithi132.stdout: "segexd": 0, 2023-07-21T22:39:04.876 INFO:teuthology.orchestra.run.smithi132.stdout: "expos": 133994271, 2023-07-21T22:39:04.876 INFO:teuthology.orchestra.run.smithi132.stdout: "wrpos": 0, 2023-07-21T22:39:04.876 INFO:teuthology.orchestra.run.smithi132.stdout: "rdpos": 134008274, 2023-07-21T22:39:04.876 INFO:teuthology.orchestra.run.smithi132.stdout: "jlat": { 2023-07-21T22:39:04.876 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.876 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.877 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.877 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.877 INFO:teuthology.orchestra.run.smithi132.stdout: "replayed": 57621 2023-07-21T22:39:04.877 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.877 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_mem": { 2023-07-21T22:39:04.877 INFO:teuthology.orchestra.run.smithi132.stdout: "ino": 17, 2023-07-21T22:39:04.877 INFO:teuthology.orchestra.run.smithi132.stdout: "ino+": 15266, 2023-07-21T22:39:04.877 INFO:teuthology.orchestra.run.smithi132.stdout: "ino-": 15249, 2023-07-21T22:39:04.878 INFO:teuthology.orchestra.run.smithi132.stdout: "dir": 16, 2023-07-21T22:39:04.878 INFO:teuthology.orchestra.run.smithi132.stdout: "dir+": 696, 2023-07-21T22:39:04.878 INFO:teuthology.orchestra.run.smithi132.stdout: "dir-": 680, 2023-07-21T22:39:04.878 INFO:teuthology.orchestra.run.smithi132.stdout: "dn": 70, 2023-07-21T22:39:04.878 INFO:teuthology.orchestra.run.smithi132.stdout: "dn+": 36790, 2023-07-21T22:39:04.878 INFO:teuthology.orchestra.run.smithi132.stdout: "dn-": 36720, 2023-07-21T22:39:04.878 INFO:teuthology.orchestra.run.smithi132.stdout: "cap": 0, 2023-07-21T22:39:04.878 INFO:teuthology.orchestra.run.smithi132.stdout: "cap+": 0, 2023-07-21T22:39:04.879 INFO:teuthology.orchestra.run.smithi132.stdout: "cap-": 0, 2023-07-21T22:39:04.879 INFO:teuthology.orchestra.run.smithi132.stdout: "rss": 61620, 2023-07-21T22:39:04.879 INFO:teuthology.orchestra.run.smithi132.stdout: "heap": 207156 2023-07-21T22:39:04.879 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.879 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_server": { 2023-07-21T22:39:04.879 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_client_request": 0, 2023-07-21T22:39:04.879 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_server_request": 0, 2023-07-21T22:39:04.879 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_request": 0, 2023-07-21T22:39:04.880 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_session": 0, 2023-07-21T22:39:04.880 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_peer_request": 0, 2023-07-21T22:39:04.880 INFO:teuthology.orchestra.run.smithi132.stdout: "req_create_latency": { 2023-07-21T22:39:04.880 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.880 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.880 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.880 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.880 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getattr_latency": { 2023-07-21T22:39:04.881 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.881 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.881 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.881 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.881 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getfilelock_latency": { 2023-07-21T22:39:04.881 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.881 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.882 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.882 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.882 INFO:teuthology.orchestra.run.smithi132.stdout: "req_link_latency": { 2023-07-21T22:39:04.882 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.882 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.882 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.882 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.882 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookup_latency": { 2023-07-21T22:39:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookuphash_latency": { 2023-07-21T22:39:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupino_latency": { 2023-07-21T22:39:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupname_latency": { 2023-07-21T22:39:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.885 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.885 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.885 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.885 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupparent_latency": { 2023-07-21T22:39:04.885 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.885 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.885 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.886 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.886 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupsnap_latency": { 2023-07-21T22:39:04.886 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.886 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.886 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.886 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.886 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lssnap_latency": { 2023-07-21T22:39:04.886 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.887 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.887 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.887 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.887 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mkdir_latency": { 2023-07-21T22:39:04.887 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.887 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.887 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.887 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.888 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mknod_latency": { 2023-07-21T22:39:04.888 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.888 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.888 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.888 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.888 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mksnap_latency": { 2023-07-21T22:39:04.888 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.888 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.889 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.889 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.889 INFO:teuthology.orchestra.run.smithi132.stdout: "req_open_latency": { 2023-07-21T22:39:04.889 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.889 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.889 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.889 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.889 INFO:teuthology.orchestra.run.smithi132.stdout: "req_readdir_latency": { 2023-07-21T22:39:04.890 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.890 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.890 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.890 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.890 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rename_latency": { 2023-07-21T22:39:04.890 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.890 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.891 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.891 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.891 INFO:teuthology.orchestra.run.smithi132.stdout: "req_renamesnap_latency": { 2023-07-21T22:39:04.891 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.891 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.891 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.891 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.891 INFO:teuthology.orchestra.run.smithi132.stdout: "req_snapdiff_latency": { 2023-07-21T22:39:04.892 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.892 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.892 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.892 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.892 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmdir_latency": { 2023-07-21T22:39:04.892 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.892 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.892 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.893 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.893 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmsnap_latency": { 2023-07-21T22:39:04.893 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.893 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.893 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.893 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.893 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmxattr_latency": { 2023-07-21T22:39:04.893 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.894 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.894 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.894 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.894 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setattr_latency": { 2023-07-21T22:39:04.894 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.894 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.894 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.894 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.895 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setdirlayout_latency": { 2023-07-21T22:39:04.895 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.895 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.895 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.895 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.895 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setfilelock_latency": { 2023-07-21T22:39:04.895 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.895 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.896 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.896 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.896 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setlayout_latency": { 2023-07-21T22:39:04.896 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.896 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.896 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.896 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.897 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setxattr_latency": { 2023-07-21T22:39:04.897 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.897 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.897 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.897 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.897 INFO:teuthology.orchestra.run.smithi132.stdout: "req_symlink_latency": { 2023-07-21T22:39:04.897 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.897 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.898 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.898 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.898 INFO:teuthology.orchestra.run.smithi132.stdout: "req_unlink_latency": { 2023-07-21T22:39:04.898 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.898 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.898 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.898 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.898 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:39:04.899 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:39:04.899 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getvxattr_latency": { 2023-07-21T22:39:04.899 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.899 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.899 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.899 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:04.899 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.900 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_sessions": { 2023-07-21T22:39:04.900 INFO:teuthology.orchestra.run.smithi132.stdout: "session_count": 1, 2023-07-21T22:39:04.900 INFO:teuthology.orchestra.run.smithi132.stdout: "session_add": 1, 2023-07-21T22:39:04.900 INFO:teuthology.orchestra.run.smithi132.stdout: "session_remove": 0, 2023-07-21T22:39:04.900 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_open": 0, 2023-07-21T22:39:04.900 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_stale": 0, 2023-07-21T22:39:04.900 INFO:teuthology.orchestra.run.smithi132.stdout: "total_load": 0, 2023-07-21T22:39:04.900 INFO:teuthology.orchestra.run.smithi132.stdout: "average_load": 0, 2023-07-21T22:39:04.901 INFO:teuthology.orchestra.run.smithi132.stdout: "avg_session_uptime": 2145 2023-07-21T22:39:04.901 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.901 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2023-07-21T22:39:04.901 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:39:04.901 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2023-07-21T22:39:04.901 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:39:04.901 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:39:04.901 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:39:04.902 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:39:04.902 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:39:04.902 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:39:04.902 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:39:04.902 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:39:04.902 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:39:04.902 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:39:04.902 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:39:04.903 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:39:04.903 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:39:04.903 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2023-07-21T22:39:04.903 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:39:04.903 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2023-07-21T22:39:04.903 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:39:04.903 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:39:04.903 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:39:04.904 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:39:04.904 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:39:04.904 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:39:04.904 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:39:04.904 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2023-07-21T22:39:04.904 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:39:04.904 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:39:04.905 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:39:04.905 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2023-07-21T22:39:04.905 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2023-07-21T22:39:04.905 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2023-07-21T22:39:04.905 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:39:04.905 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:39:04.905 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:39:04.905 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:39:04.906 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 231454, 2023-07-21T22:39:04.906 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 71, 2023-07-21T22:39:04.906 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:39:04.906 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2023-07-21T22:39:04.906 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2023-07-21T22:39:04.906 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2023-07-21T22:39:04.906 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:39:04.906 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2023-07-21T22:39:04.907 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:39:04.907 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2023-07-21T22:39:04.907 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 7720, 2023-07-21T22:39:04.907 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 162, 2023-07-21T22:39:04.907 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:39:04.907 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:39:04.907 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2023-07-21T22:39:04.907 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2023-07-21T22:39:04.908 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 107173, 2023-07-21T22:39:04.908 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 484, 2023-07-21T22:39:04.908 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2023-07-21T22:39:04.908 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2023-07-21T22:39:04.908 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2023-07-21T22:39:04.908 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2023-07-21T22:39:04.908 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.908 INFO:teuthology.orchestra.run.smithi132.stdout: "objecter": { 2023-07-21T22:39:04.909 INFO:teuthology.orchestra.run.smithi132.stdout: "op_active": 0, 2023-07-21T22:39:04.909 INFO:teuthology.orchestra.run.smithi132.stdout: "op_laggy": 0, 2023-07-21T22:39:04.909 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send": 1771, 2023-07-21T22:39:04.909 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send_bytes": 26, 2023-07-21T22:39:04.909 INFO:teuthology.orchestra.run.smithi132.stdout: "op_resend": 4, 2023-07-21T22:39:04.909 INFO:teuthology.orchestra.run.smithi132.stdout: "op_reply": 1771, 2023-07-21T22:39:04.909 INFO:teuthology.orchestra.run.smithi132.stdout: "op_latency": { 2023-07-21T22:39:04.909 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1771, 2023-07-21T22:39:04.910 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 10.764726455, 2023-07-21T22:39:04.910 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.006078332 2023-07-21T22:39:04.910 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.910 INFO:teuthology.orchestra.run.smithi132.stdout: "op_inflight": 0, 2023-07-21T22:39:04.910 INFO:teuthology.orchestra.run.smithi132.stdout: "oplen_avg": { 2023-07-21T22:39:04.910 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1771, 2023-07-21T22:39:04.910 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1772 2023-07-21T22:39:04.910 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.911 INFO:teuthology.orchestra.run.smithi132.stdout: "op": 1771, 2023-07-21T22:39:04.911 INFO:teuthology.orchestra.run.smithi132.stdout: "op_r": 1771, 2023-07-21T22:39:04.911 INFO:teuthology.orchestra.run.smithi132.stdout: "op_w": 0, 2023-07-21T22:39:04.911 INFO:teuthology.orchestra.run.smithi132.stdout: "op_rmw": 0, 2023-07-21T22:39:04.911 INFO:teuthology.orchestra.run.smithi132.stdout: "op_pg": 0, 2023-07-21T22:39:04.911 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_stat": 974, 2023-07-21T22:39:04.911 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_create": 0, 2023-07-21T22:39:04.911 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_read": 794, 2023-07-21T22:39:04.912 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_write": 0, 2023-07-21T22:39:04.912 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writefull": 0, 2023-07-21T22:39:04.912 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writesame": 0, 2023-07-21T22:39:04.912 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_append": 0, 2023-07-21T22:39:04.912 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_zero": 0, 2023-07-21T22:39:04.912 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_truncate": 0, 2023-07-21T22:39:04.912 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_delete": 0, 2023-07-21T22:39:04.913 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_mapext": 0, 2023-07-21T22:39:04.913 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_sparse_read": 0, 2023-07-21T22:39:04.913 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_clonerange": 0, 2023-07-21T22:39:04.913 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_getxattr": 2, 2023-07-21T22:39:04.913 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_setxattr": 0, 2023-07-21T22:39:04.913 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:39:04.913 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_rmxattr": 0, 2023-07-21T22:39:04.913 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:39:04.914 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_call": 0, 2023-07-21T22:39:04.914 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_watch": 0, 2023-07-21T22:39:04.914 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_notify": 0, 2023-07-21T22:39:04.914 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:39:04.914 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls": 0, 2023-07-21T22:39:04.914 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:39:04.914 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_other": 0, 2023-07-21T22:39:04.914 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_active": 0, 2023-07-21T22:39:04.915 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_send": 0, 2023-07-21T22:39:04.915 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_resend": 0, 2023-07-21T22:39:04.915 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_ping": 0, 2023-07-21T22:39:04.915 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_active": 0, 2023-07-21T22:39:04.915 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_send": 0, 2023-07-21T22:39:04.915 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_resend": 0, 2023-07-21T22:39:04.915 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_active": 0, 2023-07-21T22:39:04.915 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_send": 0, 2023-07-21T22:39:04.916 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_resend": 0, 2023-07-21T22:39:04.916 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_active": 0, 2023-07-21T22:39:04.916 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_send": 0, 2023-07-21T22:39:04.916 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_resend": 0, 2023-07-21T22:39:04.916 INFO:teuthology.orchestra.run.smithi132.stdout: "command_active": 0, 2023-07-21T22:39:04.916 INFO:teuthology.orchestra.run.smithi132.stdout: "command_send": 0, 2023-07-21T22:39:04.916 INFO:teuthology.orchestra.run.smithi132.stdout: "command_resend": 0, 2023-07-21T22:39:04.916 INFO:teuthology.orchestra.run.smithi132.stdout: "map_epoch": 0, 2023-07-21T22:39:04.917 INFO:teuthology.orchestra.run.smithi132.stdout: "map_full": 0, 2023-07-21T22:39:04.917 INFO:teuthology.orchestra.run.smithi132.stdout: "map_inc": 0, 2023-07-21T22:39:04.917 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_sessions": 10, 2023-07-21T22:39:04.917 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_open": 10, 2023-07-21T22:39:04.917 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_close": 0, 2023-07-21T22:39:04.917 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_laggy": 0, 2023-07-21T22:39:04.917 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_wr": 0, 2023-07-21T22:39:04.917 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_rd": 2, 2023-07-21T22:39:04.918 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_del": 0 2023-07-21T22:39:04.918 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.918 INFO:teuthology.orchestra.run.smithi132.stdout: "oft": { 2023-07-21T22:39:04.918 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_objs": 0, 2023-07-21T22:39:04.918 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:39:04.918 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_updates": 0, 2023-07-21T22:39:04.918 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_removes": 0 2023-07-21T22:39:04.918 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.919 INFO:teuthology.orchestra.run.smithi132.stdout: "osd-slow-ops": { 2023-07-21T22:39:04.919 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_ops_count": 0 2023-07-21T22:39:04.919 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.919 INFO:teuthology.orchestra.run.smithi132.stdout: "purge_queue": { 2023-07-21T22:39:04.919 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops": 0, 2023-07-21T22:39:04.919 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T22:39:04.919 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing": 0, 2023-07-21T22:39:04.919 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_high_water": 0, 2023-07-21T22:39:04.920 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed": 0, 2023-07-21T22:39:04.920 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_item_in_journal": 0 2023-07-21T22:39:04.920 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.920 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:39:04.920 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:04.920 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-07-21T22:39:04.920 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:04.921 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 1931, 2023-07-21T22:39:04.921 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 129866791, 2023-07-21T22:39:04.921 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:04.921 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 1931, 2023-07-21T22:39:04.921 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-07-21T22:39:04.921 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-07-21T22:39:04.921 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 1931, 2023-07-21T22:39:04.921 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 129866791, 2023-07-21T22:39:04.922 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:04.922 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.922 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.922 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.922 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:04.922 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.922 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_bytes": { 2023-07-21T22:39:04.922 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:04.923 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-07-21T22:39:04.923 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:04.923 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-07-21T22:39:04.923 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-07-21T22:39:04.923 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:04.923 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:04.923 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 1771, 2023-07-21T22:39:04.923 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 129229842, 2023-07-21T22:39:04.924 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 1771, 2023-07-21T22:39:04.924 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 129229842, 2023-07-21T22:39:04.924 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:04.924 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.924 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.924 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.924 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:04.924 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.925 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_ops": { 2023-07-21T22:39:04.925 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:04.925 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 1024, 2023-07-21T22:39:04.925 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:04.925 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-07-21T22:39:04.925 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-07-21T22:39:04.925 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:04.925 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:04.926 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 1771, 2023-07-21T22:39:04.926 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 1771, 2023-07-21T22:39:04.926 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 1771, 2023-07-21T22:39:04.926 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1771, 2023-07-21T22:39:04.926 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:04.926 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.926 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.926 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.927 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:04.927 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.927 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:39:04.927 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:04.927 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-07-21T22:39:04.927 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:04.927 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-07-21T22:39:04.927 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-07-21T22:39:04.928 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:04.928 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:04.928 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-07-21T22:39:04.928 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-07-21T22:39:04.928 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 0, 2023-07-21T22:39:04.928 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 0, 2023-07-21T22:39:04.928 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:04.929 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.929 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.929 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.929 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:04.929 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:04.929 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle-0x561cf2154320": { 2023-07-21T22:39:04.929 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:04.929 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-07-21T22:39:04.930 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:04.930 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-07-21T22:39:04.930 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-07-21T22:39:04.930 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:04.930 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:04.930 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-07-21T22:39:04.930 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-07-21T22:39:04.930 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 0, 2023-07-21T22:39:04.931 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 0, 2023-07-21T22:39:04.931 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:04.931 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:04.931 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:04.931 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:04.931 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:04.931 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:04.931 INFO:teuthology.orchestra.run.smithi132.stdout:} 2023-07-21T22:39:05.261 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-07-21T22:39:05.262 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-07-21T22:39:05.262 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2023-07-21T22:39:05.262 DEBUG:tasks.check_counter:Getting stats from k 2023-07-21T22:39:05.262 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph daemon mds.k perf dump 2023-07-21T22:39:05.814 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:39:05.994 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:05 smithi132 ceph-mon[97863]: pgmap v594: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 11 op/s 2023-07-21T22:39:06.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:05 smithi190 ceph-mon[98334]: pgmap v594: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 11 op/s 2023-07-21T22:39:06.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:05 smithi107 ceph-mon[96543]: pgmap v594: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 11 op/s 2023-07-21T22:39:07.013 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2023-07-21T22:39:07.014 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:39:07.014 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 25206, 2023-07-21T22:39:07.014 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 26128, 2023-07-21T22:39:07.014 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 12727211, 2023-07-21T22:39:07.014 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 60984541, 2023-07-21T22:39:07.015 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 12, 2023-07-21T22:39:07.015 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 6, 2023-07-21T22:39:07.015 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 1.685958951, 2023-07-21T22:39:07.015 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.700161289, 2023-07-21T22:39:07.015 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.850607988, 2023-07-21T22:39:07.015 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.261159901, 2023-07-21T22:39:07.015 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:07.015 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 26124, 2023-07-21T22:39:07.016 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1.901285714, 2023-07-21T22:39:07.016 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000072779 2023-07-21T22:39:07.016 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.016 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:07.016 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 37390, 2023-07-21T22:39:07.016 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.006721574, 2023-07-21T22:39:07.016 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000179 2023-07-21T22:39:07.017 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.017 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:39:07.017 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:39:07.017 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.017 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:39:07.017 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 16347, 2023-07-21T22:39:07.017 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 20507, 2023-07-21T22:39:07.018 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 5661334, 2023-07-21T22:39:07.018 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 27667342, 2023-07-21T22:39:07.018 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 9, 2023-07-21T22:39:07.018 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 5, 2023-07-21T22:39:07.018 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 1.156765700, 2023-07-21T22:39:07.018 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.442562919, 2023-07-21T22:39:07.018 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.563960740, 2023-07-21T22:39:07.018 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.210898068, 2023-07-21T22:39:07.019 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:07.019 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 20510, 2023-07-21T22:39:07.019 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1.626487995, 2023-07-21T22:39:07.019 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000079302 2023-07-21T22:39:07.019 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.019 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:07.019 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 27599, 2023-07-21T22:39:07.020 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.004651518, 2023-07-21T22:39:07.020 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000168 2023-07-21T22:39:07.020 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.020 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-07-21T22:39:07.020 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 321584 2023-07-21T22:39:07.020 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.021 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:39:07.021 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 116705, 2023-07-21T22:39:07.021 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 161417, 2023-07-21T22:39:07.021 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 37635698, 2023-07-21T22:39:07.021 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 113881766, 2023-07-21T22:39:07.021 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 21, 2023-07-21T22:39:07.021 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 8, 2023-07-21T22:39:07.022 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 4.279203752, 2023-07-21T22:39:07.022 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 2.106413284, 2023-07-21T22:39:07.022 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 21.921513903, 2023-07-21T22:39:07.022 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.214717998, 2023-07-21T22:39:07.022 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:07.022 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 161414, 2023-07-21T22:39:07.022 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 20.564493029, 2023-07-21T22:39:07.023 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000127402 2023-07-21T22:39:07.023 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.023 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:07.023 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 74867, 2023-07-21T22:39:07.023 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.023927978, 2023-07-21T22:39:07.023 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000319 2023-07-21T22:39:07.023 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.024 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 671152, 2023-07-21T22:39:07.024 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 116032 2023-07-21T22:39:07.024 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.024 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2023-07-21T22:39:07.024 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 1, 2023-07-21T22:39:07.024 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2023-07-21T22:39:07.025 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.025 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-MDSRank": { 2023-07-21T22:39:07.025 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-07-21T22:39:07.025 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-07-21T22:39:07.025 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 5409, 2023-07-21T22:39:07.025 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 28.416818087, 2023-07-21T22:39:07.025 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.005253617 2023-07-21T22:39:07.025 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:07.026 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.026 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-PurgeQueue": { 2023-07-21T22:39:07.026 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-07-21T22:39:07.026 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-07-21T22:39:07.026 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 19728, 2023-07-21T22:39:07.026 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 2.926103544, 2023-07-21T22:39:07.026 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000148322 2023-07-21T22:39:07.027 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:07.027 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.027 INFO:teuthology.orchestra.run.smithi132.stdout: "mds": { 2023-07-21T22:39:07.027 INFO:teuthology.orchestra.run.smithi132.stdout: "request": 52820, 2023-07-21T22:39:07.027 INFO:teuthology.orchestra.run.smithi132.stdout: "reply": 52249, 2023-07-21T22:39:07.027 INFO:teuthology.orchestra.run.smithi132.stdout: "reply_latency": { 2023-07-21T22:39:07.027 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 52249, 2023-07-21T22:39:07.027 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 867.530438616, 2023-07-21T22:39:07.028 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.016603771 2023-07-21T22:39:07.028 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.028 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_reply": 0, 2023-07-21T22:39:07.028 INFO:teuthology.orchestra.run.smithi132.stdout: "forward": 571, 2023-07-21T22:39:07.028 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_complete": 0, 2023-07-21T22:39:07.028 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_keys": 0, 2023-07-21T22:39:07.028 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_commit": 263, 2023-07-21T22:39:07.028 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_split": 28, 2023-07-21T22:39:07.029 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_merge": 0, 2023-07-21T22:39:07.029 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes": 24, 2023-07-21T22:39:07.029 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_top": 7, 2023-07-21T22:39:07.029 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_bottom": 4, 2023-07-21T22:39:07.029 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pin_tail": 13, 2023-07-21T22:39:07.029 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pinned": 13, 2023-07-21T22:39:07.029 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_expired": 3883, 2023-07-21T22:39:07.030 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_with_caps": 0, 2023-07-21T22:39:07.030 INFO:teuthology.orchestra.run.smithi132.stdout: "caps": 0, 2023-07-21T22:39:07.030 INFO:teuthology.orchestra.run.smithi132.stdout: "subtrees": 5, 2023-07-21T22:39:07.030 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse": 94082, 2023-07-21T22:39:07.030 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_hit": 78823, 2023-07-21T22:39:07.030 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_forward": 540, 2023-07-21T22:39:07.030 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_discover": 505, 2023-07-21T22:39:07.031 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:39:07.031 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_remote_ino": 0, 2023-07-21T22:39:07.031 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_lock": 1228, 2023-07-21T22:39:07.031 INFO:teuthology.orchestra.run.smithi132.stdout: "load_cent": 0, 2023-07-21T22:39:07.031 INFO:teuthology.orchestra.run.smithi132.stdout: "q": 0, 2023-07-21T22:39:07.031 INFO:teuthology.orchestra.run.smithi132.stdout: "exported": 124, 2023-07-21T22:39:07.031 INFO:teuthology.orchestra.run.smithi132.stdout: "exported_inodes": 2132, 2023-07-21T22:39:07.032 INFO:teuthology.orchestra.run.smithi132.stdout: "imported": 122, 2023-07-21T22:39:07.032 INFO:teuthology.orchestra.run.smithi132.stdout: "imported_inodes": 8034, 2023-07-21T22:39:07.032 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_dir_fetch": 0, 2023-07-21T22:39:07.032 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:39:07.032 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_peer_discover": 0, 2023-07-21T22:39:07.032 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rfiles": 5452, 2023-07-21T22:39:07.032 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rbytes": 102857566, 2023-07-21T22:39:07.032 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rsnaps": 0, 2023-07-21T22:39:07.033 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:39:07.033 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_set_tag": 0, 2023-07-21T22:39:07.033 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:39:07.033 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:39:07.033 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:39:07.033 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:39:07.033 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:39:07.033 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_file_inodes": 0, 2023-07-21T22:39:07.034 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_inode_file_caps": 134, 2023-07-21T22:39:07.034 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_revoke": 7647, 2023-07-21T22:39:07.034 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_grant": 18922, 2023-07-21T22:39:07.034 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_trunc": 1, 2023-07-21T22:39:07.034 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:39:07.034 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:39:07.034 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps": 16216, 2023-07-21T22:39:07.034 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps_dirty": 7268, 2023-07-21T22:39:07.035 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_cap_release": 317, 2023-07-21T22:39:07.035 INFO:teuthology.orchestra.run.smithi132.stdout: "process_request_cap_release": 24631 2023-07-21T22:39:07.035 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.035 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_cache": { 2023-07-21T22:39:07.035 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update": 0, 2023-07-21T22:39:07.035 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update_receipt": 14, 2023-07-21T22:39:07.035 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_try_discover": 10, 2023-07-21T22:39:07.036 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_send_discover": 583, 2023-07-21T22:39:07.036 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_handle_discover": 1106, 2023-07-21T22:39:07.036 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays": 0, 2023-07-21T22:39:07.036 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_delayed": 0, 2023-07-21T22:39:07.036 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:39:07.036 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_created": 10650, 2023-07-21T22:39:07.036 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_enqueued": 10650, 2023-07-21T22:39:07.037 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_reintegrated": 0, 2023-07-21T22:39:07.037 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_migrated": 0, 2023-07-21T22:39:07.037 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_processing": 0, 2023-07-21T22:39:07.037 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:39:07.037 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:39:07.037 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_started": 0, 2023-07-21T22:39:07.037 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_completed": 0, 2023-07-21T22:39:07.038 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:39:07.038 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_exportdir": 124, 2023-07-21T22:39:07.038 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_flush": 0, 2023-07-21T22:39:07.038 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragmentdir": 28, 2023-07-21T22:39:07.038 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragstats": 0, 2023-07-21T22:39:07.038 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_inodestats": 0 2023-07-21T22:39:07.038 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.038 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_log": { 2023-07-21T22:39:07.039 INFO:teuthology.orchestra.run.smithi132.stdout: "evadd": 59671, 2023-07-21T22:39:07.039 INFO:teuthology.orchestra.run.smithi132.stdout: "evex": 0, 2023-07-21T22:39:07.039 INFO:teuthology.orchestra.run.smithi132.stdout: "evtrm": 0, 2023-07-21T22:39:07.039 INFO:teuthology.orchestra.run.smithi132.stdout: "ev": 59671, 2023-07-21T22:39:07.039 INFO:teuthology.orchestra.run.smithi132.stdout: "evexg": 0, 2023-07-21T22:39:07.039 INFO:teuthology.orchestra.run.smithi132.stdout: "evexd": 0, 2023-07-21T22:39:07.039 INFO:teuthology.orchestra.run.smithi132.stdout: "segadd": 75, 2023-07-21T22:39:07.040 INFO:teuthology.orchestra.run.smithi132.stdout: "segex": 0, 2023-07-21T22:39:07.040 INFO:teuthology.orchestra.run.smithi132.stdout: "segtrm": 0, 2023-07-21T22:39:07.040 INFO:teuthology.orchestra.run.smithi132.stdout: "seg": 75, 2023-07-21T22:39:07.040 INFO:teuthology.orchestra.run.smithi132.stdout: "segexg": 0, 2023-07-21T22:39:07.040 INFO:teuthology.orchestra.run.smithi132.stdout: "segexd": 0, 2023-07-21T22:39:07.040 INFO:teuthology.orchestra.run.smithi132.stdout: "expos": 4194304, 2023-07-21T22:39:07.040 INFO:teuthology.orchestra.run.smithi132.stdout: "wrpos": 126344525, 2023-07-21T22:39:07.041 INFO:teuthology.orchestra.run.smithi132.stdout: "rdpos": 0, 2023-07-21T22:39:07.041 INFO:teuthology.orchestra.run.smithi132.stdout: "jlat": { 2023-07-21T22:39:07.041 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2655, 2023-07-21T22:39:07.041 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 79.596879860, 2023-07-21T22:39:07.041 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.029979992 2023-07-21T22:39:07.041 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.041 INFO:teuthology.orchestra.run.smithi132.stdout: "replayed": 0 2023-07-21T22:39:07.042 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.042 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_mem": { 2023-07-21T22:39:07.042 INFO:teuthology.orchestra.run.smithi132.stdout: "ino": 31, 2023-07-21T22:39:07.042 INFO:teuthology.orchestra.run.smithi132.stdout: "ino+": 12028, 2023-07-21T22:39:07.042 INFO:teuthology.orchestra.run.smithi132.stdout: "ino-": 11997, 2023-07-21T22:39:07.042 INFO:teuthology.orchestra.run.smithi132.stdout: "dir": 28, 2023-07-21T22:39:07.042 INFO:teuthology.orchestra.run.smithi132.stdout: "dir+": 602, 2023-07-21T22:39:07.043 INFO:teuthology.orchestra.run.smithi132.stdout: "dir-": 574, 2023-07-21T22:39:07.043 INFO:teuthology.orchestra.run.smithi132.stdout: "dn": 24, 2023-07-21T22:39:07.043 INFO:teuthology.orchestra.run.smithi132.stdout: "dn+": 30311, 2023-07-21T22:39:07.043 INFO:teuthology.orchestra.run.smithi132.stdout: "dn-": 30287, 2023-07-21T22:39:07.043 INFO:teuthology.orchestra.run.smithi132.stdout: "cap": 0, 2023-07-21T22:39:07.043 INFO:teuthology.orchestra.run.smithi132.stdout: "cap+": 14750, 2023-07-21T22:39:07.044 INFO:teuthology.orchestra.run.smithi132.stdout: "cap-": 14750, 2023-07-21T22:39:07.044 INFO:teuthology.orchestra.run.smithi132.stdout: "rss": 96888, 2023-07-21T22:39:07.044 INFO:teuthology.orchestra.run.smithi132.stdout: "heap": 198964 2023-07-21T22:39:07.044 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.044 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_server": { 2023-07-21T22:39:07.044 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_client_request": 65485, 2023-07-21T22:39:07.044 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_server_request": 4674, 2023-07-21T22:39:07.045 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_request": 52820, 2023-07-21T22:39:07.045 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_session": 306, 2023-07-21T22:39:07.045 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_peer_request": 6649, 2023-07-21T22:39:07.045 INFO:teuthology.orchestra.run.smithi132.stdout: "req_create_latency": { 2023-07-21T22:39:07.045 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 7114, 2023-07-21T22:39:07.045 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 14.711093190, 2023-07-21T22:39:07.045 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.002067907 2023-07-21T22:39:07.045 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.046 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getattr_latency": { 2023-07-21T22:39:07.046 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 183, 2023-07-21T22:39:07.046 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 43.995752995, 2023-07-21T22:39:07.046 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.240413950 2023-07-21T22:39:07.046 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.046 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getfilelock_latency": { 2023-07-21T22:39:07.046 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.046 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.047 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.047 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.047 INFO:teuthology.orchestra.run.smithi132.stdout: "req_link_latency": { 2023-07-21T22:39:07.047 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.047 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.047 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.047 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.048 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookup_latency": { 2023-07-21T22:39:07.048 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 26571, 2023-07-21T22:39:07.048 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 562.398244356, 2023-07-21T22:39:07.048 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.021165866 2023-07-21T22:39:07.048 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.048 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookuphash_latency": { 2023-07-21T22:39:07.048 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.048 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.049 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.049 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.049 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupino_latency": { 2023-07-21T22:39:07.049 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.049 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.049 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.049 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.049 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupname_latency": { 2023-07-21T22:39:07.050 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.050 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.050 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.050 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.050 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupparent_latency": { 2023-07-21T22:39:07.050 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.050 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.051 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.051 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.051 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupsnap_latency": { 2023-07-21T22:39:07.051 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.051 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.051 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.051 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.052 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lssnap_latency": { 2023-07-21T22:39:07.052 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.052 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.052 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.052 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.052 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mkdir_latency": { 2023-07-21T22:39:07.052 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 32, 2023-07-21T22:39:07.053 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1.452980394, 2023-07-21T22:39:07.053 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.045405637 2023-07-21T22:39:07.053 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.053 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mknod_latency": { 2023-07-21T22:39:07.053 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.053 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.053 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.053 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.054 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mksnap_latency": { 2023-07-21T22:39:07.054 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.054 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.054 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.054 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.054 INFO:teuthology.orchestra.run.smithi132.stdout: "req_open_latency": { 2023-07-21T22:39:07.054 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2308, 2023-07-21T22:39:07.054 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 3.787930035, 2023-07-21T22:39:07.055 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.001641217 2023-07-21T22:39:07.055 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.055 INFO:teuthology.orchestra.run.smithi132.stdout: "req_readdir_latency": { 2023-07-21T22:39:07.055 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 359, 2023-07-21T22:39:07.055 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.459441352, 2023-07-21T22:39:07.055 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.001279780 2023-07-21T22:39:07.055 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.056 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rename_latency": { 2023-07-21T22:39:07.056 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 7069, 2023-07-21T22:39:07.056 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 224.522805805, 2023-07-21T22:39:07.056 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.031761607 2023-07-21T22:39:07.056 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.056 INFO:teuthology.orchestra.run.smithi132.stdout: "req_renamesnap_latency": { 2023-07-21T22:39:07.056 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.056 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.057 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.057 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.057 INFO:teuthology.orchestra.run.smithi132.stdout: "req_snapdiff_latency": { 2023-07-21T22:39:07.057 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.057 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.057 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.057 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.057 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmdir_latency": { 2023-07-21T22:39:07.058 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 48, 2023-07-21T22:39:07.058 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 4.750798020, 2023-07-21T22:39:07.058 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.098974958 2023-07-21T22:39:07.058 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.058 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmsnap_latency": { 2023-07-21T22:39:07.058 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.059 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.059 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.059 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.059 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmxattr_latency": { 2023-07-21T22:39:07.059 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.059 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.059 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.060 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.060 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setattr_latency": { 2023-07-21T22:39:07.060 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1, 2023-07-21T22:39:07.060 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.087410955, 2023-07-21T22:39:07.060 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.087410955 2023-07-21T22:39:07.060 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.061 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setdirlayout_latency": { 2023-07-21T22:39:07.061 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.061 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.061 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.061 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.061 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setfilelock_latency": { 2023-07-21T22:39:07.061 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.061 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.062 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.062 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.062 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setlayout_latency": { 2023-07-21T22:39:07.062 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.062 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.062 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.062 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.062 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setxattr_latency": { 2023-07-21T22:39:07.063 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.063 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.063 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.063 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.063 INFO:teuthology.orchestra.run.smithi132.stdout: "req_symlink_latency": { 2023-07-21T22:39:07.063 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.064 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.064 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.064 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.064 INFO:teuthology.orchestra.run.smithi132.stdout: "req_unlink_latency": { 2023-07-21T22:39:07.064 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 8564, 2023-07-21T22:39:07.064 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 11.363981514, 2023-07-21T22:39:07.064 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.001326947 2023-07-21T22:39:07.065 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.065 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:39:07.065 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:39:07.065 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getvxattr_latency": { 2023-07-21T22:39:07.065 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.065 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.065 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.065 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:07.066 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.066 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_sessions": { 2023-07-21T22:39:07.066 INFO:teuthology.orchestra.run.smithi132.stdout: "session_count": 1, 2023-07-21T22:39:07.066 INFO:teuthology.orchestra.run.smithi132.stdout: "session_add": 1, 2023-07-21T22:39:07.066 INFO:teuthology.orchestra.run.smithi132.stdout: "session_remove": 0, 2023-07-21T22:39:07.066 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_open": 1, 2023-07-21T22:39:07.066 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_stale": 0, 2023-07-21T22:39:07.066 INFO:teuthology.orchestra.run.smithi132.stdout: "total_load": 7931, 2023-07-21T22:39:07.067 INFO:teuthology.orchestra.run.smithi132.stdout: "average_load": 7931, 2023-07-21T22:39:07.067 INFO:teuthology.orchestra.run.smithi132.stdout: "avg_session_uptime": 2148 2023-07-21T22:39:07.067 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.067 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2023-07-21T22:39:07.067 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:39:07.067 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2023-07-21T22:39:07.067 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:39:07.068 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:39:07.068 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:39:07.068 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:39:07.068 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:39:07.068 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:39:07.068 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:39:07.068 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:39:07.068 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:39:07.069 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:39:07.069 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:39:07.069 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:39:07.069 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:39:07.069 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2023-07-21T22:39:07.069 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:39:07.069 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2023-07-21T22:39:07.070 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:39:07.070 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:39:07.070 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:39:07.070 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:39:07.070 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:39:07.070 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:39:07.070 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:39:07.071 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2023-07-21T22:39:07.071 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:39:07.071 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:39:07.071 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:39:07.071 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2023-07-21T22:39:07.071 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2023-07-21T22:39:07.071 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2023-07-21T22:39:07.071 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:39:07.072 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:39:07.072 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:39:07.072 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:39:07.072 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 291057, 2023-07-21T22:39:07.072 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 120, 2023-07-21T22:39:07.072 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 528, 2023-07-21T22:39:07.072 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 6, 2023-07-21T22:39:07.073 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2023-07-21T22:39:07.073 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2023-07-21T22:39:07.073 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:39:07.073 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2023-07-21T22:39:07.073 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:39:07.073 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2023-07-21T22:39:07.073 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 7720, 2023-07-21T22:39:07.073 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 162, 2023-07-21T22:39:07.074 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:39:07.074 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:39:07.074 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2023-07-21T22:39:07.074 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2023-07-21T22:39:07.074 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 122581, 2023-07-21T22:39:07.074 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 584, 2023-07-21T22:39:07.075 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2023-07-21T22:39:07.075 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2023-07-21T22:39:07.075 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2023-07-21T22:39:07.075 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2023-07-21T22:39:07.075 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.075 INFO:teuthology.orchestra.run.smithi132.stdout: "objecter": { 2023-07-21T22:39:07.075 INFO:teuthology.orchestra.run.smithi132.stdout: "op_active": 0, 2023-07-21T22:39:07.076 INFO:teuthology.orchestra.run.smithi132.stdout: "op_laggy": 0, 2023-07-21T22:39:07.076 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send": 14985, 2023-07-21T22:39:07.076 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send_bytes": 128849666, 2023-07-21T22:39:07.076 INFO:teuthology.orchestra.run.smithi132.stdout: "op_resend": 33, 2023-07-21T22:39:07.076 INFO:teuthology.orchestra.run.smithi132.stdout: "op_reply": 14985, 2023-07-21T22:39:07.076 INFO:teuthology.orchestra.run.smithi132.stdout: "op_latency": { 2023-07-21T22:39:07.076 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 14985, 2023-07-21T22:39:07.077 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 251.959601928, 2023-07-21T22:39:07.077 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.016814120 2023-07-21T22:39:07.077 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.077 INFO:teuthology.orchestra.run.smithi132.stdout: "op_inflight": 0, 2023-07-21T22:39:07.077 INFO:teuthology.orchestra.run.smithi132.stdout: "oplen_avg": { 2023-07-21T22:39:07.078 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 14985, 2023-07-21T22:39:07.078 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 15444 2023-07-21T22:39:07.078 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.078 INFO:teuthology.orchestra.run.smithi132.stdout: "op": 14985, 2023-07-21T22:39:07.078 INFO:teuthology.orchestra.run.smithi132.stdout: "op_r": 389, 2023-07-21T22:39:07.078 INFO:teuthology.orchestra.run.smithi132.stdout: "op_w": 14596, 2023-07-21T22:39:07.079 INFO:teuthology.orchestra.run.smithi132.stdout: "op_rmw": 0, 2023-07-21T22:39:07.079 INFO:teuthology.orchestra.run.smithi132.stdout: "op_pg": 0, 2023-07-21T22:39:07.079 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_stat": 0, 2023-07-21T22:39:07.079 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_create": 10, 2023-07-21T22:39:07.079 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_read": 389, 2023-07-21T22:39:07.079 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_write": 3044, 2023-07-21T22:39:07.079 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writefull": 198, 2023-07-21T22:39:07.079 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writesame": 0, 2023-07-21T22:39:07.080 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_append": 0, 2023-07-21T22:39:07.080 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_zero": 0, 2023-07-21T22:39:07.080 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_truncate": 28, 2023-07-21T22:39:07.080 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_delete": 10963, 2023-07-21T22:39:07.080 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_mapext": 0, 2023-07-21T22:39:07.080 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_sparse_read": 0, 2023-07-21T22:39:07.080 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_clonerange": 0, 2023-07-21T22:39:07.081 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_getxattr": 0, 2023-07-21T22:39:07.081 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_setxattr": 20, 2023-07-21T22:39:07.081 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:39:07.081 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_rmxattr": 0, 2023-07-21T22:39:07.081 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:39:07.081 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_call": 0, 2023-07-21T22:39:07.081 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_watch": 0, 2023-07-21T22:39:07.082 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_notify": 0, 2023-07-21T22:39:07.082 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:39:07.082 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls": 0, 2023-07-21T22:39:07.082 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:39:07.082 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_other": 1, 2023-07-21T22:39:07.082 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_active": 0, 2023-07-21T22:39:07.082 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_send": 0, 2023-07-21T22:39:07.083 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_resend": 0, 2023-07-21T22:39:07.083 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_ping": 0, 2023-07-21T22:39:07.083 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_active": 0, 2023-07-21T22:39:07.083 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_send": 0, 2023-07-21T22:39:07.083 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_resend": 0, 2023-07-21T22:39:07.083 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_active": 0, 2023-07-21T22:39:07.083 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_send": 0, 2023-07-21T22:39:07.083 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_resend": 0, 2023-07-21T22:39:07.084 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_active": 0, 2023-07-21T22:39:07.084 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_send": 0, 2023-07-21T22:39:07.084 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_resend": 0, 2023-07-21T22:39:07.084 INFO:teuthology.orchestra.run.smithi132.stdout: "command_active": 0, 2023-07-21T22:39:07.084 INFO:teuthology.orchestra.run.smithi132.stdout: "command_send": 0, 2023-07-21T22:39:07.084 INFO:teuthology.orchestra.run.smithi132.stdout: "command_resend": 0, 2023-07-21T22:39:07.085 INFO:teuthology.orchestra.run.smithi132.stdout: "map_epoch": 85, 2023-07-21T22:39:07.085 INFO:teuthology.orchestra.run.smithi132.stdout: "map_full": 0, 2023-07-21T22:39:07.085 INFO:teuthology.orchestra.run.smithi132.stdout: "map_inc": 3, 2023-07-21T22:39:07.085 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_sessions": 12, 2023-07-21T22:39:07.085 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_open": 12, 2023-07-21T22:39:07.085 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_close": 0, 2023-07-21T22:39:07.085 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_laggy": 0, 2023-07-21T22:39:07.085 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_wr": 684, 2023-07-21T22:39:07.086 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_rd": 0, 2023-07-21T22:39:07.086 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_del": 107 2023-07-21T22:39:07.086 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.086 INFO:teuthology.orchestra.run.smithi132.stdout: "oft": { 2023-07-21T22:39:07.086 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_objs": 1, 2023-07-21T22:39:07.086 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:39:07.086 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_updates": 0, 2023-07-21T22:39:07.086 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_removes": 0 2023-07-21T22:39:07.087 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.087 INFO:teuthology.orchestra.run.smithi132.stdout: "osd-slow-ops": { 2023-07-21T22:39:07.087 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_ops_count": 53889 2023-07-21T22:39:07.087 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.087 INFO:teuthology.orchestra.run.smithi132.stdout: "purge_queue": { 2023-07-21T22:39:07.087 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops": 0, 2023-07-21T22:39:07.087 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops_high_water": 14, 2023-07-21T22:39:07.088 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing": 0, 2023-07-21T22:39:07.088 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_high_water": 6, 2023-07-21T22:39:07.088 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed": 10650, 2023-07-21T22:39:07.088 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_item_in_journal": 0 2023-07-21T22:39:07.088 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.088 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:39:07.088 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:07.089 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-07-21T22:39:07.089 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:07.089 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 158254, 2023-07-21T22:39:07.089 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 43557429, 2023-07-21T22:39:07.089 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:07.089 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 158254, 2023-07-21T22:39:07.089 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-07-21T22:39:07.090 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-07-21T22:39:07.090 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 158254, 2023-07-21T22:39:07.090 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 43557429, 2023-07-21T22:39:07.090 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:07.090 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.090 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.090 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.091 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:07.091 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.091 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_bytes": { 2023-07-21T22:39:07.091 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:07.091 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-07-21T22:39:07.091 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:07.091 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-07-21T22:39:07.092 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-07-21T22:39:07.092 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:07.092 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:07.092 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 14985, 2023-07-21T22:39:07.092 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 129925588, 2023-07-21T22:39:07.092 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 14985, 2023-07-21T22:39:07.092 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 129925588, 2023-07-21T22:39:07.093 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:07.093 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.093 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.093 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.093 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:07.093 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.093 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_ops": { 2023-07-21T22:39:07.094 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:07.094 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 1024, 2023-07-21T22:39:07.094 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:07.094 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-07-21T22:39:07.094 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-07-21T22:39:07.094 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:07.094 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:07.094 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 14985, 2023-07-21T22:39:07.095 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 14985, 2023-07-21T22:39:07.095 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 14985, 2023-07-21T22:39:07.095 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 14985, 2023-07-21T22:39:07.095 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:07.095 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.095 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.095 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.096 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:07.096 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.096 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:39:07.096 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:07.096 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-07-21T22:39:07.096 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:07.096 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 10650, 2023-07-21T22:39:07.097 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 1075922, 2023-07-21T22:39:07.097 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:07.097 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 10650, 2023-07-21T22:39:07.097 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-07-21T22:39:07.097 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-07-21T22:39:07.097 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 389, 2023-07-21T22:39:07.097 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1075922, 2023-07-21T22:39:07.097 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:07.098 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.098 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.098 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.098 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:07.098 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-07-21T22:39:07.098 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle-0x55c58465e0a0": { 2023-07-21T22:39:07.098 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-07-21T22:39:07.098 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-07-21T22:39:07.099 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-07-21T22:39:07.099 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 59671, 2023-07-21T22:39:07.099 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 122150221, 2023-07-21T22:39:07.099 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:07.099 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 59671, 2023-07-21T22:39:07.099 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-07-21T22:39:07.099 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-07-21T22:39:07.100 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 2655, 2023-07-21T22:39:07.100 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 122150221, 2023-07-21T22:39:07.100 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-07-21T22:39:07.100 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-07-21T22:39:07.100 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-07-21T22:39:07.100 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-07-21T22:39:07.100 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:07.101 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-07-21T22:39:07.101 INFO:teuthology.orchestra.run.smithi132.stdout:} 2023-07-21T22:39:07.456 INFO:tasks.check_counter:Daemon mds.k mds.exported=124 2023-07-21T22:39:07.457 INFO:tasks.check_counter:Daemon mds.k mds.imported=122 2023-07-21T22:39:07.457 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=28 2023-07-21T22:39:07.457 DEBUG:tasks.check_counter:Getting stats from c 2023-07-21T22:39:07.457 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph daemon mds.c perf dump 2023-07-21T22:39:07.887 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:07 smithi190 ceph-mon[98334]: pgmap v595: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 12 op/s 2023-07-21T22:39:08.033 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:39:08.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:07 smithi132 ceph-mon[97863]: pgmap v595: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 12 op/s 2023-07-21T22:39:08.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:07 smithi107 ceph-mon[96543]: pgmap v595: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 12 op/s 2023-07-21T22:39:09.280 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2023-07-21T22:39:09.281 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:39:09.281 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1539, 2023-07-21T22:39:09.281 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1538, 2023-07-21T22:39:09.281 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 75950275, 2023-07-21T22:39:09.281 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 477529, 2023-07-21T22:39:09.281 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 8, 2023-07-21T22:39:09.282 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2023-07-21T22:39:09.282 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.152123820, 2023-07-21T22:39:09.282 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.035144614, 2023-07-21T22:39:09.282 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.062858030, 2023-07-21T22:39:09.282 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.043176677, 2023-07-21T22:39:09.282 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:09.282 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1537, 2023-07-21T22:39:09.283 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.069006353, 2023-07-21T22:39:09.283 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000044896 2023-07-21T22:39:09.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.283 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:09.283 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.283 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.283 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.283 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.284 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-21T22:39:09.284 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T22:39:09.284 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.284 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:39:09.284 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 759, 2023-07-21T22:39:09.284 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 720, 2023-07-21T22:39:09.285 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 72128866, 2023-07-21T22:39:09.285 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 271924, 2023-07-21T22:39:09.285 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 6, 2023-07-21T22:39:09.285 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 5, 2023-07-21T22:39:09.286 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.097499755, 2023-07-21T22:39:09.286 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.018284034, 2023-07-21T22:39:09.286 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.061374557, 2023-07-21T22:39:09.286 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.016428954, 2023-07-21T22:39:09.287 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:09.287 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 719, 2023-07-21T22:39:09.287 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.023119209, 2023-07-21T22:39:09.287 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000032154 2023-07-21T22:39:09.288 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.288 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:09.288 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.289 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.289 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.289 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.289 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 683184, 2023-07-21T22:39:09.290 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 109520 2023-07-21T22:39:09.290 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.290 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:39:09.290 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 466, 2023-07-21T22:39:09.291 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 609, 2023-07-21T22:39:09.291 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 67006459, 2023-07-21T22:39:09.291 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 465399, 2023-07-21T22:39:09.292 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 6, 2023-07-21T22:39:09.292 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 5, 2023-07-21T22:39:09.292 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.072174476, 2023-07-21T22:39:09.292 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.013090682, 2023-07-21T22:39:09.293 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.043733797, 2023-07-21T22:39:09.293 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.011219690, 2023-07-21T22:39:09.293 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:09.293 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 607, 2023-07-21T22:39:09.294 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.052767526, 2023-07-21T22:39:09.294 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000086931 2023-07-21T22:39:09.294 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.294 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:09.294 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.295 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.295 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.295 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.295 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-21T22:39:09.296 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 320784 2023-07-21T22:39:09.296 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.296 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2023-07-21T22:39:09.297 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2023-07-21T22:39:09.297 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2023-07-21T22:39:09.297 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.297 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2023-07-21T22:39:09.298 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-07-21T22:39:09.298 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-07-21T22:39:09.298 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4131, 2023-07-21T22:39:09.298 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.114705891, 2023-07-21T22:39:09.299 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000027767 2023-07-21T22:39:09.299 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:09.299 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.299 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2023-07-21T22:39:09.300 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-07-21T22:39:09.300 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-07-21T22:39:09.300 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.300 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.301 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.301 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:09.301 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.302 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2023-07-21T22:39:09.302 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 0, 2023-07-21T22:39:09.302 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 0, 2023-07-21T22:39:09.302 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2023-07-21T22:39:09.303 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.303 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.303 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.303 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.304 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2023-07-21T22:39:09.304 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 0, 2023-07-21T22:39:09.304 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2023-07-21T22:39:09.304 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2023-07-21T22:39:09.305 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 0, 2023-07-21T22:39:09.305 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2023-07-21T22:39:09.305 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2023-07-21T22:39:09.305 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 19307, 2023-07-21T22:39:09.306 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 0, 2023-07-21T22:39:09.306 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 0, 2023-07-21T22:39:09.306 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 19307, 2023-07-21T22:39:09.306 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 19307, 2023-07-21T22:39:09.307 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 1032, 2023-07-21T22:39:09.307 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2023-07-21T22:39:09.307 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2023-07-21T22:39:09.308 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 1, 2023-07-21T22:39:09.308 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 0, 2023-07-21T22:39:09.308 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 0, 2023-07-21T22:39:09.308 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2023-07-21T22:39:09.309 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 0, 2023-07-21T22:39:09.309 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:39:09.309 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2023-07-21T22:39:09.309 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2023-07-21T22:39:09.310 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2023-07-21T22:39:09.310 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2023-07-21T22:39:09.310 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2023-07-21T22:39:09.310 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2023-07-21T22:39:09.311 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 0, 2023-07-21T22:39:09.311 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 0, 2023-07-21T22:39:09.311 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2023-07-21T22:39:09.312 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:39:09.312 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2023-07-21T22:39:09.312 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 0, 2023-07-21T22:39:09.312 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 0, 2023-07-21T22:39:09.313 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2023-07-21T22:39:09.313 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:39:09.313 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2023-07-21T22:39:09.314 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:39:09.314 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:39:09.314 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:39:09.314 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:39:09.315 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:39:09.315 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2023-07-21T22:39:09.315 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2023-07-21T22:39:09.315 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T22:39:09.316 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 0, 2023-07-21T22:39:09.316 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:39:09.316 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:39:09.316 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:39:09.316 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 0, 2023-07-21T22:39:09.317 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 0, 2023-07-21T22:39:09.317 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2023-07-21T22:39:09.317 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2023-07-21T22:39:09.317 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.317 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2023-07-21T22:39:09.317 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2023-07-21T22:39:09.317 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 0, 2023-07-21T22:39:09.317 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 0, 2023-07-21T22:39:09.318 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 0, 2023-07-21T22:39:09.318 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2023-07-21T22:39:09.318 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2023-07-21T22:39:09.318 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2023-07-21T22:39:09.318 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:39:09.318 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2023-07-21T22:39:09.318 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2023-07-21T22:39:09.318 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2023-07-21T22:39:09.319 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2023-07-21T22:39:09.319 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2023-07-21T22:39:09.319 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:39:09.319 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:39:09.319 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2023-07-21T22:39:09.319 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2023-07-21T22:39:09.319 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:39:09.319 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2023-07-21T22:39:09.320 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2023-07-21T22:39:09.320 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2023-07-21T22:39:09.320 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2023-07-21T22:39:09.320 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2023-07-21T22:39:09.320 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.320 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2023-07-21T22:39:09.320 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 0, 2023-07-21T22:39:09.321 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2023-07-21T22:39:09.321 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2023-07-21T22:39:09.321 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 0, 2023-07-21T22:39:09.321 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2023-07-21T22:39:09.321 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2023-07-21T22:39:09.321 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 0, 2023-07-21T22:39:09.321 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2023-07-21T22:39:09.321 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2023-07-21T22:39:09.322 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 120, 2023-07-21T22:39:09.322 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2023-07-21T22:39:09.322 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2023-07-21T22:39:09.322 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2023-07-21T22:39:09.322 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 0, 2023-07-21T22:39:09.322 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 217864571, 2023-07-21T22:39:09.322 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2023-07-21T22:39:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 97796 2023-07-21T22:39:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2023-07-21T22:39:09.323 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 16, 2023-07-21T22:39:09.324 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 27879, 2023-07-21T22:39:09.324 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 27863, 2023-07-21T22:39:09.324 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 28, 2023-07-21T22:39:09.324 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 1245, 2023-07-21T22:39:09.324 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 1217, 2023-07-21T22:39:09.324 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 19307, 2023-07-21T22:39:09.324 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 64028, 2023-07-21T22:39:09.324 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 44721, 2023-07-21T22:39:09.325 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2023-07-21T22:39:09.325 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 0, 2023-07-21T22:39:09.325 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2023-07-21T22:39:09.325 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 74904, 2023-07-21T22:39:09.325 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 207156 2023-07-21T22:39:09.325 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.325 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2023-07-21T22:39:09.326 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 0, 2023-07-21T22:39:09.326 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2023-07-21T22:39:09.326 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 0, 2023-07-21T22:39:09.326 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 0, 2023-07-21T22:39:09.326 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 0, 2023-07-21T22:39:09.326 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2023-07-21T22:39:09.326 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.326 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.327 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.327 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2023-07-21T22:39:09.327 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.327 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.327 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.327 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.327 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2023-07-21T22:39:09.328 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.328 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.328 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.328 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.329 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2023-07-21T22:39:09.329 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.329 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.329 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.330 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.330 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2023-07-21T22:39:09.330 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.331 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.331 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.331 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.331 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2023-07-21T22:39:09.332 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.332 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.332 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.333 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.333 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2023-07-21T22:39:09.333 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.333 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.334 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.334 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2023-07-21T22:39:09.334 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.334 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.334 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.334 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.335 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2023-07-21T22:39:09.335 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.335 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.335 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.335 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.335 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2023-07-21T22:39:09.335 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.335 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.336 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.336 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2023-07-21T22:39:09.336 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.336 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.336 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.336 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.337 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2023-07-21T22:39:09.337 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.337 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.337 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.337 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.337 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2023-07-21T22:39:09.337 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.337 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.338 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.338 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2023-07-21T22:39:09.338 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.338 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.338 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.338 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.339 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2023-07-21T22:39:09.339 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.339 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.339 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.339 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.339 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2023-07-21T22:39:09.339 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.339 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.340 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.340 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2023-07-21T22:39:09.340 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.340 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.340 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.340 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.341 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2023-07-21T22:39:09.341 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.341 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.341 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.341 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.341 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2023-07-21T22:39:09.341 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.341 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.342 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.342 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2023-07-21T22:39:09.342 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.342 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.342 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.342 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.342 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2023-07-21T22:39:09.343 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.343 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.343 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.343 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.343 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2023-07-21T22:39:09.343 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.343 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.344 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.344 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.344 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2023-07-21T22:39:09.344 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.344 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.344 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.345 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.345 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2023-07-21T22:39:09.345 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.345 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.346 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.346 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.346 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2023-07-21T22:39:09.347 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.347 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.347 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.347 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.348 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2023-07-21T22:39:09.348 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.348 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.349 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.349 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.349 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2023-07-21T22:39:09.349 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.350 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.350 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.350 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.351 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2023-07-21T22:39:09.351 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.351 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.352 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.352 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.352 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2023-07-21T22:39:09.352 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.352 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.353 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.353 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.353 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:39:09.353 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:39:09.353 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2023-07-21T22:39:09.354 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.354 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.354 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.355 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:09.355 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.355 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2023-07-21T22:39:09.355 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2023-07-21T22:39:09.356 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 2, 2023-07-21T22:39:09.356 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 1, 2023-07-21T22:39:09.356 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2023-07-21T22:39:09.356 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2023-07-21T22:39:09.356 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 0, 2023-07-21T22:39:09.356 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 0, 2023-07-21T22:39:09.356 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 2139 2023-07-21T22:39:09.357 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.357 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2023-07-21T22:39:09.357 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:39:09.357 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2023-07-21T22:39:09.357 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:39:09.357 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:39:09.357 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:39:09.357 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:39:09.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:39:09.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:39:09.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:39:09.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:39:09.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:39:09.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:39:09.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:39:09.358 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:39:09.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:39:09.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2023-07-21T22:39:09.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:39:09.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2023-07-21T22:39:09.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:39:09.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:39:09.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:39:09.359 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:39:09.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:39:09.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:39:09.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:39:09.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2023-07-21T22:39:09.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:39:09.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:39:09.360 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:39:09.361 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2023-07-21T22:39:09.361 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2023-07-21T22:39:09.361 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2023-07-21T22:39:09.361 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:39:09.361 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:39:09.361 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:39:09.361 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:39:09.361 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 240564, 2023-07-21T22:39:09.362 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 82, 2023-07-21T22:39:09.362 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:39:09.362 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2023-07-21T22:39:09.362 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2023-07-21T22:39:09.362 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2023-07-21T22:39:09.362 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:39:09.362 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2023-07-21T22:39:09.363 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:39:09.363 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2023-07-21T22:39:09.363 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7720, 2023-07-21T22:39:09.363 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 162, 2023-07-21T22:39:09.363 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:39:09.363 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:39:09.363 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2023-07-21T22:39:09.363 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2023-07-21T22:39:09.364 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 13666567, 2023-07-21T22:39:09.364 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 80657, 2023-07-21T22:39:09.364 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2023-07-21T22:39:09.364 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2023-07-21T22:39:09.364 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2023-07-21T22:39:09.364 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2023-07-21T22:39:09.364 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.364 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2023-07-21T22:39:09.365 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2023-07-21T22:39:09.365 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2023-07-21T22:39:09.365 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 2521, 2023-07-21T22:39:09.365 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 21, 2023-07-21T22:39:09.365 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 3, 2023-07-21T22:39:09.365 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 2521, 2023-07-21T22:39:09.365 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2023-07-21T22:39:09.366 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2521, 2023-07-21T22:39:09.366 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 11.809665292, 2023-07-21T22:39:09.366 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.004684516 2023-07-21T22:39:09.366 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.366 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2023-07-21T22:39:09.366 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2023-07-21T22:39:09.366 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2521, 2023-07-21T22:39:09.366 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 2522 2023-07-21T22:39:09.367 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.367 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 2521, 2023-07-21T22:39:09.367 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 2521, 2023-07-21T22:39:09.367 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2023-07-21T22:39:09.367 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2023-07-21T22:39:09.367 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2023-07-21T22:39:09.367 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 1494, 2023-07-21T22:39:09.367 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2023-07-21T22:39:09.368 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 1025, 2023-07-21T22:39:09.368 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2023-07-21T22:39:09.368 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2023-07-21T22:39:09.368 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2023-07-21T22:39:09.368 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2023-07-21T22:39:09.368 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2023-07-21T22:39:09.368 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2023-07-21T22:39:09.369 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2023-07-21T22:39:09.369 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2023-07-21T22:39:09.369 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2023-07-21T22:39:09.369 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2023-07-21T22:39:09.369 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 1, 2023-07-21T22:39:09.369 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2023-07-21T22:39:09.369 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:39:09.369 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2023-07-21T22:39:09.370 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:39:09.370 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2023-07-21T22:39:09.370 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2023-07-21T22:39:09.370 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2023-07-21T22:39:09.370 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:39:09.370 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2023-07-21T22:39:09.370 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:39:09.370 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2023-07-21T22:39:09.371 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2023-07-21T22:39:09.371 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2023-07-21T22:39:09.371 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2023-07-21T22:39:09.371 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2023-07-21T22:39:09.371 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2023-07-21T22:39:09.371 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2023-07-21T22:39:09.371 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2023-07-21T22:39:09.371 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2023-07-21T22:39:09.372 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2023-07-21T22:39:09.372 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2023-07-21T22:39:09.372 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2023-07-21T22:39:09.372 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2023-07-21T22:39:09.372 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2023-07-21T22:39:09.372 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2023-07-21T22:39:09.372 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2023-07-21T22:39:09.373 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2023-07-21T22:39:09.373 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 85, 2023-07-21T22:39:09.373 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2023-07-21T22:39:09.373 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 3, 2023-07-21T22:39:09.373 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 11, 2023-07-21T22:39:09.373 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 11, 2023-07-21T22:39:09.373 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2023-07-21T22:39:09.373 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2023-07-21T22:39:09.374 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2023-07-21T22:39:09.374 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 2, 2023-07-21T22:39:09.374 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2023-07-21T22:39:09.374 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.374 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2023-07-21T22:39:09.374 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 0, 2023-07-21T22:39:09.374 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:39:09.374 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2023-07-21T22:39:09.375 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2023-07-21T22:39:09.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.375 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2023-07-21T22:39:09.375 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2023-07-21T22:39:09.375 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.375 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2023-07-21T22:39:09.375 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2023-07-21T22:39:09.375 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T22:39:09.376 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2023-07-21T22:39:09.376 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2023-07-21T22:39:09.376 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2023-07-21T22:39:09.376 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2023-07-21T22:39:09.376 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.376 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:39:09.376 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:09.377 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-07-21T22:39:09.377 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:09.377 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 2760, 2023-07-21T22:39:09.377 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 214822205, 2023-07-21T22:39:09.377 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:09.377 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 2760, 2023-07-21T22:39:09.377 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-07-21T22:39:09.377 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-07-21T22:39:09.378 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2760, 2023-07-21T22:39:09.378 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 214822205, 2023-07-21T22:39:09.378 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:09.378 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.378 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.378 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.378 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:09.379 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.379 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2023-07-21T22:39:09.379 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:09.379 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-07-21T22:39:09.379 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:09.379 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-07-21T22:39:09.379 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-07-21T22:39:09.379 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:09.380 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:09.380 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 2521, 2023-07-21T22:39:09.380 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 213699268, 2023-07-21T22:39:09.380 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2521, 2023-07-21T22:39:09.380 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 213699268, 2023-07-21T22:39:09.380 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:09.380 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.381 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.381 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.381 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:09.381 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.381 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2023-07-21T22:39:09.381 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:09.381 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2023-07-21T22:39:09.381 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:09.382 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-07-21T22:39:09.382 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-07-21T22:39:09.382 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:09.382 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:09.382 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 2521, 2023-07-21T22:39:09.382 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 2521, 2023-07-21T22:39:09.382 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2521, 2023-07-21T22:39:09.382 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 2521, 2023-07-21T22:39:09.383 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:09.383 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.383 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.383 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.383 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:09.383 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.383 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:39:09.383 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:09.384 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-07-21T22:39:09.384 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:09.384 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-07-21T22:39:09.384 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-07-21T22:39:09.384 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:09.384 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:09.384 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-07-21T22:39:09.385 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-07-21T22:39:09.385 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2023-07-21T22:39:09.385 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2023-07-21T22:39:09.385 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:09.385 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.385 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.385 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.385 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:09.386 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:09.386 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55c12f610460": { 2023-07-21T22:39:09.386 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:09.386 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-07-21T22:39:09.386 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:09.386 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-07-21T22:39:09.386 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-07-21T22:39:09.386 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:09.387 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:09.387 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-07-21T22:39:09.387 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-07-21T22:39:09.387 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2023-07-21T22:39:09.387 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2023-07-21T22:39:09.387 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:09.387 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:09.387 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:09.388 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:09.388 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:09.388 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:09.388 INFO:teuthology.orchestra.run.smithi190.stdout:} 2023-07-21T22:39:09.675 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-07-21T22:39:09.676 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-07-21T22:39:09.676 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2023-07-21T22:39:09.676 DEBUG:tasks.check_counter:Getting stats from f 2023-07-21T22:39:09.677 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph daemon mds.f perf dump 2023-07-21T22:39:10.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:09 smithi190 ceph-mon[98334]: pgmap v596: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 B/s wr, 14 op/s 2023-07-21T22:39:10.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:09 smithi132 ceph-mon[97863]: pgmap v596: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 B/s wr, 14 op/s 2023-07-21T22:39:10.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:09 smithi107 ceph-mon[96543]: pgmap v596: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 B/s wr, 14 op/s 2023-07-21T22:39:10.228 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:39:11.456 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2023-07-21T22:39:11.456 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:39:11.457 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 168735, 2023-07-21T22:39:11.457 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 246945, 2023-07-21T22:39:11.457 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 55614104, 2023-07-21T22:39:11.457 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 149397422, 2023-07-21T22:39:11.458 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 15, 2023-07-21T22:39:11.458 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 3, 2023-07-21T22:39:11.458 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 5.924981730, 2023-07-21T22:39:11.459 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 3.057920998, 2023-07-21T22:39:11.459 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 14.875342171, 2023-07-21T22:39:11.459 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.116323759, 2023-07-21T22:39:11.459 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:11.460 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 246944, 2023-07-21T22:39:11.460 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 19.846847062, 2023-07-21T22:39:11.460 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000080369 2023-07-21T22:39:11.461 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.461 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:11.461 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 136123, 2023-07-21T22:39:11.461 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.035162609, 2023-07-21T22:39:11.462 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000258 2023-07-21T22:39:11.462 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.462 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 686080, 2023-07-21T22:39:11.463 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 185008 2023-07-21T22:39:11.463 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.463 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:39:11.463 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 88621, 2023-07-21T22:39:11.464 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 86093, 2023-07-21T22:39:11.464 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 23824111, 2023-07-21T22:39:11.464 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 145049419, 2023-07-21T22:39:11.465 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 10, 2023-07-21T22:39:11.465 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 6, 2023-07-21T22:39:11.465 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 3.911841234, 2023-07-21T22:39:11.465 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 1.513869666, 2023-07-21T22:39:11.466 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 2.527100000, 2023-07-21T22:39:11.466 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.330433805, 2023-07-21T22:39:11.466 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:11.467 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 86092, 2023-07-21T22:39:11.467 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 2.179185401, 2023-07-21T22:39:11.467 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000025312 2023-07-21T22:39:11.467 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.468 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:11.468 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 154564, 2023-07-21T22:39:11.468 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.015605669, 2023-07-21T22:39:11.469 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000100 2023-07-21T22:39:11.469 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.469 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-07-21T22:39:11.469 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 326032 2023-07-21T22:39:11.470 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.470 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:39:11.470 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1234187, 2023-07-21T22:39:11.471 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1227692, 2023-07-21T22:39:11.471 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 207344344, 2023-07-21T22:39:11.471 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 248709740, 2023-07-21T22:39:11.472 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 17, 2023-07-21T22:39:11.472 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 8, 2023-07-21T22:39:11.472 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 30.755377166, 2023-07-21T22:39:11.472 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 11.871038702, 2023-07-21T22:39:11.473 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 20.993542491, 2023-07-21T22:39:11.473 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.260284427, 2023-07-21T22:39:11.473 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:11.474 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1227690, 2023-07-21T22:39:11.474 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 13.264320193, 2023-07-21T22:39:11.474 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000010804 2023-07-21T22:39:11.474 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.475 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:11.475 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2375900, 2023-07-21T22:39:11.475 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.221160044, 2023-07-21T22:39:11.476 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000093 2023-07-21T22:39:11.476 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.476 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:39:11.476 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:39:11.477 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.477 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2023-07-21T22:39:11.477 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2023-07-21T22:39:11.477 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2023-07-21T22:39:11.478 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.478 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2023-07-21T22:39:11.478 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-07-21T22:39:11.479 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-07-21T22:39:11.479 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5128, 2023-07-21T22:39:11.479 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 47.766149423, 2023-07-21T22:39:11.479 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.009314771 2023-07-21T22:39:11.480 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:11.480 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.480 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2023-07-21T22:39:11.480 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-07-21T22:39:11.481 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-07-21T22:39:11.481 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 31300, 2023-07-21T22:39:11.481 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 6.381001904, 2023-07-21T22:39:11.481 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000203865 2023-07-21T22:39:11.482 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:11.482 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.482 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2023-07-21T22:39:11.483 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 85768, 2023-07-21T22:39:11.483 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 83271, 2023-07-21T22:39:11.483 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2023-07-21T22:39:11.483 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 83271, 2023-07-21T22:39:11.483 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 795.152394463, 2023-07-21T22:39:11.484 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.009548971 2023-07-21T22:39:11.484 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.484 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2023-07-21T22:39:11.485 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 2495, 2023-07-21T22:39:11.485 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 21, 2023-07-21T22:39:11.485 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2023-07-21T22:39:11.485 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 609, 2023-07-21T22:39:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 66, 2023-07-21T22:39:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 5, 2023-07-21T22:39:11.486 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 357, 2023-07-21T22:39:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 239, 2023-07-21T22:39:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 108, 2023-07-21T22:39:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 10, 2023-07-21T22:39:11.487 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 15, 2023-07-21T22:39:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 16895, 2023-07-21T22:39:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 2, 2023-07-21T22:39:11.488 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 2, 2023-07-21T22:39:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 6, 2023-07-21T22:39:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 140158, 2023-07-21T22:39:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 117642, 2023-07-21T22:39:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 2423, 2023-07-21T22:39:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 666, 2023-07-21T22:39:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:39:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2023-07-21T22:39:11.489 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 710, 2023-07-21T22:39:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 2000, 2023-07-21T22:39:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 2, 2023-07-21T22:39:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 195, 2023-07-21T22:39:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 7869, 2023-07-21T22:39:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 191, 2023-07-21T22:39:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 15559, 2023-07-21T22:39:11.490 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2023-07-21T22:39:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 576, 2023-07-21T22:39:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2023-07-21T22:39:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 5452, 2023-07-21T22:39:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 102857566, 2023-07-21T22:39:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2023-07-21T22:39:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 576, 2023-07-21T22:39:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2023-07-21T22:39:11.491 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:39:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:39:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 1, 2023-07-21T22:39:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:39:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 1, 2023-07-21T22:39:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 575, 2023-07-21T22:39:11.492 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 161, 2023-07-21T22:39:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 14166, 2023-07-21T22:39:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 32138, 2023-07-21T22:39:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:39:11.493 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:39:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:39:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 26077, 2023-07-21T22:39:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 13006, 2023-07-21T22:39:11.494 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 447, 2023-07-21T22:39:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 37091 2023-07-21T22:39:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.495 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2023-07-21T22:39:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 3, 2023-07-21T22:39:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 7, 2023-07-21T22:39:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 5, 2023-07-21T22:39:11.496 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 795, 2023-07-21T22:39:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 482, 2023-07-21T22:39:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 1, 2023-07-21T22:39:11.497 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2023-07-21T22:39:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:39:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 17207, 2023-07-21T22:39:11.498 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 17206, 2023-07-21T22:39:11.499 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2023-07-21T22:39:11.499 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2023-07-21T22:39:11.499 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2023-07-21T22:39:11.499 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:39:11.500 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:39:11.500 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2023-07-21T22:39:11.500 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2023-07-21T22:39:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:39:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 195, 2023-07-21T22:39:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2023-07-21T22:39:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 71, 2023-07-21T22:39:11.501 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2023-07-21T22:39:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2023-07-21T22:39:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2023-07-21T22:39:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 93835, 2023-07-21T22:39:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2023-07-21T22:39:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2023-07-21T22:39:11.502 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 93835, 2023-07-21T22:39:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2023-07-21T22:39:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2023-07-21T22:39:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 116, 2023-07-21T22:39:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2023-07-21T22:39:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2023-07-21T22:39:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 116, 2023-07-21T22:39:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2023-07-21T22:39:11.503 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2023-07-21T22:39:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2023-07-21T22:39:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 207918520, 2023-07-21T22:39:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2023-07-21T22:39:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2023-07-21T22:39:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2341, 2023-07-21T22:39:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 118.071921137, 2023-07-21T22:39:11.504 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.050436531 2023-07-21T22:39:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2023-07-21T22:39:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2023-07-21T22:39:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 58, 2023-07-21T22:39:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 21819, 2023-07-21T22:39:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 21761, 2023-07-21T22:39:11.505 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 57, 2023-07-21T22:39:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 971, 2023-07-21T22:39:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 914, 2023-07-21T22:39:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 357, 2023-07-21T22:39:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 59899, 2023-07-21T22:39:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 59542, 2023-07-21T22:39:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 2, 2023-07-21T22:39:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 24708, 2023-07-21T22:39:11.506 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 24706, 2023-07-21T22:39:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 110680, 2023-07-21T22:39:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 198964 2023-07-21T22:39:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2023-07-21T22:39:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 101691, 2023-07-21T22:39:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 2385, 2023-07-21T22:39:11.507 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 85768, 2023-07-21T22:39:11.508 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 254, 2023-07-21T22:39:11.508 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 3923, 2023-07-21T22:39:11.508 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2023-07-21T22:39:11.508 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 12929, 2023-07-21T22:39:11.508 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 28.650245509, 2023-07-21T22:39:11.508 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002215967 2023-07-21T22:39:11.508 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.508 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2023-07-21T22:39:11.509 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1957, 2023-07-21T22:39:11.509 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 82.079008639, 2023-07-21T22:39:11.509 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.041941241 2023-07-21T22:39:11.509 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.509 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2023-07-21T22:39:11.509 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.509 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.509 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2023-07-21T22:39:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2023-07-21T22:39:11.510 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 35227, 2023-07-21T22:39:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 451.019037202, 2023-07-21T22:39:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.012803220 2023-07-21T22:39:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2023-07-21T22:39:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.511 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2023-07-21T22:39:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2023-07-21T22:39:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000228364, 2023-07-21T22:39:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000228364 2023-07-21T22:39:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2023-07-21T22:39:11.512 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.513 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.513 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.513 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2023-07-21T22:39:11.513 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.513 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.513 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.513 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.514 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2023-07-21T22:39:11.514 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.514 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.514 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.514 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.514 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2023-07-21T22:39:11.514 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.515 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.515 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.515 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2023-07-21T22:39:11.515 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 78, 2023-07-21T22:39:11.515 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1.857696070, 2023-07-21T22:39:11.515 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.023816616 2023-07-21T22:39:11.515 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.516 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2023-07-21T22:39:11.516 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.516 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.516 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.516 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.516 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2023-07-21T22:39:11.516 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2023-07-21T22:39:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4328, 2023-07-21T22:39:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 7.877107924, 2023-07-21T22:39:11.517 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001820034 2023-07-21T22:39:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2023-07-21T22:39:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 637, 2023-07-21T22:39:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.920690515, 2023-07-21T22:39:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001445354 2023-07-21T22:39:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2023-07-21T22:39:11.518 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 12939, 2023-07-21T22:39:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 188.297492325, 2023-07-21T22:39:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.014552708 2023-07-21T22:39:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2023-07-21T22:39:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.519 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2023-07-21T22:39:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2023-07-21T22:39:11.520 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 105, 2023-07-21T22:39:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 15.449196914, 2023-07-21T22:39:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.147135208 2023-07-21T22:39:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2023-07-21T22:39:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.521 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2023-07-21T22:39:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2023-07-21T22:39:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.522 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2023-07-21T22:39:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.523 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2023-07-21T22:39:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2023-07-21T22:39:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.524 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2023-07-21T22:39:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.525 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2023-07-21T22:39:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2023-07-21T22:39:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 15070, 2023-07-21T22:39:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 19.001691001, 2023-07-21T22:39:11.526 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001260895 2023-07-21T22:39:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:39:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:39:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2023-07-21T22:39:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.527 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2023-07-21T22:39:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2023-07-21T22:39:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 1, 2023-07-21T22:39:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 0, 2023-07-21T22:39:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2023-07-21T22:39:11.528 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2023-07-21T22:39:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 16486, 2023-07-21T22:39:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 16486, 2023-07-21T22:39:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 604 2023-07-21T22:39:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2023-07-21T22:39:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:39:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2023-07-21T22:39:11.529 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:39:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:39:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:39:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:39:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:39:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:39:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:39:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:39:11.530 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:39:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:39:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:39:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:39:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:39:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2023-07-21T22:39:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:39:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2023-07-21T22:39:11.531 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:39:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:39:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:39:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:39:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:39:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:39:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:39:11.532 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2023-07-21T22:39:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:39:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:39:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:39:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2023-07-21T22:39:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2023-07-21T22:39:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2023-07-21T22:39:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:39:11.533 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:39:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:39:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:39:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 320556, 2023-07-21T22:39:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 140, 2023-07-21T22:39:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 528, 2023-07-21T22:39:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 6, 2023-07-21T22:39:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2023-07-21T22:39:11.534 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2023-07-21T22:39:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:39:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2023-07-21T22:39:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:39:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2023-07-21T22:39:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7720, 2023-07-21T22:39:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 162, 2023-07-21T22:39:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:39:11.535 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:39:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2023-07-21T22:39:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2023-07-21T22:39:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 465946, 2023-07-21T22:39:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 5467, 2023-07-21T22:39:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2023-07-21T22:39:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2023-07-21T22:39:11.536 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2023-07-21T22:39:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2023-07-21T22:39:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2023-07-21T22:39:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2023-07-21T22:39:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2023-07-21T22:39:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 22738, 2023-07-21T22:39:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 217556935, 2023-07-21T22:39:11.537 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 33, 2023-07-21T22:39:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 22738, 2023-07-21T22:39:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2023-07-21T22:39:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 22738, 2023-07-21T22:39:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 290.137816516, 2023-07-21T22:39:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.012760041 2023-07-21T22:39:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2023-07-21T22:39:11.538 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2023-07-21T22:39:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 22738, 2023-07-21T22:39:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 23607 2023-07-21T22:39:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 22738, 2023-07-21T22:39:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 1090, 2023-07-21T22:39:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 21648, 2023-07-21T22:39:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2023-07-21T22:39:11.539 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2023-07-21T22:39:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2023-07-21T22:39:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2023-07-21T22:39:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 447, 2023-07-21T22:39:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 2788, 2023-07-21T22:39:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 228, 2023-07-21T22:39:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2023-07-21T22:39:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2023-07-21T22:39:11.540 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2023-07-21T22:39:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 66, 2023-07-21T22:39:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 17852, 2023-07-21T22:39:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2023-07-21T22:39:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2023-07-21T22:39:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2023-07-21T22:39:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 576, 2023-07-21T22:39:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2023-07-21T22:39:11.541 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:39:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2023-07-21T22:39:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:39:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2023-07-21T22:39:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2023-07-21T22:39:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2023-07-21T22:39:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:39:11.542 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2023-07-21T22:39:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:39:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2023-07-21T22:39:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2023-07-21T22:39:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2023-07-21T22:39:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2023-07-21T22:39:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2023-07-21T22:39:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2023-07-21T22:39:11.543 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2023-07-21T22:39:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2023-07-21T22:39:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2023-07-21T22:39:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2023-07-21T22:39:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2023-07-21T22:39:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2023-07-21T22:39:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2023-07-21T22:39:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2023-07-21T22:39:11.544 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2023-07-21T22:39:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2023-07-21T22:39:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2023-07-21T22:39:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 85, 2023-07-21T22:39:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2023-07-21T22:39:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 3, 2023-07-21T22:39:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 12, 2023-07-21T22:39:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 12, 2023-07-21T22:39:11.545 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2023-07-21T22:39:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2023-07-21T22:39:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 1383, 2023-07-21T22:39:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 88, 2023-07-21T22:39:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 149 2023-07-21T22:39:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2023-07-21T22:39:11.546 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2023-07-21T22:39:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:39:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2023-07-21T22:39:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2023-07-21T22:39:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2023-07-21T22:39:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 86809 2023-07-21T22:39:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.547 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2023-07-21T22:39:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2023-07-21T22:39:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 14, 2023-07-21T22:39:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2023-07-21T22:39:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 6, 2023-07-21T22:39:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 17206, 2023-07-21T22:39:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2023-07-21T22:39:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.548 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:39:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-07-21T22:39:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 1491539, 2023-07-21T22:39:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 154893349, 2023-07-21T22:39:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 1491539, 2023-07-21T22:39:11.549 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-07-21T22:39:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-07-21T22:39:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 1491539, 2023-07-21T22:39:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 154893349, 2023-07-21T22:39:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.550 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2023-07-21T22:39:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-07-21T22:39:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-07-21T22:39:11.551 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-07-21T22:39:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 22738, 2023-07-21T22:39:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 219293375, 2023-07-21T22:39:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 22738, 2023-07-21T22:39:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 219293375, 2023-07-21T22:39:11.552 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2023-07-21T22:39:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:11.553 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2023-07-21T22:39:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-07-21T22:39:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-07-21T22:39:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 22738, 2023-07-21T22:39:11.554 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 22738, 2023-07-21T22:39:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 22738, 2023-07-21T22:39:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 22738, 2023-07-21T22:39:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:11.555 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:39:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-07-21T22:39:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 17206, 2023-07-21T22:39:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 1738382, 2023-07-21T22:39:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:11.556 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 17206, 2023-07-21T22:39:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-07-21T22:39:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-07-21T22:39:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 447, 2023-07-21T22:39:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1738382, 2023-07-21T22:39:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.557 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x556fd35be0a0": { 2023-07-21T22:39:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-07-21T22:39:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:11.558 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 93835, 2023-07-21T22:39:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 203724216, 2023-07-21T22:39:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 93835, 2023-07-21T22:39:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-07-21T22:39:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-07-21T22:39:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2341, 2023-07-21T22:39:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 203724216, 2023-07-21T22:39:11.559 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:11.560 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:11.560 INFO:teuthology.orchestra.run.smithi190.stdout:} 2023-07-21T22:39:11.968 INFO:tasks.check_counter:Daemon mds.f mds.exported=195 2023-07-21T22:39:11.968 INFO:tasks.check_counter:Daemon mds.f mds.imported=191 2023-07-21T22:39:11.969 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=66 2023-07-21T22:39:11.969 DEBUG:tasks.check_counter:Getting stats from i 2023-07-21T22:39:11.969 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph daemon mds.i perf dump 2023-07-21T22:39:11.995 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:11 smithi190 ceph-mon[98334]: pgmap v597: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2023-07-21T22:39:12.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:11 smithi132 ceph-mon[97863]: pgmap v597: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2023-07-21T22:39:12.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:11 smithi107 ceph-mon[96543]: pgmap v597: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2023-07-21T22:39:12.522 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:39:13.766 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2023-07-21T22:39:13.766 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:39:13.767 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 170316, 2023-07-21T22:39:13.767 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 182221, 2023-07-21T22:39:13.767 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 47656903, 2023-07-21T22:39:13.767 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 97647665, 2023-07-21T22:39:13.767 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 12, 2023-07-21T22:39:13.767 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2023-07-21T22:39:13.767 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 6.729836945, 2023-07-21T22:39:13.768 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 2.613878396, 2023-07-21T22:39:13.768 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 4.526784617, 2023-07-21T22:39:13.768 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.292616750, 2023-07-21T22:39:13.768 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:13.768 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 182221, 2023-07-21T22:39:13.768 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 4.092566438, 2023-07-21T22:39:13.769 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000022459 2023-07-21T22:39:13.769 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.769 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:13.769 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 333233, 2023-07-21T22:39:13.769 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.036996559, 2023-07-21T22:39:13.770 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000111 2023-07-21T22:39:13.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.770 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 912, 2023-07-21T22:39:13.770 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 688 2023-07-21T22:39:13.770 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.770 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:39:13.770 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 148284, 2023-07-21T22:39:13.770 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 234191, 2023-07-21T22:39:13.771 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 46325587, 2023-07-21T22:39:13.771 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 178387100, 2023-07-21T22:39:13.771 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 19, 2023-07-21T22:39:13.771 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 8, 2023-07-21T22:39:13.771 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 5.886515912, 2023-07-21T22:39:13.771 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 2.937761521, 2023-07-21T22:39:13.771 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 30.064700016, 2023-07-21T22:39:13.772 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.341709378, 2023-07-21T22:39:13.772 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:13.772 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 234188, 2023-07-21T22:39:13.772 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 18.202233049, 2023-07-21T22:39:13.772 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000077724 2023-07-21T22:39:13.772 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.772 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:13.773 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 137636, 2023-07-21T22:39:13.773 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.033759500, 2023-07-21T22:39:13.773 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000245 2023-07-21T22:39:13.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.773 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 2768, 2023-07-21T22:39:13.773 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 327456 2023-07-21T22:39:13.773 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.774 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:39:13.774 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 15911, 2023-07-21T22:39:13.774 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 16158, 2023-07-21T22:39:13.774 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 7106090, 2023-07-21T22:39:13.774 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 83554437, 2023-07-21T22:39:13.774 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 12, 2023-07-21T22:39:13.774 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 10, 2023-07-21T22:39:13.774 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 1.087240249, 2023-07-21T22:39:13.775 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.477200084, 2023-07-21T22:39:13.775 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.392280391, 2023-07-21T22:39:13.775 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.231294529, 2023-07-21T22:39:13.775 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:13.775 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 16157, 2023-07-21T22:39:13.775 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1.482787514, 2023-07-21T22:39:13.775 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000091773 2023-07-21T22:39:13.775 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.776 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:13.776 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 15234, 2023-07-21T22:39:13.776 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.002144415, 2023-07-21T22:39:13.776 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000140 2023-07-21T22:39:13.776 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.776 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 689600, 2023-07-21T22:39:13.776 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 118240 2023-07-21T22:39:13.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.777 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2023-07-21T22:39:13.777 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2023-07-21T22:39:13.777 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2023-07-21T22:39:13.777 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.777 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2023-07-21T22:39:13.777 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-07-21T22:39:13.778 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-07-21T22:39:13.778 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 5790, 2023-07-21T22:39:13.778 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 49.404759558, 2023-07-21T22:39:13.778 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.008532773 2023-07-21T22:39:13.778 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:13.778 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.778 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2023-07-21T22:39:13.778 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-07-21T22:39:13.779 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-07-21T22:39:13.779 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 34273, 2023-07-21T22:39:13.779 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 6.428101283, 2023-07-21T22:39:13.779 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000187555 2023-07-21T22:39:13.779 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:13.779 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.779 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2023-07-21T22:39:13.779 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 73510, 2023-07-21T22:39:13.780 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 72601, 2023-07-21T22:39:13.780 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2023-07-21T22:39:13.780 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 72601, 2023-07-21T22:39:13.780 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 945.924899616, 2023-07-21T22:39:13.780 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.013029089 2023-07-21T22:39:13.780 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.780 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2023-07-21T22:39:13.781 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 909, 2023-07-21T22:39:13.781 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 5, 2023-07-21T22:39:13.781 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2023-07-21T22:39:13.781 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 489, 2023-07-21T22:39:13.781 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 53, 2023-07-21T22:39:13.781 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 1, 2023-07-21T22:39:13.781 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 280, 2023-07-21T22:39:13.782 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 184, 2023-07-21T22:39:13.782 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 80, 2023-07-21T22:39:13.782 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 16, 2023-07-21T22:39:13.782 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 16, 2023-07-21T22:39:13.782 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 20765, 2023-07-21T22:39:13.782 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2023-07-21T22:39:13.782 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2023-07-21T22:39:13.783 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 4, 2023-07-21T22:39:13.783 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 120060, 2023-07-21T22:39:13.783 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 107250, 2023-07-21T22:39:13.783 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 871, 2023-07-21T22:39:13.783 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 621, 2023-07-21T22:39:13.783 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:39:13.783 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2023-07-21T22:39:13.783 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 1023, 2023-07-21T22:39:13.784 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 2000, 2023-07-21T22:39:13.784 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 2, 2023-07-21T22:39:13.784 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 98, 2023-07-21T22:39:13.784 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 16696, 2023-07-21T22:39:13.784 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 143, 2023-07-21T22:39:13.784 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 27390, 2023-07-21T22:39:13.784 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2023-07-21T22:39:13.785 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 319, 2023-07-21T22:39:13.785 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2023-07-21T22:39:13.785 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 5452, 2023-07-21T22:39:13.785 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 102857566, 2023-07-21T22:39:13.785 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2023-07-21T22:39:13.785 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 319, 2023-07-21T22:39:13.785 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2023-07-21T22:39:13.785 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:39:13.786 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:39:13.786 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:39:13.786 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:39:13.786 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:39:13.786 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 319, 2023-07-21T22:39:13.786 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 309, 2023-07-21T22:39:13.786 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 14997, 2023-07-21T22:39:13.787 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 34162, 2023-07-21T22:39:13.787 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 1, 2023-07-21T22:39:13.787 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:39:13.787 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:39:13.787 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 29382, 2023-07-21T22:39:13.787 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 12320, 2023-07-21T22:39:13.787 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 422, 2023-07-21T22:39:13.787 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 36404 2023-07-21T22:39:13.788 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.788 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2023-07-21T22:39:13.788 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 3, 2023-07-21T22:39:13.788 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 9, 2023-07-21T22:39:13.788 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 6, 2023-07-21T22:39:13.788 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 723, 2023-07-21T22:39:13.788 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 843, 2023-07-21T22:39:13.788 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 3, 2023-07-21T22:39:13.789 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2023-07-21T22:39:13.789 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:39:13.789 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 18580, 2023-07-21T22:39:13.789 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 18577, 2023-07-21T22:39:13.789 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2023-07-21T22:39:13.789 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2023-07-21T22:39:13.790 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2023-07-21T22:39:13.790 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:39:13.790 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:39:13.790 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2023-07-21T22:39:13.790 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2023-07-21T22:39:13.790 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:39:13.790 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 101, 2023-07-21T22:39:13.790 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2023-07-21T22:39:13.791 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 55, 2023-07-21T22:39:13.791 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2023-07-21T22:39:13.791 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2023-07-21T22:39:13.791 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.791 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2023-07-21T22:39:13.791 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 97796, 2023-07-21T22:39:13.791 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2023-07-21T22:39:13.791 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2023-07-21T22:39:13.792 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 97796, 2023-07-21T22:39:13.792 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2023-07-21T22:39:13.792 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2023-07-21T22:39:13.792 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 120, 2023-07-21T22:39:13.792 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2023-07-21T22:39:13.792 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2023-07-21T22:39:13.792 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 120, 2023-07-21T22:39:13.793 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2023-07-21T22:39:13.793 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2023-07-21T22:39:13.793 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2023-07-21T22:39:13.793 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 217873147, 2023-07-21T22:39:13.793 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 0, 2023-07-21T22:39:13.793 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2023-07-21T22:39:13.793 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2736, 2023-07-21T22:39:13.793 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 119.161341789, 2023-07-21T22:39:13.794 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.043553121 2023-07-21T22:39:13.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.794 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 0 2023-07-21T22:39:13.794 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.794 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2023-07-21T22:39:13.794 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 32, 2023-07-21T22:39:13.794 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 28202, 2023-07-21T22:39:13.794 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 28170, 2023-07-21T22:39:13.795 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 42, 2023-07-21T22:39:13.795 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 1328, 2023-07-21T22:39:13.795 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 1286, 2023-07-21T22:39:13.795 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 280, 2023-07-21T22:39:13.795 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 66609, 2023-07-21T22:39:13.795 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 66329, 2023-07-21T22:39:13.795 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2023-07-21T22:39:13.796 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 28677, 2023-07-21T22:39:13.796 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 28677, 2023-07-21T22:39:13.796 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 106300, 2023-07-21T22:39:13.796 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 198964 2023-07-21T22:39:13.796 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.796 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2023-07-21T22:39:13.796 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 94962, 2023-07-21T22:39:13.797 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 3467, 2023-07-21T22:39:13.797 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 73510, 2023-07-21T22:39:13.797 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 285, 2023-07-21T22:39:13.797 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 5183, 2023-07-21T22:39:13.797 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2023-07-21T22:39:13.797 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 12106, 2023-07-21T22:39:13.797 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 24.217426108, 2023-07-21T22:39:13.798 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.002000448 2023-07-21T22:39:13.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.798 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2023-07-21T22:39:13.798 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4214, 2023-07-21T22:39:13.798 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 162.742780845, 2023-07-21T22:39:13.798 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.038619549 2023-07-21T22:39:13.798 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.799 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2023-07-21T22:39:13.799 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.799 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.799 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.799 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.799 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2023-07-21T22:39:13.799 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.799 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.800 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.800 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2023-07-21T22:39:13.800 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 24299, 2023-07-21T22:39:13.800 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 460.949844844, 2023-07-21T22:39:13.800 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.018969910 2023-07-21T22:39:13.800 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.801 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2023-07-21T22:39:13.801 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.801 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.801 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.801 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.801 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2023-07-21T22:39:13.801 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.801 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.802 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.802 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.802 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2023-07-21T22:39:13.802 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.802 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.802 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.802 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.802 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2023-07-21T22:39:13.803 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.803 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.803 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.803 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.803 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2023-07-21T22:39:13.803 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.804 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.804 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.804 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.804 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2023-07-21T22:39:13.805 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.805 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.805 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.805 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.806 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2023-07-21T22:39:13.806 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 54, 2023-07-21T22:39:13.806 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1.765746369, 2023-07-21T22:39:13.806 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.032699006 2023-07-21T22:39:13.806 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.807 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2023-07-21T22:39:13.807 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.807 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.807 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.807 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.808 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2023-07-21T22:39:13.808 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.808 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.808 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.809 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.809 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2023-07-21T22:39:13.809 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 3720, 2023-07-21T22:39:13.809 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 6.992979760, 2023-07-21T22:39:13.809 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001879833 2023-07-21T22:39:13.810 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.810 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2023-07-21T22:39:13.810 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 694, 2023-07-21T22:39:13.810 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 1.116838757, 2023-07-21T22:39:13.810 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001609277 2023-07-21T22:39:13.811 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.811 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2023-07-21T22:39:13.811 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 12095, 2023-07-21T22:39:13.811 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 252.624161484, 2023-07-21T22:39:13.811 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.020886660 2023-07-21T22:39:13.812 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.812 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2023-07-21T22:39:13.812 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.812 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.813 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.813 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.813 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2023-07-21T22:39:13.813 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.813 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.814 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.814 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.814 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2023-07-21T22:39:13.814 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 116, 2023-07-21T22:39:13.814 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 15.837429455, 2023-07-21T22:39:13.815 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.136529564 2023-07-21T22:39:13.815 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.815 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2023-07-21T22:39:13.815 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.816 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.816 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.816 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.817 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2023-07-21T22:39:13.817 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.818 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.818 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.819 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.819 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2023-07-21T22:39:13.819 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1, 2023-07-21T22:39:13.820 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.013913011, 2023-07-21T22:39:13.820 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.013913011 2023-07-21T22:39:13.821 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.821 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2023-07-21T22:39:13.822 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.822 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.822 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.823 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.823 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2023-07-21T22:39:13.824 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.824 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.825 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.825 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.825 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2023-07-21T22:39:13.826 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.826 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.826 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.827 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.827 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2023-07-21T22:39:13.828 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.828 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.828 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.828 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.829 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2023-07-21T22:39:13.829 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.829 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.830 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.830 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.830 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2023-07-21T22:39:13.830 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 15302, 2023-07-21T22:39:13.831 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 19.663778983, 2023-07-21T22:39:13.831 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.001285046 2023-07-21T22:39:13.831 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.831 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:39:13.832 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:39:13.832 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2023-07-21T22:39:13.832 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.832 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.833 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.833 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:13.833 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.833 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2023-07-21T22:39:13.834 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2023-07-21T22:39:13.834 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 2, 2023-07-21T22:39:13.834 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 1, 2023-07-21T22:39:13.835 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2023-07-21T22:39:13.835 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2023-07-21T22:39:13.835 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 14528, 2023-07-21T22:39:13.835 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 14528, 2023-07-21T22:39:13.836 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 2143 2023-07-21T22:39:13.836 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.836 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2023-07-21T22:39:13.836 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:39:13.837 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2023-07-21T22:39:13.837 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:39:13.837 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:39:13.837 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:39:13.838 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:39:13.838 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:39:13.838 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:39:13.838 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:39:13.839 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:39:13.839 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:39:13.839 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:39:13.840 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:39:13.840 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:39:13.840 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:39:13.840 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2023-07-21T22:39:13.841 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:39:13.841 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2023-07-21T22:39:13.841 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:39:13.841 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:39:13.842 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:39:13.842 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:39:13.842 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:39:13.842 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:39:13.843 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:39:13.843 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2023-07-21T22:39:13.843 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:39:13.844 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:39:13.844 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:39:13.844 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2023-07-21T22:39:13.844 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2023-07-21T22:39:13.845 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2023-07-21T22:39:13.845 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:39:13.845 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:39:13.845 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:39:13.846 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:39:13.846 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 292699, 2023-07-21T22:39:13.846 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 132, 2023-07-21T22:39:13.846 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 616, 2023-07-21T22:39:13.847 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 7, 2023-07-21T22:39:13.847 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2023-07-21T22:39:13.847 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2023-07-21T22:39:13.847 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:39:13.848 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2023-07-21T22:39:13.848 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:39:13.848 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2023-07-21T22:39:13.849 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7720, 2023-07-21T22:39:13.849 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 162, 2023-07-21T22:39:13.849 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:39:13.849 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:39:13.850 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2023-07-21T22:39:13.850 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2023-07-21T22:39:13.850 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 339255, 2023-07-21T22:39:13.850 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 3605, 2023-07-21T22:39:13.851 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2023-07-21T22:39:13.851 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2023-07-21T22:39:13.851 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2023-07-21T22:39:13.851 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2023-07-21T22:39:13.852 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.852 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2023-07-21T22:39:13.852 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2023-07-21T22:39:13.852 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2023-07-21T22:39:13.853 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 24329, 2023-07-21T22:39:13.853 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 225779511, 2023-07-21T22:39:13.853 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 28, 2023-07-21T22:39:13.853 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 24329, 2023-07-21T22:39:13.854 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2023-07-21T22:39:13.854 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 24329, 2023-07-21T22:39:13.854 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 290.921772877, 2023-07-21T22:39:13.855 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.011957818 2023-07-21T22:39:13.855 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.855 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2023-07-21T22:39:13.855 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2023-07-21T22:39:13.856 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 24329, 2023-07-21T22:39:13.856 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 25047 2023-07-21T22:39:13.856 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.856 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 24329, 2023-07-21T22:39:13.857 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 839, 2023-07-21T22:39:13.857 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 23490, 2023-07-21T22:39:13.857 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2023-07-21T22:39:13.857 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2023-07-21T22:39:13.858 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 0, 2023-07-21T22:39:13.858 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 10, 2023-07-21T22:39:13.858 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 510, 2023-07-21T22:39:13.858 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 3249, 2023-07-21T22:39:13.859 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 270, 2023-07-21T22:39:13.859 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2023-07-21T22:39:13.859 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2023-07-21T22:39:13.859 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2023-07-21T22:39:13.860 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 53, 2023-07-21T22:39:13.860 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 19327, 2023-07-21T22:39:13.860 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2023-07-21T22:39:13.861 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2023-07-21T22:39:13.861 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2023-07-21T22:39:13.862 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 319, 2023-07-21T22:39:13.862 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 20, 2023-07-21T22:39:13.863 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:39:13.863 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2023-07-21T22:39:13.863 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:39:13.864 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2023-07-21T22:39:13.864 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2023-07-21T22:39:13.865 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2023-07-21T22:39:13.865 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:39:13.866 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2023-07-21T22:39:13.866 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:39:13.866 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 1, 2023-07-21T22:39:13.867 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2023-07-21T22:39:13.867 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2023-07-21T22:39:13.868 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2023-07-21T22:39:13.868 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2023-07-21T22:39:13.869 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2023-07-21T22:39:13.869 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2023-07-21T22:39:13.869 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2023-07-21T22:39:13.870 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2023-07-21T22:39:13.870 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2023-07-21T22:39:13.871 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2023-07-21T22:39:13.871 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2023-07-21T22:39:13.871 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2023-07-21T22:39:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2023-07-21T22:39:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2023-07-21T22:39:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2023-07-21T22:39:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2023-07-21T22:39:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 85, 2023-07-21T22:39:13.874 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2023-07-21T22:39:13.874 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 3, 2023-07-21T22:39:13.874 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 12, 2023-07-21T22:39:13.875 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 12, 2023-07-21T22:39:13.875 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2023-07-21T22:39:13.875 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2023-07-21T22:39:13.875 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 1136, 2023-07-21T22:39:13.876 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 15, 2023-07-21T22:39:13.876 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 137 2023-07-21T22:39:13.876 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.876 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2023-07-21T22:39:13.877 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 1, 2023-07-21T22:39:13.877 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:39:13.877 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2023-07-21T22:39:13.877 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2023-07-21T22:39:13.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.878 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2023-07-21T22:39:13.878 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 74321 2023-07-21T22:39:13.878 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.879 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2023-07-21T22:39:13.879 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2023-07-21T22:39:13.879 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 14, 2023-07-21T22:39:13.880 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2023-07-21T22:39:13.880 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 6, 2023-07-21T22:39:13.880 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 18577, 2023-07-21T22:39:13.880 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2023-07-21T22:39:13.881 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.881 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:39:13.881 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:13.881 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-07-21T22:39:13.882 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:13.882 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 334508, 2023-07-21T22:39:13.882 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 73018353, 2023-07-21T22:39:13.882 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:13.883 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 334508, 2023-07-21T22:39:13.883 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-07-21T22:39:13.883 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-07-21T22:39:13.883 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 334508, 2023-07-21T22:39:13.884 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 73018353, 2023-07-21T22:39:13.884 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:13.884 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.885 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.885 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.885 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:13.885 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.886 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2023-07-21T22:39:13.886 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:13.886 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-07-21T22:39:13.886 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:13.887 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-07-21T22:39:13.887 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-07-21T22:39:13.887 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:13.887 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:13.888 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 24329, 2023-07-21T22:39:13.888 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 227656214, 2023-07-21T22:39:13.888 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 24329, 2023-07-21T22:39:13.889 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 227656214, 2023-07-21T22:39:13.889 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:13.889 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.889 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.890 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.890 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:13.890 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.890 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2023-07-21T22:39:13.891 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:13.891 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2023-07-21T22:39:13.891 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:13.891 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-07-21T22:39:13.892 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-07-21T22:39:13.892 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:13.892 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:13.892 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 24329, 2023-07-21T22:39:13.893 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 24329, 2023-07-21T22:39:13.893 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 24329, 2023-07-21T22:39:13.893 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 24329, 2023-07-21T22:39:13.893 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:13.894 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.894 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.894 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.894 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:13.894 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.894 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:39:13.895 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:13.895 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-07-21T22:39:13.895 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:13.895 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 18577, 2023-07-21T22:39:13.895 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 1876957, 2023-07-21T22:39:13.895 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:13.895 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 18577, 2023-07-21T22:39:13.895 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-07-21T22:39:13.896 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-07-21T22:39:13.896 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 513, 2023-07-21T22:39:13.896 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 1876957, 2023-07-21T22:39:13.896 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:13.896 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.896 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.896 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.897 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:13.897 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:13.897 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x561c156780a0": { 2023-07-21T22:39:13.897 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:13.897 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-07-21T22:39:13.897 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:13.897 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 97796, 2023-07-21T22:39:13.897 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 213678843, 2023-07-21T22:39:13.898 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:13.898 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 97796, 2023-07-21T22:39:13.898 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-07-21T22:39:13.898 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-07-21T22:39:13.898 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2736, 2023-07-21T22:39:13.898 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 213678843, 2023-07-21T22:39:13.898 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:13.898 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:13.899 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:13.899 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:13.899 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:13.899 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:13.899 INFO:teuthology.orchestra.run.smithi190.stdout:} 2023-07-21T22:39:13.900 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:13 smithi190 ceph-mon[98334]: pgmap v598: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2023-07-21T22:39:14.177 INFO:tasks.check_counter:Daemon mds.i mds.exported=98 2023-07-21T22:39:14.178 INFO:tasks.check_counter:Daemon mds.i mds.imported=143 2023-07-21T22:39:14.178 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=53 2023-07-21T22:39:14.178 DEBUG:tasks.check_counter:Getting stats from l 2023-07-21T22:39:14.178 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid af99f5d0-2814-11ee-9b34-001a4aab830c -- ceph daemon mds.l perf dump 2023-07-21T22:39:14.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:13 smithi132 ceph-mon[97863]: pgmap v598: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2023-07-21T22:39:14.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:13 smithi107 ceph-mon[96543]: pgmap v598: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2023-07-21T22:39:14.770 INFO:teuthology.orchestra.run.smithi190.stderr:Inferring config /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:39:15.006 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:14 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:39:15.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:14 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:39:15.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:14 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:39:15.991 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2023-07-21T22:39:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:39:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 839, 2023-07-21T22:39:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 839, 2023-07-21T22:39:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 71529202, 2023-07-21T22:39:15.991 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 260197, 2023-07-21T22:39:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 8, 2023-07-21T22:39:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2023-07-21T22:39:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.099269908, 2023-07-21T22:39:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.015840269, 2023-07-21T22:39:15.992 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.054913585, 2023-07-21T22:39:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.022073165, 2023-07-21T22:39:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 838, 2023-07-21T22:39:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.054232028, 2023-07-21T22:39:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000064716 2023-07-21T22:39:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:15.993 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-21T22:39:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T22:39:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:39:15.994 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 660, 2023-07-21T22:39:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 807, 2023-07-21T22:39:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 84057009, 2023-07-21T22:39:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 527994, 2023-07-21T22:39:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 6, 2023-07-21T22:39:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 5, 2023-07-21T22:39:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.089507617, 2023-07-21T22:39:15.995 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.019080482, 2023-07-21T22:39:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.046536598, 2023-07-21T22:39:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.018330454, 2023-07-21T22:39:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 805, 2023-07-21T22:39:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.035369406, 2023-07-21T22:39:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000043937 2023-07-21T22:39:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:15.996 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-21T22:39:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 323040 2023-07-21T22:39:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:15.997 INFO:teuthology.orchestra.run.smithi190.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:39:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_messages": 1185, 2023-07-21T22:39:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages": 1153, 2023-07-21T22:39:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_bytes": 49497117, 2023-07-21T22:39:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_bytes": 407955, 2023-07-21T22:39:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_created_connections": 5, 2023-07-21T22:39:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_active_connections": 4, 2023-07-21T22:39:15.998 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_total_time": 0.136767109, 2023-07-21T22:39:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_send_time": 0.033096454, 2023-07-21T22:39:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_recv_time": 0.061460412, 2023-07-21T22:39:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_running_fast_dispatch_time": 0.035941964, 2023-07-21T22:39:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:39:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 1152, 2023-07-21T22:39:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.049497067, 2023-07-21T22:39:15.999 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000042966 2023-07-21T22:39:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:39:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_recv_encrypted_bytes": 659136, 2023-07-21T22:39:16.000 INFO:teuthology.orchestra.run.smithi190.stdout: "msgr_send_encrypted_bytes": 112816 2023-07-21T22:39:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: "cct": { 2023-07-21T22:39:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: "total_workers": 1, 2023-07-21T22:39:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: "unhealthy_workers": 0 2023-07-21T22:39:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-MDSRank": { 2023-07-21T22:39:16.001 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-07-21T22:39:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-07-21T22:39:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 4041, 2023-07-21T22:39:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.115648411, 2023-07-21T22:39:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000028618 2023-07-21T22:39:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "finisher-PurgeQueue": { 2023-07-21T22:39:16.002 INFO:teuthology.orchestra.run.smithi190.stdout: "queue_len": 0, 2023-07-21T22:39:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: "complete_latency": { 2023-07-21T22:39:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: "mds": { 2023-07-21T22:39:16.003 INFO:teuthology.orchestra.run.smithi190.stdout: "request": 0, 2023-07-21T22:39:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: "reply": 0, 2023-07-21T22:39:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: "reply_latency": { 2023-07-21T22:39:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.004 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_reply": 0, 2023-07-21T22:39:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "forward": 0, 2023-07-21T22:39:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_complete": 0, 2023-07-21T22:39:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_fetch_keys": 0, 2023-07-21T22:39:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_commit": 0, 2023-07-21T22:39:16.005 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_split": 0, 2023-07-21T22:39:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_merge": 0, 2023-07-21T22:39:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes": 17623, 2023-07-21T22:39:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_top": 0, 2023-07-21T22:39:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_bottom": 0, 2023-07-21T22:39:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pin_tail": 17623, 2023-07-21T22:39:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_pinned": 17623, 2023-07-21T22:39:16.006 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_expired": 1065, 2023-07-21T22:39:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "inodes_with_caps": 0, 2023-07-21T22:39:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "caps": 0, 2023-07-21T22:39:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "subtrees": 1, 2023-07-21T22:39:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse": 0, 2023-07-21T22:39:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_hit": 0, 2023-07-21T22:39:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_forward": 0, 2023-07-21T22:39:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_discover": 0, 2023-07-21T22:39:16.007 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:39:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_remote_ino": 0, 2023-07-21T22:39:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "traverse_lock": 0, 2023-07-21T22:39:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "load_cent": 0, 2023-07-21T22:39:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "q": 0, 2023-07-21T22:39:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "exported": 0, 2023-07-21T22:39:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "exported_inodes": 0, 2023-07-21T22:39:16.008 INFO:teuthology.orchestra.run.smithi190.stdout: "imported": 0, 2023-07-21T22:39:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "imported_inodes": 0, 2023-07-21T22:39:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_dir_fetch": 0, 2023-07-21T22:39:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:39:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "openino_peer_discover": 0, 2023-07-21T22:39:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rfiles": 0, 2023-07-21T22:39:16.009 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rbytes": 0, 2023-07-21T22:39:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "root_rsnaps": 0, 2023-07-21T22:39:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:39:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_set_tag": 0, 2023-07-21T22:39:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:39:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:39:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:39:16.010 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:39:16.011 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:39:16.011 INFO:teuthology.orchestra.run.smithi190.stdout: "scrub_file_inodes": 0, 2023-07-21T22:39:16.011 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_inode_file_caps": 0, 2023-07-21T22:39:16.011 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T22:39:16.011 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_grant": 0, 2023-07-21T22:39:16.011 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:39:16.011 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:39:16.011 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:39:16.012 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps": 0, 2023-07-21T22:39:16.012 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_caps_dirty": 0, 2023-07-21T22:39:16.012 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_cap_release": 0, 2023-07-21T22:39:16.012 INFO:teuthology.orchestra.run.smithi190.stdout: "process_request_cap_release": 0 2023-07-21T22:39:16.012 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.012 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_cache": { 2023-07-21T22:39:16.012 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update": 0, 2023-07-21T22:39:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_update_receipt": 0, 2023-07-21T22:39:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_try_discover": 0, 2023-07-21T22:39:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_send_discover": 0, 2023-07-21T22:39:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: "dir_handle_discover": 0, 2023-07-21T22:39:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays": 0, 2023-07-21T22:39:16.013 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_delayed": 0, 2023-07-21T22:39:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:39:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_created": 0, 2023-07-21T22:39:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_enqueued": 0, 2023-07-21T22:39:16.014 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_reintegrated": 0, 2023-07-21T22:39:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "strays_migrated": 0, 2023-07-21T22:39:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_processing": 0, 2023-07-21T22:39:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:39:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:39:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_started": 0, 2023-07-21T22:39:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "recovery_completed": 0, 2023-07-21T22:39:16.015 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:39:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_exportdir": 0, 2023-07-21T22:39:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_flush": 0, 2023-07-21T22:39:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragmentdir": 0, 2023-07-21T22:39:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_fragstats": 0, 2023-07-21T22:39:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: "ireq_inodestats": 0 2023-07-21T22:39:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_log": { 2023-07-21T22:39:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "evadd": 0, 2023-07-21T22:39:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "evex": 0, 2023-07-21T22:39:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "evtrm": 0, 2023-07-21T22:39:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "ev": 0, 2023-07-21T22:39:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "evexg": 0, 2023-07-21T22:39:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "evexd": 0, 2023-07-21T22:39:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "segadd": 0, 2023-07-21T22:39:16.017 INFO:teuthology.orchestra.run.smithi190.stdout: "segex": 0, 2023-07-21T22:39:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "segtrm": 0, 2023-07-21T22:39:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "seg": 116, 2023-07-21T22:39:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "segexg": 0, 2023-07-21T22:39:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "segexd": 0, 2023-07-21T22:39:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "expos": 4194304, 2023-07-21T22:39:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "wrpos": 0, 2023-07-21T22:39:16.018 INFO:teuthology.orchestra.run.smithi190.stdout: "rdpos": 207917382, 2023-07-21T22:39:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "jlat": { 2023-07-21T22:39:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "replayed": 93835 2023-07-21T22:39:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.019 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_mem": { 2023-07-21T22:39:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "ino": 14, 2023-07-21T22:39:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "ino+": 21423, 2023-07-21T22:39:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "ino-": 21409, 2023-07-21T22:39:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "dir": 12, 2023-07-21T22:39:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "dir+": 927, 2023-07-21T22:39:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "dir-": 915, 2023-07-21T22:39:16.020 INFO:teuthology.orchestra.run.smithi190.stdout: "dn": 17623, 2023-07-21T22:39:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "dn+": 54990, 2023-07-21T22:39:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "dn-": 37367, 2023-07-21T22:39:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "cap": 0, 2023-07-21T22:39:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "cap+": 0, 2023-07-21T22:39:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "cap-": 0, 2023-07-21T22:39:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "rss": 81544, 2023-07-21T22:39:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: "heap": 207156 2023-07-21T22:39:16.021 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_server": { 2023-07-21T22:39:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_client_request": 0, 2023-07-21T22:39:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "dispatch_server_request": 0, 2023-07-21T22:39:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_request": 0, 2023-07-21T22:39:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_client_session": 0, 2023-07-21T22:39:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "handle_peer_request": 0, 2023-07-21T22:39:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "req_create_latency": { 2023-07-21T22:39:16.022 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getattr_latency": { 2023-07-21T22:39:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.023 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getfilelock_latency": { 2023-07-21T22:39:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: "req_link_latency": { 2023-07-21T22:39:16.024 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookup_latency": { 2023-07-21T22:39:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.025 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookuphash_latency": { 2023-07-21T22:39:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.026 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupino_latency": { 2023-07-21T22:39:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.027 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupname_latency": { 2023-07-21T22:39:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.028 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupparent_latency": { 2023-07-21T22:39:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.029 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.030 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.030 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.030 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lookupsnap_latency": { 2023-07-21T22:39:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.031 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: "req_lssnap_latency": { 2023-07-21T22:39:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.032 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mkdir_latency": { 2023-07-21T22:39:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.033 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mknod_latency": { 2023-07-21T22:39:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.034 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "req_mksnap_latency": { 2023-07-21T22:39:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "req_open_latency": { 2023-07-21T22:39:16.035 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: "req_readdir_latency": { 2023-07-21T22:39:16.036 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rename_latency": { 2023-07-21T22:39:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.037 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.038 INFO:teuthology.orchestra.run.smithi190.stdout: "req_renamesnap_latency": { 2023-07-21T22:39:16.038 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.038 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.038 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.038 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.038 INFO:teuthology.orchestra.run.smithi190.stdout: "req_snapdiff_latency": { 2023-07-21T22:39:16.038 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.039 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.039 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.039 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmdir_latency": { 2023-07-21T22:39:16.039 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.039 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.039 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.039 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.040 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmsnap_latency": { 2023-07-21T22:39:16.040 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.040 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.040 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.040 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.040 INFO:teuthology.orchestra.run.smithi190.stdout: "req_rmxattr_latency": { 2023-07-21T22:39:16.040 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.041 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.041 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.041 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.041 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setattr_latency": { 2023-07-21T22:39:16.041 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.041 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.041 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.042 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setdirlayout_latency": { 2023-07-21T22:39:16.042 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.042 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.042 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.042 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.042 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setfilelock_latency": { 2023-07-21T22:39:16.042 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.043 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.043 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.043 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setlayout_latency": { 2023-07-21T22:39:16.043 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.043 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.043 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.043 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.044 INFO:teuthology.orchestra.run.smithi190.stdout: "req_setxattr_latency": { 2023-07-21T22:39:16.044 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.044 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.044 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.044 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.044 INFO:teuthology.orchestra.run.smithi190.stdout: "req_symlink_latency": { 2023-07-21T22:39:16.045 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.045 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.045 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.045 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.045 INFO:teuthology.orchestra.run.smithi190.stdout: "req_unlink_latency": { 2023-07-21T22:39:16.045 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.045 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.045 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.046 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.046 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:39:16.046 INFO:teuthology.orchestra.run.smithi190.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:39:16.046 INFO:teuthology.orchestra.run.smithi190.stdout: "req_getvxattr_latency": { 2023-07-21T22:39:16.046 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.046 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.046 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.046 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:16.047 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.047 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_sessions": { 2023-07-21T22:39:16.047 INFO:teuthology.orchestra.run.smithi190.stdout: "session_count": 1, 2023-07-21T22:39:16.047 INFO:teuthology.orchestra.run.smithi190.stdout: "session_add": 1, 2023-07-21T22:39:16.047 INFO:teuthology.orchestra.run.smithi190.stdout: "session_remove": 0, 2023-07-21T22:39:16.047 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_open": 1, 2023-07-21T22:39:16.047 INFO:teuthology.orchestra.run.smithi190.stdout: "sessions_stale": 0, 2023-07-21T22:39:16.048 INFO:teuthology.orchestra.run.smithi190.stdout: "total_load": 0, 2023-07-21T22:39:16.048 INFO:teuthology.orchestra.run.smithi190.stdout: "average_load": 0, 2023-07-21T22:39:16.048 INFO:teuthology.orchestra.run.smithi190.stdout: "avg_session_uptime": 2146 2023-07-21T22:39:16.048 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.048 INFO:teuthology.orchestra.run.smithi190.stdout: "mempool": { 2023-07-21T22:39:16.048 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:39:16.048 INFO:teuthology.orchestra.run.smithi190.stdout: "bloom_filter_items": 0, 2023-07-21T22:39:16.049 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:39:16.049 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:39:16.049 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:39:16.049 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:39:16.049 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:39:16.049 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:39:16.049 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:39:16.049 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:39:16.050 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:39:16.050 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:39:16.050 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:39:16.050 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:39:16.051 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:39:16.051 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_extent_items": 0, 2023-07-21T22:39:16.051 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:39:16.051 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_blob_items": 0, 2023-07-21T22:39:16.051 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:39:16.051 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:39:16.051 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:39:16.051 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:39:16.052 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:39:16.052 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:39:16.052 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:39:16.052 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_txc_items": 0, 2023-07-21T22:39:16.052 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:39:16.052 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:39:16.053 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:39:16.053 INFO:teuthology.orchestra.run.smithi190.stdout: "bluestore_writing_items": 0, 2023-07-21T22:39:16.053 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_bytes": 0, 2023-07-21T22:39:16.053 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_items": 0, 2023-07-21T22:39:16.053 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:39:16.053 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:39:16.053 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:39:16.053 INFO:teuthology.orchestra.run.smithi190.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:39:16.054 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_bytes": 231385, 2023-07-21T22:39:16.054 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_anon_items": 75, 2023-07-21T22:39:16.054 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:39:16.054 INFO:teuthology.orchestra.run.smithi190.stdout: "buffer_meta_items": 0, 2023-07-21T22:39:16.054 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_bytes": 0, 2023-07-21T22:39:16.054 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_items": 0, 2023-07-21T22:39:16.054 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:39:16.055 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_mapbl_items": 0, 2023-07-21T22:39:16.055 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:39:16.055 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_pglog_items": 0, 2023-07-21T22:39:16.055 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_bytes": 7720, 2023-07-21T22:39:16.055 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_items": 162, 2023-07-21T22:39:16.055 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:39:16.055 INFO:teuthology.orchestra.run.smithi190.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:39:16.055 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_bytes": 0, 2023-07-21T22:39:16.056 INFO:teuthology.orchestra.run.smithi190.stdout: "pgmap_items": 0, 2023-07-21T22:39:16.056 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_bytes": 12456935, 2023-07-21T22:39:16.056 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_co_items": 74591, 2023-07-21T22:39:16.056 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_bytes": 0, 2023-07-21T22:39:16.056 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_1_items": 0, 2023-07-21T22:39:16.056 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_bytes": 0, 2023-07-21T22:39:16.056 INFO:teuthology.orchestra.run.smithi190.stdout: "unittest_2_items": 0 2023-07-21T22:39:16.056 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.057 INFO:teuthology.orchestra.run.smithi190.stdout: "objecter": { 2023-07-21T22:39:16.057 INFO:teuthology.orchestra.run.smithi190.stdout: "op_active": 0, 2023-07-21T22:39:16.057 INFO:teuthology.orchestra.run.smithi190.stdout: "op_laggy": 0, 2023-07-21T22:39:16.057 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send": 2443, 2023-07-21T22:39:16.057 INFO:teuthology.orchestra.run.smithi190.stdout: "op_send_bytes": 21, 2023-07-21T22:39:16.057 INFO:teuthology.orchestra.run.smithi190.stdout: "op_resend": 3, 2023-07-21T22:39:16.057 INFO:teuthology.orchestra.run.smithi190.stdout: "op_reply": 2443, 2023-07-21T22:39:16.058 INFO:teuthology.orchestra.run.smithi190.stdout: "op_latency": { 2023-07-21T22:39:16.058 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2443, 2023-07-21T22:39:16.058 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 13.921605425, 2023-07-21T22:39:16.058 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.005698569 2023-07-21T22:39:16.058 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.058 INFO:teuthology.orchestra.run.smithi190.stdout: "op_inflight": 0, 2023-07-21T22:39:16.058 INFO:teuthology.orchestra.run.smithi190.stdout: "oplen_avg": { 2023-07-21T22:39:16.058 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 2443, 2023-07-21T22:39:16.059 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 2444 2023-07-21T22:39:16.059 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.059 INFO:teuthology.orchestra.run.smithi190.stdout: "op": 2443, 2023-07-21T22:39:16.059 INFO:teuthology.orchestra.run.smithi190.stdout: "op_r": 2443, 2023-07-21T22:39:16.059 INFO:teuthology.orchestra.run.smithi190.stdout: "op_w": 0, 2023-07-21T22:39:16.060 INFO:teuthology.orchestra.run.smithi190.stdout: "op_rmw": 0, 2023-07-21T22:39:16.060 INFO:teuthology.orchestra.run.smithi190.stdout: "op_pg": 0, 2023-07-21T22:39:16.060 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_stat": 1407, 2023-07-21T22:39:16.061 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_create": 0, 2023-07-21T22:39:16.061 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_read": 1034, 2023-07-21T22:39:16.061 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_write": 0, 2023-07-21T22:39:16.061 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writefull": 0, 2023-07-21T22:39:16.062 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_writesame": 0, 2023-07-21T22:39:16.062 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_append": 0, 2023-07-21T22:39:16.062 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_zero": 0, 2023-07-21T22:39:16.062 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_truncate": 0, 2023-07-21T22:39:16.063 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_delete": 0, 2023-07-21T22:39:16.063 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_mapext": 0, 2023-07-21T22:39:16.063 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_sparse_read": 0, 2023-07-21T22:39:16.063 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_clonerange": 0, 2023-07-21T22:39:16.064 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_getxattr": 1, 2023-07-21T22:39:16.064 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_setxattr": 0, 2023-07-21T22:39:16.064 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:39:16.064 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_rmxattr": 0, 2023-07-21T22:39:16.065 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:39:16.065 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_call": 0, 2023-07-21T22:39:16.065 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_watch": 0, 2023-07-21T22:39:16.066 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_notify": 0, 2023-07-21T22:39:16.066 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:39:16.066 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls": 0, 2023-07-21T22:39:16.066 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:39:16.067 INFO:teuthology.orchestra.run.smithi190.stdout: "osdop_other": 0, 2023-07-21T22:39:16.067 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_active": 0, 2023-07-21T22:39:16.067 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_send": 0, 2023-07-21T22:39:16.067 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_resend": 0, 2023-07-21T22:39:16.068 INFO:teuthology.orchestra.run.smithi190.stdout: "linger_ping": 0, 2023-07-21T22:39:16.068 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_active": 0, 2023-07-21T22:39:16.068 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_send": 0, 2023-07-21T22:39:16.068 INFO:teuthology.orchestra.run.smithi190.stdout: "poolop_resend": 0, 2023-07-21T22:39:16.069 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_active": 0, 2023-07-21T22:39:16.069 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_send": 0, 2023-07-21T22:39:16.069 INFO:teuthology.orchestra.run.smithi190.stdout: "poolstat_resend": 0, 2023-07-21T22:39:16.069 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_active": 0, 2023-07-21T22:39:16.069 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_send": 0, 2023-07-21T22:39:16.070 INFO:teuthology.orchestra.run.smithi190.stdout: "statfs_resend": 0, 2023-07-21T22:39:16.070 INFO:teuthology.orchestra.run.smithi190.stdout: "command_active": 0, 2023-07-21T22:39:16.070 INFO:teuthology.orchestra.run.smithi190.stdout: "command_send": 0, 2023-07-21T22:39:16.071 INFO:teuthology.orchestra.run.smithi190.stdout: "command_resend": 0, 2023-07-21T22:39:16.071 INFO:teuthology.orchestra.run.smithi190.stdout: "map_epoch": 85, 2023-07-21T22:39:16.071 INFO:teuthology.orchestra.run.smithi190.stdout: "map_full": 0, 2023-07-21T22:39:16.071 INFO:teuthology.orchestra.run.smithi190.stdout: "map_inc": 3, 2023-07-21T22:39:16.072 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_sessions": 10, 2023-07-21T22:39:16.072 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_open": 10, 2023-07-21T22:39:16.072 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_session_close": 0, 2023-07-21T22:39:16.072 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_laggy": 0, 2023-07-21T22:39:16.073 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_wr": 0, 2023-07-21T22:39:16.073 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_rd": 2, 2023-07-21T22:39:16.073 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_del": 0 2023-07-21T22:39:16.073 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.074 INFO:teuthology.orchestra.run.smithi190.stdout: "oft": { 2023-07-21T22:39:16.074 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_objs": 0, 2023-07-21T22:39:16.074 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:39:16.074 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_updates": 0, 2023-07-21T22:39:16.075 INFO:teuthology.orchestra.run.smithi190.stdout: "omap_total_removes": 0 2023-07-21T22:39:16.075 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.075 INFO:teuthology.orchestra.run.smithi190.stdout: "osd-slow-ops": { 2023-07-21T22:39:16.075 INFO:teuthology.orchestra.run.smithi190.stdout: "slow_ops_count": 0 2023-07-21T22:39:16.076 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.076 INFO:teuthology.orchestra.run.smithi190.stdout: "purge_queue": { 2023-07-21T22:39:16.076 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops": 0, 2023-07-21T22:39:16.077 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T22:39:16.077 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing": 0, 2023-07-21T22:39:16.077 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executing_high_water": 0, 2023-07-21T22:39:16.077 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_executed": 0, 2023-07-21T22:39:16.078 INFO:teuthology.orchestra.run.smithi190.stdout: "pq_item_in_journal": 0 2023-07-21T22:39:16.078 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.078 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:39:16.078 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:16.079 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-07-21T22:39:16.079 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:16.079 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 2680, 2023-07-21T22:39:16.079 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 204827159, 2023-07-21T22:39:16.080 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:16.080 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 2680, 2023-07-21T22:39:16.080 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-07-21T22:39:16.080 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-07-21T22:39:16.081 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2680, 2023-07-21T22:39:16.081 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 204827159, 2023-07-21T22:39:16.081 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:16.082 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.082 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.082 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.082 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:16.083 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.083 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_bytes": { 2023-07-21T22:39:16.083 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:16.083 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 104857600, 2023-07-21T22:39:16.084 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:16.084 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-07-21T22:39:16.084 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-07-21T22:39:16.084 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:16.085 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:16.085 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 2443, 2023-07-21T22:39:16.085 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 203743362, 2023-07-21T22:39:16.085 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2443, 2023-07-21T22:39:16.086 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 203743362, 2023-07-21T22:39:16.086 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:16.086 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.087 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.087 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.087 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:16.087 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.088 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-objecter_ops": { 2023-07-21T22:39:16.088 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:16.088 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 1024, 2023-07-21T22:39:16.089 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:16.089 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-07-21T22:39:16.089 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-07-21T22:39:16.089 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:16.090 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:16.090 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 2443, 2023-07-21T22:39:16.090 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 2443, 2023-07-21T22:39:16.090 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 2443, 2023-07-21T22:39:16.090 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 2443, 2023-07-21T22:39:16.091 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:16.091 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.091 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.091 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.091 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:16.091 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.091 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:39:16.092 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:16.092 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-07-21T22:39:16.092 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:16.092 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-07-21T22:39:16.092 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-07-21T22:39:16.092 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:16.092 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:16.093 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-07-21T22:39:16.093 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-07-21T22:39:16.093 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2023-07-21T22:39:16.093 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2023-07-21T22:39:16.093 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:16.093 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.093 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.094 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.094 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:16.094 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-07-21T22:39:16.094 INFO:teuthology.orchestra.run.smithi190.stdout: "throttle-write_buf_throttle-0x55d9a3c36460": { 2023-07-21T22:39:16.094 INFO:teuthology.orchestra.run.smithi190.stdout: "val": 0, 2023-07-21T22:39:16.094 INFO:teuthology.orchestra.run.smithi190.stdout: "max": 3758096384, 2023-07-21T22:39:16.094 INFO:teuthology.orchestra.run.smithi190.stdout: "get_started": 0, 2023-07-21T22:39:16.094 INFO:teuthology.orchestra.run.smithi190.stdout: "get": 0, 2023-07-21T22:39:16.095 INFO:teuthology.orchestra.run.smithi190.stdout: "get_sum": 0, 2023-07-21T22:39:16.095 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_fail": 0, 2023-07-21T22:39:16.095 INFO:teuthology.orchestra.run.smithi190.stdout: "get_or_fail_success": 0, 2023-07-21T22:39:16.095 INFO:teuthology.orchestra.run.smithi190.stdout: "take": 0, 2023-07-21T22:39:16.095 INFO:teuthology.orchestra.run.smithi190.stdout: "take_sum": 0, 2023-07-21T22:39:16.095 INFO:teuthology.orchestra.run.smithi190.stdout: "put": 0, 2023-07-21T22:39:16.095 INFO:teuthology.orchestra.run.smithi190.stdout: "put_sum": 0, 2023-07-21T22:39:16.095 INFO:teuthology.orchestra.run.smithi190.stdout: "wait": { 2023-07-21T22:39:16.096 INFO:teuthology.orchestra.run.smithi190.stdout: "avgcount": 0, 2023-07-21T22:39:16.096 INFO:teuthology.orchestra.run.smithi190.stdout: "sum": 0.000000000, 2023-07-21T22:39:16.096 INFO:teuthology.orchestra.run.smithi190.stdout: "avgtime": 0.000000000 2023-07-21T22:39:16.096 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:16.096 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-07-21T22:39:16.096 INFO:teuthology.orchestra.run.smithi190.stdout:} 2023-07-21T22:39:16.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:15 smithi190 ceph-mon[98334]: pgmap v599: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T22:39:16.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:39:16.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:39:16.097 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:15 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:39:16.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:15 smithi132 ceph-mon[97863]: pgmap v599: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T22:39:16.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:39:16.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:39:16.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:15 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:39:16.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:15 smithi107 ceph-mon[96543]: pgmap v599: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-21T22:39:16.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:39:16.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:39:16.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:15 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' 2023-07-21T22:39:16.413 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-07-21T22:39:16.414 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-07-21T22:39:16.414 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2023-07-21T22:39:16.414 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-21T22:39:16.424 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-21T22:39:16.424 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:39:16.425 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:39:16.451 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:39:16.452 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:39:16.514 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:39:16.691 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 -- 172.21.15.107:0/1987236247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59340ffbf0 msgr2=0x7f5934100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:16.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 --2- 172.21.15.107:0/1987236247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59340ffbf0 0x7f5934100040 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5930009990 tx=0x7f5930009ca0 comp rx=0 tx=0).stop 2023-07-21T22:39:16.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 -- 172.21.15.107:0/1987236247 shutdown_connections 2023-07-21T22:39:16.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 --2- 172.21.15.107:0/1987236247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5934100580 0x7f5934106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:16.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 --2- 172.21.15.107:0/1987236247 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59340fea00 0x7f59340fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:16.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 --2- 172.21.15.107:0/1987236247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59340ffbf0 0x7f5934100040 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:16.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 -- 172.21.15.107:0/1987236247 >> 172.21.15.107:0/1987236247 conn(0x7f59340f9fe0 msgr2=0x7f59340fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:16.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 -- 172.21.15.107:0/1987236247 shutdown_connections 2023-07-21T22:39:16.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 -- 172.21.15.107:0/1987236247 wait complete. 2023-07-21T22:39:16.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 Processor -- start 2023-07-21T22:39:16.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 -- start start 2023-07-21T22:39:16.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59340fea00 0x7f5934102990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59340ffbf0 0x7f5934100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5934100580 0x7f5934101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5934101cb0 con 0x7f59340ffbf0 2023-07-21T22:39:16.695 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5934101e20 con 0x7f59340fea00 2023-07-21T22:39:16.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.645+0000 7f593bc08700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593419f830 con 0x7f5934100580 2023-07-21T22:39:16.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f59391a3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59340ffbf0 0x7f5934100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:16.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f59391a3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59340ffbf0 0x7f5934100fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51776/0 (socket says 172.21.15.107:51776) 2023-07-21T22:39:16.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f59391a3700 1 -- 172.21.15.107:0/865106020 learned_addr learned my addr 172.21.15.107:0/865106020 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:39:16.696 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f593a1a5700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5934100580 0x7f5934101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:16.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f59399a4700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59340fea00 0x7f5934102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:16.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f59391a3700 1 -- 172.21.15.107:0/865106020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5934100580 msgr2=0x7f5934101510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:16.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f59391a3700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5934100580 0x7f5934101510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:16.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f59391a3700 1 -- 172.21.15.107:0/865106020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59340fea00 msgr2=0x7f5934102990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:16.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f59391a3700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59340fea00 0x7f5934102990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:16.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f59391a3700 1 -- 172.21.15.107:0/865106020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59300095f0 con 0x7f59340ffbf0 2023-07-21T22:39:16.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f59399a4700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59340fea00 0x7f5934102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:39:16.697 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f59391a3700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59340ffbf0 0x7f5934100fd0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5930004760 tx=0x7f5930004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:16.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f592affd700 1 -- 172.21.15.107:0/865106020 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593001d070 con 0x7f59340ffbf0 2023-07-21T22:39:16.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f592affd700 1 -- 172.21.15.107:0/865106020 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5930004cf0 con 0x7f59340ffbf0 2023-07-21T22:39:16.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f593bc08700 1 -- 172.21.15.107:0/865106020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f593419fab0 con 0x7f59340ffbf0 2023-07-21T22:39:16.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f592affd700 1 -- 172.21.15.107:0/865106020 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930005280 con 0x7f59340ffbf0 2023-07-21T22:39:16.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.646+0000 7f593bc08700 1 -- 172.21.15.107:0/865106020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593419ff00 con 0x7f59340ffbf0 2023-07-21T22:39:16.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.647+0000 7f593bc08700 1 -- 172.21.15.107:0/865106020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593410a1d0 con 0x7f59340ffbf0 2023-07-21T22:39:16.698 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.648+0000 7f592affd700 1 -- 172.21.15.107:0/865106020 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f59300053e0 con 0x7f59340ffbf0 2023-07-21T22:39:16.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.649+0000 7f592affd700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f59200a11a0 0x7f59200a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:16.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.649+0000 7f592affd700 1 -- 172.21.15.107:0/865106020 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f59300cde90 con 0x7f59340ffbf0 2023-07-21T22:39:16.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.649+0000 7f59399a4700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f59200a11a0 0x7f59200a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:16.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.649+0000 7f59399a4700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f59200a11a0 0x7f59200a3650 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5924005950 tx=0x7f59240058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:16.700 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.652+0000 7f592affd700 1 -- 172.21.15.107:0/865106020 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5930097650 con 0x7f59340ffbf0 2023-07-21T22:39:16.887 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.844+0000 7f593bc08700 1 -- 172.21.15.107:0/865106020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59341a0570 con 0x7f59340ffbf0 2023-07-21T22:39:16.888 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.846+0000 7f592affd700 1 -- 172.21.15.107:0/865106020 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22727 (secure 0 0 0) 0x7f5930004e60 con 0x7f59340ffbf0 2023-07-21T22:39:16.889 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:39:16.889 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":85,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:31:05.959573+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:39:16.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.848+0000 7f593bc08700 1 -- 172.21.15.107:0/865106020 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f59200a11a0 msgr2=0x7f59200a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:16.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.848+0000 7f593bc08700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f59200a11a0 0x7f59200a3650 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5924005950 tx=0x7f59240058e0 comp rx=0 tx=0).stop 2023-07-21T22:39:16.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.848+0000 7f593bc08700 1 -- 172.21.15.107:0/865106020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59340ffbf0 msgr2=0x7f5934100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:16.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.848+0000 7f593bc08700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59340ffbf0 0x7f5934100fd0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5930004760 tx=0x7f5930004790 comp rx=0 tx=0).stop 2023-07-21T22:39:16.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.848+0000 7f593bc08700 1 -- 172.21.15.107:0/865106020 shutdown_connections 2023-07-21T22:39:16.891 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.848+0000 7f593bc08700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f59200a11a0 0x7f59200a3650 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:16.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.848+0000 7f593bc08700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5934100580 0x7f5934101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:16.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.848+0000 7f593bc08700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59340fea00 0x7f5934102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:16.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.848+0000 7f593bc08700 1 --2- 172.21.15.107:0/865106020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59340ffbf0 0x7f5934100fd0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:16.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.848+0000 7f593bc08700 1 -- 172.21.15.107:0/865106020 >> 172.21.15.107:0/865106020 conn(0x7f59340f9fe0 msgr2=0x7f5934068720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:16.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.848+0000 7f593bc08700 1 -- 172.21.15.107:0/865106020 shutdown_connections 2023-07-21T22:39:16.892 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.848+0000 7f593bc08700 1 -- 172.21.15.107:0/865106020 wait complete. 2023-07-21T22:39:16.903 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:23:52.031819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:27:39.494209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:27:40.672303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:39:16.904 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:39:17.029 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.985+0000 7fafdbb56700 1 -- 172.21.15.107:0/2878602148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd40fffe0 msgr2=0x7fafd41003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.029 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.985+0000 7fafdbb56700 1 --2- 172.21.15.107:0/2878602148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd40fffe0 0x7fafd41003f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009990 tx=0x7fafc4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.029 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.985+0000 7fafdbb56700 1 -- 172.21.15.107:0/2878602148 shutdown_connections 2023-07-21T22:39:17.029 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.985+0000 7fafdbb56700 1 --2- 172.21.15.107:0/2878602148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafd40ff5e0 0x7fafd4106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.029 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.985+0000 7fafdbb56700 1 --2- 172.21.15.107:0/2878602148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd40fffe0 0x7fafd41003f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.985+0000 7fafdbb56700 1 --2- 172.21.15.107:0/2878602148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fafd40fec30 0x7fafd40ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.985+0000 7fafdbb56700 1 -- 172.21.15.107:0/2878602148 >> 172.21.15.107:0/2878602148 conn(0x7fafd40f9770 msgr2=0x7fafd40fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.985+0000 7fafdbb56700 1 -- 172.21.15.107:0/2878602148 shutdown_connections 2023-07-21T22:39:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.985+0000 7fafdbb56700 1 -- 172.21.15.107:0/2878602148 wait complete. 2023-07-21T22:39:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafdbb56700 1 Processor -- start 2023-07-21T22:39:17.030 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafdbb56700 1 -- start start 2023-07-21T22:39:17.032 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafdbb56700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd40fec30 0x7fafd40fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:17.032 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafdbb56700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fafd40ff5e0 0x7fafd419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:17.032 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafdbb56700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafd40fffe0 0x7fafd419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:17.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafdbb56700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd419d450 con 0x7fafd40fec30 2023-07-21T22:39:17.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafdbb56700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd419d5c0 con 0x7fafd40ff5e0 2023-07-21T22:39:17.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafdbb56700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd419d780 con 0x7fafd40fffe0 2023-07-21T22:39:17.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafd98f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd40fec30 0x7fafd40fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:17.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafd98f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd40fec30 0x7fafd40fe6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51788/0 (socket says 172.21.15.107:51788) 2023-07-21T22:39:17.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafd98f2700 1 -- 172.21.15.107:0/3307028089 learned_addr learned my addr 172.21.15.107:0/3307028089 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:39:17.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafd90f1700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fafd40ff5e0 0x7fafd419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:17.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafda0f3700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafd40fffe0 0x7fafd419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:17.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafd98f2700 1 -- 172.21.15.107:0/3307028089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafd40fffe0 msgr2=0x7fafd419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafd98f2700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafd40fffe0 0x7fafd419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafd98f2700 1 -- 172.21.15.107:0/3307028089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fafd40ff5e0 msgr2=0x7fafd419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafd98f2700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fafd40ff5e0 0x7fafd419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafd98f2700 1 -- 172.21.15.107:0/3307028089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc40095f0 con 0x7fafd40fec30 2023-07-21T22:39:17.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.986+0000 7fafd90f1700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fafd40ff5e0 0x7fafd419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:39:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.987+0000 7fafd98f2700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd40fec30 0x7fafd40fe6b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fafc40095c0 tx=0x7fafc400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.987+0000 7fafcaffd700 1 -- 172.21.15.107:0/3307028089 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc401d070 con 0x7fafd40fec30 2023-07-21T22:39:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.987+0000 7fafcaffd700 1 -- 172.21.15.107:0/3307028089 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafc4003e20 con 0x7fafd40fec30 2023-07-21T22:39:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.987+0000 7fafdbb56700 1 -- 172.21.15.107:0/3307028089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd419da00 con 0x7fafd40fec30 2023-07-21T22:39:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.987+0000 7fafcaffd700 1 -- 172.21.15.107:0/3307028089 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc40173f0 con 0x7fafd40fec30 2023-07-21T22:39:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.987+0000 7fafdbb56700 1 -- 172.21.15.107:0/3307028089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd419df00 con 0x7fafd40fec30 2023-07-21T22:39:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.988+0000 7fafdbb56700 1 -- 172.21.15.107:0/3307028089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd41059d0 con 0x7fafd40fec30 2023-07-21T22:39:17.035 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.989+0000 7fafcaffd700 1 -- 172.21.15.107:0/3307028089 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fafc4003960 con 0x7fafd40fec30 2023-07-21T22:39:17.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.989+0000 7fafcaffd700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fafc00a11d0 0x7fafc00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:17.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.990+0000 7fafd90f1700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fafc00a11d0 0x7fafc00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:17.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.990+0000 7fafcaffd700 1 -- 172.21.15.107:0/3307028089 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fafc40cdcf0 con 0x7fafd40fec30 2023-07-21T22:39:17.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.990+0000 7fafd90f1700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fafc00a11d0 0x7fafc00a3680 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fafd40fe420 tx=0x7fafd0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:17.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:16.993+0000 7fafcaffd700 1 -- 172.21.15.107:0/3307028089 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fafc40974b0 con 0x7fafd40fec30 2023-07-21T22:39:17.225 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.182+0000 7fafdbb56700 1 -- 172.21.15.107:0/3307028089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fafd402c9e0 con 0x7fafd40fec30 2023-07-21T22:39:17.226 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.183+0000 7fafcaffd700 1 -- 172.21.15.107:0/3307028089 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fafc409aad0 con 0x7fafd40fec30 2023-07-21T22:39:17.226 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2023-07-21T22:39:17.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.185+0000 7fafdbb56700 1 -- 172.21.15.107:0/3307028089 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fafc00a11d0 msgr2=0x7fafc00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.185+0000 7fafdbb56700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fafc00a11d0 0x7fafc00a3680 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fafd40fe420 tx=0x7fafd0008040 comp rx=0 tx=0).stop 2023-07-21T22:39:17.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.185+0000 7fafdbb56700 1 -- 172.21.15.107:0/3307028089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd40fec30 msgr2=0x7fafd40fe6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.185+0000 7fafdbb56700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd40fec30 0x7fafd40fe6b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fafc40095c0 tx=0x7fafc400f740 comp rx=0 tx=0).stop 2023-07-21T22:39:17.228 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.185+0000 7fafdbb56700 1 -- 172.21.15.107:0/3307028089 shutdown_connections 2023-07-21T22:39:17.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.185+0000 7fafdbb56700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fafc00a11d0 0x7fafc00a3680 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.185+0000 7fafdbb56700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafd40fffe0 0x7fafd419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.185+0000 7fafdbb56700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafd40fec30 0x7fafd40fe6b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.185+0000 7fafdbb56700 1 --2- 172.21.15.107:0/3307028089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fafd40ff5e0 0x7fafd419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.185+0000 7fafdbb56700 1 -- 172.21.15.107:0/3307028089 >> 172.21.15.107:0/3307028089 conn(0x7fafd40f9770 msgr2=0x7fafd40fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:17.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.185+0000 7fafdbb56700 1 -- 172.21.15.107:0/3307028089 shutdown_connections 2023-07-21T22:39:17.229 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.185+0000 7fafdbb56700 1 -- 172.21.15.107:0/3307028089 wait complete. 2023-07-21T22:39:17.240 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:39:17.365 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.322+0000 7f0034a9a700 1 -- 172.21.15.107:0/2229468964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0030100560 msgr2=0x7f0030106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.322+0000 7f0034a9a700 1 --2- 172.21.15.107:0/2229468964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0030100560 0x7f0030106e20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f002800b0a0 tx=0x7f002800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.322+0000 7f0034a9a700 1 -- 172.21.15.107:0/2229468964 shutdown_connections 2023-07-21T22:39:17.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.322+0000 7f0034a9a700 1 --2- 172.21.15.107:0/2229468964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0030100560 0x7f0030106e20 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.366 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.322+0000 7f0034a9a700 1 --2- 172.21.15.107:0/2229468964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00300fe9e0 0x7f00300fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.367 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.322+0000 7f0034a9a700 1 --2- 172.21.15.107:0/2229468964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00300ffbd0 0x7f0030100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.367 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.322+0000 7f0034a9a700 1 -- 172.21.15.107:0/2229468964 >> 172.21.15.107:0/2229468964 conn(0x7f00300fa000 msgr2=0x7f00300fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:17.367 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.322+0000 7f0034a9a700 1 -- 172.21.15.107:0/2229468964 shutdown_connections 2023-07-21T22:39:17.367 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.322+0000 7f0034a9a700 1 -- 172.21.15.107:0/2229468964 wait complete. 2023-07-21T22:39:17.367 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f0034a9a700 1 Processor -- start 2023-07-21T22:39:17.367 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f0034a9a700 1 -- start start 2023-07-21T22:39:17.369 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f0034a9a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00300fe9e0 0x7f0030197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:17.369 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f0034a9a700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00300ffbd0 0x7f0030198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:17.369 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f0034a9a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0030100560 0x7f003019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:17.369 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f0034a9a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003019f660 con 0x7f0030100560 2023-07-21T22:39:17.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f0034a9a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003019f7a0 con 0x7f00300fe9e0 2023-07-21T22:39:17.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f0034a9a700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003019f930 con 0x7f00300ffbd0 2023-07-21T22:39:17.370 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f002ed9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0030100560 0x7f003019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:17.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f002ed9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0030100560 0x7f003019f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51806/0 (socket says 172.21.15.107:51806) 2023-07-21T22:39:17.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f002ed9d700 1 -- 172.21.15.107:0/3436469840 learned_addr learned my addr 172.21.15.107:0/3436469840 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:39:17.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f002e59c700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00300fe9e0 0x7f0030197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:17.371 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f002dd9b700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00300ffbd0 0x7f0030198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:17.372 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f002ed9d700 1 -- 172.21.15.107:0/3436469840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00300ffbd0 msgr2=0x7f0030198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.372 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f002ed9d700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00300ffbd0 0x7f0030198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.372 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f002ed9d700 1 -- 172.21.15.107:0/3436469840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00300fe9e0 msgr2=0x7f0030197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f002ed9d700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00300fe9e0 0x7f0030197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f002ed9d700 1 -- 172.21.15.107:0/3436469840 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0028009d00 con 0x7f0030100560 2023-07-21T22:39:17.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f002dd9b700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00300ffbd0 0x7f0030198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:39:17.373 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f002e59c700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00300fe9e0 0x7f0030197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:39:17.374 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.323+0000 7f002ed9d700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0030100560 0x7f003019f090 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f002800b830 tx=0x7f0028004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:17.374 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.324+0000 7f001f7fe700 1 -- 172.21.15.107:0/3436469840 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002800e040 con 0x7f0030100560 2023-07-21T22:39:17.374 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.324+0000 7f001f7fe700 1 -- 172.21.15.107:0/3436469840 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00280094c0 con 0x7f0030100560 2023-07-21T22:39:17.375 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.324+0000 7f0034a9a700 1 -- 172.21.15.107:0/3436469840 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003019fbb0 con 0x7f0030100560 2023-07-21T22:39:17.375 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.324+0000 7f001f7fe700 1 -- 172.21.15.107:0/3436469840 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002801bc70 con 0x7f0030100560 2023-07-21T22:39:17.375 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.324+0000 7f0034a9a700 1 -- 172.21.15.107:0/3436469840 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00301a0030 con 0x7f0030100560 2023-07-21T22:39:17.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.324+0000 7f0034a9a700 1 -- 172.21.15.107:0/3436469840 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003010a1c0 con 0x7f0030100560 2023-07-21T22:39:17.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.326+0000 7f001f7fe700 1 -- 172.21.15.107:0/3436469840 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f002801d890 con 0x7f0030100560 2023-07-21T22:39:17.376 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.326+0000 7f001f7fe700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f00180a1150 0x7f00180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:17.377 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.327+0000 7f002e59c700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f00180a1150 0x7f00180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:17.377 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.327+0000 7f001f7fe700 1 -- 172.21.15.107:0/3436469840 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f00280cec10 con 0x7f0030100560 2023-07-21T22:39:17.377 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.327+0000 7f002e59c700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f00180a1150 0x7f00180a3600 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f00300ffa30 tx=0x7f0024005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:17.377 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.330+0000 7f001f7fe700 1 -- 172.21.15.107:0/3436469840 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f00280983d0 con 0x7f0030100560 2023-07-21T22:39:17.564 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.521+0000 7f0034a9a700 1 -- 172.21.15.107:0/3436469840 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00301a05e0 con 0x7f0030100560 2023-07-21T22:39:17.565 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.522+0000 7f001f7fe700 1 -- 172.21.15.107:0/3436469840 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0028021020 con 0x7f0030100560 2023-07-21T22:39:17.565 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2023-07-21T22:39:17.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.524+0000 7f0034a9a700 1 -- 172.21.15.107:0/3436469840 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f00180a1150 msgr2=0x7f00180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.524+0000 7f0034a9a700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f00180a1150 0x7f00180a3600 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f00300ffa30 tx=0x7f0024005f90 comp rx=0 tx=0).stop 2023-07-21T22:39:17.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.524+0000 7f0034a9a700 1 -- 172.21.15.107:0/3436469840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0030100560 msgr2=0x7f003019f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.524+0000 7f0034a9a700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0030100560 0x7f003019f090 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f002800b830 tx=0x7f0028004d30 comp rx=0 tx=0).stop 2023-07-21T22:39:17.567 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.524+0000 7f0034a9a700 1 -- 172.21.15.107:0/3436469840 shutdown_connections 2023-07-21T22:39:17.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.524+0000 7f0034a9a700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0030100560 0x7f003019f090 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.524+0000 7f0034a9a700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00300fe9e0 0x7f0030197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.524+0000 7f0034a9a700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f00180a1150 0x7f00180a3600 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.524+0000 7f0034a9a700 1 --2- 172.21.15.107:0/3436469840 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00300ffbd0 0x7f0030198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.524+0000 7f0034a9a700 1 -- 172.21.15.107:0/3436469840 >> 172.21.15.107:0/3436469840 conn(0x7f00300fa000 msgr2=0x7f0030068700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:17.568 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.524+0000 7f0034a9a700 1 -- 172.21.15.107:0/3436469840 shutdown_connections 2023-07-21T22:39:17.569 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.524+0000 7f0034a9a700 1 -- 172.21.15.107:0/3436469840 wait complete. 2023-07-21T22:39:17.579 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:39:17.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.659+0000 7fd71c74e700 1 -- 172.21.15.107:0/452182005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd714100ce0 msgr2=0x7fd714101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.702 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.659+0000 7fd71c74e700 1 --2- 172.21.15.107:0/452182005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd714100ce0 0x7fd714101130 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd710009990 tx=0x7fd710009ca0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.659+0000 7fd71c74e700 1 -- 172.21.15.107:0/452182005 shutdown_connections 2023-07-21T22:39:17.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.659+0000 7fd71c74e700 1 --2- 172.21.15.107:0/452182005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd714101670 0x7fd714107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.659+0000 7fd71c74e700 1 --2- 172.21.15.107:0/452182005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7140ffaf0 0x7fd7140fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.659+0000 7fd71c74e700 1 --2- 172.21.15.107:0/452182005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd714100ce0 0x7fd714101130 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.659+0000 7fd71c74e700 1 -- 172.21.15.107:0/452182005 >> 172.21.15.107:0/452182005 conn(0x7fd7140fb130 msgr2=0x7fd7140fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:17.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.659+0000 7fd71c74e700 1 -- 172.21.15.107:0/452182005 shutdown_connections 2023-07-21T22:39:17.703 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.659+0000 7fd71c74e700 1 -- 172.21.15.107:0/452182005 wait complete. 2023-07-21T22:39:17.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.659+0000 7fd71c74e700 1 Processor -- start 2023-07-21T22:39:17.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71c74e700 1 -- start start 2023-07-21T22:39:17.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71c74e700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7140ffaf0 0x7fd714197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:17.705 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71c74e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd714100ce0 0x7fd714197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:17.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71c74e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd714101670 0x7fd71419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:17.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71c74e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71419f5c0 con 0x7fd714101670 2023-07-21T22:39:17.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71c74e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71419f700 con 0x7fd714100ce0 2023-07-21T22:39:17.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71c74e700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71419f890 con 0x7fd7140ffaf0 2023-07-21T22:39:17.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71aceb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd714101670 0x7fd71419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:17.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71aceb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd714101670 0x7fd71419eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51820/0 (socket says 172.21.15.107:51820) 2023-07-21T22:39:17.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71aceb700 1 -- 172.21.15.107:0/2316229591 learned_addr learned my addr 172.21.15.107:0/2316229591 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:39:17.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd719ce9700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd714100ce0 0x7fd714197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:17.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71a4ea700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7140ffaf0 0x7fd714197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:17.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71aceb700 1 -- 172.21.15.107:0/2316229591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7140ffaf0 msgr2=0x7fd714197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71aceb700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7140ffaf0 0x7fd714197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71aceb700 1 -- 172.21.15.107:0/2316229591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd714100ce0 msgr2=0x7fd714197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71aceb700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd714100ce0 0x7fd714197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71aceb700 1 -- 172.21.15.107:0/2316229591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7100095f0 con 0x7fd714101670 2023-07-21T22:39:17.707 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71aceb700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd714101670 0x7fd71419eff0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd70c007ee0 tx=0x7fd70c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd70b7fe700 1 -- 172.21.15.107:0/2316229591 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c017070 con 0x7fd714101670 2023-07-21T22:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd70b7fe700 1 -- 172.21.15.107:0/2316229591 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd70c00f040 con 0x7fd714101670 2023-07-21T22:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd71c74e700 1 -- 172.21.15.107:0/2316229591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71419fb70 con 0x7fd714101670 2023-07-21T22:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.660+0000 7fd70b7fe700 1 -- 172.21.15.107:0/2316229591 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c005740 con 0x7fd714101670 2023-07-21T22:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.661+0000 7fd71c74e700 1 -- 172.21.15.107:0/2316229591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7141a0030 con 0x7fd714101670 2023-07-21T22:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.661+0000 7fd71c74e700 1 -- 172.21.15.107:0/2316229591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71410b310 con 0x7fd714101670 2023-07-21T22:39:17.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.662+0000 7fd70b7fe700 1 -- 172.21.15.107:0/2316229591 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd70c00dc60 con 0x7fd714101670 2023-07-21T22:39:17.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.663+0000 7fd70b7fe700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd7000a1150 0x7fd7000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:17.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.663+0000 7fd70b7fe700 1 -- 172.21.15.107:0/2316229591 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd70c0cc790 con 0x7fd714101670 2023-07-21T22:39:17.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.663+0000 7fd71a4ea700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd7000a1150 0x7fd7000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:17.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.663+0000 7fd71a4ea700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd7000a1150 0x7fd7000a3600 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd704005950 tx=0x7fd7040058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:17.710 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.667+0000 7fd70b7fe700 1 -- 172.21.15.107:0/2316229591 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd70c095ec0 con 0x7fd714101670 2023-07-21T22:39:17.899 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.856+0000 7fd71c74e700 1 -- 172.21.15.107:0/2316229591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7141a05e0 con 0x7fd714101670 2023-07-21T22:39:17.900 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.857+0000 7fd70b7fe700 1 -- 172.21.15.107:0/2316229591 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd70c0994e0 con 0x7fd714101670 2023-07-21T22:39:17.900 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2023-07-21T22:39:17.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.859+0000 7fd71c74e700 1 -- 172.21.15.107:0/2316229591 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd7000a1150 msgr2=0x7fd7000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.859+0000 7fd71c74e700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd7000a1150 0x7fd7000a3600 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd704005950 tx=0x7fd7040058e0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.859+0000 7fd71c74e700 1 -- 172.21.15.107:0/2316229591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd714101670 msgr2=0x7fd71419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:17.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.859+0000 7fd71c74e700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd714101670 0x7fd71419eff0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd70c007ee0 tx=0x7fd70c00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.902 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.859+0000 7fd71c74e700 1 -- 172.21.15.107:0/2316229591 shutdown_connections 2023-07-21T22:39:17.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.859+0000 7fd71c74e700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd714101670 0x7fd71419eff0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.859+0000 7fd71c74e700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fd7000a1150 0x7fd7000a3600 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.859+0000 7fd71c74e700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd7140ffaf0 0x7fd714197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.859+0000 7fd71c74e700 1 --2- 172.21.15.107:0/2316229591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd714100ce0 0x7fd714197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:17.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.859+0000 7fd71c74e700 1 -- 172.21.15.107:0/2316229591 >> 172.21.15.107:0/2316229591 conn(0x7fd7140fb130 msgr2=0x7fd7140fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:17.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.859+0000 7fd71c74e700 1 -- 172.21.15.107:0/2316229591 shutdown_connections 2023-07-21T22:39:17.903 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.859+0000 7fd71c74e700 1 -- 172.21.15.107:0/2316229591 wait complete. 2023-07-21T22:39:17.913 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:17 smithi107 ceph-mon[96543]: pgmap v600: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2023-07-21T22:39:17.913 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:17 smithi107 ceph-mon[96543]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:17.914 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:17 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/865106020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:39:17.914 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:17 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3307028089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:39:17.914 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:17 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3436469840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:39:17.915 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-21T22:39:18.036 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.993+0000 7f58ef507700 1 -- 172.21.15.107:0/4234225352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e8074550 msgr2=0x7f58e80729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:18.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.993+0000 7f58ef507700 1 --2- 172.21.15.107:0/4234225352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e8074550 0x7f58e80729a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f58d800b0a0 tx=0x7f58d800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:39:18.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.993+0000 7f58ef507700 1 -- 172.21.15.107:0/4234225352 shutdown_connections 2023-07-21T22:39:18.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.993+0000 7f58ef507700 1 --2- 172.21.15.107:0/4234225352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58e810c120 0x7f58e810e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:18.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.993+0000 7f58ef507700 1 --2- 172.21.15.107:0/4234225352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e8074550 0x7f58e80729a0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:18.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.993+0000 7f58ef507700 1 --2- 172.21.15.107:0/4234225352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58e8072f70 0x7f58e80733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:18.037 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.993+0000 7f58ef507700 1 -- 172.21.15.107:0/4234225352 >> 172.21.15.107:0/4234225352 conn(0x7f58e80fb170 msgr2=0x7f58e80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:18.038 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.993+0000 7f58ef507700 1 -- 172.21.15.107:0/4234225352 shutdown_connections 2023-07-21T22:39:18.038 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.993+0000 7f58ef507700 1 -- 172.21.15.107:0/4234225352 wait complete. 2023-07-21T22:39:18.039 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.994+0000 7f58ef507700 1 Processor -- start 2023-07-21T22:39:18.039 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.994+0000 7f58ef507700 1 -- start start 2023-07-21T22:39:18.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.994+0000 7f58ef507700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58e8072f70 0x7f58e819c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:18.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.994+0000 7f58ef507700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58e8074550 0x7f58e819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:18.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.994+0000 7f58ef507700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e810c120 0x7f58e81a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:18.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.994+0000 7f58ef507700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e81a3bb0 con 0x7f58e810c120 2023-07-21T22:39:18.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.994+0000 7f58ef507700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e81a3cf0 con 0x7f58e8074550 2023-07-21T22:39:18.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.994+0000 7f58ef507700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e81a3e80 con 0x7f58e8072f70 2023-07-21T22:39:18.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.994+0000 7f58edaa4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e810c120 0x7f58e81a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:18.041 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.994+0000 7f58edaa4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e810c120 0x7f58e81a35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51832/0 (socket says 172.21.15.107:51832) 2023-07-21T22:39:18.041 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.994+0000 7f58edaa4700 1 -- 172.21.15.107:0/2265811553 learned_addr learned my addr 172.21.15.107:0/2265811553 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:39:18.041 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.994+0000 7f58ed2a3700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58e8072f70 0x7f58e819c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:18.041 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.994+0000 7f58ecaa2700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58e8074550 0x7f58e819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:18.041 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.995+0000 7f58edaa4700 1 -- 172.21.15.107:0/2265811553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58e8072f70 msgr2=0x7f58e819c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:18.041 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.995+0000 7f58edaa4700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58e8072f70 0x7f58e819c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:18.041 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.995+0000 7f58edaa4700 1 -- 172.21.15.107:0/2265811553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58e8074550 msgr2=0x7f58e819c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:18.042 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.995+0000 7f58edaa4700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58e8074550 0x7f58e819c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:18.042 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.995+0000 7f58edaa4700 1 -- 172.21.15.107:0/2265811553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d8009d00 con 0x7f58e810c120 2023-07-21T22:39:18.042 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.995+0000 7f58edaa4700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e810c120 0x7f58e81a35e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f58e000e940 tx=0x7f58e000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:18.042 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.995+0000 7f58e67fc700 1 -- 172.21.15.107:0/2265811553 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e00097b0 con 0x7f58e810c120 2023-07-21T22:39:18.042 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.995+0000 7f58ef507700 1 -- 172.21.15.107:0/2265811553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e81a4160 con 0x7f58e810c120 2023-07-21T22:39:18.042 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.995+0000 7f58e67fc700 1 -- 172.21.15.107:0/2265811553 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58e0005020 con 0x7f58e810c120 2023-07-21T22:39:18.042 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.995+0000 7f58e67fc700 1 -- 172.21.15.107:0/2265811553 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e000cbf0 con 0x7f58e810c120 2023-07-21T22:39:18.042 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.995+0000 7f58ef507700 1 -- 172.21.15.107:0/2265811553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e81a4670 con 0x7f58e810c120 2023-07-21T22:39:18.043 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.996+0000 7f58ef507700 1 -- 172.21.15.107:0/2265811553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e8101c70 con 0x7f58e810c120 2023-07-21T22:39:18.043 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.997+0000 7f58e67fc700 1 -- 172.21.15.107:0/2265811553 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f58e0005400 con 0x7f58e810c120 2023-07-21T22:39:18.043 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.997+0000 7f58e67fc700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f58d00a1150 0x7f58d00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:18.044 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.998+0000 7f58ed2a3700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f58d00a1150 0x7f58d00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:18.044 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.998+0000 7f58e67fc700 1 -- 172.21.15.107:0/2265811553 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f58e0014070 con 0x7f58e810c120 2023-07-21T22:39:18.044 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:17.998+0000 7f58ed2a3700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f58d00a1150 0x7f58d00a3600 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f58d80062a0 tx=0x7f58d8006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:18.044 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.001+0000 7f58e67fc700 1 -- 172.21.15.107:0/2265811553 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f58e0096210 con 0x7f58e810c120 2023-07-21T22:39:18.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:17 smithi190 ceph-mon[98334]: pgmap v600: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2023-07-21T22:39:18.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:17 smithi190 ceph-mon[98334]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:18.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:17 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/865106020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:39:18.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:17 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3307028089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:39:18.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:17 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3436469840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:39:18.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:17 smithi132 ceph-mon[97863]: pgmap v600: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2023-07-21T22:39:18.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:17 smithi132 ceph-mon[97863]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-21T22:39:18.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:17 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/865106020' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:39:18.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:17 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3307028089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:39:18.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:17 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3436469840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:39:18.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.212+0000 7f58ef507700 1 -- 172.21.15.107:0/2265811553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f58e802c850 con 0x7f58e810c120 2023-07-21T22:39:18.255 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.212+0000 7f58e67fc700 1 -- 172.21.15.107:0/2265811553 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) v1 ==== 81+0+600 (secure 0 0 0) 0x7f58e0005190 con 0x7f58e810c120 2023-07-21T22:39:18.255 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3494357734 2023-07-22T22:21:45.714358+0000 2023-07-21T22:39:18.256 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2305207412 2023-07-22T22:21:45.714358+0000 2023-07-21T22:39:18.256 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/659042147 2023-07-22T22:21:45.714358+0000 2023-07-21T22:39:18.256 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3404483652 2023-07-22T22:21:24.460216+0000 2023-07-21T22:39:18.256 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/659042147 2023-07-22T22:21:45.714358+0000 2023-07-21T22:39:18.256 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3802710476 2023-07-22T22:21:24.460216+0000 2023-07-21T22:39:18.256 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3814985795 2023-07-22T22:21:24.460216+0000 2023-07-21T22:39:18.257 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3802710476 2023-07-22T22:21:24.460216+0000 2023-07-21T22:39:18.257 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2067809774 2023-07-22T22:21:24.460216+0000 2023-07-21T22:39:18.257 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2344232960 2023-07-22T22:21:45.714358+0000 2023-07-21T22:39:18.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.214+0000 7f58ef507700 1 -- 172.21.15.107:0/2265811553 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f58d00a1150 msgr2=0x7f58d00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:18.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.214+0000 7f58ef507700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f58d00a1150 0x7f58d00a3600 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f58d80062a0 tx=0x7f58d8006210 comp rx=0 tx=0).stop 2023-07-21T22:39:18.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.214+0000 7f58ef507700 1 -- 172.21.15.107:0/2265811553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e810c120 msgr2=0x7f58e81a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:18.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.214+0000 7f58ef507700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e810c120 0x7f58e81a35e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f58e000e940 tx=0x7f58e000ec50 comp rx=0 tx=0).stop 2023-07-21T22:39:18.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.214+0000 7f58ef507700 1 -- 172.21.15.107:0/2265811553 shutdown_connections 2023-07-21T22:39:18.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.214+0000 7f58ef507700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e810c120 0x7f58e81a35e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:18.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.214+0000 7f58ef507700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f58d00a1150 0x7f58d00a3600 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:18.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.214+0000 7f58ef507700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58e8072f70 0x7f58e819c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:18.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.214+0000 7f58ef507700 1 --2- 172.21.15.107:0/2265811553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58e8074550 0x7f58e819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:18.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.214+0000 7f58ef507700 1 -- 172.21.15.107:0/2265811553 >> 172.21.15.107:0/2265811553 conn(0x7f58e80fb170 msgr2=0x7f58e80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:18.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.214+0000 7f58ef507700 1 -- 172.21.15.107:0/2265811553 shutdown_connections 2023-07-21T22:39:18.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:18.214+0000 7f58ef507700 1 -- 172.21.15.107:0/2265811553 wait complete. 2023-07-21T22:39:18.259 INFO:teuthology.orchestra.run.smithi107.stderr:listed 10 entries 2023-07-21T22:39:18.270 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-21T22:39:18.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:39:18.270 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-07-21T22:39:18.397 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2023-07-21T22:39:18.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:39:18.398 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-21T22:39:18.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:39:18.509 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-21T22:39:18.612 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-21T22:39:18.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:39:18.613 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-21T22:39:18.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:39:18.685 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2023-07-21T22:39:18.778 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-21T22:39:18.779 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-21T22:39:18.779 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:39:18.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284104k,nr_inodes=4071026,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14007 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/a13aff2add16e54ad0cc514c516d04ab04f712fbacda2764d75fc3a357cdf9f4/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/FPA6GE5C2WJTUNDWSEBKDAIPH7:/var/lib/containers/storage/overlay/l/BLXKUOFOIQO42JUHT7XE4FJ3ZZ,upperdir=/var/lib/containers/storage/overlay/a13aff2add16e54ad0cc514c516d04ab04f712fbacda2764d75fc3a357cdf9f4/diff,workdir=/var/lib/containers/storage/overlay/a13aff2add16e54ad0cc514c516d04ab04f712fbacda2764d75fc3a357cdf9f4/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9c5ee4e28e87a50eb213f1d0cdec40b41e557e34ea8a29b311d6db4792a0f408/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/FPA6GE5C2WJTUNDWSEBKDAIPH7:/var/lib/containers/storage/overlay/l/BLXKUOFOIQO42JUHT7XE4FJ3ZZ,upperdir=/var/lib/containers/storage/overlay/9c5ee4e28e87a50eb213f1d0cdec40b41e557e34ea8a29b311d6db4792a0f408/diff,workdir=/var/lib/containers/storage/overlay/9c5ee4e28e87a50eb213f1d0cdec40b41e557e34ea8a29b311d6db4792a0f408/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260520k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/4f46312327aa05c84a91a914908bc0eb89a9e49eec8d7d05be40a5fc7ca08ba1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/FPA6GE5C2WJTUNDWSEBKDAIPH7:/var/lib/containers/storage/overlay/l/BLXKUOFOIQO42JUHT7XE4FJ3ZZ,upperdir=/var/lib/containers/storage/overlay/4f46312327aa05c84a91a914908bc0eb89a9e49eec8d7d05be40a5fc7ca08ba1/diff,workdir=/var/lib/containers/storage/overlay/4f46312327aa05c84a91a914908bc0eb89a9e49eec8d7d05be40a5fc7ca08ba1/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/52602cd2d066d2bbe70365ef7cddcc213d074b148615c259f9f429b104c8a59c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/FPA6GE5C2WJTUNDWSEBKDAIPH7:/var/lib/containers/storage/overlay/l/BLXKUOFOIQO42JUHT7XE4FJ3ZZ,upperdir=/var/lib/containers/storage/overlay/52602cd2d066d2bbe70365ef7cddcc213d074b148615c259f9f429b104c8a59c/diff,workdir=/var/lib/containers/storage/overlay/52602cd2d066d2bbe70365ef7cddcc213d074b148615c259f9f429b104c8a59c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9799969a9a3f125dba10f0750d61fffdb9e4a00f842aff2f021f06fc8c5cac9b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/FPA6GE5C2WJTUNDWSEBKDAIPH7:/var/lib/containers/storage/overlay/l/BLXKUOFOIQO42JUHT7XE4FJ3ZZ,upperdir=/var/lib/containers/storage/overlay/9799969a9a3f125dba10f0750d61fffdb9e4a00f842aff2f021f06fc8c5cac9b/diff,workdir=/var/lib/containers/storage/overlay/9799969a9a3f125dba10f0750d61fffdb9e4a00f842aff2f021f06fc8c5cac9b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/05a1c9d1ea89f7c6a63aeb612c912939fee714cb620a035ef7e15aa64e7c97f5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/FPA6GE5C2WJTUNDWSEBKDAIPH7:/var/lib/containers/storage/overlay/l/BLXKUOFOIQO42JUHT7XE4FJ3ZZ,upperdir=/var/lib/containers/storage/overlay/05a1c9d1ea89f7c6a63aeb612c912939fee714cb620a035ef7e15aa64e7c97f5/diff,workdir=/var/lib/containers/storage/overlay/05a1c9d1ea89f7c6a63aeb612c912939fee714cb620a035ef7e15aa64e7c97f5/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ec1bc8392ff848e8107708edcd8d29db65ca96c9eee531265c84e7cff9838cf5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/FPA6GE5C2WJTUNDWSEBKDAIPH7:/var/lib/containers/storage/overlay/l/BLXKUOFOIQO42JUHT7XE4FJ3ZZ,upperdir=/var/lib/containers/storage/overlay/ec1bc8392ff848e8107708edcd8d29db65ca96c9eee531265c84e7cff9838cf5/diff,workdir=/var/lib/containers/storage/overlay/ec1bc8392ff848e8107708edcd8d29db65ca96c9eee531265c84e7cff9838cf5/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/bc6cb9cc010bd47975e278c3d102ca5d6970e38dbdc0ba30d9276a94e5372c43/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/FPA6GE5C2WJTUNDWSEBKDAIPH7:/var/lib/containers/storage/overlay/l/BLXKUOFOIQO42JUHT7XE4FJ3ZZ,upperdir=/var/lib/containers/storage/overlay/bc6cb9cc010bd47975e278c3d102ca5d6970e38dbdc0ba30d9276a94e5372c43/diff,workdir=/var/lib/containers/storage/overlay/bc6cb9cc010bd47975e278c3d102ca5d6970e38dbdc0ba30d9276a94e5372c43/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9a75b5df76b594b660cea9224016a2bec5d2a9865af5eab83bee239183d27a7a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/FPA6GE5C2WJTUNDWSEBKDAIPH7:/var/lib/containers/storage/overlay/l/BLXKUOFOIQO42JUHT7XE4FJ3ZZ,upperdir=/var/lib/containers/storage/overlay/9a75b5df76b594b660cea9224016a2bec5d2a9865af5eab83bee239183d27a7a/diff,workdir=/var/lib/containers/storage/overlay/9a75b5df76b594b660cea9224016a2bec5d2a9865af5eab83bee239183d27a7a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8eca63e6f6e785d0c771f17b9a152cc0cdd6baae5f4927d864b9d2f74844dbc4/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/FPA6GE5C2WJTUNDWSEBKDAIPH7:/var/lib/containers/storage/overlay/l/BLXKUOFOIQO42JUHT7XE4FJ3ZZ,upperdir=/var/lib/containers/storage/overlay/8eca63e6f6e785d0c771f17b9a152cc0cdd6baae5f4927d864b9d2f74844dbc4/diff,workdir=/var/lib/containers/storage/overlay/8eca63e6f6e785d0c771f17b9a152cc0cdd6baae5f4927d864b9d2f74844dbc4/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-07-21T22:39:18.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:39:18.798 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-21T22:39:18.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:39:18.854 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2023-07-21T22:39:18.922 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2023-07-21T22:39:18.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:39:18.924 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-21T22:39:18.934 INFO:tasks.cephadm:Teardown begin 2023-07-21T22:39:18.935 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-21T22:39:18.935 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:39:18.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:18 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2316229591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:39:18.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:18 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2265811553' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:39:19.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.014+0000 7fe32e306700 1 -- 172.21.15.107:0/4282955769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe328100cf0 msgr2=0x7fe328101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:19.057 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.014+0000 7fe32e306700 1 --2- 172.21.15.107:0/4282955769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe328100cf0 0x7fe328101140 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009990 tx=0x7fe31c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.014+0000 7fe32e306700 1 -- 172.21.15.107:0/4282955769 shutdown_connections 2023-07-21T22:39:19.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.014+0000 7fe32e306700 1 --2- 172.21.15.107:0/4282955769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe328101680 0x7fe328107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.014+0000 7fe32e306700 1 --2- 172.21.15.107:0/4282955769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3280ffb00 0x7fe3280fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.014+0000 7fe32e306700 1 --2- 172.21.15.107:0/4282955769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe328100cf0 0x7fe328101140 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.014+0000 7fe32e306700 1 -- 172.21.15.107:0/4282955769 >> 172.21.15.107:0/4282955769 conn(0x7fe3280fb0e0 msgr2=0x7fe3280fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:19.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.014+0000 7fe32e306700 1 -- 172.21.15.107:0/4282955769 shutdown_connections 2023-07-21T22:39:19.058 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.014+0000 7fe32e306700 1 -- 172.21.15.107:0/4282955769 wait complete. 2023-07-21T22:39:19.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.014+0000 7fe32e306700 1 Processor -- start 2023-07-21T22:39:19.059 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.014+0000 7fe32e306700 1 -- start start 2023-07-21T22:39:19.060 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32e306700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3280ffb00 0x7fe328195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:19.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32e306700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe328100cf0 0x7fe328195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:19.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32e306700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe328101680 0x7fe32819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:19.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32e306700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32819d2b0 con 0x7fe328101680 2023-07-21T22:39:19.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32e306700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32819d3f0 con 0x7fe3280ffb00 2023-07-21T22:39:19.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32e306700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32819d580 con 0x7fe328100cf0 2023-07-21T22:39:19.061 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32c8a3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe328101680 0x7fe32819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:19.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32c8a3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe328101680 0x7fe32819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51842/0 (socket says 172.21.15.107:51842) 2023-07-21T22:39:19.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32c8a3700 1 -- 172.21.15.107:0/3028684417 learned_addr learned my addr 172.21.15.107:0/3028684417 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:39:19.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe3277fe700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe328100cf0 0x7fe328195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:19.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe327fff700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3280ffb00 0x7fe328195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:19.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32c8a3700 1 -- 172.21.15.107:0/3028684417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe328100cf0 msgr2=0x7fe328195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:19.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32c8a3700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe328100cf0 0x7fe328195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.062 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32c8a3700 1 -- 172.21.15.107:0/3028684417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3280ffb00 msgr2=0x7fe328195780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:19.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32c8a3700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3280ffb00 0x7fe328195780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32c8a3700 1 -- 172.21.15.107:0/3028684417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c0095f0 con 0x7fe328101680 2023-07-21T22:39:19.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe327fff700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3280ffb00 0x7fe328195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:39:19.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe32c8a3700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe328101680 0x7fe32819cce0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe320007d70 tx=0x7fe32000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:19.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.015+0000 7fe3257fa700 1 -- 172.21.15.107:0/3028684417 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32000ad40 con 0x7fe328101680 2023-07-21T22:39:19.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.016+0000 7fe3257fa700 1 -- 172.21.15.107:0/3028684417 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3200128e0 con 0x7fe328101680 2023-07-21T22:39:19.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.016+0000 7fe32e306700 1 -- 172.21.15.107:0/3028684417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32819d860 con 0x7fe328101680 2023-07-21T22:39:19.063 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.016+0000 7fe3257fa700 1 -- 172.21.15.107:0/3028684417 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3200116b0 con 0x7fe328101680 2023-07-21T22:39:19.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.016+0000 7fe32e306700 1 -- 172.21.15.107:0/3028684417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32819dd70 con 0x7fe328101680 2023-07-21T22:39:19.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.016+0000 7fe32e306700 1 -- 172.21.15.107:0/3028684417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe328106fd0 con 0x7fe328101680 2023-07-21T22:39:19.064 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.018+0000 7fe3257fa700 1 -- 172.21.15.107:0/3028684417 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe320012420 con 0x7fe328101680 2023-07-21T22:39:19.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.018+0000 7fe3257fa700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe3100a11a0 0x7fe3100a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:19.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.018+0000 7fe327fff700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe3100a11a0 0x7fe3100a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:19.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.018+0000 7fe3257fa700 1 -- 172.21.15.107:0/3028684417 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe3200ccf70 con 0x7fe328101680 2023-07-21T22:39:19.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.018+0000 7fe327fff700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe3100a11a0 0x7fe3100a3650 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe318005fd0 tx=0x7fe318005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:19.065 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.021+0000 7fe3257fa700 1 -- 172.21.15.107:0/3028684417 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe3200966a0 con 0x7fe328101680 2023-07-21T22:39:19.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:18 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2316229591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:39:19.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:18 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2265811553' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:39:19.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:18 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2316229591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:39:19.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:18 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2265811553' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:39:19.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.254+0000 7fe32e306700 1 -- 172.21.15.107:0/3028684417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe32802c850 con 0x7fe328101680 2023-07-21T22:39:19.297 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.255+0000 7fe3257fa700 1 -- 172.21.15.107:0/3028684417 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+10903 (secure 0 0 0) 0x7fe320099cc0 con 0x7fe328101680 2023-07-21T22:39:19.298 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:39:19.298 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:39:16.710188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":130,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25553":{"gid":25553,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2828468744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2828468744},{"type":"v1","addr":"172.21.15.132:6837","nonce":2828468744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:39:19.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.257+0000 7fe32e306700 1 -- 172.21.15.107:0/3028684417 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe3100a11a0 msgr2=0x7fe3100a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:19.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.257+0000 7fe32e306700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe3100a11a0 0x7fe3100a3650 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe318005fd0 tx=0x7fe318005f00 comp rx=0 tx=0).stop 2023-07-21T22:39:19.300 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.257+0000 7fe32e306700 1 -- 172.21.15.107:0/3028684417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe328101680 msgr2=0x7fe32819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:19.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.257+0000 7fe32e306700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe328101680 0x7fe32819cce0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe320007d70 tx=0x7fe32000a340 comp rx=0 tx=0).stop 2023-07-21T22:39:19.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.257+0000 7fe32e306700 1 -- 172.21.15.107:0/3028684417 shutdown_connections 2023-07-21T22:39:19.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.257+0000 7fe32e306700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe328101680 0x7fe32819cce0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.257+0000 7fe32e306700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3280ffb00 0x7fe328195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.257+0000 7fe32e306700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fe3100a11a0 0x7fe3100a3650 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.257+0000 7fe32e306700 1 --2- 172.21.15.107:0/3028684417 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe328100cf0 0x7fe328195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.257+0000 7fe32e306700 1 -- 172.21.15.107:0/3028684417 >> 172.21.15.107:0/3028684417 conn(0x7fe3280fb0e0 msgr2=0x7fe328104110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:19.301 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.257+0000 7fe32e306700 1 -- 172.21.15.107:0/3028684417 shutdown_connections 2023-07-21T22:39:19.302 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.257+0000 7fe32e306700 1 -- 172.21.15.107:0/3028684417 wait complete. 2023-07-21T22:39:19.302 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 57 2023-07-21T22:39:19.312 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:39:19.436 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.390+0000 7f65ed70d700 1 -- 172.21.15.107:0/2945641797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e80fe9d0 msgr2=0x7f65e80fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:19.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.390+0000 7f65ed70d700 1 --2- 172.21.15.107:0/2945641797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e80fe9d0 0x7f65e80fede0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009990 tx=0x7f65dc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.390+0000 7f65ed70d700 1 -- 172.21.15.107:0/2945641797 shutdown_connections 2023-07-21T22:39:19.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.390+0000 7f65ed70d700 1 --2- 172.21.15.107:0/2945641797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65e8100550 0x7f65e8106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.390+0000 7f65ed70d700 1 --2- 172.21.15.107:0/2945641797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e80fe9d0 0x7f65e80fede0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.390+0000 7f65ed70d700 1 --2- 172.21.15.107:0/2945641797 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65e80ffbc0 0x7f65e8100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.390+0000 7f65ed70d700 1 -- 172.21.15.107:0/2945641797 >> 172.21.15.107:0/2945641797 conn(0x7f65e80f9fd0 msgr2=0x7f65e80fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:19.437 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.390+0000 7f65ed70d700 1 -- 172.21.15.107:0/2945641797 shutdown_connections 2023-07-21T22:39:19.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.390+0000 7f65ed70d700 1 -- 172.21.15.107:0/2945641797 wait complete. 2023-07-21T22:39:19.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.390+0000 7f65ed70d700 1 Processor -- start 2023-07-21T22:39:19.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.390+0000 7f65ed70d700 1 -- start start 2023-07-21T22:39:19.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65ed70d700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65e80fe9d0 0x7f65e8195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:19.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65ed70d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e80ffbc0 0x7f65e8195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:19.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65ed70d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65e8100550 0x7f65e819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:19.438 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65ed70d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e819d2f0 con 0x7f65e80ffbc0 2023-07-21T22:39:19.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65ed70d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e819d430 con 0x7f65e8100550 2023-07-21T22:39:19.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65ed70d700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e819d5c0 con 0x7f65e80fe9d0 2023-07-21T22:39:19.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65e67fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e80ffbc0 0x7f65e8195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:19.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65e67fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e80ffbc0 0x7f65e8195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51856/0 (socket says 172.21.15.107:51856) 2023-07-21T22:39:19.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65e67fc700 1 -- 172.21.15.107:0/3874284721 learned_addr learned my addr 172.21.15.107:0/3874284721 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:39:19.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65e77fe700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65e8100550 0x7f65e819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:19.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65e6ffd700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65e80fe9d0 0x7f65e8195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:19.439 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65e67fc700 1 -- 172.21.15.107:0/3874284721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65e80fe9d0 msgr2=0x7f65e8195770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:19.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65e67fc700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65e80fe9d0 0x7f65e8195770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65e67fc700 1 -- 172.21.15.107:0/3874284721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65e8100550 msgr2=0x7f65e819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:19.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65e67fc700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65e8100550 0x7f65e819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65e67fc700 1 -- 172.21.15.107:0/3874284721 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc0095f0 con 0x7f65e80ffbc0 2023-07-21T22:39:19.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65e67fc700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e80ffbc0 0x7f65e8195cb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f65d800d950 tx=0x7f65d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:19.440 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65cffff700 1 -- 172.21.15.107:0/3874284721 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d800f890 con 0x7f65e80ffbc0 2023-07-21T22:39:19.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65cffff700 1 -- 172.21.15.107:0/3874284721 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65d8010460 con 0x7f65e80ffbc0 2023-07-21T22:39:19.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65ed70d700 1 -- 172.21.15.107:0/3874284721 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e819d8a0 con 0x7f65e80ffbc0 2023-07-21T22:39:19.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65cffff700 1 -- 172.21.15.107:0/3874284721 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d800e610 con 0x7f65e80ffbc0 2023-07-21T22:39:19.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.391+0000 7f65ed70d700 1 -- 172.21.15.107:0/3874284721 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e819ddb0 con 0x7f65e80ffbc0 2023-07-21T22:39:19.441 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.392+0000 7f65ed70d700 1 -- 172.21.15.107:0/3874284721 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e8106a10 con 0x7f65e80ffbc0 2023-07-21T22:39:19.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.394+0000 7f65cffff700 1 -- 172.21.15.107:0/3874284721 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f65d80108d0 con 0x7f65e80ffbc0 2023-07-21T22:39:19.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.394+0000 7f65cffff700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f65d00a11a0 0x7f65d00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:19.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.394+0000 7f65e6ffd700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f65d00a11a0 0x7f65d00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:19.442 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.394+0000 7f65cffff700 1 -- 172.21.15.107:0/3874284721 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f65d80cc6f0 con 0x7f65e80ffbc0 2023-07-21T22:39:19.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.395+0000 7f65e6ffd700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f65d00a11a0 0x7f65d00a3650 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00b5c0 tx=0x7f65dc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:19.443 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.397+0000 7f65cffff700 1 -- 172.21.15.107:0/3874284721 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f65d8095e20 con 0x7f65e80ffbc0 2023-07-21T22:39:19.672 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.629+0000 7f65ed70d700 1 -- 172.21.15.107:0/3874284721 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65e802c850 con 0x7f65e80ffbc0 2023-07-21T22:39:19.673 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.630+0000 7f65cffff700 1 -- 172.21.15.107:0/3874284721 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+10903 (secure 0 0 0) 0x7f65d8016090 con 0x7f65e80ffbc0 2023-07-21T22:39:19.673 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:39:19.673 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/219129239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":219129239},{"type":"v1","addr":"172.21.15.107:6835","nonce":219129239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2102134479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2102134479},{"type":"v1","addr":"172.21.15.107:6837","nonce":2102134479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:27:41.478391+0000","modified":"2023-07-21T22:39:16.710188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24533,"mds_1":24493,"mds_2":24466,"mds_3":24455,"mds_4":24437},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14646":{"gid":14646,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/1956222571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1956222571},{"type":"v1","addr":"172.21.15.107:6839","nonce":1956222571}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14658":{"gid":14658,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/1878776680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1878776680},{"type":"v1","addr":"172.21.15.107:6841","nonce":1878776680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6833/253562772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":253562772},{"type":"v1","addr":"172.21.15.190:6833","nonce":253562772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24437":{"gid":24437,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.132:6833/3293513462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3293513462},{"type":"v1","addr":"172.21.15.132:6833","nonce":3293513462}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24455":{"gid":24455,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.132:6835/1022724305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":1022724305},{"type":"v1","addr":"172.21.15.132:6835","nonce":1022724305}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":130,"addr":"172.21.15.190:6835/2286851849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2286851849},{"type":"v1","addr":"172.21.15.190:6835","nonce":2286851849}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.190:6837/3774629948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6836","nonce":3774629948},{"type":"v1","addr":"172.21.15.190:6837","nonce":3774629948}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.190:6839/2960500982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6838","nonce":2960500982},{"type":"v1","addr":"172.21.15.190:6839","nonce":2960500982}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/523672176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":523672176},{"type":"v1","addr":"172.21.15.132:6839","nonce":523672176}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25553":{"gid":25553,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6837/2828468744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2828468744},{"type":"v1","addr":"172.21.15.132:6837","nonce":2828468744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:39:19.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.632+0000 7f65ed70d700 1 -- 172.21.15.107:0/3874284721 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f65d00a11a0 msgr2=0x7f65d00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:19.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.632+0000 7f65ed70d700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f65d00a11a0 0x7f65d00a3650 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00b5c0 tx=0x7f65dc005960 comp rx=0 tx=0).stop 2023-07-21T22:39:19.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.632+0000 7f65ed70d700 1 -- 172.21.15.107:0/3874284721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e80ffbc0 msgr2=0x7f65e8195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:19.675 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.632+0000 7f65ed70d700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e80ffbc0 0x7f65e8195cb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f65d800d950 tx=0x7f65d800dc60 comp rx=0 tx=0).stop 2023-07-21T22:39:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.632+0000 7f65ed70d700 1 -- 172.21.15.107:0/3874284721 shutdown_connections 2023-07-21T22:39:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.632+0000 7f65ed70d700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65e8100550 0x7f65e819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.632+0000 7f65ed70d700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f65d00a11a0 0x7f65d00a3650 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.632+0000 7f65ed70d700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65e80fe9d0 0x7f65e8195770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.632+0000 7f65ed70d700 1 --2- 172.21.15.107:0/3874284721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e80ffbc0 0x7f65e8195cb0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.632+0000 7f65ed70d700 1 -- 172.21.15.107:0/3874284721 >> 172.21.15.107:0/3874284721 conn(0x7f65e80f9fd0 msgr2=0x7f65e80fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.632+0000 7f65ed70d700 1 -- 172.21.15.107:0/3874284721 shutdown_connections 2023-07-21T22:39:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.632+0000 7f65ed70d700 1 -- 172.21.15.107:0/3874284721 wait complete. 2023-07-21T22:39:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 57 2023-07-21T22:39:19.687 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:39:19.808 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.765+0000 7fc3d8850700 1 -- 172.21.15.107:0/2750934739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3d4100e30 msgr2=0x7fc3d4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:19.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.765+0000 7fc3d8850700 1 --2- 172.21.15.107:0/2750934739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3d4100e30 0x7fc3d4101280 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009990 tx=0x7fc3c4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.765+0000 7fc3d8850700 1 -- 172.21.15.107:0/2750934739 shutdown_connections 2023-07-21T22:39:19.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.765+0000 7fc3d8850700 1 --2- 172.21.15.107:0/2750934739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3d41017c0 0x7fc3d4108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.765+0000 7fc3d8850700 1 --2- 172.21.15.107:0/2750934739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3d40fde60 0x7fc3d40fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.809 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.765+0000 7fc3d8850700 1 --2- 172.21.15.107:0/2750934739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3d4100e30 0x7fc3d4101280 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.765+0000 7fc3d8850700 1 -- 172.21.15.107:0/2750934739 >> 172.21.15.107:0/2750934739 conn(0x7fc3d40f9860 msgr2=0x7fc3d40fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:19.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.765+0000 7fc3d8850700 1 -- 172.21.15.107:0/2750934739 shutdown_connections 2023-07-21T22:39:19.810 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.765+0000 7fc3d8850700 1 -- 172.21.15.107:0/2750934739 wait complete. 2023-07-21T22:39:19.812 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d8850700 1 Processor -- start 2023-07-21T22:39:19.812 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d8850700 1 -- start start 2023-07-21T22:39:19.812 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d8850700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3d40fde60 0x7fc3d41957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:19.812 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d8850700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3d4100e30 0x7fc3d4195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:19.813 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d8850700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3d41017c0 0x7fc3d419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:19.813 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d8850700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419d350 con 0x7fc3d4100e30 2023-07-21T22:39:19.813 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d8850700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419d490 con 0x7fc3d40fde60 2023-07-21T22:39:19.813 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d8850700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419d620 con 0x7fc3d41017c0 2023-07-21T22:39:19.813 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d1d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3d4100e30 0x7fc3d4195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:19.813 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d1d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3d4100e30 0x7fc3d4195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51870/0 (socket says 172.21.15.107:51870) 2023-07-21T22:39:19.813 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d1d9b700 1 -- 172.21.15.107:0/3195339910 learned_addr learned my addr 172.21.15.107:0/3195339910 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:39:19.813 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d2d9d700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3d41017c0 0x7fc3d419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d259c700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3d40fde60 0x7fc3d41957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d1d9b700 1 -- 172.21.15.107:0/3195339910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3d41017c0 msgr2=0x7fc3d419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d1d9b700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3d41017c0 0x7fc3d419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d1d9b700 1 -- 172.21.15.107:0/3195339910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3d40fde60 msgr2=0x7fc3d41957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.766+0000 7fc3d1d9b700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3d40fde60 0x7fc3d41957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.767+0000 7fc3d1d9b700 1 -- 172.21.15.107:0/3195339910 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c40095f0 con 0x7fc3d4100e30 2023-07-21T22:39:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.767+0000 7fc3d1d9b700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3d4100e30 0x7fc3d4195d10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4000c00 tx=0x7fc3c400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.767+0000 7fc3c37fe700 1 -- 172.21.15.107:0/3195339910 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c401d070 con 0x7fc3d4100e30 2023-07-21T22:39:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.767+0000 7fc3c37fe700 1 -- 172.21.15.107:0/3195339910 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3c400fb50 con 0x7fc3d4100e30 2023-07-21T22:39:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.767+0000 7fc3d8850700 1 -- 172.21.15.107:0/3195339910 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d419d8a0 con 0x7fc3d4100e30 2023-07-21T22:39:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.767+0000 7fc3c37fe700 1 -- 172.21.15.107:0/3195339910 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4017650 con 0x7fc3d4100e30 2023-07-21T22:39:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.767+0000 7fc3d8850700 1 -- 172.21.15.107:0/3195339910 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d419dd50 con 0x7fc3d4100e30 2023-07-21T22:39:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.768+0000 7fc3d8850700 1 -- 172.21.15.107:0/3195339910 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d410b370 con 0x7fc3d4100e30 2023-07-21T22:39:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.769+0000 7fc3c37fe700 1 -- 172.21.15.107:0/3195339910 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc3c400fcc0 con 0x7fc3d4100e30 2023-07-21T22:39:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.770+0000 7fc3c37fe700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc3bc0a11a0 0x7fc3bc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.770+0000 7fc3c37fe700 1 -- 172.21.15.107:0/3195339910 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc3c40ce020 con 0x7fc3d4100e30 2023-07-21T22:39:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.770+0000 7fc3d259c700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc3bc0a11a0 0x7fc3bc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.771+0000 7fc3d259c700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc3bc0a11a0 0x7fc3bc0a3650 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8005950 tx=0x7fc3c80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.773+0000 7fc3c37fe700 1 -- 172.21.15.107:0/3195339910 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc3c40977e0 con 0x7fc3d4100e30 2023-07-21T22:39:20.004 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.962+0000 7fc3d8850700 1 -- 172.21.15.107:0/3195339910 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3d4106af0 con 0x7fc3d4100e30 2023-07-21T22:39:20.006 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.963+0000 7fc3c37fe700 1 -- 172.21.15.107:0/3195339910 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22727 (secure 0 0 0) 0x7fc3c409ae00 con 0x7fc3d4100e30 2023-07-21T22:39:20.006 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-21T22:39:20.007 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":85,"fsid":"af99f5d0-2814-11ee-9b34-001a4aab830c","created":"2023-07-21T22:20:56.799654+0000","modified":"2023-07-21T22:31:05.959573+0000","last_up_change":"2023-07-21T22:27:01.223917+0000","last_in_change":"2023-07-21T22:26:43.399226+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:23:52.031819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:27:39.494209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:27:40.672303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dc595a3a-edf0-47ac-b533-25c80518c634","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6803","nonce":4235328012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6805","nonce":4235328012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6809","nonce":4235328012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4235328012},{"type":"v1","addr":"172.21.15.107:6807","nonce":4235328012}]},"public_addr":"172.21.15.107:6803/4235328012","cluster_addr":"172.21.15.107:6805/4235328012","heartbeat_back_addr":"172.21.15.107:6809/4235328012","heartbeat_front_addr":"172.21.15.107:6807/4235328012","state":["exists","up"]},{"osd":1,"uuid":"6d5fd94a-6288-42ed-9f8a-413b2e687e26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6801","nonce":1473726199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6803","nonce":1473726199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6807","nonce":1473726199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":1473726199},{"type":"v1","addr":"172.21.15.132:6805","nonce":1473726199}]},"public_addr":"172.21.15.132:6801/1473726199","cluster_addr":"172.21.15.132:6803/1473726199","heartbeat_back_addr":"172.21.15.132:6807/1473726199","heartbeat_front_addr":"172.21.15.132:6805/1473726199","state":["exists","up"]},{"osd":2,"uuid":"e5ce38db-d9b7-4f49-9945-fe7e1aacf27d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6801","nonce":1160922995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6803","nonce":1160922995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6807","nonce":1160922995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":1160922995},{"type":"v1","addr":"172.21.15.190:6805","nonce":1160922995}]},"public_addr":"172.21.15.190:6801/1160922995","cluster_addr":"172.21.15.190:6803/1160922995","heartbeat_back_addr":"172.21.15.190:6807/1160922995","heartbeat_front_addr":"172.21.15.190:6805/1160922995","state":["exists","up"]},{"osd":3,"uuid":"eaefb60a-452f-4eb7-a984-5b459aa49d4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6811","nonce":1375259004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6813","nonce":1375259004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6817","nonce":1375259004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1375259004},{"type":"v1","addr":"172.21.15.107:6815","nonce":1375259004}]},"public_addr":"172.21.15.107:6811/1375259004","cluster_addr":"172.21.15.107:6813/1375259004","heartbeat_back_addr":"172.21.15.107:6817/1375259004","heartbeat_front_addr":"172.21.15.107:6815/1375259004","state":["exists","up"]},{"osd":4,"uuid":"8f114d7a-5079-4c48-a9fa-c477d9447aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6809","nonce":2021101216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6811","nonce":2021101216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6815","nonce":2021101216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2021101216},{"type":"v1","addr":"172.21.15.132:6813","nonce":2021101216}]},"public_addr":"172.21.15.132:6809/2021101216","cluster_addr":"172.21.15.132:6811/2021101216","heartbeat_back_addr":"172.21.15.132:6815/2021101216","heartbeat_front_addr":"172.21.15.132:6813/2021101216","state":["exists","up"]},{"osd":5,"uuid":"93ba2f1d-2b22-4199-9cae-05fef817b637","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6809","nonce":1950039533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6811","nonce":1950039533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6815","nonce":1950039533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":1950039533},{"type":"v1","addr":"172.21.15.190:6813","nonce":1950039533}]},"public_addr":"172.21.15.190:6809/1950039533","cluster_addr":"172.21.15.190:6811/1950039533","heartbeat_back_addr":"172.21.15.190:6815/1950039533","heartbeat_front_addr":"172.21.15.190:6813/1950039533","state":["exists","up"]},{"osd":6,"uuid":"59ce5822-cb79-41f8-bc1e-de34c37311d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6819","nonce":3497265368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6821","nonce":3497265368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6825","nonce":3497265368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":3497265368},{"type":"v1","addr":"172.21.15.107:6823","nonce":3497265368}]},"public_addr":"172.21.15.107:6819/3497265368","cluster_addr":"172.21.15.107:6821/3497265368","heartbeat_back_addr":"172.21.15.107:6825/3497265368","heartbeat_front_addr":"172.21.15.107:6823/3497265368","state":["exists","up"]},{"osd":7,"uuid":"0013f445-b592-48e8-9620-ca2ff64a8e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6817","nonce":2500130834}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6819","nonce":2500130834}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6823","nonce":2500130834}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2500130834},{"type":"v1","addr":"172.21.15.132:6821","nonce":2500130834}]},"public_addr":"172.21.15.132:6817/2500130834","cluster_addr":"172.21.15.132:6819/2500130834","heartbeat_back_addr":"172.21.15.132:6823/2500130834","heartbeat_front_addr":"172.21.15.132:6821/2500130834","state":["exists","up"]},{"osd":8,"uuid":"f0beb03d-307c-4f41-8e17-6e0418063c75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6817","nonce":3597513159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6819","nonce":3597513159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6823","nonce":3597513159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":3597513159},{"type":"v1","addr":"172.21.15.190:6821","nonce":3597513159}]},"public_addr":"172.21.15.190:6817/3597513159","cluster_addr":"172.21.15.190:6819/3597513159","heartbeat_back_addr":"172.21.15.190:6823/3597513159","heartbeat_front_addr":"172.21.15.190:6821/3597513159","state":["exists","up"]},{"osd":9,"uuid":"b6203384-ccdb-419c-aa4a-0f64c3bdcba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6827","nonce":458109252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6829","nonce":458109252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6833","nonce":458109252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":458109252},{"type":"v1","addr":"172.21.15.107:6831","nonce":458109252}]},"public_addr":"172.21.15.107:6827/458109252","cluster_addr":"172.21.15.107:6829/458109252","heartbeat_back_addr":"172.21.15.107:6833/458109252","heartbeat_front_addr":"172.21.15.107:6831/458109252","state":["exists","up"]},{"osd":10,"uuid":"fc031611-c261-490b-ab22-31fd52691970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6825","nonce":2698708367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6827","nonce":2698708367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6831","nonce":2698708367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":2698708367},{"type":"v1","addr":"172.21.15.132:6829","nonce":2698708367}]},"public_addr":"172.21.15.132:6825/2698708367","cluster_addr":"172.21.15.132:6827/2698708367","heartbeat_back_addr":"172.21.15.132:6831/2698708367","heartbeat_front_addr":"172.21.15.132:6829/2698708367","state":["exists","up"]},{"osd":11,"uuid":"b9d95a8c-e6d0-49a6-9c50-2adee825168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6825","nonce":1655561112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6827","nonce":1655561112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6831","nonce":1655561112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":1655561112},{"type":"v1","addr":"172.21.15.190:6829","nonce":1655561112}]},"public_addr":"172.21.15.190:6825/1655561112","cluster_addr":"172.21.15.190:6827/1655561112","heartbeat_back_addr":"172.21.15.190:6831/1655561112","heartbeat_front_addr":"172.21.15.190:6829/1655561112","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:07.262375+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:28.323680+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:23:49.095296+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:10.431401+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:31.952353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:24:52.647721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:13.344492+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:34.470052+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:25:55.722367+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:17.042775+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:38.522562+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:26:59.330545+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/3494357734":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/2305207412":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6801/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:0/3404483652":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6800/659042147":"2023-07-22T22:21:45.714358+0000","172.21.15.107:6800/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/3814985795":"2023-07-22T22:21:24.460216+0000","172.21.15.107:6801/3802710476":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2067809774":"2023-07-22T22:21:24.460216+0000","172.21.15.107:0/2344232960":"2023-07-22T22:21:45.714358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:39:20.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.965+0000 7fc3d8850700 1 -- 172.21.15.107:0/3195339910 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc3bc0a11a0 msgr2=0x7fc3bc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:20.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.965+0000 7fc3d8850700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc3bc0a11a0 0x7fc3bc0a3650 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8005950 tx=0x7fc3c80058e0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.008 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.965+0000 7fc3d8850700 1 -- 172.21.15.107:0/3195339910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3d4100e30 msgr2=0x7fc3d4195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:20.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.965+0000 7fc3d8850700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3d4100e30 0x7fc3d4195d10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4000c00 tx=0x7fc3c400f710 comp rx=0 tx=0).stop 2023-07-21T22:39:20.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.965+0000 7fc3d8850700 1 -- 172.21.15.107:0/3195339910 shutdown_connections 2023-07-21T22:39:20.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.965+0000 7fc3d8850700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fc3bc0a11a0 0x7fc3bc0a3650 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.965+0000 7fc3d8850700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3d41017c0 0x7fc3d419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.965+0000 7fc3d8850700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3d40fde60 0x7fc3d41957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.965+0000 7fc3d8850700 1 --2- 172.21.15.107:0/3195339910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3d4100e30 0x7fc3d4195d10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.009 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.965+0000 7fc3d8850700 1 -- 172.21.15.107:0/3195339910 >> 172.21.15.107:0/3195339910 conn(0x7fc3d40f9860 msgr2=0x7fc3d40fba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:20.010 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.965+0000 7fc3d8850700 1 -- 172.21.15.107:0/3195339910 shutdown_connections 2023-07-21T22:39:20.010 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:19.965+0000 7fc3d8850700 1 -- 172.21.15.107:0/3195339910 wait complete. 2023-07-21T22:39:20.019 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:19 smithi107 ceph-mon[96543]: pgmap v601: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-07-21T22:39:20.019 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:19 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3028684417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:39:20.020 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:19 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3874284721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:39:20.021 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-21T22:39:20.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:19 smithi190 ceph-mon[98334]: pgmap v601: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-07-21T22:39:20.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:19 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3028684417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:39:20.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:19 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3874284721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:39:20.142 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.099+0000 7f3e231b7700 1 -- 172.21.15.107:0/607231696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c0fec10 msgr2=0x7f3e1c0ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:20.143 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.099+0000 7f3e231b7700 1 --2- 172.21.15.107:0/607231696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c0fec10 0x7f3e1c0ff080 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009990 tx=0x7f3e08009ca0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.143 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.099+0000 7f3e231b7700 1 -- 172.21.15.107:0/607231696 shutdown_connections 2023-07-21T22:39:20.143 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.099+0000 7f3e231b7700 1 --2- 172.21.15.107:0/607231696 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e1c0ff5c0 0x7f3e1c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.143 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.099+0000 7f3e231b7700 1 --2- 172.21.15.107:0/607231696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c100050 0x7f3e1c100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.143 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.099+0000 7f3e231b7700 1 --2- 172.21.15.107:0/607231696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c0fec10 0x7f3e1c0ff080 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.143 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.099+0000 7f3e231b7700 1 -- 172.21.15.107:0/607231696 >> 172.21.15.107:0/607231696 conn(0x7f3e1c0f9770 msgr2=0x7f3e1c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:20.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.099+0000 7f3e231b7700 1 -- 172.21.15.107:0/607231696 shutdown_connections 2023-07-21T22:39:20.144 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.099+0000 7f3e231b7700 1 -- 172.21.15.107:0/607231696 wait complete. 2023-07-21T22:39:20.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.100+0000 7f3e231b7700 1 Processor -- start 2023-07-21T22:39:20.145 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.100+0000 7f3e231b7700 1 -- start start 2023-07-21T22:39:20.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.100+0000 7f3e231b7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c0fec10 0x7f3e1c0fe690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:20.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.100+0000 7f3e231b7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e1c0ff5c0 0x7f3e1c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:20.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.100+0000 7f3e231b7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c100050 0x7f3e1c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:20.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.100+0000 7f3e231b7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c19d370 con 0x7f3e1c100050 2023-07-21T22:39:20.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.100+0000 7f3e231b7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c19d4e0 con 0x7f3e1c0fec10 2023-07-21T22:39:20.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.100+0000 7f3e231b7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c19d6a0 con 0x7f3e1c0ff5c0 2023-07-21T22:39:20.146 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.100+0000 7f3e21754700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c100050 0x7f3e1c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:20.147 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.100+0000 7f3e21754700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c100050 0x7f3e1c19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51886/0 (socket says 172.21.15.107:51886) 2023-07-21T22:39:20.147 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.100+0000 7f3e21754700 1 -- 172.21.15.107:0/3178806835 learned_addr learned my addr 172.21.15.107:0/3178806835 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:39:20.147 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.100+0000 7f3e1bfff700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e1c0ff5c0 0x7f3e1c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:20.147 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.101+0000 7f3e20f53700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c0fec10 0x7f3e1c0fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:20.147 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.101+0000 7f3e21754700 1 -- 172.21.15.107:0/3178806835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e1c0ff5c0 msgr2=0x7f3e1c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:20.147 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.101+0000 7f3e21754700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e1c0ff5c0 0x7f3e1c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.147 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.101+0000 7f3e21754700 1 -- 172.21.15.107:0/3178806835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c0fec10 msgr2=0x7f3e1c0fe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:20.147 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.101+0000 7f3e21754700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c0fec10 0x7f3e1c0fe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.148 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.101+0000 7f3e21754700 1 -- 172.21.15.107:0/3178806835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e080095f0 con 0x7f3e1c100050 2023-07-21T22:39:20.148 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.101+0000 7f3e21754700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c100050 0x7f3e1c19cba0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400ef30 tx=0x7f3e1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:20.148 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.101+0000 7f3e19ffb700 1 -- 172.21.15.107:0/3178806835 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14018070 con 0x7f3e1c100050 2023-07-21T22:39:20.148 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.101+0000 7f3e231b7700 1 -- 172.21.15.107:0/3178806835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1c19d920 con 0x7f3e1c100050 2023-07-21T22:39:20.148 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.101+0000 7f3e19ffb700 1 -- 172.21.15.107:0/3178806835 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e1400f040 con 0x7f3e1c100050 2023-07-21T22:39:20.148 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.101+0000 7f3e19ffb700 1 -- 172.21.15.107:0/3178806835 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1400cdb0 con 0x7f3e1c100050 2023-07-21T22:39:20.148 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.101+0000 7f3e231b7700 1 -- 172.21.15.107:0/3178806835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1c19de30 con 0x7f3e1c100050 2023-07-21T22:39:20.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.102+0000 7f3e231b7700 1 -- 172.21.15.107:0/3178806835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1c10a1f0 con 0x7f3e1c100050 2023-07-21T22:39:20.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.103+0000 7f3e19ffb700 1 -- 172.21.15.107:0/3178806835 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3e14003a00 con 0x7f3e1c100050 2023-07-21T22:39:20.149 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.104+0000 7f3e19ffb700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3e0c0a1150 0x7f3e0c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:20.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.104+0000 7f3e20f53700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3e0c0a1150 0x7f3e0c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:20.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.104+0000 7f3e19ffb700 1 -- 172.21.15.107:0/3178806835 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3e140cc540 con 0x7f3e1c100050 2023-07-21T22:39:20.150 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.104+0000 7f3e20f53700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3e0c0a1150 0x7f3e0c0a3600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3e10005fd0 tx=0x7f3e10005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:20.151 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.108+0000 7f3e19ffb700 1 -- 172.21.15.107:0/3178806835 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3e14095ca0 con 0x7f3e1c100050 2023-07-21T22:39:20.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:19 smithi132 ceph-mon[97863]: pgmap v601: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-07-21T22:39:20.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:19 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3028684417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:39:20.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:19 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3874284721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:39:20.386 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.343+0000 7f3e231b7700 1 -- 172.21.15.107:0/3178806835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3e1c105930 con 0x7f3e1c100050 2023-07-21T22:39:20.395 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.352+0000 7f3e19ffb700 1 -- 172.21.15.107:0/3178806835 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3e140992c0 con 0x7f3e1c100050 2023-07-21T22:39:20.397 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.354+0000 7f3e231b7700 1 -- 172.21.15.107:0/3178806835 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3e0c0a1150 msgr2=0x7f3e0c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.354+0000 7f3e231b7700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3e0c0a1150 0x7f3e0c0a3600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3e10005fd0 tx=0x7f3e10005f60 comp rx=0 tx=0).stop 2023-07-21T22:39:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.354+0000 7f3e231b7700 1 -- 172.21.15.107:0/3178806835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c100050 msgr2=0x7f3e1c19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.354+0000 7f3e231b7700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c100050 0x7f3e1c19cba0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400ef30 tx=0x7f3e1400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.354+0000 7f3e231b7700 1 -- 172.21.15.107:0/3178806835 shutdown_connections 2023-07-21T22:39:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.355+0000 7f3e231b7700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c100050 0x7f3e1c19cba0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.355+0000 7f3e231b7700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e1c0fec10 0x7f3e1c0fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.355+0000 7f3e231b7700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f3e0c0a1150 0x7f3e0c0a3600 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.355+0000 7f3e231b7700 1 --2- 172.21.15.107:0/3178806835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e1c0ff5c0 0x7f3e1c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.355+0000 7f3e231b7700 1 -- 172.21.15.107:0/3178806835 >> 172.21.15.107:0/3178806835 conn(0x7f3e1c0f9770 msgr2=0x7f3e1c0fb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.355+0000 7f3e231b7700 1 -- 172.21.15.107:0/3178806835 shutdown_connections 2023-07-21T22:39:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.355+0000 7f3e231b7700 1 -- 172.21.15.107:0/3178806835 wait complete. 2023-07-21T22:39:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-21T22:39:20.411 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-21T22:39:20.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.496+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/3146471327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ffcd0 msgr2=0x7f4ff0106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:20.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.496+0000 7f4ff4fb7700 1 --2- 172.21.15.107:0/3146471327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ffcd0 0x7f4ff0106590 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800b0a0 tx=0x7f4fe800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.496+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/3146471327 shutdown_connections 2023-07-21T22:39:20.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.496+0000 7f4ff4fb7700 1 --2- 172.21.15.107:0/3146471327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ffcd0 0x7f4ff0106590 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.540 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.496+0000 7f4ff4fb7700 1 --2- 172.21.15.107:0/3146471327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff00fe150 0x7f4ff00fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.496+0000 7f4ff4fb7700 1 --2- 172.21.15.107:0/3146471327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ff00ff340 0x7f4ff00ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.496+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/3146471327 >> 172.21.15.107:0/3146471327 conn(0x7f4ff00f9770 msgr2=0x7f4ff00fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:20.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.496+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/3146471327 shutdown_connections 2023-07-21T22:39:20.541 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.496+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/3146471327 wait complete. 2023-07-21T22:39:20.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.497+0000 7f4ff4fb7700 1 Processor -- start 2023-07-21T22:39:20.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.497+0000 7f4ff4fb7700 1 -- start start 2023-07-21T22:39:20.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.497+0000 7f4ff4fb7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ff00fe150 0x7f4ff0194800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:20.543 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.497+0000 7f4ff4fb7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ff340 0x7f4ff0194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:20.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.497+0000 7f4ff4fb7700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff00ffcd0 0x7f4ff019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:20.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.497+0000 7f4ff4fb7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff019c380 con 0x7f4ff00ff340 2023-07-21T22:39:20.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.497+0000 7f4ff4fb7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff019c4c0 con 0x7f4ff00fe150 2023-07-21T22:39:20.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.497+0000 7f4ff4fb7700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff019c650 con 0x7f4ff00ffcd0 2023-07-21T22:39:20.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.497+0000 7f4fedd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ff340 0x7f4ff0194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:20.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.497+0000 7f4fedd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ff340 0x7f4ff0194d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51920/0 (socket says 172.21.15.107:51920) 2023-07-21T22:39:20.544 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.497+0000 7f4fedd9b700 1 -- 172.21.15.107:0/2479741814 learned_addr learned my addr 172.21.15.107:0/2479741814 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:39:20.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.497+0000 7f4feed9d700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff00ffcd0 0x7f4ff019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:20.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.498+0000 7f4fee59c700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ff00fe150 0x7f4ff0194800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:20.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.498+0000 7f4fedd9b700 1 -- 172.21.15.107:0/2479741814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff00ffcd0 msgr2=0x7f4ff019bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:20.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.498+0000 7f4fedd9b700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff00ffcd0 0x7f4ff019bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.498+0000 7f4fedd9b700 1 -- 172.21.15.107:0/2479741814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ff00fe150 msgr2=0x7f4ff0194800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:20.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.498+0000 7f4fedd9b700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ff00fe150 0x7f4ff0194800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:20.545 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.498+0000 7f4fedd9b700 1 -- 172.21.15.107:0/2479741814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe8009d00 con 0x7f4ff00ff340 2023-07-21T22:39:20.546 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.498+0000 7f4fee59c700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ff00fe150 0x7f4ff0194800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:39:20.546 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.498+0000 7f4fedd9b700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ff340 0x7f4ff0194d40 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000e9a0 tx=0x7f4fe000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:20.546 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.498+0000 7f4fdf7fe700 1 -- 172.21.15.107:0/2479741814 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0009790 con 0x7f4ff00ff340 2023-07-21T22:39:20.546 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.498+0000 7f4fdf7fe700 1 -- 172.21.15.107:0/2479741814 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fe000cbd0 con 0x7f4ff00ff340 2023-07-21T22:39:20.546 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.498+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/2479741814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff019c930 con 0x7f4ff00ff340 2023-07-21T22:39:20.546 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.498+0000 7f4fdf7fe700 1 -- 172.21.15.107:0/2479741814 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0018680 con 0x7f4ff00ff340 2023-07-21T22:39:20.546 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.498+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/2479741814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff019ce40 con 0x7f4ff00ff340 2023-07-21T22:39:20.546 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.499+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/2479741814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff010a1b0 con 0x7f4ff00ff340 2023-07-21T22:39:20.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.500+0000 7f4fdf7fe700 1 -- 172.21.15.107:0/2479741814 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4fe000cd40 con 0x7f4ff00ff340 2023-07-21T22:39:20.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.501+0000 7f4fdf7fe700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4fd80a1150 0x7f4fd80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:20.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.501+0000 7f4fdf7fe700 1 -- 172.21.15.107:0/2479741814 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8699+0+0 (secure 0 0 0) 0x7f4fe0014070 con 0x7f4ff00ff340 2023-07-21T22:39:20.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.501+0000 7f4fee59c700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4fd80a1150 0x7f4fd80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:20.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.501+0000 7f4fee59c700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4fd80a1150 0x7f4fd80a3600 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00ff1a0 tx=0x7f4fe4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:20.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.504+0000 7f4fdf7fe700 1 -- 172.21.15.107:0/2479741814 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4fe0096590 con 0x7f4ff00ff340 2023-07-21T22:39:20.684 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-a[96519]: 2023-07-21T22:39:20.344+0000 7f3b40cda700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T22:39:20.685 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-a[131224]: ignoring --setuser ceph since I am not root 2023-07-21T22:39:20.685 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-a[131224]: ignoring --setgroup ceph since I am not root 2023-07-21T22:39:20.685 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-mds[131248]: -- 172.21.15.107:0/1810144994 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cffdefd1e0 con 0x55cffded3000 2023-07-21T22:39:20.685 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-a[131224]: starting mds.a at 2023-07-21T22:39:20.685 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-g[130839]: ignoring --setuser ceph since I am not root 2023-07-21T22:39:20.686 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-g[130839]: ignoring --setgroup ceph since I am not root 2023-07-21T22:39:20.686 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-mds[130863]: -- 172.21.15.107:0/4176641845 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5655080991e0 con 0x56550806f000 2023-07-21T22:39:20.686 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-g[130839]: starting mds.g at 2023-07-21T22:39:20.771 INFO:journalctl@ceph.mds.i.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-i[124538]: ignoring --setuser ceph since I am not root 2023-07-21T22:39:20.772 INFO:journalctl@ceph.mds.i.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-i[124538]: ignoring --setgroup ceph since I am not root 2023-07-21T22:39:20.772 INFO:journalctl@ceph.mds.i.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-mds[124562]: -- 172.21.15.190:0/906577607 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b6c6c291e0 con 0x55b6c6bfec00 2023-07-21T22:39:20.772 INFO:journalctl@ceph.mds.i.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-i[124538]: starting mds.i at 2023-07-21T22:39:20.772 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-c[124200]: ignoring --setuser ceph since I am not root 2023-07-21T22:39:20.773 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-c[124200]: ignoring --setgroup ceph since I am not root 2023-07-21T22:39:20.773 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-mds[124224]: -- 172.21.15.190:0/3378749580 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563cfd2131e0 con 0x563cfd1e8c00 2023-07-21T22:39:20.773 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-c[124200]: starting mds.c at 2023-07-21T22:39:20.773 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: ignoring --setuser ceph since I am not root 2023-07-21T22:39:20.773 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: ignoring --setgroup ceph since I am not root 2023-07-21T22:39:20.774 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-mds[123882]: -- 172.21.15.190:0/1817848250 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5560830091e0 con 0x556082fdf000 2023-07-21T22:39:20.774 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: starting mds.f at 2023-07-21T22:39:20.774 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-l[123519]: ignoring --setuser ceph since I am not root 2023-07-21T22:39:20.774 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-l[123519]: ignoring --setgroup ceph since I am not root 2023-07-21T22:39:20.774 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-mds[123543]: -- 172.21.15.190:0/2139787059 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5638a1c2b1e0 con 0x5638a1c01000 2023-07-21T22:39:20.774 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-l[123519]: starting mds.l at 2023-07-21T22:39:20.775 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-e[122519]: ignoring --setuser ceph since I am not root 2023-07-21T22:39:20.775 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-e[122519]: ignoring --setgroup ceph since I am not root 2023-07-21T22:39:20.775 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-mds[122543]: -- 172.21.15.132:0/2079314134 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56079bdd11e0 con 0x56079bda6800 2023-07-21T22:39:20.775 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-e[122519]: starting mds.e at 2023-07-21T22:39:20.776 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-h[125464]: ignoring --setuser ceph since I am not root 2023-07-21T22:39:20.776 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-h[125464]: ignoring --setgroup ceph since I am not root 2023-07-21T22:39:20.776 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-mds[125488]: -- 172.21.15.132:0/3208868989 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d74b00f1e0 con 0x55d74afe4800 2023-07-21T22:39:20.776 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-h[125464]: starting mds.h at 2023-07-21T22:39:20.776 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-k[122181]: ignoring --setuser ceph since I am not root 2023-07-21T22:39:20.777 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-k[122181]: ignoring --setgroup ceph since I am not root 2023-07-21T22:39:20.777 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-mds[122205]: -- 172.21.15.132:0/3545828442 <== mon.1 v2:172.21.15.190:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5563474491e0 con 0x55634741e800 2023-07-21T22:39:20.777 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-k[122181]: starting mds.k at 2023-07-21T22:39:20.777 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-b[123210]: ignoring --setuser ceph since I am not root 2023-07-21T22:39:20.777 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-b[123210]: ignoring --setgroup ceph since I am not root 2023-07-21T22:39:20.777 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-b[123210]: starting mds.b at 2023-07-21T22:39:20.783 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:20.740+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/2479741814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4ff0105000 con 0x7f4ff00ff340 2023-07-21T22:39:21.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3195339910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:39:21.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3178806835' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T22:39:21.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-mon[98334]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-21T22:39:21.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-mon[98334]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T22:39:21.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-mon[98334]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-07-21T22:39:21.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-mon[98334]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T22:39:21.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/3178806835' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-21T22:39:21.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:20 smithi190 ceph-mon[98334]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-21T22:39:21.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3195339910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:39:21.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3178806835' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T22:39:21.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-mon[97863]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-21T22:39:21.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-mon[97863]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T22:39:21.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-mon[97863]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-07-21T22:39:21.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-mon[97863]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T22:39:21.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/3178806835' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-21T22:39:21.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:20 smithi132 ceph-mon[97863]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-21T22:39:21.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3195339910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:39:21.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3178806835' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T22:39:21.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-mon[96543]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-21T22:39:21.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-mon[96543]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T22:39:21.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-mon[96543]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-07-21T22:39:21.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-mon[96543]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T22:39:21.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/3178806835' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-21T22:39:21.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:20 smithi107 ceph-mon[96543]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-21T22:39:21.394 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.351+0000 7f4fdf7fe700 1 -- 172.21.15.107:0/2479741814 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4fe0094e60 con 0x7f4ff00ff340 2023-07-21T22:39:21.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.356+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/2479741814 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4fd80a1150 msgr2=0x7f4fd80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:21.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.356+0000 7f4ff4fb7700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4fd80a1150 0x7f4fd80a3600 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00ff1a0 tx=0x7f4fe4009260 comp rx=0 tx=0).stop 2023-07-21T22:39:21.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.356+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/2479741814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ff340 msgr2=0x7f4ff0194d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:21.400 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.356+0000 7f4ff4fb7700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ff340 0x7f4ff0194d40 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000e9a0 tx=0x7f4fe000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:39:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.356+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/2479741814 shutdown_connections 2023-07-21T22:39:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.356+0000 7f4ff4fb7700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7f4fd80a1150 0x7f4fd80a3600 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.356+0000 7f4ff4fb7700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ff00ffcd0 0x7f4ff019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.356+0000 7f4ff4fb7700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ff00fe150 0x7f4ff0194800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.356+0000 7f4ff4fb7700 1 --2- 172.21.15.107:0/2479741814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff00ff340 0x7f4ff0194d40 secure :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000e9a0 tx=0x7f4fe000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:39:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.356+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/2479741814 >> 172.21.15.107:0/2479741814 conn(0x7f4ff00f9770 msgr2=0x7f4ff00fb980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:21.401 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.357+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/2479741814 shutdown_connections 2023-07-21T22:39:21.402 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.357+0000 7f4ff4fb7700 1 -- 172.21.15.107:0/2479741814 wait complete. 2023-07-21T22:39:21.416 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-21T22:39:21.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.504+0000 7feb39cdf700 1 -- 172.21.15.107:0/2733334605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb340fe9e0 msgr2=0x7feb340fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:21.547 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.504+0000 7feb39cdf700 1 --2- 172.21.15.107:0/2733334605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb340fe9e0 0x7feb340fedf0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009990 tx=0x7feb1c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:39:21.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.504+0000 7feb39cdf700 1 -- 172.21.15.107:0/2733334605 shutdown_connections 2023-07-21T22:39:21.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.504+0000 7feb39cdf700 1 --2- 172.21.15.107:0/2733334605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb34100560 0x7feb34106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:21.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.504+0000 7feb39cdf700 1 --2- 172.21.15.107:0/2733334605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb340fe9e0 0x7feb340fedf0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:21.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.504+0000 7feb39cdf700 1 --2- 172.21.15.107:0/2733334605 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb340ffbd0 0x7feb34100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:21.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.504+0000 7feb39cdf700 1 -- 172.21.15.107:0/2733334605 >> 172.21.15.107:0/2733334605 conn(0x7feb340fa000 msgr2=0x7feb340fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:21.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.504+0000 7feb39cdf700 1 -- 172.21.15.107:0/2733334605 shutdown_connections 2023-07-21T22:39:21.548 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.504+0000 7feb39cdf700 1 -- 172.21.15.107:0/2733334605 wait complete. 2023-07-21T22:39:21.549 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.504+0000 7feb39cdf700 1 Processor -- start 2023-07-21T22:39:21.550 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb39cdf700 1 -- start start 2023-07-21T22:39:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb39cdf700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb340fe9e0 0x7feb34195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb39cdf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb340ffbd0 0x7feb34195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb39cdf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb34100560 0x7feb3419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb39cdf700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3419d3c0 con 0x7feb34100560 2023-07-21T22:39:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb39cdf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3419d500 con 0x7feb340ffbd0 2023-07-21T22:39:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb39cdf700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3419d690 con 0x7feb340fe9e0 2023-07-21T22:39:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb33fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb34100560 0x7feb3419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:21.552 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb33fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb34100560 0x7feb3419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51940/0 (socket says 172.21.15.107:51940) 2023-07-21T22:39:21.552 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb33fff700 1 -- 172.21.15.107:0/642923049 learned_addr learned my addr 172.21.15.107:0/642923049 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:39:21.552 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb337fe700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb340fe9e0 0x7feb34195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:21.552 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb32ffd700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb340ffbd0 0x7feb34195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:21.552 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb33fff700 1 -- 172.21.15.107:0/642923049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb340fe9e0 msgr2=0x7feb34195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:21.552 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb33fff700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb340fe9e0 0x7feb34195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:21.552 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb33fff700 1 -- 172.21.15.107:0/642923049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb340ffbd0 msgr2=0x7feb34195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:21.552 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb33fff700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb340ffbd0 0x7feb34195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:21.553 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb33fff700 1 -- 172.21.15.107:0/642923049 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1c0095f0 con 0x7feb34100560 2023-07-21T22:39:21.553 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.505+0000 7feb33fff700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb34100560 0x7feb3419cdf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7feb2c007ee0 tx=0x7feb2c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:21.553 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.506+0000 7feb30ff9700 1 -- 172.21.15.107:0/642923049 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c00ddd0 con 0x7feb34100560 2023-07-21T22:39:21.553 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.506+0000 7feb30ff9700 1 -- 172.21.15.107:0/642923049 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb2c00f040 con 0x7feb34100560 2023-07-21T22:39:21.553 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.506+0000 7feb39cdf700 1 -- 172.21.15.107:0/642923049 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3419d970 con 0x7feb34100560 2023-07-21T22:39:21.553 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.506+0000 7feb30ff9700 1 -- 172.21.15.107:0/642923049 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0048e0 con 0x7feb34100560 2023-07-21T22:39:21.553 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.506+0000 7feb39cdf700 1 -- 172.21.15.107:0/642923049 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3419de30 con 0x7feb34100560 2023-07-21T22:39:21.554 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.506+0000 7feb39cdf700 1 -- 172.21.15.107:0/642923049 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb34105960 con 0x7feb34100560 2023-07-21T22:39:21.554 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.507+0000 7feb30ff9700 1 -- 172.21.15.107:0/642923049 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7feb2c003a00 con 0x7feb34100560 2023-07-21T22:39:21.554 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.508+0000 7feb30ff9700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7feb200a1150 0x7feb200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:21.554 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.508+0000 7feb30ff9700 1 -- 172.21.15.107:0/642923049 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8699+0+0 (secure 0 0 0) 0x7feb2c0cc430 con 0x7feb34100560 2023-07-21T22:39:21.555 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.508+0000 7feb337fe700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7feb200a1150 0x7feb200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:21.555 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.508+0000 7feb337fe700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7feb200a1150 0x7feb200a3600 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00b5c0 tx=0x7feb1c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:21.555 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.512+0000 7feb30ff9700 1 -- 172.21.15.107:0/642923049 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7feb2c04e770 con 0x7feb34100560 2023-07-21T22:39:21.744 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:21.701+0000 7feb39cdf700 1 -- 172.21.15.107:0/642923049 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feb3402c850 con 0x7feb34100560 2023-07-21T22:39:22.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: pgmap v602: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2023-07-21T22:39:22.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2479741814' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:39:22.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-21T22:39:22.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-21T22:39:22.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: Cluster is now healthy 2023-07-21T22:39:22.138 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.107:6838/3935525730,v1:172.21.15.107:6840/3935525730] up:boot 2023-07-21T22:39:22.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.132:6832/3885866659,v1:172.21.15.132:6834/3885866659] up:boot 2023-07-21T22:39:22.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.132:6833/3838086245,v1:172.21.15.132:6835/3838086245] up:boot 2023-07-21T22:39:22.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.190:6836/1359887428,v1:172.21.15.190:6837/1359887428] up:boot 2023-07-21T22:39:22.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.107:6839/1484651691,v1:172.21.15.107:6841/1484651691] up:boot 2023-07-21T22:39:22.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.190:6838/2836453453,v1:172.21.15.190:6839/2836453453] up:boot 2023-07-21T22:39:22.139 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.132:6836/2679483363,v1:172.21.15.132:6837/2679483363] up:boot 2023-07-21T22:39:22.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.132:6838/4077943736,v1:172.21.15.132:6839/4077943736] up:boot 2023-07-21T22:39:22.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.190:6832/3748254961,v1:172.21.15.190:6833/3748254961] up:boot 2023-07-21T22:39:22.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: mds.? [v2:172.21.15.190:6834/807692966,v1:172.21.15.190:6835/807692966] up:boot 2023-07-21T22:39:22.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/2479741814' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:39:22.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: fsmap 12 up:standby 2023-07-21T22:39:22.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:39:22.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:39:22.140 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:39:22.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:39:22.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:39:22.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:39:22.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:39:22.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:39:22.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:39:22.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:39:22.141 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:21 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/642923049' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:39:22.178 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: pgmap v602: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2023-07-21T22:39:22.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2479741814' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:39:22.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-21T22:39:22.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-21T22:39:22.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: Cluster is now healthy 2023-07-21T22:39:22.179 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.107:6838/3935525730,v1:172.21.15.107:6840/3935525730] up:boot 2023-07-21T22:39:22.180 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.132:6832/3885866659,v1:172.21.15.132:6834/3885866659] up:boot 2023-07-21T22:39:22.180 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.132:6833/3838086245,v1:172.21.15.132:6835/3838086245] up:boot 2023-07-21T22:39:22.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.190:6836/1359887428,v1:172.21.15.190:6837/1359887428] up:boot 2023-07-21T22:39:22.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.107:6839/1484651691,v1:172.21.15.107:6841/1484651691] up:boot 2023-07-21T22:39:22.181 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.190:6838/2836453453,v1:172.21.15.190:6839/2836453453] up:boot 2023-07-21T22:39:22.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.132:6836/2679483363,v1:172.21.15.132:6837/2679483363] up:boot 2023-07-21T22:39:22.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.132:6838/4077943736,v1:172.21.15.132:6839/4077943736] up:boot 2023-07-21T22:39:22.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.190:6832/3748254961,v1:172.21.15.190:6833/3748254961] up:boot 2023-07-21T22:39:22.182 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: mds.? [v2:172.21.15.190:6834/807692966,v1:172.21.15.190:6835/807692966] up:boot 2023-07-21T22:39:22.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/2479741814' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:39:22.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: fsmap 12 up:standby 2023-07-21T22:39:22.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:39:22.183 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:39:22.184 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:39:22.184 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:39:22.184 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:39:22.184 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:39:22.185 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:39:22.185 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:39:22.185 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:39:22.185 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:39:22.186 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:21 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/642923049' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:39:22.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: pgmap v602: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2023-07-21T22:39:22.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2479741814' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:39:22.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-21T22:39:22.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-21T22:39:22.187 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: Cluster is now healthy 2023-07-21T22:39:22.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.107:6838/3935525730,v1:172.21.15.107:6840/3935525730] up:boot 2023-07-21T22:39:22.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.132:6832/3885866659,v1:172.21.15.132:6834/3885866659] up:boot 2023-07-21T22:39:22.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.132:6833/3838086245,v1:172.21.15.132:6835/3838086245] up:boot 2023-07-21T22:39:22.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.190:6836/1359887428,v1:172.21.15.190:6837/1359887428] up:boot 2023-07-21T22:39:22.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.107:6839/1484651691,v1:172.21.15.107:6841/1484651691] up:boot 2023-07-21T22:39:22.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.190:6838/2836453453,v1:172.21.15.190:6839/2836453453] up:boot 2023-07-21T22:39:22.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.132:6836/2679483363,v1:172.21.15.132:6837/2679483363] up:boot 2023-07-21T22:39:22.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.132:6838/4077943736,v1:172.21.15.132:6839/4077943736] up:boot 2023-07-21T22:39:22.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.190:6832/3748254961,v1:172.21.15.190:6833/3748254961] up:boot 2023-07-21T22:39:22.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: mds.? [v2:172.21.15.190:6834/807692966,v1:172.21.15.190:6835/807692966] up:boot 2023-07-21T22:39:22.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/2479741814' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:39:22.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: fsmap 12 up:standby 2023-07-21T22:39:22.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:39:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:39:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:39:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:39:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:39:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:39:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:39:22.190 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:39:22.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:39:22.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: from='mgr.14150 172.21.15.107:0/875525258' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:39:22.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:21 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/642923049' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:39:22.496 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.452+0000 7feb30ff9700 1 -- 172.21.15.107:0/642923049 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) v1 ==== 181+0+0 (secure 0 0 0) 0x7feb2c099440 con 0x7feb34100560 2023-07-21T22:39:22.499 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.456+0000 7feb39cdf700 1 -- 172.21.15.107:0/642923049 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7feb200a1150 msgr2=0x7feb200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.456+0000 7feb39cdf700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7feb200a1150 0x7feb200a3600 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00b5c0 tx=0x7feb1c005960 comp rx=0 tx=0).stop 2023-07-21T22:39:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.456+0000 7feb39cdf700 1 -- 172.21.15.107:0/642923049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb34100560 msgr2=0x7feb3419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.456+0000 7feb39cdf700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb34100560 0x7feb3419cdf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7feb2c007ee0 tx=0x7feb2c00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:39:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.456+0000 7feb39cdf700 1 -- 172.21.15.107:0/642923049 shutdown_connections 2023-07-21T22:39:22.500 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.456+0000 7feb39cdf700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb34100560 0x7feb3419cdf0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.456+0000 7feb39cdf700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7feb200a1150 0x7feb200a3600 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.456+0000 7feb39cdf700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feb340fe9e0 0x7feb34195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.456+0000 7feb39cdf700 1 --2- 172.21.15.107:0/642923049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb340ffbd0 0x7feb34195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.456+0000 7feb39cdf700 1 -- 172.21.15.107:0/642923049 >> 172.21.15.107:0/642923049 conn(0x7feb340fa000 msgr2=0x7feb340fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:22.501 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.457+0000 7feb39cdf700 1 -- 172.21.15.107:0/642923049 shutdown_connections 2023-07-21T22:39:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.457+0000 7feb39cdf700 1 -- 172.21.15.107:0/642923049 wait complete. 2023-07-21T22:39:22.502 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_metadata' removed 2023-07-21T22:39:22.518 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-21T22:39:22.648 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.605+0000 7fbd00666700 1 -- 172.21.15.107:0/3349996784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcf80fe3c0 msgr2=0x7fbcf80fe7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.605+0000 7fbd00666700 1 --2- 172.21.15.107:0/3349996784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcf80fe3c0 0x7fbcf80fe7d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009990 tx=0x7fbcf4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:39:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.605+0000 7fbd00666700 1 -- 172.21.15.107:0/3349996784 shutdown_connections 2023-07-21T22:39:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.605+0000 7fbd00666700 1 --2- 172.21.15.107:0/3349996784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcf80ff6c0 0x7fbcf80ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.605+0000 7fbd00666700 1 --2- 172.21.15.107:0/3349996784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcf80fe3c0 0x7fbcf80fe7d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.605+0000 7fbd00666700 1 --2- 172.21.15.107:0/3349996784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcf80fed10 0x7fbcf80ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.605+0000 7fbd00666700 1 -- 172.21.15.107:0/3349996784 >> 172.21.15.107:0/3349996784 conn(0x7fbcf80fa030 msgr2=0x7fbcf80fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.605+0000 7fbd00666700 1 -- 172.21.15.107:0/3349996784 shutdown_connections 2023-07-21T22:39:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.605+0000 7fbd00666700 1 -- 172.21.15.107:0/3349996784 wait complete. 2023-07-21T22:39:22.651 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.606+0000 7fbd00666700 1 Processor -- start 2023-07-21T22:39:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.606+0000 7fbd00666700 1 -- start start 2023-07-21T22:39:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.606+0000 7fbd00666700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcf80fe3c0 0x7fbcf81979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.606+0000 7fbd00666700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcf80fed10 0x7fbcf8197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.606+0000 7fbd00666700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcf80ff6c0 0x7fbcf819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.606+0000 7fbd00666700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf819f570 con 0x7fbcf80fe3c0 2023-07-21T22:39:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.606+0000 7fbd00666700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf819f6b0 con 0x7fbcf80fed10 2023-07-21T22:39:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.606+0000 7fbd00666700 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf819f840 con 0x7fbcf80ff6c0 2023-07-21T22:39:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.606+0000 7fbcfe402700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcf80fe3c0 0x7fbcf81979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.606+0000 7fbcfe402700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcf80fe3c0 0x7fbcf81979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51954/0 (socket says 172.21.15.107:51954) 2023-07-21T22:39:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.606+0000 7fbcfec03700 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcf80ff6c0 0x7fbcf819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.606+0000 7fbcfe402700 1 -- 172.21.15.107:0/4279001406 learned_addr learned my addr 172.21.15.107:0/4279001406 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-21T22:39:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.606+0000 7fbcfdc01700 1 --2- 172.21.15.107:0/4279001406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcf80fed10 0x7fbcf8197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.607+0000 7fbcfe402700 1 -- 172.21.15.107:0/4279001406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcf80ff6c0 msgr2=0x7fbcf819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.607+0000 7fbcfe402700 1 --2- 172.21.15.107:0/4279001406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcf80ff6c0 0x7fbcf819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.607+0000 7fbcfe402700 1 -- 172.21.15.107:0/4279001406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcf80fed10 msgr2=0x7fbcf8197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.607+0000 7fbcfe402700 1 --2- 172.21.15.107:0/4279001406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcf80fed10 0x7fbcf8197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.607+0000 7fbcfe402700 1 -- 172.21.15.107:0/4279001406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf40095f0 con 0x7fbcf80fe3c0 2023-07-21T22:39:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.607+0000 7fbcfe402700 1 --2- 172.21.15.107:0/4279001406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcf80fe3c0 0x7fbcf81979f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40059d0 tx=0x7fbcf40047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.607+0000 7fbcef7fe700 1 -- 172.21.15.107:0/4279001406 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf401d070 con 0x7fbcf80fe3c0 2023-07-21T22:39:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.607+0000 7fbd00666700 1 -- 172.21.15.107:0/4279001406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf819fac0 con 0x7fbcf80fe3c0 2023-07-21T22:39:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.607+0000 7fbcef7fe700 1 -- 172.21.15.107:0/4279001406 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcf400fb90 con 0x7fbcf80fe3c0 2023-07-21T22:39:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.607+0000 7fbcef7fe700 1 -- 172.21.15.107:0/4279001406 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4017650 con 0x7fbcf80fe3c0 2023-07-21T22:39:22.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.607+0000 7fbd00666700 1 -- 172.21.15.107:0/4279001406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf819fee0 con 0x7fbcf80fe3c0 2023-07-21T22:39:22.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.608+0000 7fbd00666700 1 -- 172.21.15.107:0/4279001406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf81091a0 con 0x7fbcf80fe3c0 2023-07-21T22:39:22.655 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.609+0000 7fbcef7fe700 1 -- 172.21.15.107:0/4279001406 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fbcf400fd00 con 0x7fbcf80fe3c0 2023-07-21T22:39:22.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.609+0000 7fbcef7fe700 1 --2- 172.21.15.107:0/4279001406 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fbce40a1150 0x7fbce40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:39:22.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.610+0000 7fbcfdc01700 1 --2- 172.21.15.107:0/4279001406 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fbce40a1150 0x7fbce40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:39:22.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.610+0000 7fbcef7fe700 1 -- 172.21.15.107:0/4279001406 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8145+0+0 (secure 0 0 0) 0x7fbcf40ce3e0 con 0x7fbcf80fe3c0 2023-07-21T22:39:22.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.610+0000 7fbcfdc01700 1 --2- 172.21.15.107:0/4279001406 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fbce40a1150 0x7fbce40a3600 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbce8007c60 tx=0x7fbce80073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:39:22.656 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.613+0000 7fbcef7fe700 1 -- 172.21.15.107:0/4279001406 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbcf4097c60 con 0x7fbcf80fe3c0 2023-07-21T22:39:22.869 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:22.826+0000 7fbd00666700 1 -- 172.21.15.107:0/4279001406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbcf802c850 con 0x7fbcf80fe3c0 2023-07-21T22:39:23.779 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:23.736+0000 7fbcef7fe700 1 -- 172.21.15.107:0/4279001406 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbcf409b280 con 0x7fbcf80fe3c0 2023-07-21T22:39:23.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:23.741+0000 7fbd00666700 1 -- 172.21.15.107:0/4279001406 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fbce40a1150 msgr2=0x7fbce40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:23.784 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:23.741+0000 7fbd00666700 1 --2- 172.21.15.107:0/4279001406 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fbce40a1150 0x7fbce40a3600 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbce8007c60 tx=0x7fbce80073d0 comp rx=0 tx=0).stop 2023-07-21T22:39:23.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:23.741+0000 7fbd00666700 1 -- 172.21.15.107:0/4279001406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcf80fe3c0 msgr2=0x7fbcf81979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:39:23.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:23.741+0000 7fbd00666700 1 --2- 172.21.15.107:0/4279001406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcf80fe3c0 0x7fbcf81979f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40059d0 tx=0x7fbcf40047a0 comp rx=0 tx=0).stop 2023-07-21T22:39:23.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:23.741+0000 7fbd00666700 1 -- 172.21.15.107:0/4279001406 shutdown_connections 2023-07-21T22:39:23.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:23.741+0000 7fbd00666700 1 --2- 172.21.15.107:0/4279001406 >> [v2:172.21.15.107:6800/1657248266,v1:172.21.15.107:6801/1657248266] conn(0x7fbce40a1150 0x7fbce40a3600 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:23.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:23.741+0000 7fbd00666700 1 --2- 172.21.15.107:0/4279001406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcf80ff6c0 0x7fbcf819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:23.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:23.741+0000 7fbd00666700 1 --2- 172.21.15.107:0/4279001406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcf80fe3c0 0x7fbcf81979f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:23.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:23.741+0000 7fbd00666700 1 --2- 172.21.15.107:0/4279001406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcf80fed10 0x7fbcf8197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:39:23.785 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:23.741+0000 7fbd00666700 1 -- 172.21.15.107:0/4279001406 >> 172.21.15.107:0/4279001406 conn(0x7fbcf80fa030 msgr2=0x7fbcf80fbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:39:23.786 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:23.741+0000 7fbd00666700 1 -- 172.21.15.107:0/4279001406 shutdown_connections 2023-07-21T22:39:23.786 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-21T22:39:23.741+0000 7fbd00666700 1 -- 172.21.15.107:0/4279001406 wait complete. 2023-07-21T22:39:23.786 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data' removed 2023-07-21T22:39:23.803 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:23 smithi107 ceph-mon[96543]: pgmap v604: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T22:39:23.803 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:23 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/642923049' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T22:39:23.803 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:23 smithi107 ceph-mon[96543]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T22:39:23.803 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:23 smithi107 ceph-mon[96543]: from='client.? 172.21.15.107:0/4279001406' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:39:23.804 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:39:23.832 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:39:23.857 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:23 smithi132 ceph-mon[97863]: pgmap v604: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T22:39:23.858 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:23 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/642923049' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T22:39:23.858 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:23 smithi132 ceph-mon[97863]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T22:39:23.858 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:23 smithi132 ceph-mon[97863]: from='client.? 172.21.15.107:0/4279001406' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:39:23.862 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:39:23.887 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:23 smithi190 ceph-mon[98334]: pgmap v604: 97 pgs: 97 active+clean; 766 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-21T22:39:23.887 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:23 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/642923049' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T22:39:23.887 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:23 smithi190 ceph-mon[98334]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T22:39:23.887 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:23 smithi190 ceph-mon[98334]: from='client.? 172.21.15.107:0/4279001406' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:39:23.893 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-21T22:39:23.893 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T22:39:23.909 DEBUG:teuthology.orchestra.run.smithi132:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T22:39:23.925 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T22:39:23.949 INFO:tasks.cephadm:Stopping all daemons... 2023-07-21T22:39:23.950 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-21T22:39:23.950 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.a 2023-07-21T22:39:24.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:23 smithi107 systemd[1]: Stopping Ceph mon.a for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:39:24.663 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:24 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-a[96519]: 2023-07-21T22:39:24.330+0000 7f3b464e5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:24.663 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:24 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-a[96519]: 2023-07-21T22:39:24.330+0000 7f3b464e5700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-21T22:39:24.909 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.a.service' 2023-07-21T22:39:24.934 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:24 smithi107 bash[157959]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-a 2023-07-21T22:39:24.934 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:24 smithi107 systemd[1]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.a.service: Succeeded. 2023-07-21T22:39:24.934 INFO:journalctl@ceph.mon.a.smithi107.stdout:Jul 21 22:39:24 smithi107 systemd[1]: Stopped Ceph mon.a for af99f5d0-2814-11ee-9b34-001a4aab830c. 2023-07-21T22:39:24.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:24.958 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-21T22:39:24.959 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-21T22:39:24.959 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.b 2023-07-21T22:39:25.325 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:24 smithi132 systemd[1]: Stopping Ceph mon.b for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:39:25.626 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:25 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-b[97839]: 2023-07-21T22:39:25.282+0000 7f72130ef700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:25.627 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:25 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-b[97839]: 2023-07-21T22:39:25.282+0000 7f72130ef700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-21T22:39:25.928 INFO:journalctl@ceph.mon.b.smithi132.stdout:Jul 21 22:39:25 smithi132 bash[127972]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-b 2023-07-21T22:39:26.035 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.b.service' 2023-07-21T22:39:26.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:26.086 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-21T22:39:26.086 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-21T22:39:26.087 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.c 2023-07-21T22:39:26.387 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:26 smithi190 systemd[1]: Stopping Ceph mon.c for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:39:27.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:26 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-c[98309]: 2023-07-21T22:39:26.673+0000 7fea37fc0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:27.137 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:26 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-c[98309]: 2023-07-21T22:39:26.673+0000 7fea37fc0700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-21T22:39:27.524 INFO:journalctl@ceph.mon.c.smithi190.stdout:Jul 21 22:39:27 smithi190 bash[129437]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mon-c 2023-07-21T22:39:27.531 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mon.c.service' 2023-07-21T22:39:27.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:27.587 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-21T22:39:27.587 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-21T22:39:27.588 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.x 2023-07-21T22:39:27.934 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:39:27 smithi107 systemd[1]: Stopping Ceph mgr.x for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:39:28.262 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:39:27 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:39:27.895+0000 7f936a294700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:28.263 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Jul 21 22:39:27 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-x[96785]: 2023-07-21T22:39:27.896+0000 7f936a294700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-07-21T22:39:28.475 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.x.service' 2023-07-21T22:39:28.510 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:28.511 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-21T22:39:28.511 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-21T22:39:28.511 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.y 2023-07-21T22:39:28.928 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:39:28 smithi132 systemd[1]: Stopping Ceph mgr.y for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:39:29.223 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Jul 21 22:39:29 smithi132 bash[128139]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-y 2023-07-21T22:39:29.408 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.y.service' 2023-07-21T22:39:29.446 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:29.447 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-21T22:39:29.447 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-21T22:39:29.447 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.z 2023-07-21T22:39:29.874 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:39:29 smithi190 systemd[1]: Stopping Ceph mgr.z for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:39:30.137 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:39:29 smithi190 bash[129594]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mgr-z 2023-07-21T22:39:30.137 INFO:journalctl@ceph.mgr.z.smithi190.stdout:Jul 21 22:39:30 smithi190 systemd[1]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-07-21T22:39:30.238 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mgr.z.service' 2023-07-21T22:39:30.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:30.275 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-21T22:39:30.275 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-21T22:39:30.275 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.0 2023-07-21T22:39:30.630 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:39:30 smithi107 systemd[1]: Stopping Ceph osd.0 for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:39:30.934 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:39:30 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-0[109479]: 2023-07-21T22:39:30.587+0000 7f1d5e806700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:30.934 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:39:30 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-0[109479]: 2023-07-21T22:39:30.587+0000 7f1d5e806700 -1 osd.0 88 *** Got signal Terminated *** 2023-07-21T22:39:30.935 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:39:30 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-0[109479]: 2023-07-21T22:39:30.587+0000 7f1d5e806700 -1 osd.0 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:39:36.684 INFO:journalctl@ceph.osd.0.smithi107.stdout:Jul 21 22:39:36 smithi107 bash[158243]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-0 2023-07-21T22:39:37.754 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.0.service' 2023-07-21T22:39:37.795 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:37.796 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-21T22:39:37.796 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-21T22:39:37.797 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.1 2023-07-21T22:39:38.141 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:39:37 smithi132 systemd[1]: Stopping Ceph osd.1 for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:39:38.428 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:39:38 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-1[102848]: 2023-07-21T22:39:38.098+0000 7f57fe28c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:38.428 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:39:38 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-1[102848]: 2023-07-21T22:39:38.098+0000 7f57fe28c700 -1 osd.1 88 *** Got signal Terminated *** 2023-07-21T22:39:38.428 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:39:38 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-1[102848]: 2023-07-21T22:39:38.098+0000 7f57fe28c700 -1 osd.1 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:39:43.927 INFO:journalctl@ceph.osd.1.smithi132.stdout:Jul 21 22:39:43 smithi132 bash[128278]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-1 2023-07-21T22:39:44.989 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.1.service' 2023-07-21T22:39:45.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:45.029 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-21T22:39:45.029 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-21T22:39:45.030 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.2 2023-07-21T22:39:45.387 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:39:45 smithi190 systemd[1]: Stopping Ceph osd.2 for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:39:45.887 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:39:45 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-2[104149]: 2023-07-21T22:39:45.344+0000 7f1f80456700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:45.887 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:39:45 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-2[104149]: 2023-07-21T22:39:45.344+0000 7f1f80456700 -1 osd.2 88 *** Got signal Terminated *** 2023-07-21T22:39:45.887 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:39:45 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-2[104149]: 2023-07-21T22:39:45.344+0000 7f1f80456700 -1 osd.2 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:39:51.269 INFO:journalctl@ceph.osd.2.smithi190.stdout:Jul 21 22:39:51 smithi190 bash[129748]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-2 2023-07-21T22:39:52.533 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.2.service' 2023-07-21T22:39:52.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:52.574 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-21T22:39:52.574 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-21T22:39:52.575 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.3 2023-07-21T22:39:52.934 INFO:journalctl@ceph.osd.3.smithi107.stdout:Jul 21 22:39:52 smithi107 systemd[1]: Stopping Ceph osd.3 for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:39:53.434 INFO:journalctl@ceph.osd.3.smithi107.stdout:Jul 21 22:39:52 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-3[114835]: 2023-07-21T22:39:52.901+0000 7f92bd2de700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:53.434 INFO:journalctl@ceph.osd.3.smithi107.stdout:Jul 21 22:39:52 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-3[114835]: 2023-07-21T22:39:52.901+0000 7f92bd2de700 -1 osd.3 88 *** Got signal Terminated *** 2023-07-21T22:39:53.434 INFO:journalctl@ceph.osd.3.smithi107.stdout:Jul 21 22:39:52 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-3[114835]: 2023-07-21T22:39:52.901+0000 7f92bd2de700 -1 osd.3 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:39:56.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:39:56 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:39:56.446+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:39:57.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:39:57 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:39:57.263+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:39:57.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:39:57 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:39:57.470+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:39:57.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:39:57 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:39:57.389+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:39:57.886 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:39:57 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:39:57.579+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:39:58.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:39:58 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:39:58.260+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:39:58.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:39:58 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:39:58.340+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:39:58.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:39:58 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:39:58.519+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:39:58.887 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:39:58 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:39:58.574+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:39:58.887 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:39:58 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:39:58.378+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:39:58.888 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:39:58 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:39:58.485+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:39:59.183 INFO:journalctl@ceph.osd.3.smithi107.stdout:Jul 21 22:39:58 smithi107 bash[158540]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-3 2023-07-21T22:39:59.652 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:39:59 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:39:59.330+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:39:59.652 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:39:59 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:39:59.471+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:39:59.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:39:59 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:39:59.233+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:39:59.886 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:39:59 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:39:59.421+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:39:59.887 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:39:59 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:39:59.603+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:39:59.887 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:39:59 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:39:59.527+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:39:59.978 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.3.service' 2023-07-21T22:40:00.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:00.018 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-21T22:40:00.018 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-21T22:40:00.018 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.4 2023-07-21T22:40:00.371 INFO:journalctl@ceph.osd.4.smithi132.stdout:Jul 21 22:40:00 smithi132 systemd[1]: Stopping Ceph osd.4 for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:40:00.371 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:00 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:00.186+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:00.677 INFO:journalctl@ceph.osd.4.smithi132.stdout:Jul 21 22:40:00 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-4[108301]: 2023-07-21T22:40:00.328+0000 7f575119a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:00.678 INFO:journalctl@ceph.osd.4.smithi132.stdout:Jul 21 22:40:00 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-4[108301]: 2023-07-21T22:40:00.328+0000 7f575119a700 -1 osd.4 88 *** Got signal Terminated *** 2023-07-21T22:40:00.678 INFO:journalctl@ceph.osd.4.smithi132.stdout:Jul 21 22:40:00 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-4[108301]: 2023-07-21T22:40:00.328+0000 7f575119a700 -1 osd.4 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:40:00.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:00 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:00.321+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:00.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:00 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:00.431+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:00.886 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:00 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:00.445+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:40:00.887 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:00 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:00.573+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:00.887 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:00 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:00.614+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:01.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:01 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:01.171+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:01.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:01 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:01.349+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:01.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:01 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:01.437+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:01.886 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:01 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:01.440+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:40:01.887 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:01 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:01.611+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:01.887 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:01 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:01.621+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:02.427 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:02 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:02.132+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:02.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:02 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:02.374+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:02.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:02 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:02.409+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:02.886 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:02 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:02.392+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:40:02.887 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:02 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:02.585+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:02.887 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:02 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:02.567+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:03.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:03 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:03.170+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:03.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:03 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:03.354+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:03.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:03 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:03.433+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:03.886 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:03 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:03.362+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:40:03.887 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:03 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:03.585+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:03.887 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:03 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:03.599+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:04.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:04 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:04.203+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:04.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:04 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:04.389+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:04.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:04 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:04.456+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:04.687 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:04 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:04.558+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:04.688 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:04 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:04.374+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:40:05.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:04 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:04.646+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:05.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:05 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:05.213+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:05.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:05 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:05.376+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:05.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:05 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:05.436+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:05.722 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:05 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:05.534+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:05.723 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:05 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:05.371+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:40:06.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:05 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:05.680+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:06.258 INFO:journalctl@ceph.osd.4.smithi132.stdout:Jul 21 22:40:06 smithi132 bash[128555]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-4 2023-07-21T22:40:06.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:06 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:06.227+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:06.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:06 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:06.227+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:06.765 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:06 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:06.549+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:06.766 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:06 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:06.394+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:40:06.934 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:06 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:06.395+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:06.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:06 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:06.410+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:07.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:06 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:06.723+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:07.531 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:07 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:07.257+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:07.531 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:07 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:07.218+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:07.532 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:07 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:07.218+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:07.538 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.4.service' 2023-07-21T22:40:07.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:07.578 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-21T22:40:07.579 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-21T22:40:07.579 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.5 2023-07-21T22:40:07.815 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:07 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:07.537+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:07.816 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:07 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:07.350+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:40:07.816 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:07 smithi190 systemd[1]: Stopping Ceph osd.5 for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:40:07.934 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:07 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:07.437+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:07.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:07 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:07.392+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:07.935 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:07 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:07.392+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:08.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:07 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:07.773+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:08.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:07 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:07.773+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:08.137 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:07 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:07.900+0000 7f5685bea700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:08.137 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:07 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:07.900+0000 7f5685bea700 -1 osd.5 88 *** Got signal Terminated *** 2023-07-21T22:40:08.137 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:07 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:07.900+0000 7f5685bea700 -1 osd.5 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:40:08.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:08 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:08.275+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:08.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:08 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:08.207+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:08.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:08 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:08.207+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:08.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:08 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:08.472+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:08.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:08 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:08.472+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.576190+0000 front 2023-07-21T22:39:41.576381+0000 (oldest deadline 2023-07-21T22:40:07.476375+0000) 2023-07-21T22:40:08.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:08 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:08.383+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:08.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:08 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:08.383+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:08.773 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:08 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:08.497+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:08.774 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:08 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:08.389+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:40:08.774 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:08 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:08.389+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.736525+0000 front 2023-07-21T22:39:42.736671+0000 (oldest deadline 2023-07-21T22:40:07.436626+0000) 2023-07-21T22:40:09.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:08 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:08.732+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:09.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:08 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:08.732+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:09.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:09 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:09.292+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:09.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:09 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:09.202+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:09.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:09 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:09.202+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:09.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:09 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:09.488+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:09.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:09 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:09.488+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.576190+0000 front 2023-07-21T22:39:41.576381+0000 (oldest deadline 2023-07-21T22:40:07.476375+0000) 2023-07-21T22:40:09.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:09 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:09.349+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:09.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:09 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:09.349+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:09.780 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:09 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:09.466+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:09.780 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:09 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:09.466+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:09.781 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:09 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:09.377+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:40:09.781 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:09 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:09.377+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.736525+0000 front 2023-07-21T22:39:42.736671+0000 (oldest deadline 2023-07-21T22:40:07.436626+0000) 2023-07-21T22:40:10.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:09 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:09.738+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:10.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:09 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:09.738+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:10.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:10 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:10.325+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:10.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:10 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:10.208+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:10.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:10 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:10.208+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:10.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:10 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:10.494+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:10.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:10 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:10.494+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.576190+0000 front 2023-07-21T22:39:41.576381+0000 (oldest deadline 2023-07-21T22:40:07.476375+0000) 2023-07-21T22:40:10.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:10 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:10.335+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:10.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:10 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:10.335+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:10.769 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:10 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:10.472+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:10.769 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:10 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:10.472+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:10.770 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:10 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:10.379+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:40:10.770 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:10 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:10.379+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.736525+0000 front 2023-07-21T22:39:42.736671+0000 (oldest deadline 2023-07-21T22:40:07.436626+0000) 2023-07-21T22:40:11.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:10 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:10.727+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:11.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:10 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:10.727+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:11.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:11 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:11.334+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:11.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:11 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:11.197+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:11.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:11 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:11.197+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:11.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:11 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:11.197+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:11.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:11 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:11.326+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:11.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:11 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:11.326+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:11.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:11 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:11.461+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:11.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:11 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:11.461+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.576190+0000 front 2023-07-21T22:39:41.576381+0000 (oldest deadline 2023-07-21T22:40:07.476375+0000) 2023-07-21T22:40:11.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:11 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:11.461+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.476831+0000 front 2023-07-21T22:39:47.476952+0000 (oldest deadline 2023-07-21T22:40:10.976914+0000) 2023-07-21T22:40:11.797 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:11 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:11.428+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:11.797 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:11 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:11.428+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:11.798 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:11 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:11.358+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:40:11.798 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:11 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:11.358+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.736525+0000 front 2023-07-21T22:39:42.736671+0000 (oldest deadline 2023-07-21T22:40:07.436626+0000) 2023-07-21T22:40:12.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:11 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:11.755+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:12.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:11 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:11.755+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:12.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:12 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:12.309+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:12.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:12 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:12.188+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:12.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:12 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:12.188+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:12.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:12 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:12.188+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:12.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:12 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:12.366+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:12.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:12 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:12.366+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:12.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:12 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:12.488+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:12.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:12 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:12.488+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.576190+0000 front 2023-07-21T22:39:41.576381+0000 (oldest deadline 2023-07-21T22:40:07.476375+0000) 2023-07-21T22:40:12.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:12 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:12.488+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.476831+0000 front 2023-07-21T22:39:47.476952+0000 (oldest deadline 2023-07-21T22:40:10.976914+0000) 2023-07-21T22:40:12.779 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:12 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:12.398+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.234715+0000 front 2023-07-21T22:39:32.234695+0000 (oldest deadline 2023-07-21T22:39:57.534765+0000) 2023-07-21T22:40:12.779 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:12 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5[109671]: 2023-07-21T22:40:12.398+0000 7f56801dc700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.736525+0000 front 2023-07-21T22:39:42.736671+0000 (oldest deadline 2023-07-21T22:40:07.436626+0000) 2023-07-21T22:40:12.779 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:12 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:12.469+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:12.779 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:12 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:12.469+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:13.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:12 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:12.736+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:13.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:12 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:12.736+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:13.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:12 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:12.736+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:13.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:13 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:13.303+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:13.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:13 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:13.186+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:13.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:13 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:13.186+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:13.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:13 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:13.186+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:13.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:13 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:13.477+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:13.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:13 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:13.477+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.576190+0000 front 2023-07-21T22:39:41.576381+0000 (oldest deadline 2023-07-21T22:40:07.476375+0000) 2023-07-21T22:40:13.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:13 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:13.477+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.476831+0000 front 2023-07-21T22:39:47.476952+0000 (oldest deadline 2023-07-21T22:40:10.976914+0000) 2023-07-21T22:40:13.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:13 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:13.352+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:13.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:13 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:13.352+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:13.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:13 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:13.352+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:13.716 INFO:journalctl@ceph.osd.5.smithi190.stdout:Jul 21 22:40:13 smithi190 bash[130010]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-5 2023-07-21T22:40:13.717 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:13 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:13.445+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:13.717 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:13 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:13.445+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:13.717 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:13 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:13.445+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:14.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:13 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:13.778+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:14.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:13 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:13.778+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:14.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:13 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:13.778+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:14.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:14 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:14.317+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:14.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:14 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:14.201+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:14.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:14 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:14.201+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:14.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:14 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:14.201+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:14.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:14 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:14.428+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:14.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:14 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:14.428+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.576190+0000 front 2023-07-21T22:39:41.576381+0000 (oldest deadline 2023-07-21T22:40:07.476375+0000) 2023-07-21T22:40:14.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:14 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:14.428+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.476831+0000 front 2023-07-21T22:39:47.476952+0000 (oldest deadline 2023-07-21T22:40:10.976914+0000) 2023-07-21T22:40:14.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:14 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:14.333+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:14.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:14 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:14.333+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:14.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:14 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:14.333+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:14.809 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:14 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:14.483+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:14.809 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:14 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:14.483+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:14.810 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:14 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:14.483+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:14.977 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.5.service' 2023-07-21T22:40:15.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:15.017 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-21T22:40:15.017 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-21T22:40:15.018 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.6 2023-07-21T22:40:15.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:14 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:14.766+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:15.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:14 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:14.766+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:15.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:14 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:14.766+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:15.387 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:15 smithi107 systemd[1]: Stopping Ceph osd.6 for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:40:15.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:15 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:15.306+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:15.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:15 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:15.243+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:15.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:15 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:15.243+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:15.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:15 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:15.243+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:15.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:15 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:15.382+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:15.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:15 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:15.382+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:15.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:15 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:15.382+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:15.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:15 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:15.345+0000 7f300f845700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:15.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:15 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:15.345+0000 7f300f845700 -1 osd.6 88 *** Got signal Terminated *** 2023-07-21T22:40:15.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:15 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:15.345+0000 7f300f845700 -1 osd.6 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:40:15.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:15 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:15.400+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:15.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:15 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:15.400+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.576190+0000 front 2023-07-21T22:39:41.576381+0000 (oldest deadline 2023-07-21T22:40:07.476375+0000) 2023-07-21T22:40:15.685 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:15 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:15.400+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.476831+0000 front 2023-07-21T22:39:47.476952+0000 (oldest deadline 2023-07-21T22:40:10.976914+0000) 2023-07-21T22:40:15.783 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:15 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:15.450+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:15.783 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:15 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:15.450+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:15.783 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:15 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:15.450+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:16.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:15 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:15.741+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:16.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:15 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:15.741+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:16.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:15 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:15.741+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:16.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:16 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:16.293+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:16.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:16 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:16.293+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.228026+0000 front 2023-07-21T22:39:47.228059+0000 (oldest deadline 2023-07-21T22:40:15.428733+0000) 2023-07-21T22:40:16.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:16 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:16.196+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:16.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:16 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:16.196+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:16.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:16 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:16.196+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:16.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:16 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:16.370+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:16.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:16 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:16.370+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.576190+0000 front 2023-07-21T22:39:41.576381+0000 (oldest deadline 2023-07-21T22:40:07.476375+0000) 2023-07-21T22:40:16.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:16 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:16.370+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.476831+0000 front 2023-07-21T22:39:47.476952+0000 (oldest deadline 2023-07-21T22:40:10.976914+0000) 2023-07-21T22:40:16.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:16 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:16.429+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:16.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:16 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:16.429+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:16.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:16 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:16.429+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:16.814 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:16 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:16.437+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:16.815 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:16 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:16.437+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:16.815 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:16 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:16.437+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:17.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:16 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:16.772+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:17.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:16 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:16.772+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:17.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:16 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:16.772+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:17.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:17 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:17.304+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:17.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:17 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:17.304+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.228026+0000 front 2023-07-21T22:39:47.228059+0000 (oldest deadline 2023-07-21T22:40:15.428733+0000) 2023-07-21T22:40:17.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:17 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:17.212+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:17.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:17 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:17.212+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:17.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:17 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:17.212+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:17.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:17 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:17.324+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:17.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:17 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:17.324+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.576190+0000 front 2023-07-21T22:39:41.576381+0000 (oldest deadline 2023-07-21T22:40:07.476375+0000) 2023-07-21T22:40:17.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:17 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:17.324+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.476831+0000 front 2023-07-21T22:39:47.476952+0000 (oldest deadline 2023-07-21T22:40:10.976914+0000) 2023-07-21T22:40:17.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:17 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:17.443+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:17.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:17 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:17.443+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:17.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:17 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:17.443+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:17.787 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:17 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:17.435+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:17.788 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:17 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:17.435+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:17.788 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:17 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:17.435+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:18.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:17 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:17.745+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:18.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:17 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:17.745+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:18.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:17 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:17.745+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:18.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:18 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:18.316+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:18.678 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:18 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:18.316+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.228026+0000 front 2023-07-21T22:39:47.228059+0000 (oldest deadline 2023-07-21T22:40:15.428733+0000) 2023-07-21T22:40:18.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:18 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:18.234+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:18.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:18 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:18.234+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:18.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:18 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:18.234+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:18.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:18.300+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:18.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:18.300+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.576190+0000 front 2023-07-21T22:39:41.576381+0000 (oldest deadline 2023-07-21T22:40:07.476375+0000) 2023-07-21T22:40:18.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:18.300+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.476831+0000 front 2023-07-21T22:39:47.476952+0000 (oldest deadline 2023-07-21T22:40:10.976914+0000) 2023-07-21T22:40:18.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:18.404+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:18.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:18.404+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:18.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:18 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:18.404+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:18.834 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:18 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:18.410+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:18.923 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:18 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:18.410+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:18.924 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:18 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:18.410+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:19.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:18 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:18.792+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:19.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:18 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:18.792+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:19.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:18 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:18.792+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:19.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:19 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:19.301+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:19.678 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:19 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:19.301+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.228026+0000 front 2023-07-21T22:39:47.228059+0000 (oldest deadline 2023-07-21T22:40:15.428733+0000) 2023-07-21T22:40:19.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:19 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:19.214+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:19.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:19 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:19.214+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:19.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:19 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:19.214+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:19.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:19 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:19.324+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:19.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:19 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:19.324+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.576190+0000 front 2023-07-21T22:39:41.576381+0000 (oldest deadline 2023-07-21T22:40:07.476375+0000) 2023-07-21T22:40:19.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:19 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:19.324+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.476831+0000 front 2023-07-21T22:39:47.476952+0000 (oldest deadline 2023-07-21T22:40:10.976914+0000) 2023-07-21T22:40:19.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:19 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:19.413+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:19.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:19 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:19.413+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:19.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:19 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:19.413+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:19.871 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:19 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:19.368+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:19.871 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:19 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:19.368+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:19.871 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:19 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:19.368+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:19.871 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:19 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:19.368+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:20.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:19 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:19.828+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:20.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:19 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:19.828+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:20.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:19 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:19.828+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:20.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:20.269+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:20.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:20.269+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.228026+0000 front 2023-07-21T22:39:47.228059+0000 (oldest deadline 2023-07-21T22:40:15.428733+0000) 2023-07-21T22:40:20.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:20.251+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:20.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:20.251+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:20.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:20.251+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:20.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:20 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:20.251+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:20.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:20.303+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:33.672980+0000 front 2023-07-21T22:39:33.672907+0000 (oldest deadline 2023-07-21T22:39:57.073840+0000) 2023-07-21T22:40:20.683 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:20.303+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.576190+0000 front 2023-07-21T22:39:41.576381+0000 (oldest deadline 2023-07-21T22:40:07.476375+0000) 2023-07-21T22:40:20.684 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6[120407]: 2023-07-21T22:40:20.303+0000 7f3009e37700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.476831+0000 front 2023-07-21T22:39:47.476952+0000 (oldest deadline 2023-07-21T22:40:10.976914+0000) 2023-07-21T22:40:20.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:20.397+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:20.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:20.397+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:20.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:20 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:20.397+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:20.872 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:20.408+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:20.872 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:20.408+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:20.873 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:20.408+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:20.873 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:20.408+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:21.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:20.830+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:21.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:20.830+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:21.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:20 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:20.830+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:21.258 INFO:journalctl@ceph.osd.6.smithi107.stdout:Jul 21 22:40:21 smithi107 bash[158805]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-6 2023-07-21T22:40:21.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:21 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:21.292+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:21.678 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:21 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:21.292+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.228026+0000 front 2023-07-21T22:39:47.228059+0000 (oldest deadline 2023-07-21T22:40:15.428733+0000) 2023-07-21T22:40:21.678 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:21 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:21.292+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.930020+0000 front 2023-07-21T22:39:55.929876+0000 (oldest deadline 2023-07-21T22:40:21.229793+0000) 2023-07-21T22:40:21.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:21 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:21.274+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:21.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:21 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:21.274+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:21.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:21 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:21.274+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:21.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:21 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:21.274+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:21.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:21 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:21.367+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:21.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:21 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:21.367+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:21.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:21 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:21.368+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:21.824 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:21 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:21.418+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:21.824 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:21 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:21.418+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:21.824 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:21 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:21.418+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:21.824 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:21 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:21.418+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:22.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:21 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:21.782+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:22.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:21 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:21.782+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:22.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:21 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:21.782+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:22.510 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:22 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:22.384+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:22.510 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:22 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:22.384+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:22.511 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:22 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:22.384+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:22.511 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:22 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:22.384+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:22.518 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.6.service' 2023-07-21T22:40:22.557 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:22.557 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-21T22:40:22.557 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-21T22:40:22.558 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.7 2023-07-21T22:40:22.587 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:22 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:22.311+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:22.587 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:22 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:22.311+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.228026+0000 front 2023-07-21T22:39:47.228059+0000 (oldest deadline 2023-07-21T22:40:15.428733+0000) 2023-07-21T22:40:22.587 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:22 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:22.311+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.930020+0000 front 2023-07-21T22:39:55.929876+0000 (oldest deadline 2023-07-21T22:40:21.229793+0000) 2023-07-21T22:40:22.587 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:22 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:22.225+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:22.588 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:22 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:22.225+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:22.588 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:22 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:22.225+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:22.588 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:22 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:22.225+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:22.829 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:22 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:22.421+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:22.830 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:22 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:22.421+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:22.830 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:22 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:22.421+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:22.830 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:22 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:22.421+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:22.908 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:22 smithi132 systemd[1]: Stopping Ceph osd.7 for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:40:23.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:22 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:22.787+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:23.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:22 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:22.787+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:23.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:22 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:22.787+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:23.177 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:22 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:22.866+0000 7fd381a09700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:23.177 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:22 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:22.866+0000 7fd381a09700 -1 osd.7 88 *** Got signal Terminated *** 2023-07-21T22:40:23.177 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:22 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:22.866+0000 7fd381a09700 -1 osd.7 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:40:23.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:23 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:23.294+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:23.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:23 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:23.294+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.228026+0000 front 2023-07-21T22:39:47.228059+0000 (oldest deadline 2023-07-21T22:40:15.428733+0000) 2023-07-21T22:40:23.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:23 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:23.294+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.930020+0000 front 2023-07-21T22:39:55.929876+0000 (oldest deadline 2023-07-21T22:40:21.229793+0000) 2023-07-21T22:40:23.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:23 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:23.270+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:23.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:23 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:23.270+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:23.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:23 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:23.270+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:23.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:23 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:23.270+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:23.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:23 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:23.388+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:23.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:23 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:23.388+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:23.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:23 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:23.388+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:23.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:23 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:23.388+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:23.804 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:23 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:23.379+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:23.804 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:23 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:23.379+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:23.804 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:23 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:23.379+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:23.804 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:23 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:23.379+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:24.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:23 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:23.762+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:24.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:23 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:23.762+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:24.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:23 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:23.762+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:24.636 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:24 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:24.342+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:24.636 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:24 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:24.342+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:24.637 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:24 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:24.342+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:24.637 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:24 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:24.342+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:24.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:24 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:24.344+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:24.678 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:24 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:24.344+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.228026+0000 front 2023-07-21T22:39:47.228059+0000 (oldest deadline 2023-07-21T22:40:15.428733+0000) 2023-07-21T22:40:24.678 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:24 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:24.344+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.930020+0000 front 2023-07-21T22:39:55.929876+0000 (oldest deadline 2023-07-21T22:40:21.229793+0000) 2023-07-21T22:40:24.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:24 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:24.265+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:24.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:24 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:24.265+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:24.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:24 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:24.265+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:24.679 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:24 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:24.265+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:24.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:24 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:24.345+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:24.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:24 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:24.345+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:24.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:24 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:24.345+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:24.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:24 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:24.345+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:25.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:24 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:24.795+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:25.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:24 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:24.795+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:25.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:24 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:24.795+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:25.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:25 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:25.318+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:25.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:25 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:25.318+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.228026+0000 front 2023-07-21T22:39:47.228059+0000 (oldest deadline 2023-07-21T22:40:15.428733+0000) 2023-07-21T22:40:25.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:25 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:25.318+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.930020+0000 front 2023-07-21T22:39:55.929876+0000 (oldest deadline 2023-07-21T22:40:21.229793+0000) 2023-07-21T22:40:25.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:25 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:25.275+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:25.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:25 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:25.276+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:25.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:25 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:25.276+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:25.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:25 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:25.276+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:25.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:25 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:25.317+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:25.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:25 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:25.317+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:25.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:25 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:25.317+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:25.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:25 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:25.317+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:25.879 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:25 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:25.352+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:25.879 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:25 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:25.352+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:25.880 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:25 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:25.352+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:25.880 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:25 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:25.352+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:26.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:25 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:25.837+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:26.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:25 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:25.837+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:26.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:25 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:25.837+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:26.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:26 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:26.324+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:26.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:26 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:26.324+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.228026+0000 front 2023-07-21T22:39:47.228059+0000 (oldest deadline 2023-07-21T22:40:15.428733+0000) 2023-07-21T22:40:26.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:26 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:26.324+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.930020+0000 front 2023-07-21T22:39:55.929876+0000 (oldest deadline 2023-07-21T22:40:21.229793+0000) 2023-07-21T22:40:26.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:26 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:26.272+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:26.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:26 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:26.272+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:26.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:26 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:26.272+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:26.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:26 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:26.272+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:26.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:26 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:26.301+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:26.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:26 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:26.301+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:26.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:26 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:26.301+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:26.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:26 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:26.301+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:26.855 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:26 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:26.351+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:26.855 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:26 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:26.351+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:26.855 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:26 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:26.351+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:26.855 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:26 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:26.351+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:27.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:26 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:26.813+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:27.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:26 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:26.813+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:27.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:26 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:26.813+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:27.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:26 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:26.813+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:27.677 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:27 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:27.336+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:37.826415+0000 front 2023-07-21T22:39:37.826509+0000 (oldest deadline 2023-07-21T22:40:07.227237+0000) 2023-07-21T22:40:27.678 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:27 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:27.336+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.228026+0000 front 2023-07-21T22:39:47.228059+0000 (oldest deadline 2023-07-21T22:40:15.428733+0000) 2023-07-21T22:40:27.678 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:27 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7[113776]: 2023-07-21T22:40:27.336+0000 7fd37d7fe700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.930020+0000 front 2023-07-21T22:39:55.929876+0000 (oldest deadline 2023-07-21T22:40:21.229793+0000) 2023-07-21T22:40:27.679 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:27 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:27.243+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:27.679 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:27 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:27.243+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:27.679 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:27 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:27.243+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:27.680 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:27 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:27.243+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:27.680 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:27 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:27.243+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:27.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:27 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:27.347+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:27.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:27 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:27.347+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:27.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:27 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:27.347+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:27.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:27 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:27.347+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:27.860 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:27 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:27.395+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:27.861 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:27 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:27.395+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:27.861 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:27 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:27.395+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:27.861 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:27 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:27.395+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:28.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:27 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:27.818+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:28.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:27 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:27.818+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:28.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:27 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:27.818+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:28.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:27 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:27.818+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:28.549 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:28 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:28.253+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:28.549 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:28 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:28.253+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:28.550 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:28 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:28.253+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:28.550 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:28 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:28.253+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:28.550 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:28 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:28.253+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:28.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:28.325+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:28.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:28.325+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:28.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:28.325+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:28.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:28 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:28.325+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:28.869 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:28 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:28.435+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:28.869 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:28 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:28.435+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:28.869 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:28 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:28.435+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:28.870 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:28 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:28.435+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:28.927 INFO:journalctl@ceph.osd.7.smithi132.stdout:Jul 21 22:40:28 smithi132 bash[128821]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-7 2023-07-21T22:40:29.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:28 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:28.827+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:29.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:28 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:28.827+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:29.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:28 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:28.827+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:29.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:28 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:28.827+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:29.657 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:29 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:29.279+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:29.657 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:29 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:29.279+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:29.657 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:29 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:29.279+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:29.658 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:29 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:29.279+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:29.658 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:29 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:29.279+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:29.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:29 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:29.295+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:29.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:29 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:29.295+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:29.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:29 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:29.295+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:29.685 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:29 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:29.295+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:29.685 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:29 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:29.295+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:29.847 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:29 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:29.483+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:29.847 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:29 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:29.483+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:29.848 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:29 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:29.483+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:29.848 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:29 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:29.483+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:29.987 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.7.service' 2023-07-21T22:40:30.067 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:30.067 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-21T22:40:30.067 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-21T22:40:30.068 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.8 2023-07-21T22:40:30.098 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:29 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:29.806+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:30.099 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:29 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:29.806+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:30.099 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:29 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:29.806+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:30.099 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:29 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:29.806+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:30.386 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:30 smithi190 systemd[1]: Stopping Ceph osd.8 for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:40:30.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:30 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:30.297+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:30.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:30 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:30.297+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:30.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:30 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:30.297+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:30.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:30 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:30.297+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:30.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:30 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:30.297+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:30.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:30 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:30.331+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:30.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:30 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:30.331+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:30.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:30 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:30.331+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:30.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:30 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:30.331+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:30.685 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:30 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:30.331+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:30.806 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:30 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:30.380+0000 7fb282d6f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:30.806 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:30 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:30.380+0000 7fb282d6f700 -1 osd.8 88 *** Got signal Terminated *** 2023-07-21T22:40:30.806 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:30 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:30.380+0000 7fb282d6f700 -1 osd.8 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:40:30.807 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:30 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:30.499+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:30.807 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:30 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:30.500+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:30.807 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:30 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:30.500+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:30.807 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:30 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:30.500+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:31.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:30 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:30.764+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:31.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:30 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:30.764+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:31.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:30 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:30.764+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:31.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:30 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:30.764+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:31.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:31 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:31.347+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:31.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:31 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:31.347+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:31.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:31 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:31.347+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:31.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:31 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:31.347+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:31.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:31 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:31.347+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:31.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:31 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:31.353+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:31.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:31 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:31.353+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:31.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:31 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:31.353+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:31.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:31 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:31.353+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:31.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:31 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:31.353+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:31.788 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:31 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:31.481+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:31.788 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:31 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:31.481+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:31.788 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:31 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:31.481+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:31.788 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:31 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:31.481+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:32.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:31 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:31.746+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:32.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:31 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:31.746+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:32.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:31 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:31.746+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:32.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:31 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:31.746+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:32.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:32 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:32.313+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:32.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:32 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:32.313+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:32.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:32 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:32.313+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:32.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:32 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:32.313+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:32.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:32 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:32.313+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:32.766 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:32 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:32.433+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:32.767 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:32 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:32.433+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:32.767 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:32 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:32.433+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:32.767 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:32 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:32.433+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:32.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:32 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:32.398+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:32.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:32 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:32.398+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:32.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:32 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:32.398+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:32.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:32 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:32.398+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:32.935 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:32 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:32.398+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:33.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:32 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:32.725+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:33.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:32 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:32.725+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:33.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:32 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:32.725+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:33.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:32 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:32.725+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:33.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:33 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:33.312+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:33.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:33 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:33.312+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:33.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:33 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:33.312+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:33.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:33 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:33.312+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:33.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:33 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:33.312+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:33.798 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:33 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:33.412+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:33.798 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:33 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:33.412+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:33.799 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:33 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:33.412+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:33.799 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:33 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:33.412+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:33.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:33 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:33.409+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:33.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:33 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:33.409+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:33.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:33 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:33.409+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:33.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:33 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:33.409+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:33.935 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:33 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:33.409+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:34.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:33 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:33.757+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:34.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:33 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:33.757+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:34.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:33 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:33.757+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:34.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:33 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:33.757+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:34.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:34 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:34.291+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:34.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:34 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:34.291+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:34.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:34 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:34.291+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:34.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:34 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:34.291+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:34.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:34 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:34.291+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:34.825 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:34 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:34.418+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.498749+0000 front 2023-07-21T22:39:30.498678+0000 (oldest deadline 2023-07-21T22:39:58.099328+0000) 2023-07-21T22:40:34.825 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:34 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:34.418+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:38.099701+0000 front 2023-07-21T22:39:38.099828+0000 (oldest deadline 2023-07-21T22:40:08.700444+0000) 2023-07-21T22:40:34.825 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:34 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:34.418+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:48.700963+0000 front 2023-07-21T22:39:48.700913+0000 (oldest deadline 2023-07-21T22:40:13.401115+0000) 2023-07-21T22:40:34.825 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:34 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:34.418+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:53.401580+0000 front 2023-07-21T22:39:53.401623+0000 (oldest deadline 2023-07-21T22:40:19.301631+0000) 2023-07-21T22:40:34.825 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:34 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8[115107]: 2023-07-21T22:40:34.418+0000 7fb27bb5e700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:11.003613+0000 front 2023-07-21T22:40:11.003640+0000 (oldest deadline 2023-07-21T22:40:33.903780+0000) 2023-07-21T22:40:34.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:34 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:34.394+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:34.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:34 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:34.394+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:34.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:34 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:34.394+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:34.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:34 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:34.394+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:34.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:34 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:34.394+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:34.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:34 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:34.394+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:11.181869+0000 front 2023-07-21T22:40:11.181751+0000 (oldest deadline 2023-07-21T22:40:34.081759+0000) 2023-07-21T22:40:35.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:34 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:34.783+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:35.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:34 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:34.783+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:35.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:34 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:34.783+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:35.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:34 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:34.783+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:35.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:34 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:34.783+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:35.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:35 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:35.314+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:35.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:35 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:35.314+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:35.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:35 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:35.314+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:35.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:35 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:35.314+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:35.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:35 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:35.314+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:35.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:35 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:35.427+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:35.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:35 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:35.427+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:35.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:35 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:35.427+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:35.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:35 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:35.427+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:35.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:35 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:35.427+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:35.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:35 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:35.427+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:11.181869+0000 front 2023-07-21T22:40:11.181751+0000 (oldest deadline 2023-07-21T22:40:34.081759+0000) 2023-07-21T22:40:36.044 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:35 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:35.741+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:36.044 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:35 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:35.741+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:36.045 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:35 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:35.742+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:36.045 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:35 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:35.742+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:36.045 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:35 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:35.742+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:36.386 INFO:journalctl@ceph.osd.8.smithi190.stdout:Jul 21 22:40:36 smithi190 bash[130275]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-8 2023-07-21T22:40:36.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:36 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:36.358+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:36.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:36 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:36.358+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:36.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:36 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:36.358+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:36.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:36 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:36.358+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:36.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:36 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:36.358+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:36.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:36 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:36.378+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:36.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:36 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:36.378+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:36.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:36 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:36.378+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:36.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:36 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:36.378+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:36.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:36 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:36.378+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:36.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:36 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:36.378+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:11.181869+0000 front 2023-07-21T22:40:11.181751+0000 (oldest deadline 2023-07-21T22:40:34.081759+0000) 2023-07-21T22:40:37.046 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:36 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:36.741+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:37.046 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:36 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:36.741+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:37.046 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:36 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:36.741+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:37.046 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:36 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:36.741+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:37.047 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:36 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:36.741+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:37.373 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.8.service' 2023-07-21T22:40:37.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:37.454 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-21T22:40:37.455 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-21T22:40:37.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.9 2023-07-21T22:40:37.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:37 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:37.373+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:37.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:37 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:37.373+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:37.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:37 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:37.373+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:37.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:37 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:37.373+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:37.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:37 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:37.373+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:37.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:37 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:37.384+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:37.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:37 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:37.384+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:37.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:37 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:37.384+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:37.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:37 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:37.384+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:37.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:37 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:37.384+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:37.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:37 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:37.384+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:11.181869+0000 front 2023-07-21T22:40:11.181751+0000 (oldest deadline 2023-07-21T22:40:34.081759+0000) 2023-07-21T22:40:37.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:37 smithi107 systemd[1]: Stopping Ceph osd.9 for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:40:38.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:37 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:37.719+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:38.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:37 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:37.719+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:38.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:37 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:37.719+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:38.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:37 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:37.719+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:38.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:37 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:37.719+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:38.183 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:37 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:37.755+0000 7f3969838700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:38.183 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:37 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:37.755+0000 7f3969838700 -1 osd.9 88 *** Got signal Terminated *** 2023-07-21T22:40:38.184 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:37 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:37.755+0000 7f3969838700 -1 osd.9 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:40:38.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:38 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:38.377+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:38.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:38 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:38.377+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:38.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:38 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:38.377+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:38.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:38 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:38.377+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:38.677 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:38 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:38.377+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:38.678 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:38 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:38.377+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:10.401771+0000 front 2023-07-21T22:40:10.401804+0000 (oldest deadline 2023-07-21T22:40:38.002535+0000) 2023-07-21T22:40:38.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:38 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:38.361+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:38.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:38 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:38.361+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:38.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:38 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:38.361+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:38.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:38 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:38.361+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:38.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:38 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:38.361+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:38.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:38 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:38.361+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:11.181869+0000 front 2023-07-21T22:40:11.181751+0000 (oldest deadline 2023-07-21T22:40:34.081759+0000) 2023-07-21T22:40:39.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:38 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:38.712+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:39.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:38 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:38.712+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:39.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:38 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:38.712+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:39.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:38 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:38.712+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:39.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:38 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:38.712+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:39.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:39 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:39.388+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:39.683 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:39 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:39.388+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:39.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:39 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:39.388+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:39.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:39 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:39.388+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:39.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:39 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:39.388+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:39.684 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:39 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:39.388+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:11.181869+0000 front 2023-07-21T22:40:11.181751+0000 (oldest deadline 2023-07-21T22:40:34.081759+0000) 2023-07-21T22:40:39.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:39 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:39.407+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:39.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:39 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:39.407+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:39.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:39 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:39.407+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:39.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:39 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:39.407+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:39.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:39 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:39.407+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:39.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:39 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:39.407+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:10.401771+0000 front 2023-07-21T22:40:10.401804+0000 (oldest deadline 2023-07-21T22:40:38.002535+0000) 2023-07-21T22:40:40.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:39 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:39.746+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:40.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:39 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:39.746+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:40.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:39 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:39.746+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:40.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:39 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:39.746+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:40.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:39 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:39.746+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:40.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:40 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:40.425+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:40.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:40 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:40.425+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:40.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:40 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:40.425+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:40.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:40 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:40.426+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:40.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:40 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:40.426+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:40.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:40 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:40.426+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:10.401771+0000 front 2023-07-21T22:40:10.401804+0000 (oldest deadline 2023-07-21T22:40:38.002535+0000) 2023-07-21T22:40:40.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:40 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:40.433+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:40.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:40 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:40.433+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:40.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:40 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:40.433+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:40.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:40 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:40.433+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:40.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:40 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:40.433+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:40.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:40 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:40.433+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:11.181869+0000 front 2023-07-21T22:40:11.181751+0000 (oldest deadline 2023-07-21T22:40:34.081759+0000) 2023-07-21T22:40:41.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:40 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:40.712+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:41.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:40 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:40.712+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:41.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:40 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:40.712+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:41.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:40 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:40.712+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:41.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:40 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:40.712+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:41.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:41 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:41.433+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:41.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:41 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:41.433+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:41.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:41 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:41.433+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:41.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:41 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:41.433+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:41.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:41 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:41.433+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:41.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:41 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:41.433+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:10.401771+0000 front 2023-07-21T22:40:10.401804+0000 (oldest deadline 2023-07-21T22:40:38.002535+0000) 2023-07-21T22:40:41.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:41 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:41.420+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:41.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:41 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:41.420+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:41.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:41 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:41.420+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:41.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:41 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:41.420+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:41.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:41 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:41.420+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:41.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:41 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:41.420+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:11.181869+0000 front 2023-07-21T22:40:11.181751+0000 (oldest deadline 2023-07-21T22:40:34.081759+0000) 2023-07-21T22:40:41.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:41 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:41.420+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:18.083045+0000 front 2023-07-21T22:40:18.083084+0000 (oldest deadline 2023-07-21T22:40:40.983226+0000) 2023-07-21T22:40:42.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:41 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:41.684+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:42.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:41 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:41.684+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:42.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:41 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:41.684+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:42.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:41 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:41.684+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:42.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:41 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:41.684+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:42.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:41 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:41.684+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:42.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:42 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:42.480+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:42.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:42 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:42.480+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:42.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:42 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:42.480+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:42.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:42 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:42.480+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:42.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:42 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:42.480+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:42.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:42 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:42.480+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:10.401771+0000 front 2023-07-21T22:40:10.401804+0000 (oldest deadline 2023-07-21T22:40:38.002535+0000) 2023-07-21T22:40:42.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:42 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:42.459+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:30.474008+0000 front 2023-07-21T22:39:30.474096+0000 (oldest deadline 2023-07-21T22:39:56.374248+0000) 2023-07-21T22:40:42.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:42 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:42.459+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:42.176133+0000 front 2023-07-21T22:39:42.176107+0000 (oldest deadline 2023-07-21T22:40:06.876099+0000) 2023-07-21T22:40:42.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:42 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:42.459+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:47.977224+0000 front 2023-07-21T22:39:47.977240+0000 (oldest deadline 2023-07-21T22:40:12.677343+0000) 2023-07-21T22:40:42.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:42 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:42.459+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:56.178461+0000 front 2023-07-21T22:39:56.178416+0000 (oldest deadline 2023-07-21T22:40:22.078525+0000) 2023-07-21T22:40:42.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:42 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:42.459+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:03.080186+0000 front 2023-07-21T22:40:03.080235+0000 (oldest deadline 2023-07-21T22:40:28.980322+0000) 2023-07-21T22:40:42.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:42 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:42.459+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:11.181869+0000 front 2023-07-21T22:40:11.181751+0000 (oldest deadline 2023-07-21T22:40:34.081759+0000) 2023-07-21T22:40:42.934 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:42 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9[125953]: 2023-07-21T22:40:42.459+0000 7f3962e28700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:18.083045+0000 front 2023-07-21T22:40:18.083084+0000 (oldest deadline 2023-07-21T22:40:40.983226+0000) 2023-07-21T22:40:43.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:42 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:42.656+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:43.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:42 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:42.656+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:43.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:42 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:42.656+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:43.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:42 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:42.656+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:43.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:42 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:42.656+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:43.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:42 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:42.656+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:43.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:43 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:43.481+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:43.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:43 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:43.481+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:43.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:43 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:43.481+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:43.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:43 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:43.481+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:43.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:43 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:43.481+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:43.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:43 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:43.481+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:10.401771+0000 front 2023-07-21T22:40:10.401804+0000 (oldest deadline 2023-07-21T22:40:38.002535+0000) 2023-07-21T22:40:43.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:43 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:43.481+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:18.503512+0000 front 2023-07-21T22:40:18.503625+0000 (oldest deadline 2023-07-21T22:40:42.603998+0000) 2023-07-21T22:40:43.933 INFO:journalctl@ceph.osd.9.smithi107.stdout:Jul 21 22:40:43 smithi107 bash[159069]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-9 2023-07-21T22:40:44.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:43 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:43.682+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:44.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:43 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:43.682+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:44.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:43 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:43.682+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:44.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:43 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:43.682+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:44.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:43 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:43.682+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:44.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:43 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:43.682+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:44.859 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.9.service' 2023-07-21T22:40:44.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:44.911 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-21T22:40:44.912 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-21T22:40:44.913 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.10 2023-07-21T22:40:44.926 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:44 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:44.480+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:44.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:44 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:44.480+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:44.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:44 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:44.480+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:44.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:44 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:44.480+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:44.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:44 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:44.480+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:44.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:44 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:44.480+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:10.401771+0000 front 2023-07-21T22:40:10.401804+0000 (oldest deadline 2023-07-21T22:40:38.002535+0000) 2023-07-21T22:40:44.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:44 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:44.480+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:18.503512+0000 front 2023-07-21T22:40:18.503625+0000 (oldest deadline 2023-07-21T22:40:42.603998+0000) 2023-07-21T22:40:45.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:44 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:44.645+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:45.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:44 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:44.645+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:45.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:44 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:44.645+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:45.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:44 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:44.645+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:45.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:44 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:44.645+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:45.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:44 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:44.645+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:45.279 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:44 smithi132 systemd[1]: Stopping Ceph osd.10 for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:40:45.547 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:45 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:45.238+0000 7f3e1f53e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:45.548 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:45 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:45.238+0000 7f3e1f53e700 -1 osd.10 88 *** Got signal Terminated *** 2023-07-21T22:40:45.548 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:45 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:45.238+0000 7f3e1f53e700 -1 osd.10 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:40:45.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:45 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:45.506+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:45.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:45 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:45.506+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:45.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:45 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:45.506+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:45.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:45 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:45.506+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:45.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:45 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:45.506+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:45.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:45 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:45.506+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:10.401771+0000 front 2023-07-21T22:40:10.401804+0000 (oldest deadline 2023-07-21T22:40:38.002535+0000) 2023-07-21T22:40:45.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:45 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:45.506+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:18.503512+0000 front 2023-07-21T22:40:18.503625+0000 (oldest deadline 2023-07-21T22:40:42.603998+0000) 2023-07-21T22:40:46.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:45 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:45.597+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:46.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:45 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:45.597+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:46.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:45 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:45.597+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:46.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:45 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:45.597+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:46.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:45 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:45.597+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:46.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:45 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:45.597+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:46.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:46.499+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:46.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:46.499+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:46.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:46.499+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:46.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:46.499+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:46.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:46.499+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:46.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:46.499+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:10.401771+0000 front 2023-07-21T22:40:10.401804+0000 (oldest deadline 2023-07-21T22:40:38.002535+0000) 2023-07-21T22:40:46.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:46 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:46.499+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:18.503512+0000 front 2023-07-21T22:40:18.503625+0000 (oldest deadline 2023-07-21T22:40:42.603998+0000) 2023-07-21T22:40:47.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:46 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:46.639+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:47.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:46 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:46.639+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:47.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:46 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:46.639+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:47.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:46 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:46.639+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:47.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:46 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:46.639+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:47.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:46 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:46.639+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:47.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:47 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:47.520+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:47.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:47 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:47.520+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:47.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:47 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:47.520+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:47.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:47 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:47.520+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:47.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:47 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:47.520+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:47.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:47 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:47.520+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:10.401771+0000 front 2023-07-21T22:40:10.401804+0000 (oldest deadline 2023-07-21T22:40:38.002535+0000) 2023-07-21T22:40:47.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:47 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:47.520+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:18.503512+0000 front 2023-07-21T22:40:18.503625+0000 (oldest deadline 2023-07-21T22:40:42.603998+0000) 2023-07-21T22:40:48.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:47 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:47.636+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:48.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:47 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:47.636+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:48.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:47 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:47.636+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:48.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:47 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:47.636+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:48.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:47 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:47.636+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:48.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:47 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:47.636+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:48.926 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:48 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:48.552+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:48.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:48 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:48.552+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:48.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:48 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:48.552+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:48.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:48 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:48.552+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:48.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:48 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:48.552+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:48.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:48 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:48.552+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:10.401771+0000 front 2023-07-21T22:40:10.401804+0000 (oldest deadline 2023-07-21T22:40:38.002535+0000) 2023-07-21T22:40:48.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:48 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:48.552+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:18.503512+0000 front 2023-07-21T22:40:18.503625+0000 (oldest deadline 2023-07-21T22:40:42.603998+0000) 2023-07-21T22:40:49.135 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:48 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:48.641+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:49.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:48 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:48.641+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:49.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:48 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:48.641+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:49.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:48 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:48.641+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:49.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:48 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:48.641+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:49.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:48 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:48.641+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:49.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:49 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:49.551+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:35.395650+0000 front 2023-07-21T22:39:35.395579+0000 (oldest deadline 2023-07-21T22:39:56.495696+0000) 2023-07-21T22:40:49.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:49 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:49.551+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:39.897408+0000 front 2023-07-21T22:39:39.897436+0000 (oldest deadline 2023-07-21T22:40:05.797532+0000) 2023-07-21T22:40:49.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:49 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:49.551+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:45.798119+0000 front 2023-07-21T22:39:45.798145+0000 (oldest deadline 2023-07-21T22:40:11.098274+0000) 2023-07-21T22:40:49.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:49 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:49.551+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2023-07-21T22:39:55.199451+0000 front 2023-07-21T22:39:55.199466+0000 (oldest deadline 2023-07-21T22:40:19.899539+0000) 2023-07-21T22:40:49.927 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:49 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:49.551+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:02.800674+0000 front 2023-07-21T22:40:02.800612+0000 (oldest deadline 2023-07-21T22:40:26.300869+0000) 2023-07-21T22:40:49.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:49 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:49.551+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:10.401771+0000 front 2023-07-21T22:40:10.401804+0000 (oldest deadline 2023-07-21T22:40:38.002535+0000) 2023-07-21T22:40:49.928 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:49 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10[119392]: 2023-07-21T22:40:49.551+0000 7f3e1832d700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:18.503512+0000 front 2023-07-21T22:40:18.503625+0000 (oldest deadline 2023-07-21T22:40:42.603998+0000) 2023-07-21T22:40:50.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:49 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:49.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:50.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:49 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:49.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:50.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:49 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:49.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:50.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:49 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:49.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:50.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:49 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:49.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:50.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:49 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:49.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:50.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:49 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:49.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-07-21T22:40:26.258071+0000 front 2023-07-21T22:40:26.258109+0000 (oldest deadline 2023-07-21T22:40:49.158048+0000) 2023-07-21T22:40:51.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:50 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:50.622+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:51.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:50 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:50.622+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:51.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:50 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:50.622+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:51.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:50 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:50.622+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:51.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:50 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:50.622+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:51.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:50 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:50.622+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:51.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:50 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:50.622+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-07-21T22:40:26.258071+0000 front 2023-07-21T22:40:26.258109+0000 (oldest deadline 2023-07-21T22:40:49.158048+0000) 2023-07-21T22:40:51.177 INFO:journalctl@ceph.osd.10.smithi132.stdout:Jul 21 22:40:50 smithi132 bash[129097]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-10 2023-07-21T22:40:52.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:51 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:51.666+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:52.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:51 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:51.666+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:52.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:51 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:51.666+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:52.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:51 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:51.666+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:52.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:51 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:51.666+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:52.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:51 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:51.666+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:52.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:51 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:51.666+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-07-21T22:40:26.258071+0000 front 2023-07-21T22:40:26.258109+0000 (oldest deadline 2023-07-21T22:40:49.158048+0000) 2023-07-21T22:40:52.227 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.10.service' 2023-07-21T22:40:52.306 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:52.307 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-21T22:40:52.307 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-21T22:40:52.307 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.11 2023-07-21T22:40:52.636 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:52 smithi190 systemd[1]: Stopping Ceph osd.11 for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:40:53.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:52 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:52.617+0000 7f5771ee2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:53.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:52 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:52.617+0000 7f5771ee2700 -1 osd.11 88 *** Got signal Terminated *** 2023-07-21T22:40:53.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:52 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:52.617+0000 7f5771ee2700 -1 osd.11 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:40:53.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:52 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:52.652+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:53.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:52 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:52.652+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:53.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:52 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:52.652+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:53.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:52 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:52.652+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:53.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:52 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:52.652+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:53.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:52 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:52.652+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:53.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:52 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:52.652+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-07-21T22:40:26.258071+0000 front 2023-07-21T22:40:26.258109+0000 (oldest deadline 2023-07-21T22:40:49.158048+0000) 2023-07-21T22:40:54.135 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:53 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:53.667+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:54.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:53 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:53.667+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:54.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:53 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:53.667+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:54.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:53 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:53.667+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:54.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:53 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:53.667+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:54.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:53 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:53.667+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:54.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:53 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:53.667+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-07-21T22:40:26.258071+0000 front 2023-07-21T22:40:26.258109+0000 (oldest deadline 2023-07-21T22:40:49.158048+0000) 2023-07-21T22:40:55.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:54 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:54.633+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:55.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:54 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:54.633+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:55.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:54 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:54.633+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:55.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:54 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:54.633+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:55.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:54 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:54.633+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:55.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:54 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:54.633+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:55.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:54 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:54.633+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-07-21T22:40:26.258071+0000 front 2023-07-21T22:40:26.258109+0000 (oldest deadline 2023-07-21T22:40:49.158048+0000) 2023-07-21T22:40:56.135 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:55 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:55.643+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:56.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:55 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:55.643+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:56.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:55 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:55.643+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:56.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:55 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:55.643+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:56.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:55 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:55.643+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:56.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:55 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:55.643+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:56.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:55 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:55.643+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-07-21T22:40:26.258071+0000 front 2023-07-21T22:40:26.258109+0000 (oldest deadline 2023-07-21T22:40:49.158048+0000) 2023-07-21T22:40:57.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:56 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:56.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2023-07-21T22:39:32.148797+0000 front 2023-07-21T22:39:32.148526+0000 (oldest deadline 2023-07-21T22:39:56.749276+0000) 2023-07-21T22:40:57.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:56 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:56.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-07-21T22:39:41.450847+0000 front 2023-07-21T22:39:41.450803+0000 (oldest deadline 2023-07-21T22:40:06.750645+0000) 2023-07-21T22:40:57.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:56 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:56.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6804 osd.2 since back 2023-07-21T22:39:46.751050+0000 front 2023-07-21T22:39:46.751020+0000 (oldest deadline 2023-07-21T22:40:12.651231+0000) 2023-07-21T22:40:57.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:56 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:56.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-07-21T22:40:01.353762+0000 front 2023-07-21T22:40:01.353822+0000 (oldest deadline 2023-07-21T22:40:26.653714+0000) 2023-07-21T22:40:57.136 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:56 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:56.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.190:6812 osd.5 since back 2023-07-21T22:40:09.455049+0000 front 2023-07-21T22:40:09.454971+0000 (oldest deadline 2023-07-21T22:40:34.755105+0000) 2023-07-21T22:40:57.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:56 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:56.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2023-07-21T22:40:15.856338+0000 front 2023-07-21T22:40:15.856171+0000 (oldest deadline 2023-07-21T22:40:41.056577+0000) 2023-07-21T22:40:57.137 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:56 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11[120679]: 2023-07-21T22:40:56.619+0000 7f576acd1700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-07-21T22:40:26.258071+0000 front 2023-07-21T22:40:26.258109+0000 (oldest deadline 2023-07-21T22:40:49.158048+0000) 2023-07-21T22:40:58.635 INFO:journalctl@ceph.osd.11.smithi190.stdout:Jul 21 22:40:58 smithi190 bash[130545]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-osd-11 2023-07-21T22:40:59.706 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@osd.11.service' 2023-07-21T22:40:59.746 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:59.747 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-21T22:40:59.747 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-21T22:40:59.747 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.a 2023-07-21T22:41:00.101 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:40:59 smithi107 systemd[1]: Stopping Ceph mds.a for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:41:00.433 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:41:00 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-a[131224]: 2023-07-21T22:41:00.059+0000 7f91d02c4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:41:00.433 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:41:00 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-a[131224]: 2023-07-21T22:41:00.059+0000 7f91d02c4700 -1 mds.a *** got signal Terminated *** 2023-07-21T22:41:04.582 INFO:journalctl@ceph.mds.a.smithi107.stdout:Jul 21 22:41:04 smithi107 bash[159346]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-a 2023-07-21T22:41:04.588 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.a.service' 2023-07-21T22:41:04.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:41:04.625 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-21T22:41:04.625 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-21T22:41:04.626 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.d 2023-07-21T22:41:04.933 INFO:journalctl@ceph.mds.d.smithi107.stdout:Jul 21 22:41:04 smithi107 systemd[1]: Stopping Ceph mds.d for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:41:05.433 INFO:journalctl@ceph.mds.d.smithi107.stdout:Jul 21 22:41:04 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-d[130227]: 2023-07-21T22:41:04.949+0000 7fc942ce5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:41:05.433 INFO:journalctl@ceph.mds.d.smithi107.stdout:Jul 21 22:41:04 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-d[130227]: 2023-07-21T22:41:04.949+0000 7fc942ce5700 -1 mds.d *** got signal Terminated *** 2023-07-21T22:41:06.845 INFO:journalctl@ceph.mds.d.smithi107.stdout:Jul 21 22:41:06 smithi107 bash[159492]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-d 2023-07-21T22:41:06.850 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.d.service' 2023-07-21T22:41:06.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:41:06.885 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-21T22:41:06.885 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-21T22:41:06.886 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.g 2023-07-21T22:41:07.182 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:41:06 smithi107 systemd[1]: Stopping Ceph mds.g for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:41:07.683 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:41:07 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-g[130839]: 2023-07-21T22:41:07.214+0000 7f01f6f32700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:41:07.683 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:41:07 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-g[130839]: 2023-07-21T22:41:07.214+0000 7f01f6f32700 -1 mds.g *** got signal Terminated *** 2023-07-21T22:41:08.901 INFO:journalctl@ceph.mds.g.smithi107.stdout:Jul 21 22:41:08 smithi107 bash[159636]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-g 2023-07-21T22:41:08.907 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.g.service' 2023-07-21T22:41:08.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:41:08.941 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-21T22:41:08.942 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-21T22:41:08.942 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.j 2023-07-21T22:41:09.182 INFO:journalctl@ceph.mds.j.smithi107.stdout:Jul 21 22:41:08 smithi107 systemd[1]: Stopping Ceph mds.j for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:41:09.683 INFO:journalctl@ceph.mds.j.smithi107.stdout:Jul 21 22:41:09 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-j[129837]: 2023-07-21T22:41:09.272+0000 7fb0b8894700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:41:09.683 INFO:journalctl@ceph.mds.j.smithi107.stdout:Jul 21 22:41:09 smithi107 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-j[129837]: 2023-07-21T22:41:09.272+0000 7fb0b8894700 -1 mds.j *** got signal Terminated *** 2023-07-21T22:41:12.584 INFO:journalctl@ceph.mds.j.smithi107.stdout:Jul 21 22:41:12 smithi107 bash[159776]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-j 2023-07-21T22:41:12.790 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.j.service' 2023-07-21T22:41:12.822 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:41:12.823 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-21T22:41:12.823 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-21T22:41:12.824 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.b 2023-07-21T22:41:13.175 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:41:12 smithi132 systemd[1]: Stopping Ceph mds.b for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:41:13.426 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:41:13 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-b[123210]: 2023-07-21T22:41:13.133+0000 7fbd08a32700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:41:13.426 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:41:13 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-b[123210]: 2023-07-21T22:41:13.133+0000 7fbd08a32700 -1 mds.b *** got signal Terminated *** 2023-07-21T22:41:16.926 INFO:journalctl@ceph.mds.b.smithi132.stdout:Jul 21 22:41:16 smithi132 bash[129374]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-b 2023-07-21T22:41:16.935 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.b.service' 2023-07-21T22:41:16.972 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:41:16.972 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-21T22:41:16.972 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-21T22:41:16.973 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.e 2023-07-21T22:41:17.337 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:41:17 smithi132 systemd[1]: Stopping Ceph mds.e for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:41:17.676 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:41:17 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-e[122519]: 2023-07-21T22:41:17.296+0000 7f40598f3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:41:17.677 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:41:17 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-e[122519]: 2023-07-21T22:41:17.296+0000 7f40598f3700 -1 mds.e *** got signal Terminated *** 2023-07-21T22:41:20.899 INFO:journalctl@ceph.mds.e.smithi132.stdout:Jul 21 22:41:20 smithi132 bash[129518]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-e 2023-07-21T22:41:20.905 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.e.service' 2023-07-21T22:41:20.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:41:20.939 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-21T22:41:20.940 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-21T22:41:20.940 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.h 2023-07-21T22:41:21.176 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:41:20 smithi132 systemd[1]: Stopping Ceph mds.h for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:41:21.676 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:41:21 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-h[125464]: 2023-07-21T22:41:21.282+0000 7f1672e84700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:41:21.677 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:41:21 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-h[125464]: 2023-07-21T22:41:21.282+0000 7f1672e84700 -1 mds.h *** got signal Terminated *** 2023-07-21T22:41:24.788 INFO:journalctl@ceph.mds.h.smithi132.stdout:Jul 21 22:41:24 smithi132 bash[129659]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-h 2023-07-21T22:41:24.977 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.h.service' 2023-07-21T22:41:25.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:41:25.011 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-21T22:41:25.011 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-21T22:41:25.011 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.k 2023-07-21T22:41:25.388 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:41:25 smithi132 systemd[1]: Stopping Ceph mds.k for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:41:25.676 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:41:25 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-k[122181]: 2023-07-21T22:41:25.347+0000 7fd60ac6c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:41:25.676 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:41:25 smithi132 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-k[122181]: 2023-07-21T22:41:25.347+0000 7fd60ac6c700 -1 mds.k *** got signal Terminated *** 2023-07-21T22:41:28.927 INFO:journalctl@ceph.mds.k.smithi132.stdout:Jul 21 22:41:28 smithi132 bash[129799]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-k 2023-07-21T22:41:29.054 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.k.service' 2023-07-21T22:41:29.090 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:41:29.091 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-21T22:41:29.091 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-21T22:41:29.092 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.c 2023-07-21T22:41:29.385 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:41:29 smithi190 systemd[1]: Stopping Ceph mds.c for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:41:29.885 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:41:29 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-c[124200]: 2023-07-21T22:41:29.486+0000 7f1fc0420700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:41:29.886 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:41:29 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-c[124200]: 2023-07-21T22:41:29.486+0000 7f1fc0420700 -1 mds.c *** got signal Terminated *** 2023-07-21T22:41:32.815 INFO:journalctl@ceph.mds.c.smithi190.stdout:Jul 21 22:41:32 smithi190 bash[130808]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-c 2023-07-21T22:41:33.004 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.c.service' 2023-07-21T22:41:33.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:41:33.040 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-21T22:41:33.040 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-21T22:41:33.041 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.f 2023-07-21T22:41:33.385 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:41:33 smithi190 systemd[1]: Stopping Ceph mds.f for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:41:33.885 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:41:33 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:41:33.386+0000 7fdd7329c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:41:33.885 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:41:33 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f[123858]: 2023-07-21T22:41:33.386+0000 7fdd7329c700 -1 mds.f *** got signal Terminated *** 2023-07-21T22:41:36.785 INFO:journalctl@ceph.mds.f.smithi190.stdout:Jul 21 22:41:36 smithi190 bash[130949]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-f 2023-07-21T22:41:36.974 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.f.service' 2023-07-21T22:41:37.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:41:37.010 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-21T22:41:37.011 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-21T22:41:37.011 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.i 2023-07-21T22:41:37.385 INFO:journalctl@ceph.mds.i.smithi190.stdout:Jul 21 22:41:37 smithi190 systemd[1]: Stopping Ceph mds.i for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:41:37.885 INFO:journalctl@ceph.mds.i.smithi190.stdout:Jul 21 22:41:37 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-i[124538]: 2023-07-21T22:41:37.352+0000 7f1076f63700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:41:37.885 INFO:journalctl@ceph.mds.i.smithi190.stdout:Jul 21 22:41:37 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-i[124538]: 2023-07-21T22:41:37.352+0000 7f1076f63700 -1 mds.i *** got signal Terminated *** 2023-07-21T22:41:41.040 INFO:journalctl@ceph.mds.i.smithi190.stdout:Jul 21 22:41:40 smithi190 bash[131097]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-i 2023-07-21T22:41:41.047 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.i.service' 2023-07-21T22:41:41.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:41:41.083 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-21T22:41:41.084 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-21T22:41:41.084 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.l 2023-07-21T22:41:41.385 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:41:41 smithi190 systemd[1]: Stopping Ceph mds.l for af99f5d0-2814-11ee-9b34-001a4aab830c... 2023-07-21T22:41:41.885 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:41:41 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-l[123519]: 2023-07-21T22:41:41.409+0000 7f8e5cc12700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:41:41.885 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:41:41 smithi190 ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-l[123519]: 2023-07-21T22:41:41.409+0000 7f8e5cc12700 -1 mds.l *** got signal Terminated *** 2023-07-21T22:41:44.814 INFO:journalctl@ceph.mds.l.smithi190.stdout:Jul 21 22:41:44 smithi190 bash[131241]: ceph-af99f5d0-2814-11ee-9b34-001a4aab830c-mds-l 2023-07-21T22:41:44.934 DEBUG:teuthology.orchestra.run.smithi190:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af99f5d0-2814-11ee-9b34-001a4aab830c@mds.l.service' 2023-07-21T22:41:44.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:41:44.968 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-21T22:41:44.969 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af99f5d0-2814-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T22:41:46.829 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af99f5d0-2814-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T22:41:48.739 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af99f5d0-2814-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T22:41:50.600 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:41:50.636 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:41:50.672 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:41:50.703 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-21T22:41:50.705 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737/remote/smithi107/crash 2023-07-21T22:41:50.705 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T22:41:50.735 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T22:41:50.735 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2023-07-21T22:41:50.737 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737/remote/smithi132/crash 2023-07-21T22:41:50.738 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T22:41:50.792 INFO:teuthology.orchestra.run.smithi132.stderr:tar: /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T22:41:50.792 INFO:teuthology.orchestra.run.smithi132.stderr:tar: Error is not recoverable: exiting now 2023-07-21T22:41:50.794 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737/remote/smithi190/crash 2023-07-21T22:41:50.795 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T22:41:50.830 INFO:teuthology.orchestra.run.smithi190.stderr:tar: /var/lib/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T22:41:50.831 INFO:teuthology.orchestra.run.smithi190.stderr:tar: Error is not recoverable: exiting now 2023-07-21T22:41:50.831 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-21T22:41:50.832 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-21T22:41:50.865 INFO:teuthology.orchestra.run.smithi107.stderr:grep: /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph.log: No such file or directory 2023-07-21T22:41:50.867 INFO:tasks.cephadm:Compressing logs... 2023-07-21T22:41:50.867 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T22:41:50.910 DEBUG:teuthology.orchestra.run.smithi132:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T22:41:50.912 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T22:41:50.936 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-21T22:41:50.936 INFO:teuthology.orchestra.run.smithi107.stderr:: No such file or directory 2023-07-21T22:41:50.936 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.0.log 2023-07-21T22:41:50.937 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.g.log 2023-07-21T22:41:50.937 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.a.log 2023-07-21T22:41:50.938 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.0.log: /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.6.log 2023-07-21T22:41:50.938 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.3.log 2023-07-21T22:41:50.939 INFO:teuthology.orchestra.run.smithi190.stderr:find: ‘/var/log/rbd-target-api’gzip : No such file or directory-5 2023-07-21T22:41:50.939 INFO:teuthology.orchestra.run.smithi190.stderr:--verbose -- /var/log/ceph/cephadm.log 2023-07-21T22:41:50.940 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.5.log 2023-07-21T22:41:50.940 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.2.log 2023-07-21T22:41:50.940 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.11.log 2023-07-21T22:41:50.940 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mgr.z.log 2023-07-21T22:41:50.940 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.8.log 2023-07-21T22:41:50.941 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.11.log: /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mgr.z.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T22:41:50.942 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.6.log: /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.3.log: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T22:41:50.945 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.8.log: 92.7% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mgr.z.log.gz 2023-07-21T22:41:50.946 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.f.log 2023-07-21T22:41:50.946 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.c.log 2023-07-21T22:41:50.946 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.i.log 2023-07-21T22:41:50.950 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.d.log 2023-07-21T22:41:50.950 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mgr.x.log 2023-07-21T22:41:50.951 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.9.log 2023-07-21T22:41:50.952 INFO:teuthology.orchestra.run.smithi132.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-07-21T22:41:50.953 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-21T22:41:50.956 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.c.log: /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mon.c.log 2023-07-21T22:41:50.958 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mgr.x.log: /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.d.log.gz 2023-07-21T22:41:50.958 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T22:41:50.959 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.10.log 2023-07-21T22:41:50.960 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.h.log 2023-07-21T22:41:50.960 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.e.log 2023-07-21T22:41:50.961 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.4.log 2023-07-21T22:41:50.961 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.1.log 2023-07-21T22:41:50.961 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.e.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T22:41:50.961 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.k.log 2023-07-21T22:41:50.961 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T22:41:50.963 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T22:41:50.963 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.7.log 2023-07-21T22:41:50.963 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mon.b.log 2023-07-21T22:41:50.963 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.b.log 2023-07-21T22:41:50.963 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mgr.y.log 2023-07-21T22:41:50.964 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.j.log 2023-07-21T22:41:50.964 INFO:teuthology.orchestra.run.smithi107.stderr: 91.6% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T22:41:50.968 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mon.a.log 2023-07-21T22:41:50.968 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.b.log: /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mgr.y.log: 92.6% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mgr.y.log.gz 2023-07-21T22:41:50.969 INFO:teuthology.orchestra.run.smithi107.stderr: 94.5% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.j.log.gz 2023-07-21T22:41:50.982 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.l.log 2023-07-21T22:41:50.992 INFO:teuthology.orchestra.run.smithi190.stderr: 91.5% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T22:41:51.180 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mon.a.log: 89.8% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mgr.x.log.gz 2023-07-21T22:41:51.375 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.l.log: 92.0% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mon.c.log.gz 2023-07-21T22:41:51.960 INFO:teuthology.orchestra.run.smithi132.stderr: 91.5% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T22:41:52.078 INFO:teuthology.orchestra.run.smithi107.stderr: 90.6% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mon.a.log.gz 2023-07-21T22:41:52.402 INFO:teuthology.orchestra.run.smithi132.stderr: 92.0% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mon.b.log.gz 2023-07-21T22:41:53.958 INFO:teuthology.orchestra.run.smithi107.stderr: 95.5% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.g.log.gz 2023-07-21T22:41:56.274 INFO:teuthology.orchestra.run.smithi107.stderr: 93.8% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.9.log.gz 2023-07-21T22:41:58.966 INFO:teuthology.orchestra.run.smithi190.stderr: 95.6% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.l.log.gz 2023-07-21T22:41:59.110 INFO:teuthology.orchestra.run.smithi107.stderr: 93.7% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.3.log.gz 2023-07-21T22:41:59.209 INFO:teuthology.orchestra.run.smithi190.stderr: 95.5% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.c.log.gz 2023-07-21T22:41:59.233 INFO:teuthology.orchestra.run.smithi107.stderr: 93.7% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.6.log.gz 2023-07-21T22:41:59.282 INFO:teuthology.orchestra.run.smithi132.stderr: 95.2% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.h.log.gz 2023-07-21T22:41:59.703 INFO:teuthology.orchestra.run.smithi190.stderr: 93.7% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.8.log.gz 2023-07-21T22:41:59.763 INFO:teuthology.orchestra.run.smithi107.stderr: 95.5% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.a.log.gz 2023-07-21T22:41:59.925 INFO:teuthology.orchestra.run.smithi107.stderr: 93.7% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.0.log.gz 2023-07-21T22:41:59.926 INFO:teuthology.orchestra.run.smithi190.stderr: 93.7% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.5.log.gz 2023-07-21T22:41:59.927 INFO:teuthology.orchestra.run.smithi107.stderr: 2023-07-21T22:41:59.928 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m9.002s 2023-07-21T22:41:59.928 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m42.535s 2023-07-21T22:41:59.928 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m1.392s 2023-07-21T22:42:00.179 INFO:teuthology.orchestra.run.smithi190.stderr: 93.8% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.2.log.gz 2023-07-21T22:42:00.339 INFO:teuthology.orchestra.run.smithi190.stderr: 93.8% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.11.log.gz 2023-07-21T22:42:01.423 INFO:teuthology.orchestra.run.smithi132.stderr: 93.8% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.10.log.gz 2023-07-21T22:42:01.465 INFO:teuthology.orchestra.run.smithi132.stderr: 93.8% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.4.log.gz 2023-07-21T22:42:04.032 INFO:teuthology.orchestra.run.smithi132.stderr: 93.7% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.1.log.gz 2023-07-21T22:42:04.776 INFO:teuthology.orchestra.run.smithi132.stderr: 93.7% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-osd.7.log.gz 2023-07-21T22:42:47.420 INFO:teuthology.orchestra.run.smithi132.stderr: 93.7% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.k.log.gz 2023-07-21T22:43:04.491 INFO:teuthology.orchestra.run.smithi190.stderr: 93.5% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.i.log.gz 2023-07-21T22:43:16.903 INFO:teuthology.orchestra.run.smithi132.stderr: 94.7% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.b.log.gz 2023-07-21T22:43:18.971 INFO:teuthology.orchestra.run.smithi190.stderr: 94.5% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.f.log.gz 2023-07-21T22:43:18.973 INFO:teuthology.orchestra.run.smithi190.stderr: 2023-07-21T22:43:18.973 INFO:teuthology.orchestra.run.smithi190.stderr:real 1m28.046s 2023-07-21T22:43:18.973 INFO:teuthology.orchestra.run.smithi190.stderr:user 3m26.725s 2023-07-21T22:43:18.973 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m7.055s 2023-07-21T22:44:10.637 INFO:teuthology.orchestra.run.smithi132.stderr: 93.3% -- replaced with /var/log/ceph/af99f5d0-2814-11ee-9b34-001a4aab830c/ceph-mds.e.log.gz 2023-07-21T22:44:10.639 INFO:teuthology.orchestra.run.smithi132.stderr: 2023-07-21T22:44:10.640 INFO:teuthology.orchestra.run.smithi132.stderr:real 2m19.714s 2023-07-21T22:44:10.640 INFO:teuthology.orchestra.run.smithi132.stderr:user 4m15.101s 2023-07-21T22:44:10.640 INFO:teuthology.orchestra.run.smithi132.stderr:sys 0m9.521s 2023-07-21T22:44:10.640 INFO:tasks.cephadm:Archiving logs... 2023-07-21T22:44:10.641 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737/remote/smithi107/log 2023-07-21T22:44:10.642 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T22:44:14.830 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737/remote/smithi132/log 2023-07-21T22:44:14.831 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T22:44:44.378 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737/remote/smithi190/log 2023-07-21T22:44:44.380 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T22:45:06.037 INFO:tasks.cephadm:Removing cluster... 2023-07-21T22:45:06.037 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af99f5d0-2814-11ee-9b34-001a4aab830c --force 2023-07-21T22:45:06.844 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af99f5d0-2814-11ee-9b34-001a4aab830c --force 2023-07-21T22:45:08.223 DEBUG:teuthology.orchestra.run.smithi190:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af99f5d0-2814-11ee-9b34-001a4aab830c --force 2023-07-21T22:45:09.208 INFO:tasks.cephadm:Removing cephadm ... 2023-07-21T22:45:09.208 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T22:45:09.229 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T22:45:09.264 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T22:45:09.281 INFO:tasks.cephadm:Teardown complete 2023-07-21T22:45:09.282 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-21T22:45:09.293 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-21T22:45:09.293 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T22:45:09.296 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T22:45:09.306 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T22:45:09.434 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-21T22:45:09.444 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-21T22:45:09.444 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:45:09.447 DEBUG:teuthology.orchestra.run.smithi132:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:45:09.450 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:45:09.463 INFO:teuthology.orchestra.run.smithi107.stderr:bash: ntpq: command not found 2023-07-21T22:45:09.466 INFO:teuthology.orchestra.run.smithi132.stderr:bash: ntpq: command not found 2023-07-21T22:45:09.467 INFO:teuthology.orchestra.run.smithi190.stderr:bash: ntpq: command not found 2023-07-21T22:45:09.467 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:45:09.468 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2023-07-21T22:45:09.468 INFO:teuthology.orchestra.run.smithi107.stdout:^- hv01.front.sepia.ceph.com 2 6 377 29 -699us[ -656us] +/- 267ms 2023-07-21T22:45:09.468 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv02.front.sepia.ceph.com 3 6 377 29 -1181us[-1139us] +/- 66ms 2023-07-21T22:45:09.468 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +1491us[+1533us] +/- 81ms 2023-07-21T22:45:09.468 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:45:09.471 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:45:09.471 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2023-07-21T22:45:09.471 INFO:teuthology.orchestra.run.smithi190.stdout:^- hv01.front.sepia.ceph.com 2 6 377 30 -742us[ -742us] +/- 267ms 2023-07-21T22:45:09.471 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv02.front.sepia.ceph.com 3 6 377 31 -1177us[-1157us] +/- 66ms 2023-07-21T22:45:09.471 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 +1485us[+1485us] +/- 81ms 2023-07-21T22:45:09.471 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:45:09.495 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:45:09.495 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2023-07-21T22:45:09.495 INFO:teuthology.orchestra.run.smithi132.stdout:^- hv01.front.sepia.ceph.com 2 6 377 32 -737us[ -737us] +/- 267ms 2023-07-21T22:45:09.495 INFO:teuthology.orchestra.run.smithi132.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -1192us[-1157us] +/- 66ms 2023-07-21T22:45:09.496 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +1485us[+1519us] +/- 81ms 2023-07-21T22:45:09.496 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:45:09.497 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-21T22:45:09.512 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-21T22:45:09.513 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-21T22:45:09.529 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T22:45:09.632 DEBUG:teuthology.orchestra.run.smithi132:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T22:45:09.755 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T22:45:09.847 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T22:45:09.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:45:09.881 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T22:45:09.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:45:09.915 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T22:45:09.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:45:09.947 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-21T22:45:09.959 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-21T22:45:09.974 INFO:teuthology.task.internal:Duration was 2308.816784 seconds 2023-07-21T22:45:09.974 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-21T22:45:09.985 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-21T22:45:09.985 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T22:45:09.988 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T22:45:09.990 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T22:45:10.032 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T22:45:10.035 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T22:45:10.087 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T22:45:10.442 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-21T22:45:10.443 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2023-07-21T22:45:10.443 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T22:45:10.465 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi132.front.sepia.ceph.com 2023-07-21T22:45:10.466 DEBUG:teuthology.orchestra.run.smithi132:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T22:45:10.494 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2023-07-21T22:45:10.495 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T22:45:10.518 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-21T22:45:10.519 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T22:45:10.522 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T22:45:10.538 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T22:45:10.609 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-21T22:45:10.610 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T22:45:10.612 DEBUG:teuthology.orchestra.run.smithi132:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T22:45:10.614 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T22:45:10.716 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-21T22:45:10.731 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-21T22:45:10.731 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T22:45:10.758 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T22:45:10.793 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T22:45:10.819 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-21T22:45:10.831 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:45:10.834 DEBUG:teuthology.orchestra.run.smithi132:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:45:10.837 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:45:10.861 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2023-07-21T22:45:10.875 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = core 2023-07-21T22:45:10.885 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2023-07-21T22:45:10.918 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:45:10.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:45:10.935 DEBUG:teuthology.orchestra.run.smithi132:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:45:10.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:45:10.976 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:45:10.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:45:10.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-21T22:45:11.004 INFO:teuthology.task.internal:Transferring archived files... 2023-07-21T22:45:11.005 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737/remote/smithi107 2023-07-21T22:45:11.005 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T22:45:11.059 DEBUG:teuthology.misc:Transferring archived files from smithi132:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737/remote/smithi132 2023-07-21T22:45:11.059 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T22:45:11.096 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737/remote/smithi190 2023-07-21T22:45:11.097 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T22:45:11.134 INFO:teuthology.task.internal:Removing archive directory... 2023-07-21T22:45:11.135 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T22:45:11.138 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T22:45:11.140 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T22:45:11.185 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-21T22:45:11.199 INFO:teuthology.task.internal:Not uploading archives. 2023-07-21T22:45:11.200 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-21T22:45:11.213 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-21T22:45:11.213 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T22:45:11.216 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T22:45:11.218 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T22:45:11.230 INFO:teuthology.orchestra.run.smithi107.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 22:45 /home/ubuntu/cephtest 2023-07-21T22:45:11.232 INFO:teuthology.orchestra.run.smithi132.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 22:45 /home/ubuntu/cephtest 2023-07-21T22:45:11.242 INFO:teuthology.orchestra.run.smithi190.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 22:45 /home/ubuntu/cephtest 2023-07-21T22:45:11.243 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-21T22:45:11.261 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-21T22:45:11.285 INFO:teuthology.nuke:Checking targets against current locks 2023-07-21T22:45:11.315 DEBUG:teuthology.nuke:shortname: smithi107 2023-07-21T22:45:11.316 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T22:45:11.342 DEBUG:teuthology.nuke:shortname: smithi132 2023-07-21T22:45:11.343 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T22:45:11.346 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:59:02.935156', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-21T22:45:11.367 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:59:02.933574', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T22:45:11.370 DEBUG:teuthology.nuke:shortname: smithi190 2023-07-21T22:45:11.370 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T22:45:11.395 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347737', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:59:02.931783', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-21T22:45:11.405 INFO:teuthology.orchestra.console:Power off smithi107 2023-07-21T22:45:11.406 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T22:45:11.432 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:11.456 INFO:teuthology.orchestra.console:Power off smithi132 2023-07-21T22:45:11.457 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T22:45:11.482 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:11.500 INFO:teuthology.orchestra.console:Power off smithi190 2023-07-21T22:45:11.501 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T22:45:11.526 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:15.470 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:15.514 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:15.554 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:19.584 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:19.625 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:19.666 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:19.879 INFO:teuthology.orchestra.console:Power off for smithi190 completed 2023-07-21T22:45:23.700 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:23.738 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:23.915 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2023-07-21T22:45:23.952 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2023-07-21T22:45:24.057 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}} duration: 2308.8167839050293 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2023-07-21T22:45:24.057 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T22:45:24.161 INFO:teuthology.run:FAIL